Subject: [homenet] Routing protocol comparison document

From: Mark Townsley <[email-masked]>
Date: Sun, 15 Feb 2015 19:33:46 +0100

Dear WG,

In Hawaii, Margaret offered to pull together a document providing a summary
of ISIS and Babel within the context of homenet. Working with Chris Hopps
and Juliusz Chroboczek, Margaret just posted
draft-mrw-homenet-rtg-comparison-01.txt. Many thanks to all 3 authors for
this input, I'm sure it will help the WG make a more informed decision.

While there was a great deal of discussion and collaboration amongst the
authors, this document does not represent the authors' consensus in its
current form. This document is not intended to make a final recommendation
one way or another at this time, rather it is a starting point for those
who want more information but do not want to dive into the entire
repertoire of documents (or code) for ISIS or Babel.

As such, I would like to invite everyone to use this as a starting point to
familiarize yourselves with these two options. State your questions,
comments, and opinions with at as much lively debate as we heard at the
microphone and in the hallways in Hawaii.

Thank you,

- Mark

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sun, 15 Feb 2015 20:21:23 +0100

> Working with Chris Hopps and Juliusz Chroboczek, Margaret just posted
> draft-mrw-homenet-rtg-comparison-01.txt.

This is on

    https://tools.ietf.org/html/draft-mrw-homenet-rtg-comparison

As Mark mentioned, we didn't attempt to reach consensus with this
document -- we tried to produce something that's reasonably informative as
quickly as possible.

Here's my initial list of things that we might consider tweaking for -02:

1. The table in Section 13.3 compares the available implementation of
   Source-Specific Babel with the available implementation of
   Source-Specific IS-IS, which is written in Erlang and requires massive
   resources.  We should add a column with the sizes of Quagga's C version
   of IS-IS -- while this version is incomplete and not source-specific,
   this should give a rough idea of how much we can expect to be able to
   scale down IS-IS.

   (This is a little more work than you might expect, since it requires
   isolating just the isisd, zebra and libzebra bits of Quagga.  I hope
   Steven volunteers ;-) )

2. We didn't discuss the fact that Babel runs over UDP, while IS-IS runs
   directly over layer 2.  This has a number of consequences, most notably
   related to ease of implementation, portability, and the ability to run
   over tunnels (say, GRE or OpenVPN in tun mode).  I refer you to my
   previous posting to this list:

     http://mid.gmane.org/[email-masked]

3. Section 3.3 makes some rather strong claims about IS-IS scalability.
   Since we later argue in favour of a single-area implementation of
   IS-IS, I feel that these claims require some justification.  However,
   since I'm not convinced that scalability beyond 200 nodes or so is an
   important issue for Homenet, I won't try to push this particular point.

-- Juliusz

---

From: Steven Barth <[email-masked]>
Date: Sun, 15 Feb 2015 21:02:44 +0100

Thanks for writing up the document.

As a general comment:
I think it should be added that there is a huge difference in building 
enterprise routers and home routers in terms of budgets for development 
/ support and margins.
This should somehow reflect in the complexity of the specifications and 
reference software we choose or produce in this WG.

A few more specific ones.

wrt. Link State Algorithm
"This information can be used by other applications outside the routing 
protocol itself. Additionally the flooding algorithm has been found as 
an efficient method for other applications to distribute node-specific 
application data, although some care must be taken with this use so as 
not to disrupt the fundamental routing function."

Is this relevant for the homenet case, since this additional information 
(Prefix Assignment, Naming, Security, etc.) is exchanged using HNCP already?

wrt. Algorithm Comparison
"Loop-Avoiding Distance Vector [...] scales badly in extremely dense 
deployments, where a single node has thousands of direct neighbours"

Are thousands of direct neighbors relevant for the homenet usecase?
In general if we are talking about 100-200 routers here we've built OLSR 
and BATMAN mesh networks in that range with WRT54GLs and similar devices 
back then.

wrt. Link Metrics
Do the protocols assign any metrics by default (e.g. based on link speed 
/ delay / reliability)? For Babel it is mentioned that it does so for 
WiFi however is there a generic case? Does IS-IS do something similar?
I mean in the end what good does a 16-bit or 24-bit metric space do us 
in a more or less unmanged environment if all hops "cost" the same by 
default unless otherwise configured?

wrt. Security Features
"We should just state whether each protocol supports auth or encryption, 
and whether it supports symmetric or something more exciting."

I agree. To my knowledge it seemed that none of the protocols supported 
encryption or asymmetric authentication. While we can manage PSKs in 
HNCP to solve the authentication issue we might want to mention the 
potential insight that someone might gain from sniffing the unencrypted 
traffic of either routing protocol.

wrt. IS-IS
"all major router vendors implement IS-IS" "As all major routing vendors 
have (proprietary) IS-IS implementations"

Is this really true for home router vendors? I don't doubt it is for 
enterprise routers but really the only routing protocol I've seen in 
home router firmwares so far is RIP and even that only in very few 
cases. Somewhat related: is source-specific routing in general deployed 
anywhere in the enterprise (or similar) space already?

Cheers,

Steven

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sun, 15 Feb 2015 22:00:30 +0100

> Are thousands of direct neighbors relevant for the homenet usecase?

It's not relevant to any reasonable use case.  There is no widely
deplyed, cheap link technology that can support that many neighbours.

More generally, I think -01 puts undue stress on scaling and convergence
speed.  Sections 3 and 4 can be summarised as "any reasonable routing
protocol scales sufficiently well and converges sufficiently fast for the
needs of Homenet".

-- Juliusz

---

From: Margaret Wasserman <[email-masked]>
Date: Sun, 15 Feb 2015 16:04:27 -0500

> protocol scales sufficiently well and converges sufficiently fast for the

The sections in the document started from a list of routing features/properties that were discussed in the WG meeting.  There was quite a bit of discussion of scaling.  In particular, there was a lot of discussion of how these protocols will scale "on Wi-Fi".  I wasn't entirely sure, during the meeting, what special requirements there are for a routing protocol to scale well on Wi-Fi vs. other link types, but perhaps someone on the WG mailing list can clarify?

Margaret

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sun, 15 Feb 2015 22:10:24 +0100

> The sections in the document started from a list of routing
> features/properties that were discussed in the WG meeting.  There was
> quite a bit of discussion of scaling.  In particular, there was a lot of
> discussion of how these protocols will scale "on Wi-Fi".

Performance and scaling on WiFi is discussed in Section 13.  That's
a different issue from the algorithmic scaling discussed in Section 3.

-- Juliusz

---

From: Markus Stenberg <[email-masked]>
Date: Mon, 16 Feb 2015 11:16:17 +0200

On 15.2.2015, at 20.33, Mark Townsley <[email-masked]> wrote:
> In Hawaii, Margaret offered to pull together a document providing a summary of ISIS and Babel within the context of homenet. Working with Chris Hopps and Juliusz Chroboczek, Margaret just posted draft-mrw-homenet-rtg-comparison-01.txt. Many thanks to all 3 authors for this input, I’m sure it will help the WG make a more informed decision. 

Finally. Thanks to the authors :)

Section 3.3: 

As link state algorithms distribute all state, their (computation, state synchronization) cost is essentially a function of network size and network churn. 

Quote:
   Link state algorithms scales to very large, very dense networks.

I would like to add, ’static’ there, because given dynamic network, it will not simply work, at least not in One Huge Area(tm) mode we seem to currently envision for the homenet. 

I am also not sure what this logical LSP stuff really tells us here, nor why is it relevant. (Low-level protocol detail.)

Section 4:

Convergence of any real routing protocol is ‘fast enough’. 

I find it amusing that it is even discussed, as the main problem is hardware support of detecting link loss in the very heterogenous environment that is the home network.. Think dumb switches, wireless, weird L2s without carrier sense; hell, most _Linux home routers_ do not do good carrier sense on their physical LAN ports.

If you assume the need for L3+ keep-alive, convergence is a rounding error, unless you are very wasteful with the keep-alives.

Section 6.1:

We assume a ‘cleartext password’ for IS-IS in a home? What does this mean?

Section 7.1:

Woah, 24 bits of link metric? Show me the metric, baby! (And as noted by Steven, the extra stuff for prefixes is just not needed assuming HNCP.)

Anyway, I would like to hear in Section 7 in general what the metrics are used for, and how they are computed, in a zero-configuration home network. I hope the answer is not ‘operator can configure wireless to be not preferred over wired’.

Section 10.1:

Is someone actually using IS-IS multicast? Even outside homenet case? Considering there is only one homenet-y IS-IS implementation currently, I assume ‘available implementations’ here refers to all IS-IS implementations? 

Section 11:

Source-specific routing IS required thanks to BCP-38, we do not want to (auto-)tunnel in the home due to crippled routing protocol choice. Is the HOMENET IS-IS production-quality? E.g. is someone using it in production? 

Section 13.1:

How much churn is there in those ‘hundreds of nodes’ deployments, and do they run on $20 hardware per node?

Section 15.1/2:

Marketing, please make it stop.. (I could poke fun at some of the claims but I choose not to.)

Section 15.2.{1,2}:

(Speaking only for myself based on marginal knowledge of what is inside $CHEAP and $NOT-SO-CHEAP routers) 

The major hurdle for any existing vendor is NOT the routing protocol, but the support for source specific routing. For those using Linux, it is relatively cheap => adding support for RP with it is not bad. If using old Linux, or something crippled, it may be horrendously expensive if it involves e.g. drastic changes to how RIB, FIB, or both of them work (FIB may also bey on ASIC and relatively limited in terms of what you can change before it starts costing $$$).

Section {15.3.*, 15.8.1}

What do “operational” networks and my home network have in common? I am not sure.

Section 15.5.1:

I would like to see the significantly pared version of IS-IS, with an implementation, before making any claims. Devil IS in the details sometimes.

To summarize my thinking based on reading the draft:

On high level, looking at the routing protocols’ _implementations_ I could pick up and use right now, so far, I see only one ‘big’ IS-IS implementation with zero users that I know of, and Babel that is used by some hobbyists and has small footprint.

We can make academic choice of IS-IS and hope that a small, robust open-source implementation comes up some day (and works without ‘operator’ twiddling metrics etc); or we can make practical choice and ship stuff this year.

Cheers,

-Markus

---

From: Gert Doering <[email-masked]>
Date: Mon, 16 Feb 2015 10:26:18 +0100

Hi,

On Sun, Feb 15, 2015 at 09:02:44PM +0100, Steven Barth wrote:
> Somewhat related: is source-specific routing in general deployed 

Not in the way homenet/babels tackle this.

Enterprise/ISP space does VRF and per-VRF routing tables, and all that,
or policy based routing with manual configuration.  But not "we have a
single routing protocol that transports (src,dest) routing info" - at
least I've never seen anything in $C or $J land.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 16 Feb 2015 11:10:29 +0100 (CET)

On Mon, 16 Feb 2015, Markus Stenberg wrote:

> The major hurdle for any existing vendor is NOT the routing protocol, 
> but the support for source specific routing. For those using Linux, it 
> is relatively cheap => adding support for RP with it is not bad. If 
> using old Linux, or something crippled, it may be horrendously expensive 
> if it involves e.g. drastic changes to how RIB, FIB, or both of them 
> work (FIB may also bey on ASIC and relatively limited in terms of what 
> you can change before it starts costing $$$).

With current routers out there, they either do forwarding based on pure 
CPU (this would be the ones that can handle up to around 100-200 
megabit/s), or they use packet forwarding hardware acceleration to achieve 
gig speeds.

The first class is "relatively cheap" to create src/dst extensions for, 
especially since if they target OpenWRT that seems to have 3.10 as its 
kernel (and I've seen this kernel pop up in more places).

The second class with hardware acceleration is more complicated, the 
changes they've done to the Linux kernel (often 3.2) is extensive, and 
creating newer kernels is harder.

However, it's my firm belief that when homenet is going to become more 
prevalent (a few years down the line), we're going to have SoCs that do 
gig speed in pure CPU where flash/ram/cpu resources are going to be big by 
todays standards.

On the protocol implementation part, let's not forget that ISIS was 
running on 68k processors in the 90-ties. This on routers that did CPU 
based forwarding at the same time. The current Auto-ISIS implementation 
requires an Erlang environment. This is what uses most of the resources, 
not the Auto-ISIS code itself. If you need more Erlang based software on 
the same box, then the Erlang environment is shared.

If ISIS is chosen, I am sure we'll see additional implementations of this 
in C or similar programming language. I know some rudimentary 
implementations in Python as well, that people wrote in very little time.

More to the point, the L2 way ISIS works seems to not be a significant 
problem.

There are numerous factors that would influence our choice, but one I 
would like to put to rest anyway, both protocols scale CPU-wise to the 
homenet use-case of number of nodes and expected topology we're going to 
see.

---

From: Markus Stenberg <[email-masked]>
Date: Mon, 16 Feb 2015 12:35:47 +0200

On 16.2.2015, at 12.10, Mikael Abrahamsson <[email-masked]> wrote:
> With current routers out there, they either do forwarding based on pure CPU (this would be the ones that can handle up to around 100-200 megabit/s), or they use packet forwarding hardware acceleration to achieve gig speeds.
> 
> The first class is “relatively cheap" to create src/dst extensions for, especially since if they target OpenWRT that seems to have 3.10 as its kernel (and I've seen this kernel pop up in more places).

The extension is also backported even to 2.6.32 stable series (.6something), and also all other currently active stable trees. If a router vendor is using Linux but not basing on stable tree, it is their problem ;)

> The second class with hardware acceleration is more complicated, the changes they've done to the Linux kernel (often 3.2) is extensive, and creating newer kernels is harder.
> 
> However, it’s my firm belief that when homenet is going to become more prevalent (a few years down the line), we're going to have SoCs that do gig speed in pure CPU where flash/ram/cpu resources are going to be big by todays standards.

The trend has been so far reverse; it used to be all-software, and now some hardware forwarding even in homes. Of course, adding e.g. routing to the mix might reverse the trend. Currently lan-lan port traffic is handled usually purely as L2 bridging on about _all_ hardware, but homenet-y router would require routing of the traffic in software or much more complex hardware FIB.

I am not sure ram/flash will grow automatically. Currently there is lots of stuff that ships with 4MB flash still (unlike the optimistic document), because the distro of the vendor’s choice fits into 3.*MB with the set of features the vendor wants to support. As long as there is a market for those low-end chips (and no compelling reason to upgrade for the router vendors), the baseline numbers will not increase despite the Moore’s law. The savings will of course grow less and less, and eventually the economies of scale may force an upgrade, but I am not holding my breath with that. 

For an amusing anecdote of 12 years of home router progress, look at https://en.wikipedia.org/wiki/Linksys_WRT54G_series .. both ram and flash have decreased. (TL;DR: 16->8, 4->2MB respectively)

> On the protocol implementation part, let's not forget that ISIS was running on 68k processors in the 90-ties. This on routers that did CPU based forwarding at the same time. The current Auto-ISIS implementation requires an Erlang environment. This is what uses most of the resources, not the Auto-ISIS code itself. If you need more Erlang based software on the same box, then the Erlang environment is shared.

I am not sure my home devices could afford the memory, but point taken. I guess we can toast to world domination of Erlang soon too ;)

> If ISIS is chosen, I am sure we'll see additional implementations of this in C or similar programming language. I know some rudimentary implementations in Python as well, that people wrote in very little time.

I have written fast hacks in $scripting-language-du-jour, but I am not convinced e.g. IS-IS implementation in C can be considered one. Considering the effort various parties have poured into the two OS IS-IS implementations that seem to be maintained nowadays, and OS IS-IS world domination is still waiting to happen, I am not perhaps as optimistic as you are.

> More to the point, the L2 way ISIS works seems to not be a significant problem.

Agreed. Most of the sections in the document are ’not significant problem’.

> There are numerous factors that would influence our choice, but one I would like to put to rest anyway, both protocols scale CPU-wise to the homenet use-case of number of nodes and expected topology we’re going to see.

Do you care to enumerate the relevant factors from your point of view?

From my point of view, they are more or less:

[1] does it work ‘well’ zeroconf + do source-specific routing (both are MUST)
[2] does it work in the topologies it will get thrown at.
[3] can I ship it this year if I feel like it, using OS code that fits on the devices
- also,  is it ‘economical enough’ in it’s use of computing/memory/bandwidth of the router 

but I am very results oriented. [1] is addressed by both at least on paper. [2] I am not sure about (IS-IS metrics without operator = ? - all I know is the fancy 24 bits). [3] seems to favor Babel currently.

Cheers,

-Markus

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 16 Feb 2015 11:50:23 +0100 (CET)

On Mon, 16 Feb 2015, Markus Stenberg wrote:

> The extension is also backported even to 2.6.32 stable series 
> (.6something), and also all other currently active stable trees. If a 
> router vendor is using Linux but not basing on stable tree, it is their 
> problem ;)

Well, considering the push-back I've been getting when asking for newer 
kernels, I'd say it's their problem, but also our problem.

> The trend has been so far reverse; it used to be all-software, and now 
> some hardware forwarding even in homes. Of course, adding e.g. routing 
> to the mix might reverse the trend. Currently lan-lan port traffic is 
> handled usually purely as L2 bridging on about _all_ hardware, but 
> homenet-y router would require routing of the traffic in software or 
> much more complex hardware FIB.

I didn't even count the L2 switch chip there, that's just for cheap 
connectivity. Homenet won't fit (at least initially) in cheap hardware at 
all. Just the fact that you need src/dst routing means most current 
mid/higher end hardware won't work at all (or will require significant 
microcode update to work).

> I am not sure ram/flash will grow automatically. Currently there is lots 
> of stuff that ships with 4MB flash still (unlike the optimistic 
> document), because the distro of the vendor’s choice fits into 3.*MB 
> with the set of features the vendor wants to support. As long as there 
> is a market for those low-end chips (and no compelling reason to upgrade 
> for the router vendors), the baseline numbers will not increase despite 
> the Moore’s law. The savings will of course grow less and less, and 
> eventually the economies of scale may force an upgrade, but I am not 
> holding my breath with that.

In the race to the bottom, Homenet is unneccearily complicated and for 
those who want to save on the cent level for hardware, for them Homenet is 
completely useless. So for me, this point isn't valid. Looking forward, I 
believe the mobile SoC:s will start to show up in routers as well and this 
is where we'll be headed.

> For an amusing anecdote of 12 years of home router progress, look at 
> https://en.wikipedia.org/wiki/Linksys_WRT54G_series .. both ram and 
> flash have decreased. (TL;DR: 16->8, 4->2MB respectively)

Was WRT54G ever a device an ISP would have bought and deployed to their 
customers? Isn't it the 150USD enthusiast product of today?

> I am not sure my home devices could afford the memory, but point taken. 
> I guess we can toast to world domination of Erlang soon too ;)

Well, most non-compiled languages have the same behaviour. Do you need 
Python on your HGW? Just the pyton executable on my debian system is 2.7 
megabyte in size.

>> homenet use-case of number of nodes and expected topology we’re going 
>> to see.
>

Both protocols seems to handle hundreds of links and hundreds of routers, 
which for me seems like overkill for a homenet.

> but I am very results oriented. [1] is addressed by both at least on 
> paper. [2] I am not sure about (IS-IS metrics without operator = ? - all 
> I know is the fancy 24 bits). [3] seems to favor Babel currently.

ISIS metrics is a tool. If we don't want it, then we don't need it. We can 
just do a fixed metric, or interface speed metric, or interface type 
metric, or combination, or whatever we want.

I would not deploy homenet router in my home if it didn't at least 
mimic:ed my current setup, ie could do gigabit/s forwarding within my 
home. (this is what the L2 switch chip does currently). So for me, it's a 
moot point to be able to deploy immediately becase we have no hardware 
that makes sense to deploy on.

I am not aware of any device that fits your requirements, ie src/dst 
routing at gig speeds (well, that's my requirement, you didn't 
specifically write that), can run mainline Linux kernel, has small flash, 
small CPU, and not a lot of RAM.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Markus Stenberg <[email-masked]>
Date: Mon, 16 Feb 2015 13:00:46 +0200

On 16.2.2015, at 12.50, Mikael Abrahamsson <[email-masked]> wrote:
> Was WRT54G ever a device an ISP would have bought and deployed to their customers? Isn’t it the 150USD enthusiast product of today?

Do you consider ISPs more or less cost conscious than the home users? And yes, I know some ISPs have deployed it at some point, and the non-GL was definitely non-enthusiast product at some point as it transitioned from Linux to VxWorks while chasing smaller footprint.

> Well, most non-compiled languages have the same behaviour. Do you need Python on your HGW? Just the pyton executable on my debian system is 2.7 megabyte in size.

OpenWrt can fit Lua UI equipped OS to 4MB of flash..

Sure, sticking a (full) JVM in a router might be bad idea. Jury is still out on Erlang.

> ISIS metrics is a tool. If we don't want it, then we don't need it. We can just do a fixed metric, or interface speed metric, or interface type metric, or combination, or whatever we want.

Right. But Babel does sensible things by default. So I am not sure ’24 bits’ is really ‘pro’ IS-IS here. (And sensible things in this case are sufficiently dynamic that they do not work well in a link state routing protocol on scale, at least.)

> I would not deploy homenet router in my home if it didn't at least mimic:ed my current setup, ie could do gigabit/s forwarding within my home. (this is what the L2 switch chip does currently). So for me, it's a moot point to be able to deploy immediately becase we have no hardware that makes sense to deploy on.
> I am not aware of any device that fits your requirements, ie src/dst routing at gig speeds (well, that's my requirement, you didn't specifically write that), can run mainline Linux kernel, has small flash, small CPU, and not a lot of RAM.

As long as my ISP does not offer gigabit, I can L2 bridge the LANs still, and route (e.g. wlans, wan port) and get useful performance given current hardware. (Not quite gigabit, but as my upstream is 100mbit and real wireless is also sub-gigabit, it is enough.) I do have 2 upstream connections, and run homenet stuff on existing hardware.

The year most home connections are gigabit I guess this is no longer a valid choice, but I have some optimism about the hardware at that point too.

Cheers,

-Markus

P.S. Still waiting for your criteria list ;)

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 16 Feb 2015 13:31:19 +0100

> Just the fact that you need src/dst routing means most current

I'd like to point out that one of the conclusions of the work of Matthieu
is that source-specific routing can be easily implemented on any
hardware/kernel that supports policy routing -- see Section V.B of
http://arxiv.org/abs/1403.0445 .  This is implemented in Babel, of course.

(Sorry for the blatant self-promotion.  I couldn't resist.)

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 16 Feb 2015 13:44:27 +0100

> implementations in Python as well, that people wrote in very little time.

The Python implementation I'm aware of is not just rudimentary, it's not
done (as in not even having an implementation of SPF).

If you know of any reasonably complete open source implementations of
IS-IS other than the Erlang implementation and the (alpha quality) Quagga
implementation, please let me know (perhaps by private mail), so I can
mention them in -02.

-- Juliusz

---

From: "Toerless Eckert (eckert)" <[email-masked]>
Date: Mon, 16 Feb 2015 12:46:13 +0000

If you allow a clueless question here by someone who hasen't tracked the
details of this space too much:

If I am thinking longer term, I would like to see simple L2 switches in
the home be replaced with homenet switches. Or lets say at least I would
like to make sure that the homenet/OSS work makes it possible to go that
way.

What I don't know is how the PI (platform independent) / PD (platform
dependent) stack should look like to enable this. How useful is the
internal structure in linux for example such that the upper layers can
nicely act as PI enabling white box hardware to  build out easily a PD
layer that uses actual HW-acceleration ?

So far I am mostly aware of API sets like OpenFlow being used as a target
to talk to PD. But I don't think (famous last words ?) that OpenFlow can
be used as a PD below the PI linux kernel code, or can it ?

The other option people seem to like are a bunch of (probably IETF
developed) YANG models specifying sufficient data models to provide the
desired forwarding (I2RS comes to mind). But is there work to define that
for the (S/M,D/M) requirements of homenet and/or do folks think this is a
realistic path to use  for whitebox hardware / PD code ?

Thanks
   Toerless

On 2/16/15 11:50 AM, "Mikael Abrahamsson" <[email-masked]> wrote:

>> document), because the distro of the vendor¹s choice fits into 3.*MB
>> the Moore¹s law. The savings will of course grow less and less, and
>>> homenet use-case of number of nodes and expected topology we¹re going
>>> to see.
>>
>> [1] does it work Œwell¹ zeroconf + do source-specific routing (both are
>>MUST)
>> - also,  is it Œeconomical enough¹ in it¹s use of
>>computing/memory/bandwidth of the router
>>

---

From: Ray Bellis <[email-masked]>
Date: Mon, 16 Feb 2015 13:17:41 +0000

> On 16 Feb 2015, at 12:44, Juliusz Chroboczek <[email-masked]> wrote:
> 

Actually, the Chairs would like to know of this too :)

thanks,

Ray

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 16 Feb 2015 14:25:20 +0100 (CET)

On Mon, 16 Feb 2015, Markus Stenberg wrote:

> Do you consider ISPs more or less cost conscious than the home users? 
> And yes, I know some ISPs have deployed it at some point, and the non-GL 
> was definitely non-enthusiast product at some point as it transitioned 
> from Linux to VxWorks while chasing smaller footprint.

ISPs are more cost conscious than home users. Home users do not chose 
providers based on 5-10 cent difference because they're buying millions of 
them.

> Right. But Babel does sensible things by default. So I am not sure ’24 
> bits’ is really ‘pro’ IS-IS here. (And sensible things in this case are 
> sufficiently dynamic that they do not work well in a link state routing 
> protocol on scale, at least.)

I agree that 24 bit ISIS metrics isn't a great argument for ISIS.

> As long as my ISP does not offer gigabit, I can L2 bridge the LANs 
> still, and route (e.g. wlans, wan port) and get useful performance given 
> current hardware. (Not quite gigabit, but as my upstream is 100mbit and 
> real wireless is also sub-gigabit, it is enough.) I do have 2 upstream 
> connections, and run homenet stuff on existing hardware.

I for instance do backups within my home. I would like to have wifi and 
LAN on different subnets, but then I need gig speed routing between these 
LANs.

So since there is nothing (reasonable) I can buy that'll do homenet at gig 
speeds, I don't see the rush. Sure, your use case where you only need to 
load share between two different ISPs and you don't have high speed (= 
more than 100 megabit/s) traffic outside the L2, then current hw works for 
you.

>
> The year most home connections are gigabit I guess this is no longer a 
> valid choice, but I have some optimism about the hardware at that point 
> too.

I do too.

> P.S. Still waiting for your criteria list ;)

What critera list is that?

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 16 Feb 2015 14:26:03 +0100 (CET)

> http://arxiv.org/abs/1403.0445 .  This is implemented in Babel, of course.

Again, the hardware accelerators used in current hw will not necessarily 
support policy routing just because you can configure it in the kernel.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 16 Feb 2015 14:28:17 +0100 (CET)

On Mon, 16 Feb 2015, Juliusz Chroboczek wrote:

> IS-IS other than the Erlang implementation and the (alpha quality) 
> Quagga implementation, please let me know (perhaps by private mail), so 
> I can mention them in -02.

No, they are not complete.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 16 Feb 2015 14:31:53 +0100 (CET)

> Actually, the Chairs would like to know of this too :)

My point was not to say that there were other ISIS implementations (I 
would have said so earlier if that was the case), but that you can write 
ISIS in basically any language (including Erlang and Pyton), so the 
comment about ISIS needing L2 access (and that it isn't IP), isn't 
something that seems to be a real issue in real life.

ISIS can provide topology and get brought up before HNCP has even come up, 
and can provide topology. If this is a feature or a duplication of 
functionality is up to each individual to decide :P

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 16 Feb 2015 14:48:21 +0100

> ISIS needing L2 access (and that it isn't IP), isn't something that
> seems to be a real issue in real life.

As I mentioned previously, it has two consequences:

  - IS-IS cannot be implemented using the portable and familiar RFC 3493 APIs;
  - IS-IS cannot run over a GRE tunnel (without specific support).

Whether these are "real issues in real life" is open to discussion, but
I think we should be mentioning this in -02.

> and can provide topology.

Just like IS-IS, Babel can establish adjacencies and start routing IPv6
before it gets an address, and doesn't need to reconverge when IPv6
renumbers: link-local IPv6 addresses are almost as stable as MAC
addresses.

(Interestingly enough, getting the notion of neighbour association just
right is what caused the last incompatible protocol change, back in 2008.)

-- Juliusz

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 16 Feb 2015 14:58:23 +0100 (CET)

> renumbers: link-local IPv6 addresses are almost as stable as MAC

That is not the same thing as I wrote. Babel can't provide topology, 
correct? You need to be a link-state protocol to do that as far as I have 
understood?

HNCP contains the same thing as a link-state protocol to find out 
topology. HNCP could theoretically use the ISIS topology instead of its 
own, but it couldn't rely on Babel to do the same job, right?

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 16 Feb 2015 15:10:17 +0100

> own, but it couldn't rely on Babel to do the same job, right?

Correct.

-- Juliusz

---

From: Steven Barth <[email-masked]>
Date: Mon, 16 Feb 2015 15:11:52 +0100

> Actually, the Chairs would like to know of this too :)
I think it would be equally interesting to see a specification of that 
homenet-variant/subset of IS-IS. The comparison draft is very vague on 
that front and judging by some on-list and off-list discussions there 
seems to be a lack of consensus for at least some of the details. I mean 
not (yet) having an implementation is one thing, but not even knowing 
what the protocol will look like doesn't really help here either.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 16 Feb 2015 15:26:19 +0100 (CET)

> what the protocol will look like doesn't really help here either.

You mean what the TLVs look like exactly for the src/dst extensions to 
ISIS?

That is being worked on together with the ISIS WG. There are a few 
different ways of doing it, nothing that would fundamentally change the 
inner workings of it as far as I know. It's mostly what TLVs would be 
used.

Or do you want to know what way the current implementation does it?

---

From: Gert Doering <[email-masked]>
Date: Mon, 16 Feb 2015 15:34:31 +0100

Hi,

> something that seems to be a real issue in real life.

Out of interest: how is ISIS done on "Linuxish" devices?  Grabbing 
ISIS packets off the link with libpcap?

This is one of the nice aspects of HNCP and Babel - "just use plain UDP
over link-local v6", so no need to mess with packet capturing etc.

Gert Doering
        -- NetMaster

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 16 Feb 2015 15:43:27 +0100

>> that front

> You mean what the TLVs look like exactly for the src/dst extensions to

No, I don't think that's what Steve means.

Sections 12 and 15.5 of the comparison draft claim that Homenet will use
a single-area subset of IS-IS without the overly exciting features of the
protocol (partition repair, multi-topology, etc.).  To quote:

   For HOMENET we need a level-1 only implementation supporting a common
   topology for IPv4 and IPv6 over broadcast (i.e., ethernet) link
   types.  Additionally, we only require support of the latest extended
   metric TLVs (i.e., not implement legacy metric support).

Now, "everyone" knows that this can be done -- since there is no such
thing as a complete implementation of IS-IS, the IS-IS crowd have a lot of
experience implementing unspecified subsets of the protocol.  However,
those of us with more of an IETF background would feel somewhat more
comfortable if the advocated subset were clearly defined, or even
implemented.  As Markus is keen on repeating, the devil is often in the
details.

-- Juliusz

---

From: Steven Barth <[email-masked]>
Date: Mon, 16 Feb 2015 15:49:34 +0100

> You mean what the TLVs look like exactly for the src/dst extensions to 
Well not all too specific. I want to read or at least scan through the 
specifications that tell me how this homenet-variant of IS-IS would look 
like, i.e. what drafts / RFCs or chapters of these are relevant and 
which aren't.

For IS-IS the comparison draft points me to RFC 1142 which seems to be 
the obsoleted OSI-standard and then straight to the auto-conf and 
source-specific drafts which doesn't really help (at least me) very much.

For Babel I can read: RFC 6126 and draft-boutier-babel-source-specific 
about 50-60 pages and that seems to be it.
What's the equivalent of that for the homenet IS-IS variant?

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 16 Feb 2015 15:53:14 +0100 (CET)

> source-specific drafts which doesn't really help (at least me) very 

Ok, understood. I will investigate.

> For Babel I can read: RFC 6126 and draft-boutier-babel-source-specific about 
> 50-60 pages and that seems to be it.

Check. In the meantime, could we get confirmation that your above "that 
seems to be it" assumption is true for babel? Juliusz?

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 16 Feb 2015 16:07:38 +0100

> ISIS packets off the link with libpcap?

The Erlang version uses a PF_PACKET, SOCK_RAW socket.

The alpha Quagga version appears to have three implementations:

  - on Linux, it uses a PF_PACKET, SOCK_DGRAM socket;
  - on Solaris, it uses STREAMS DLPI;
  - otherwise, it uses BPF.

(I have no idea which of these implementations are functional, I'm just
looking at the source.)

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 16 Feb 2015 16:12:42 +0100

> seems to be it" assumption is true for babel? Juliusz?

There's also draft-chroboczek-babel-extension-mechanism, which is 5 pages
not counting IANA considerations.

-- Juliusz

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 16 Feb 2015 10:16:39 -0500

Hi Juliusz,

> implementation, please let me know (perhaps by private mail), so I can

You keep mentioning that Quagga is "alpha quality" but i am not sure that provides any pertinent information.  The fact that Quagga make a distinction between different levels of quality and maturity and has tagged their current IS-IS release to indicate that it is "alpha quality is only pertinent if you make the same types of distinctions for the Babel code, and the Babel code is at a higher level of stability and maturity.

When you add features, such as source specific routing, to your Babel codebase, do you go through a full formal release process with Alpha, Beta and Production versions?  How many beta testers need to return positive feedback before you have a production release?  If you don't go through that sort of process, do you have some other way of indicating the maturity and stability of the code base?  If so, what stage is the version of Babel that contains the source-specific routing in now, and what level of regression testing, testbed operational testing and success in production environments does that represent?

If I understood correctly, you mentioned in a private thread that the Quagga IS-IS codebase is not being maintained right now.  Is that correct?  I co consider that to be a pertinent fact.

I would also be interested in knowing whether the Quagga version of IS-IS works.  Does anyone have any experience with it? 

You have mentioned that Babel is "in OpenWRT" or something along those lines…  Is the version with source-specific routing included in OpenWRT yet?  If not, what is the process by which that will be included in an OpenWRT release?

How many commercial router vendors have it enabled in their products?  Is it being used on any operational networks.  [BTW, for the person who asked elsewhere "operational network"  means "a network that is currently working to allow people do real work, not just to test your software", it doesn't mean that the network is managed or involves an "operator".  The Verizon network in my home is quite "operational", as is the one here at my hotel.

It is quite possible that your specific Babel codebase is better maintained, more completely tested, in use on more operational networks, and more functional than the Alpha-quality Quagga release of IS-IS, but I would like to understand that in terms that are meaningful.

Thanks,
Margaret

---

From: Markus Stenberg <[email-masked]>
Date: Mon, 16 Feb 2015 17:57:25 +0200

> You have mentioned that Babel is "in OpenWRT" or something along those lines…  Is the version with source-specific routing included in OpenWRT yet?  If not, what is the process by which that will b…

It is - in routing feed, there is both traditional, as well as source-specific Babel version package. I assume this will go away once source-specific code shows up in the next main release.

> How many commercial router vendors have it enabled in their products?  Is it being used on any operational networks.  [BTW, for the person who asked elsewhere "operational network"  means "a networ…

If you use that wide ‘operational’ definition, almost any routing daemon ever written (minus AutoISIS perhaps) is in (for some definition) wide operational use.

Very few people write them just for fun and never use them. I believe whole Babel effort started ‘operational’, using that definition.. I am not sure if the meaningful is useful if it is really that broad. 

Cheers,

-Markus

P.S. I have babels (source-specific OpenWrt package) in operational use.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 16 Feb 2015 16:58:46 +0100

> You keep mentioning that Quagga is "alpha quality" but i am not sure that
> provides any pertinent information.

Oh, sorry if I wasn't clear.  Quagga has excellent, production quality
implementations of OSPF, RIPv2 and BGP, and a simplistic but workable
implementation of RIPng.  On the other hand, their IS-IS documentation
says:

    isisd is ALPHA quality at the moment and hasn't any way ready for  pro-
    duction use.

    isisd eats bugs for breakfast. If you have food for the maintainers try
    http://bugzilla.quagga.net

> If I understood correctly, you mentioned in a private thread that the
> Quagga IS-IS codebase is not being maintained right now.  Is that correct?

According to their commit history, the last significant changes date from
November 2012.  Of course, it is possible that work is being done out of
tree, perhaps David can clarify.

> When you add features, such as source specific routing, to your Babel
> codebase, do you go through a full formal release process with Alpha, Beta
> and Production versions?

Significant new features live in a separate branch, usually for months or
even years.  They are carefully reviewed, then merged into trunk.  After
they've had some time to (not) break, I make a release.

For example, the RTT-based metric was implemented by Baptiste Jonglez
during the summer of 2013.  It was exhaustively tested, and almost
completely rewritten in November 2013.  It was carefully reviewed by
myself in April-May 2014, and merged into trunk on 16 May 2014.  It was
further tweaked by both Baptiste and myself, all the tests were redone,
and it was released as babeld 1.5.0 on 22 May 2014.  So that took almost
a year, notwithstanding the friendly bullying by the company who wanted
the feature (no, they don't pay me, and I'm not under NDA; full disclaimer:
they bought me lunch once).

Similarly, the source-specific extension was first implemented by Matthieu
Boutier in the spring of 2013, then extensively modified.  We have been
doing serious testing since early 2014.  I've started systematically
reviewing his patches in the summer of 2014 (which involved some changes
to the data structures, Matthieu wasn't pleased), but then I ran out of
time.  I've set aside some time in February to finish the merge, at which
point I plan to let it simmer for a few weeks before releasing 1.6.0.

> You have mentioned that Babel is "in OpenWRT" or something along those
> lines… Is the version with source-specific routing included in OpenWRT
> yet?

Yes, in the package babels, which is separate from the mainline babeld
package.  Users have been complaining about this situation, which is why
I need to finish the merge as soon as I have the necessary peace of mind.

> what is the process by which that will be included in an OpenWRT release?

After I release babeld-1.6.0, hopefully in early March, Gabriel Kerneis
will include it in OpenWRT (he usually does so within a couple of days)
and Stéphane Glondu will include it in Debian (he usually takes longer).
At that point, we'll discuss on babel-users whether we want to delete the
OpenWRT babels package, or make a dummy package to ease the transition,
and Gabriel will make the final decision.

> How many commercial router vendors have it enabled in their products?

None, as far as I'm aware.  It is widely available for researchers to
experiment with, and is enabled by default in CeroWRT.

I hope that this clarifies things, please do not hesitate to tell me if
you feel that I make claims that are not backed by hard data.

-- Juliusz

---

From: Markus Stenberg <[email-masked]>
Date: Mon, 16 Feb 2015 18:11:36 +0200

> Very few people write them just for fun and never use them. I believe whole Babel effort started ‘operational’, using that definition.. I am not sure if the meaningful is useful if it is really tha…

Argh. ‘meaning is useful’ is what I planned to type, sorry for spam. At any rate, 15.3.1 definition is then rather suspicious as it says essentially only ‘widely used’ (and ‘wildly successful’ and other earlier related claims seem to stress that point too).

As Mikael says, the number of homenet WG installations is 0 right now, and it remains to be seen what is wildly successful there.

Cheers,

-Markus

---

From: Gert Doering <[email-masked]>
Date: Mon, 16 Feb 2015 19:08:48 +0100

Hi,

>   - on Linux, it uses a PF_PACKET, SOCK_DGRAM socket;

I was aware that you can use that to *send* raw frames, but didn't know
you could use that to receive as well (done via joining the [ethernet]
multicast groups).

Interesting.

thanks,

Gert Doering
        -- NetMaster

---

From: Steven Barth <[email-masked]>
Date: Mon, 16 Feb 2015 19:27:24 +0100

> Out of interest: how is ISIS done on "Linuxish" devices? Grabbing ISIS 
> packets off the link with libpcap?
libpcap is just the higher-level userspace interface to PF_PACKET on 
Linux. That said in any case you have to be a bit careful with PF_PACKET 
sockets and their settings especially if you use them to broadly (e.g. 
capture all L3 messages) your forwarding performance will go down even 
if you have a BPF-filter attached since the kernel has to copy / inspect 
a lot of packages.

---

From: Margaret Wasserman <[email-masked]>
Date: Tue, 17 Feb 2015 02:49:25 -0500

Thanks, Juliusz.  This was very helpful to me in understanding the status of these implementations.

Margaret

---

From: Ray Hunter <[email-masked]>
Date: Wed, 18 Feb 2015 13:45:23 +0100

Mark Townsley wrote:
>
> Dear WG,
>
> In Hawaii, Margaret offered to pull together a document providing a 
> summary of ISIS and Babel within the context of homenet. Working with 
> Chris Hopps and Juliusz Chroboczek, Margaret just posted 
> draft-mrw-homenet-rtg-comparison-01.txt. Many thanks to all 3 authors 
> for this input, I'm sure it will help the WG make a more informed 
> decision.
>
> While there was a great deal of discussion and collaboration amongst 
> the authors, this document does not represent the authors' consensus 
> in its current form. This document is not intended to make a final 
> recommendation one way or another at this time, rather it is a 
> starting point for those who want more information but do not want to 
> dive into the entire repertoire of documents (or code) for ISIS or Babel.
>
> As such, I would like to invite everyone to use this as a starting 
> point to familiarize yourselves with these two options. State your 
> questions, comments, and opinions with at as much lively debate as we 
> heard at the microphone and in the hallways in Hawaii.
>
> Thank you,
>
> - Mark
>
>
I have read this draft, as well as the WG discussion up until 18/2. 
Thanks to the authors for compiling this document.

I think there are still several clarification/improvement points to be 
made, even if we assume that this draft will never be published/serve as 
a useful reference beyond the life of the WG.

Section 4. seems to be largely irrelevant as currently written as far as 
I can see.

What would interest me more is whether there is a practical difference 
in behaviour given an unstable link in the Homenet network e.g. a 
powerline link that suffers from interference and bounces regularly.

Does IS IS behave differently to Babel when one link is unstable in the 
Homenet?

Is the forwarding freeze during link-state calculation at all 
significant, given the claimed convergence times?

Is there a significant difference in the (native) detection of neighbor 
down between the two protocols?

Or do they rely heavily on interface/link/carrier down?

I also don't currently see anything about interfaces to low power devices.

Is there a significant difference between Babel and IS IS behaviour in a 
completely stable network, especially with respect to battery life?

Section 12.

 >  The code size of IS-IS depends greatly on what aspects of the 
protocol have been implemented.

When talking about IS IS, I'm not entirely sure what we would be 
importing into Homenet if we did a #include IS-IS , and also what 
incompatibilities that might generate given multiple implementations 
with varying feature sets.

This comment has already been raised by others and I agree it requires 
further clarification. Exactly which feature set of IS IS are we talking 
about?

I also think there should also be more explicit links back into the 
Homenet Architecture document 
https://datatracker.ietf.org/doc/rfc7368/?include_text=1.

Especially section 3.5 where we agreed requirements for the routing 
protocol.

Perhaps a table with "complies" "does not comply" or "partially 
complies" per paragraph or phrase?

I'd also be interested in reading about the practicality of my 
smartphone being able to participate as a (stub) Homenet router. I think 
that's a nice use case for "running code."

Would I need to jailbreak my phone to be able to load IS IS? or Babel? 
Especially re: access to operating system APIs?

If it's not there today, how difficult would it be for a 3rd party App 
Developer to build such a solution?

Or would users have to depend on the phone manufacturer incorporating 
Homenet out of the shrink wrap?

-- 
Regards,
RayH

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 18 Feb 2015 15:19:28 +0100

> Homenet Architecture document
[...]
> Perhaps a table with "complies" "does not comply" or "partially complies"
> per paragraph or phrase?

Good point.  Here's a quick reading of Section 3.5 of RFC 7368:

  - border discovery: out of scope, done by HNCP.

  - self configuring: yes for both, see Section 5.

  - behaviour during reconfiguration/reconvergence:
     - both protocols are able to route IPv6 before they get an address;
     - Babel will avoid creating loops even when reconverging, but might
       create blackholes; IS-IS will collapse during reconvergence;
     - HNCP doesn't depend on unicast, so it's not impacted by routing
       reconvergence; however, the loops created by IS-IS might create
       enough wireless interference to impact the performance of HNCP.
    This could be a useful addition to Section 4.

  - the homenet routing protocol should be based on a previously deployed
    protocol: IS-IS is more widely deployed in carrier networks, Babel is
    more widely deployed in small networks.  As for the source-specific
    extensions, only the Babel implementation has seen deployment.

  - v4 support: both protocols are double-stack.

  - only Babel is able to deal intelligently with multiple link
    technologies.  This argues for expansion of Section 7.

  - multihoming support: yes for both, see Section 6.

  - LLNs and other non-transit networks: a tiny stub implementation of
    Babel is available.  It is not known whether a small stub-only
    implementation of IS-IS is possible (I think it isn't -- an IS-IS
    router must participate in reliable flooding, and must perform SPF).
    This argues for expansion of Section 8.

-- Juliusz

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Wed, 18 Feb 2015 14:28:29 +0000

>       create blackholes; IS-IS will collapse during reconvergence;

I would say the IS-IS will create micro-loops and blackholes during
reconvergence. I wouldn¹t say it would ³collapse². Note that these issues
have been addressed with extensions in SP networks but I¹m not sure we
should consider this for the homenet discussion. Also, I¹d point out that
ISIS would converge extremely fast in smaller networks.

>_______________________________________________
>homenet mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/homenet

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 18 Feb 2015 15:52:46 +0100

> ISIS would converge extremely fast in smaller networks.

On wired networks, that's certainly true -- IS-IS and OSPF don't need to
guarantee good behaviour during reconvergence, since the combination of
stable links with a reliable flooding protocol gives high probability
bounds on reconvergence time.

Is there any experimental data that measures the behaviour of IS-IS's
reliable multicast subprotocol over the 802.11 MAC, in the presence of
marginal links?  Recall that 802.11 multicast is unprotected by ARQ, and
that excessive distributed traffic may cause beaconing to fail, causing
stations to disassiociate.

FWIW, I've had some unpleasant surprises with earlier versions of Babel
causing the 802.11 MAC to collapse for minutes (!) after a major network
disruption (rebooting the whole network simultaneously).  Fixing that
particular issue involved making reconvergence less aggressive, which of
course harmed convergence speed on wired links (which I don't care
about -- anything converges fast enough on Ethernet).  Granted, that was
in IBSS mode, infra mode probably behaves better.

-- Juliusz

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Wed, 18 Feb 2015 15:04:14 +0000

>stations to disassiociate.

I’ve only heard the stories of these problems in IEEE 802.11 networks. One
would hope that basic multicast problems at Layer-2 wouldn’t permeate up
into the routing protocol…  Having said that, I know of no standard
extensions to ISIS to address this problem. I‘ve also heard there are
proprietary fixes that haven’t made it into the IEEE standards but by no
means claim to be an expert in this area.

Thanks,
Acee

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 18 Feb 2015 16:07:07 +0100

> I'd also be interested in reading about the practicality of my smartphone
> being able to participate as a (stub) Homenet router.

Babel has been ported to Android, but most of the Android work is being
done with OLSR, notably within the Commotion Wireless project.

    https://commotionwireless.net/

There are two fundamental reasons why the Commotion software requires
a rooted phone, and only works on selected hardware:

  - manipulating the kernel's routing table requires being root;
  - stock Android doesn't support IBSS (ad-hoc) mode.

The latter point is hopefully not relevant to Homenet, but the former is
probably impossible to solve without action from Google.

-- Juliusz

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 18 Feb 2015 16:50:40 +0100 (CET)

>       create blackholes; IS-IS will collapse during reconvergence;

Collapse?

>    This argues for expansion of Section 8.

This is being investigated, I hope to have an answer by next week.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 18 Feb 2015 17:13:25 +0100

> Collapse?

Okay.  I'll reformulate that.

I am not aware of any results in the open litterature that describe the
behaviour of single-area IS-IS during reconvergence.

I am not aware of any results in the open litterature that describe the
performance of IS-IS's reliable flooding, and hence its convergence speed,
over the 802.11 MAC in the presence of marginal links.

-- Juliusz

---

From: Jim Gettys <[email-masked]>
Date: Wed, 18 Feb 2015 11:20:01 -0500

> over the 802.11 MAC in the presence of marginal links.
​Note that it is exactly collapse that ​
​worries me most (having seen it first hand, in the most extreme possible
way, in the 802.11s mesh we used at OLPC).​  In that case, there was (maybe
still is) an N squared algorithmic issue in a "dense mesh", so scaling the
network up failed entirely at dismayingly few number of nodes.
                                                  - Jim

---

From: Dave Taht <[email-masked]>
Date: Wed, 18 Feb 2015 15:22:59 -0800

I wanted to note that I embrace and endorse Juliusz´s and Markus´s comments
on this thread, and most of the rest of the discussion seems pretty
sensible.

Some random comments:

* I miss the days when rip was ubiquitous. When you needed a routing
protocol, you
generally *really* needed one, and being reduced to bridging everything is
a PITA.
Sure, rip sucked, but it was helpful to always have around.

* While ram and flash have grown to be essentially "free" I really dont see
home router and cpe makers rushing to embrace slower languages or bigger
flash
and memory requirements anytime soon.

Requiring Erlang in a router is a complete non starter for me. Proposing
java is almost palatable - a headless android, for example -
but these are devices that are presently struggling to forward packets at
speeds faster than 100mbit.

I incidentally happen to LIKE Erlang - hot patching is useful (and recently
entered the linux kernel itself). But the market has spoken here. (It also
seems to be speaking for the resurgence of c-replacement compiled languages
like rust and go - I was
amazed to see folk reimplementing coreutils in rust recently)

* There are other routing protocols out there - OLSR, already mentioned, is
popular, and there are others at layer 2, notably batman and 802.11s.

I would like *all* these routing protocols to gain support for source
specific
routing. It is a really worthy feature; it is really needed.

I have generally ruled out batman because it is A) linux-only and B) at
layer 2.

My experience, like jim´s with 802.11s, was horrible, but for all I know it
can be improved, and there is pretty general support for it out there. At
least some testing with 802.11s + another routing protocol connected to
ethernet connected to 802.14 would be sane.

* I, to this day, have never really got the whole link state argument. With
DV, if I want to know the topology, I do a traceroute.

* Moving forward, there are a couple concerns I would like to see
addressed, or declared out of scope. Maybe they already have, I honestly
havent been reading much homenet stuff for a year now. I got stuck on
waiting for hnetd to stablize before more fully rolling out source specific
routing than I have (3 exit nodes, not 6 as planned, and a hacky non-hnetd
solution for address distribution). I do plan a bigger deployment as part
of the make-wifi-fast project.

A) I would like, for one, to actually be able to use a routing protocol to
do multi-homing to multiple ISPs.

The advantages of BGP multi-homing for very large businesses are
undeniable, and I do dearly wish the same capabilities were available to
smaller business lacking the resources to obtain and manage an AS number,
but desiring to mitigate their risks of an internet outage. And there is an
advantage of having such an ability to the ISPs themselves - they net more
customers.

But I know I am dreaming.

B) Still non-discussed here is any sane interface to RPL or whatever the
IoT world is going to do if it doesnt just all jump on wifi. I really do
not know how that is going to go, I have now seen wifi enabled devices in
the 2 dollar range. Not that they have any room for even a babel stub.

C) I would like to see more corporate, and academic, and other ongoing
financial support dedicated to improving and maintaining the code.

The level of investment into the people working on core infrastructure like
quagga, babel, bird, ISIS, etc is pathetic. I am seeing a bit of an uptick
in things with the increased popularity of SDN (cumulus is doing some good
work), but still nothing even remotely close to the level of investment
into switching technologies like VXLANS and trill and so on.

If mainstream manufacturers such as d-link, netgear, etc, do not have
something useful and easy to deploy, they will continue to ignore homenet.

D) Certainly the IS-IS folk would be welcomed to try their stuff at the
upcoming battlemesh. The ospf people also. Under test there are OLSR,
batman, BMX, 802.11s, and static routing.

(There was recently some AWESOME work done on improving linux´s fib table
handling, btw.
https://plus.google.com/u/0/107942175615993706558/posts/aLHkUTqpzkR I am
dying to see the results on home routers)

I do keep hoping, with bake-offs like that, more resources poured into
getting it right, and more testing, that better metrics, techniques, and
code will emerge that make for clear winners among the protocols, and
across the various media types of concern. Thus consensus can be reached,
and world peace, achieved.

---

From: Michael Thomas <[email-masked]>
Date: Wed, 18 Feb 2015 22:27:04 -0800

On 02/18/2015 03:22 PM, Dave Taht wrote:
> generally *really* needed one, and being reduced to bridging 
> everything is a PITA.
> home router and cpe makers rushing to embrace slower languages or 
> bigger flash

Please, please make me a headless android router. I'll even write an app 
for it :)

But we're not talking about an interpreted language in the forwarding 
plane, right? Is the load
from routing protocols we're talking about likely to have any noticeable 
effect on the the forwarding
rate? even when you're running hot on the routing daemon side, you're 
not too likely to be running hot
on the forwarding side because something is hosed, right?

Mike, in the specific homenet context, of course

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 08:43:26 +0100 (CET)

On Wed, 18 Feb 2015, Dave Taht wrote:

> * While ram and flash have grown to be essentially "free" I really dont 
> see home router and cpe makers rushing to embrace slower languages or 
> bigger flash and memory requirements anytime soon.

That's because there has been no requirement to do so, most of the time. 
Basically a device has been sold with a certain amount of features, and 
this featureset hasn't changed over time, thus there is no need to 
future-proof. I see this changing.

Also, currently most routers consists of mostly L2 high speed forwarding, 
with some L3 thrown in between two ports (the WAN port, and the 5th 
internal port to the 5 port switch chip with 4 external ports). With 
homenet, all this changes. Now all ports need to be L3. This is a huge 
change both when it comes to bandwidth needed from the CPU to the ports, 
and also L3 performance needed. This will require home gateway 
manufacturers to make very different devices.

We also have the (previous) mobile SOC:s trickling down into the home 
router space, and these are very price/performance efficient.

> java is almost palatable - a headless android, for example - but these 
> are devices that are presently struggling to forward packets at speeds 
> faster than 100mbit.

We're talking about a protocol decision here. People seem to focus a lot 
on the "running code" part here. ISIS is used for numerous things of apart 
frmo the MPLS and Traffic engineering space, we also have IEEE 802.1aq 
(SPB) and TRILL, it's also used in the GMPLS control plane. There are 
probably 10-20+ commercial implementations of ISIS, if not perhaps the 
exact TLVs we're talking about here. Most of not all of these standards 
are described in standards track RFCs.

Against this, we have Babel, which as far as I can tell has a single 
implementation that has been forked into two, and is based on a few 
experimental RFCs.

> I have generally ruled out batman because it is A) linux-only and B) at 

What platforms do the current implementations of Babel support?

> My experience, like jim´s with 802.11s, was horrible, but for all I know 
> it can be improved, and there is pretty general support for it out 
> there. At least some testing with 802.11s + another routing protocol 
> connected to ethernet connected to 802.14 would be sane.

Do we really believe people are going to use wifi links to connect devices 
within their homenet? Also, won't these be 5GHz and avoid a lot of the 
problems we've seen historically with Wifi? Also, don't we think IEEE will 
improve the behaviour of 802.11? It seems to me people are advocating for 
routing protocol selection based on yesteryears problems.

> * I, to this day, have never really got the whole link state argument. 
> With DV, if I want to know the topology, I do a traceroute.

That will give you the path there. That is not the same as topology. We 
need topology for some things, that's why HNCP is a "link state" protocol.

> A) I would like, for one, to actually be able to use a routing protocol 
> to do multi-homing to multiple ISPs.

That is already a requirement as far as I know.

> smaller business lacking the resources to obtain and manage an AS 
> number, but desiring to mitigate their risks of an internet outage. And 
> there is an advantage of having such an ability to the ISPs themselves - 
> they net more customers.

That is exactly what homenet is all about. You're not dreaming.

> financial support dedicated to improving and maintaining the code.

What code? Routing protocol? Hnetd?

> The level of investment into the people working on core infrastructure 
> like quagga, babel, bird, ISIS, etc is pathetic. I am seeing a bit of an 
> uptick in things with the increased popularity of SDN (cumulus is doing 
> some good work), but still nothing even remotely close to the level of 
> investment into switching technologies like VXLANS and trill and so on.

The Erlang ISIS code is aimed also for this kind of market. I don't have 
details, but I've heard talk that it's also running on SDN kind of 
devices.

> batman, BMX, 802.11s, and static routing.

Is the "wifi mesh" really the same thing as "homenet"? I'd say the 
requirements are vastly different.

> across the various media types of concern. Thus consensus can be 
> reached, and world peace, achieved.

My employer has so far been very supportive of these kinds of 
implementations. However, we want an extensible protocol (we want to use 
ISIS for more functionality than it's used for currently) and we need 
topology, so babel won't work for us.

I see homenet as primarily wired between the homenet routers. I don't 
believe in wifi for connecting different devices. People don't build 
mobile networks by using the mobile network to connect base stations. They 
use fiber or dedicated point-to-point highspeed microwave links to connect 
the base stations. If you want the homenet to work properly and 
efficiently, then the same thing applies here.

If people are making routing protocol decision based on the fact that they 
think most of the homenet links are going to be current incarnation of 
802.11, then we're lacking consensus on a lot wider range of requirements 
than just the routing protocol. We have different opinions on what homenet 
actually is and what it's going to look like.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 08:54:43 +0100 (CET)

On Wed, 18 Feb 2015, Michael Thomas wrote:

> plane, right? Is the load from routing protocols we're talking about 
> likely to have any noticeable effect on the the forwarding rate? even 
> when you're running hot on the routing daemon side, you're not too 
> likely to be running hot on the forwarding side because something is 
> hosed, right?

The complaints about the Erlang implementation has so far been on memory 
and flash footprint, not CPU resources and affecting performance. The 
above is true for most interpreted languages, such as python etc. When you 
install the environment, they're going to use noticable memory and disk 
space. Next application that needs the environment won't add much to the 
footprint though, it's the environment itself that takes space.

My Python executable on my debian box is 2.7 megabyte. I don't know about 
the rest. So if one wants to run Python on the home gateway, that would 
also use significant space.

But forwarding will be done the same way regardless of what routing 
protocol we use, that'll be done by the kernel. The routing protocol just 
programs the RIB/FIB.

---

From: Gert Doering <[email-masked]>
Date: Thu, 19 Feb 2015 09:25:54 +0100

Hi,

> implementation that has been forked into two, and is based on a few 

Just to voice something from the opposite side of the spectrum - I'm a
fan of "do one thing and do it well" approaches, and not so much of a
"oh, we have a kitchen sink here, let's see if we can turn it into a 
living room table".

So for me, the fact that ISIS is used for umpteen other things outside
the homenet environment isn't exactly a plus side.  Yes, it is understood
that basic ISIS works (exchanging TLVs, establishing topology) - but if
you look at it from an implementors point of view that does not have
many years of ISIS background, just figuring out which parts of the
heap of potential ISIS use cases they need to implement sounds much more 
time-consuming than just taking the Babels implementation and compiling it 
for the platform of choice...

We're not talking about a routing protocol for every possible use case
here - we're talking about a fairly well defined environment (aka "fairly
small number of devices, IPv4 and IPv6 only, and implementations constrained
by lack of clue on the manufacturer side").

Gert Doering
        -- NetMaster

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 09:52:34 +0100 (CET)

On Thu, 19 Feb 2015, Gert Doering wrote:

> here - we're talking about a fairly well defined environment (aka 
> "fairly small number of devices, IPv4 and IPv6 only, and implementations 
> constrained by lack of clue on the manufacturer side").

Well, to further explain my concern regarding that and current approach, I 
now see that current proposals is to have the HNCP and $ROUTINGPROTOCOL 
split. If we choose Babel, then we've locked outselves into this split and 
HNCP needs to be extended to handle every future functionality needed 
since babel can't do much more than it already does.

For instance the security work now being done on HNCP. Does ISIS already 
offer the same functionality? I can't evaluate security very well, I don't 
know how many active in this working group that can. I would rather use an 
already standardized mechanism for doing this.

It's been a while since the decision to create HNCP (which was originally 
designed so it could work inside a TLV carrying routing protocol) instead 
of carrying the required "homenet" information in the routing protocol. 
The basis for this decision, is that still true (that the routing protocol 
WGs and implementors refused to accept the TLV types needed and the APIs 
needed)?

It's been what, 1.5-2 years since then? We've already noted that HNCP 
looks awfully close to a link state protocol and duplicates quite a lot of 
of functionality.

There is talk about how complicated ISIS is. How complicated is DNCP/HNCP 
going to be when we're done with it? How many LoC is it now?

---

From: Dave Taht <[email-masked]>
Date: Thu, 19 Feb 2015 01:00:32 -0800

On Thu, Feb 19, 2015 at 12:52 AM, Mikael Abrahamsson <[email-masked]> wrote:
> HNCP needs to be extended to handle every future functionality needed since
> babel can't do much more than it already does.
>
> designed so it could work inside a TLV carrying routing protocol) instead of
> carrying the required "homenet" information in the routing protocol. The
> basis for this decision, is that still true (that the routing protocol WGs
> and implementors refused to accept the TLV types needed and the APIs
> needed)?
>
> It's been what, 1.5-2 years since then? We've already noted that HNCP looks
> awfully close to a link state protocol and duplicates quite a lot of of
> functionality.
>

hnetd exclusive of the library dependencies (which I could easily run
a sloccount for, also, if you care)

d@nuc-client:~/git/hnetd$ sloccount --personcost 110000 generic src test openwrt

SLOC    Directory    SLOC-by-Language (Sorted)
12936   src             ansic=12936
5806    test            ansic=5806
417     generic         sh=417
99      openwrt         sh=99

Totals grouped by language (dominant language first):
ansic:        18742 (97.32%)
sh:             516 (2.68%)

Total Physical Source Lines of Code (SLOC)                = 19,258
Development Effort Estimate, Person-Years (Person-Months) = 4.47 (53.59)
 (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
Schedule Estimate, Years (Months)                         = 0.95 (11.35)
 (Basic COCOMO model, Months = 2.5 * (person-months**0.38))
Estimated Average Number of Developers (Effort/Schedule)  = 4.72
Total Estimated Cost to Develop                           = $ 1,178,896
 (average salary = $110,000/year, overhead = 2.40).
SLOCCount, Copyright (C) 2001-2004 David A. Wheeler
SLOCCount is Open Source Software/Free Software, licensed under the GNU GPL.
SLOCCount comes with ABSOLUTELY NO WARRANTY, and you are welcome to
redistribute it under certain conditions as specified by the GNU GPL license;
see the documentation for details.
Please credit this data as "generated using David A. Wheeler's 'SLOCCount'."

Babel on the other hand:

SLOC    Directory    SLOC-by-Language (Sorted)
10737   babeld          ansic=10474,sh=263

Totals grouped by language (dominant language first):
ansic:        10474 (97.55%)
sh:             263 (2.45%)

Total Physical Source Lines of Code (SLOC)                = 10,737
Development Effort Estimate, Person-Years (Person-Months) = 2.42 (29.02)
 (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
Schedule Estimate, Years (Months)                         = 0.75 (8.99)
 (Basic COCOMO model, Months = 2.5 * (person-months**0.38))
Estimated Average Number of Developers (Effort/Schedule)  = 3.23
Total Estimated Cost to Develop                           = $ 638,353
 (average salary = $110,000/year, overhead = 2.40).
SLOCCount, Copyright (C) 2001-2004 David A. Wheeler
SLOCCount is Open Source Software/Free Software, licensed under the GNU GPL.
SLOCCount comes with ABSOLUTELY NO WARRANTY, and you are welcome to
redistribute it under certain conditions as specified by the GNU GPL license;
see the documentation for details.
Please credit this data as "generated using David A. Wheeler's 'SLOCCount'."

-- 
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks

---

From: Markus Stenberg <[email-masked]>
Date: Thu, 19 Feb 2015 11:15:38 +0200

> On 19.2.2015, at 10.52, Mikael Abrahamsson <[email-masked]> wrote:
>> We're not talking about a routing protocol for every possible use case here - we're talking about a fairly well defined environment (aka "fairly small number of devices, IPv4 and IPv6 only, and implementations constrained by lack of clue on the manufacturer side").
> Well, to further explain my concern regarding that and current approach, I now see that current proposals is to have the HNCP and $ROUTINGPROTOCOL split. If we choose Babel, then we've locked outselves into this split and HNCP needs to be extended to handle every future functionality needed since babel can't do much more than it already does.
> 
> For instance the security work now being done on HNCP. Does ISIS already offer the same functionality? I can't evaluate security very well, I don't know how many active in this working group that can. I would rather use an already standardized mechanism for doing this.

Luckily I can. The comparison document unfortunately lacks references, and I am not motivated enough to look it up, but based on the comparison document, it does not. 

IS-IS essentially offers just the good old ‘password’ scheme (and so does Babel). It is not user-friendly, that’s why DNCP has 2 other PKI based modes too (CA hierarchy, trust consensus).

Certainly, we could argue that we use something like mini-HNCP to ‘bootstrap’ IS-IS, but I am not sure that is a clever idea either.

> It's been a while since the decision to create HNCP (which was originally designed so it could work inside a TLV carrying routing protocol) instead of carrying the required "homenet" information in the routing protocol. The basis for this decision, is that still true (that the routing protocol WGs and implementors refused to accept the TLV types needed and the APIs needed)?

At the time there were fears of routing protocol peeing in it’s pants, given sufficiently churny data, and/or political process stalling us for years. I think we have seen the later effect already. I do not believe in the former.

> It's been what, 1.5-2 years since then? We've already noted that HNCP looks awfully close to a link state protocol and duplicates quite a lot of of functionality.

Right. It is essentially bit more modern take on a link state routing protocol. So if you bring it up, I bring up the another argument - why not route using it? Cost of doing _that_ is ~100 LoC (+ whatever fancy thing we want to do with metrics).

Cheers,

-Markus

---

From: Steven Barth <[email-masked]>
Date: Thu, 19 Feb 2015 10:28:24 +0100

> SLOC    Directory    SLOC-by-Language (Sorted)
As a rough estimate of those 12-13 KLOC only about one sixth should be 
related to DNCP. So I very much doubt that hooking it up to a 3rd-party 
daemon for topology/TLV-management would make any noticable impact.

---

From: Gert Doering <[email-masked]>
Date: Thu, 19 Feb 2015 10:29:57 +0100

Hi,

On Thu, Feb 19, 2015 at 11:15:38AM +0200, Markus Stenberg wrote:
> [ HNCP ]
> 
> Right. It is essentially bit more modern take on a link state
> routing protocol. So if you bring it up, I bring up the another
> argument - why not route using it? Cost of doing _that_ is ~100 LoC
> (+ whatever fancy thing we want to do with metrics).

This is actually something I have been wondering about.  Why not use
HNCP to do all the work, when it's already nicely establishing a 
communication infrastructure?

This decision happened before I joined this list...

Gert Doering
        -- NetMaster

---

From: Markus Stenberg <[email-masked]>
Date: Thu, 19 Feb 2015 11:43:35 +0200

On 19.2.2015, at 11.28, Steven Barth <[email-masked]> wrote:
> As a rough estimate of those 12-13 KLOC only about one sixth should be related to DNCP. So I very much doubt that hooking it up to a 3rd-party daemon for topology/TLV-management would make any noticable impact.

Do also note that the current codebase _does_ include the ‘routing’ part too, smirk..

(from dncp-00 branch):

src/dncp*.[ch]: 2.7k LoC
the rest: 12.5k LoC
(’the rest’ seems to be gradually growing but includes e.g. both supported platforms’ C code)

src/hncp_routing.[ch] has 309 lines of code, but it also does routing protocol election (that is being phased out but remains implementation-specific, as current $FOO routing protocol is not really usable in the real world).

Cheers,

-Markus

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 06:22:24 -0500

On Feb 19, 2015, at 2:43 AM, Mikael Abrahamsson <[email-masked]> wrote:
> If people are making routing protocol decision based on the fact that they think most of the homenet links are going to be current incarnation of 802.11, then we're lacking consensus on a lot wider range of requirements than just the routing protocol. We have different opinions on what homenet actually is and what it's going to look like.

Mikael, it is common practice to do air-to-air bridging on home networks now.   We have discussed whether we want to support this in homenet, and concluded that we must.   You are the first person as far as I can recall to suggest that this was optional.   Yes, we do need a routing protocol that works well when some of the transit networks are wifi networks.   This is not an edge case.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 06:26:12 -0500

> For instance the security work now being done on HNCP. Does ISIS already offer the same functionality? I can't evaluate security very well, I don't know how many active in this working group that c…

IS-IS is a routing protocol, not a configuration protocol.   Its security needs are completely orthogonal to what we are doing in homenet.   HNCP can probably provide the tokens required to do IS-IS security, but IS-IS doesn't bring anything to the table as far as homenet security is concerned.   This is basically a red herring: Babel as far as I know doesn't bring anything to the table either.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 12:41:02 +0100 (CET)

On Thu, 19 Feb 2015, Ted Lemon wrote:

> Mikael, it is common practice to do air-to-air bridging on home networks 
> now.  We have discussed whether we want to support this in homenet, and 
> concluded that we must.  You are the first person as far as I can recall 
> to suggest that this was optional.  Yes, we do need a routing protocol 
> that works well when some of the transit networks are wifi networks. 
> This is not an edge case.

Then we need to decide what kind of Wifi network we need to handle.

Because if we're trying to support marginal performance mesh networks that 
might change all the time, lose packets, drop multicast packets randomly, 
etc, then those requirements need to be brought to the discussion.

Basically, Dave Taht and Jim Gettys have been working a lot in these 
"marginal networks". They have a lot of experience. Personally, I don't 
see their kind of networks as something Homenet needs to support. I can 
see us needing to support a few wifi bridging hops, but what they're 
referring to is a quite different beast.

Also, how would HNCP perform on those kinds of networks? If we put that as 
an requirement on the routing protocol, then we need to make sure HNCP is 
also resilient facing these kinds of networks.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 06:52:50 -0500

On Feb 19, 2015, at 6:41 AM, Mikael Abrahamsson <[email-masked]> wrote:
> Basically, Dave Taht and Jim Gettys have been working a lot in these "marginal networks". They have a lot of experience. Personally, I don't see their kind of networks as something Homenet needs to support. I can see us needing to support a few wifi bridging hops, but what they're referring to is a quite different beast.

I think it's reasonable to assume that the network will be somewhat marginal; if it's really seriously marginal though maybe the right thing to do is have a UI for reporting that to the user so that they can try to fix it.   But we want it to work to the extent possible regardless, because we are targeting end-users who may not be competent to respond to UI indications of that sort.

Dealing with marginal networks obviously isn't something that's going to happen in an apartment in Stockholm, but it's pretty easy to imagine something like that where I live out in the country, and both Dave and Jim have similar situations.   This is exacerbated in my case because we used a lot of building materials with metal foils in them, and that messes with radio propagation.   I think in Dave's case it's just distance.

The point is that I can completely understand your skepticism about this, but there's a reason why some working group participants have been pushing for functionality in the more marginal cases.

I don't think HNCP has a particular problem here, but maybe Markus can speak to that.

---

From: Antonio Querubin <[email-masked]>
Date: Thu, 19 Feb 2015 03:17:43 -1000 (HST)

On Thu, 19 Feb 2015, Mikael Abrahamsson wrote:

> "marginal networks". They have a lot of experience. Personally, I don't see 
> their kind of networks as something Homenet needs to support. I can see us 
> needing to support a few wifi bridging hops, but what they're referring to is 
> a quite different beast.

Mesh networks are not necessarilly 'marginal'.  They're about extending 
range and mobility.

Antonio Querubin
e-mail:  [email-masked]
xmpp:  [email-masked]

---

From: Pierre Pfister <[email-masked]>
Date: Thu, 19 Feb 2015 15:50:05 +0100

> Right. It is essentially bit more modern take on a link state routing protocol. So if you bring it up, I bring up the another argument - why not route using it? Cost of doing _that_ is ~100 LoC (+…

First of all, because there are differences between assigning a prefix and using it for routing. 
- Assigning a prefix does not mean you are the best router to route toward it. It may not even mean it is assigned to a link you are directly connected to. It may also be an excluded prefix. Assigning a prefix just means you reserve that prefix for some usage. One of this usage happens to be ‘prefix assignment to a link’, but it may be others.
- There is a delay between the assignment and the actual use of a prefix. This is called ‘Apply Delay’ in the prefix assignment draft. I don’t think routing based on assigned prefix TLVs is a good idea because you would not try to handle assigned prefixes collisions correctly.

Second point is that DNCP is going to transport multiple things. This will imply churn. As Gert said, I think protocols should do one thing and do it well. Babel is a loop-avoiding routing protocol. Do we want to put loop-avoidance mechanism in HNCP ? Of course not. Would HNCP be able to do routing ? Yes. But would it be able to do it if a buggy node constantly sends DNCP updates… It is less sure, and depends on the quality of the implementation. 

Last point is that the routing protocol we will agree on will not be the last we’ll ever come-up with. Some other will be invented. Better ones. DNCP is an element with a set of requirements. Let’s keep DNCP simple. A routing protocol has other requirements. And maybe some day we will decide to move to another routing protocol because we made a mistake or a new better routing protocol was designed. We need to make sure that this transition will be possible. And I think having an independent routing protocol running today will make this transition easier.

Cheers,

- Pierre

---

From: Gert Doering <[email-masked]>
Date: Thu, 19 Feb 2015 15:58:44 +0100

Hi,

On Thu, Feb 19, 2015 at 03:50:05PM +0100, Pierre Pfister wrote:
> Last point is that the routing protocol we will agree on will not
> be the last we?ll ever come-up with. Some other will be invented.
> [..] And
> maybe some day we will decide to move to another routing protocol
> because we made a mistake or a new better routing protocol was
> designed. We need to make sure that this transition will be possible.
> And I think having an independent routing protocol running today
> will make this transition easier.

I disagree with the "it needs to be totally open, extendable, flexible"
line of argument.

Homenet is something that needs to be implemented by *home* CPE vendors,
and interoperate between all those implementations.

Having too much options will make it look like IPSEC.  Nice standards, but
to get two different implementations interoperate, you need a diploma in
troubleshooting.

Gert Doering
        -- NetMaster

---

From: Markus Stenberg <[email-masked]>
Date: Thu, 19 Feb 2015 17:11:24 +0200

> - Assigning a prefix does not mean you are the best router to route toward it. It may not even mean it is assigned to a link you are directly connected to. It may also be an excluded prefix. Assign…

Well, we can officially add ‘I haz this address on this interface’ TLV if we care about it and then semantics would be exactly same as with e.g. IS-IS. Down to both using link state protocol to transmit the information.

> - There is a delay between the assignment and the actual use of a prefix. This is called ‘Apply Delay’ in the prefix assignment draft. I don’t think routing based on assigned prefix TLVs is a good…

Again, detail. Which can be addressed if we care enough. And at much smaller footprint than second link state database.

> Second point is that DNCP is going to transport multiple things. This will imply churn. As Gert said, I think protocols should do one thing and do it well. Babel is a loop-avoiding routing protocol…

Link state _with_ loops is hard. So I am not sure how this applies.

If buggy nodes happen _for anything_, very little is guaranteed. So strawman argument from my point of view.

> Last point is that the routing protocol we will agree on will not be the last we’ll ever come-up with. Some other will be invented. Better ones. DNCP is an element with a set of requirements. Let’s…

I thought the whole thing was to aim for bad transition mechanics; we originally proposed ‘voting’ mechanism which was extensible. Also, at some point, Steven suggested ‘fallback as low priority default’ (while running 0-N routing protocols on top of it with disjoint metric spaces). Again, not accepted. ‘One routing protocol’ does not sound like the option that is extensible in any direction. Except by saying that ‘yeah, we have HNCP2’ with zero-interoperability in either direction.

Cheers,

-Markus

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 16:17:06 +0100

> Is the load from routing protocols we're talking about likely to have
> any noticeable effect on the the forwarding rate?

Here are the figures given by H. Gredler on p.266 of "The Complete IS-IS
Routing Protocol" for a Cisco GRP 1200:

  Routers      Links   SPF time (ms)
    100         250       4,80
    200         500      12,42
    400        1000      31,22
    600        1500      52,94
    800        2000      76,67
    1000       2500     101,94

I have no idea how a Cisco GRP 1200 compares to a non-superscalar, 250 MHz
MIPS processor with a tiny cache.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 16:21:10 +0100

> above is true for most interpreted languages, such as python etc.

Just to clarify -- nobody's complaininng about the Erlang implementation,
which is a fine prototype.  People are complaining about the fact that
there is no implementation other than the Erlang implementation.

While we're willing to believe that it's not difficult to sit down and
write a relatively tight implementation of the (unspecified) Homenet
subset of IS-IS, it is a little worrying that nobody has actually done so.

-- Juliusz

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 16:21:51 +0100 (CET)

On Thu, 19 Feb 2015, Pierre Pfister wrote:

> - There is a delay between the assignment and the actual use of a 
> prefix. This is called ‘Apply Delay’ in the prefix assignment draft. I 
> don’t think routing based on assigned prefix TLVs is a good idea because 
> you would not try to handle assigned prefixes collisions correctly.

Hi,

I was not proposing to use the HNCP TLVs for routing. Actually, I wasn't 
proposing to change the internal use of TLV movement much at all in HNCP, 
I was proposing to move them to IS-IS and use that to synchronize them 
instead of using the current transport. I do not see the fundamental 
difference in syncronizing TLVs between machines by means of HNCP (trickle 
based) and by means of a link state protocol (ISIS).

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 16:35:21 +0100

> homenet, all this changes. Now all ports need to be L3.

I'm possibly missing something, but I see no requirement to perform L3
routing between the LAN ports.

> What platforms do the current implementations of Babel support?

The reference implementation supports Linux and multiple BSD variants
(tested on OpenBSD and Mac OS X).  The Quagga implementation supports
whatever Quagga supports.

At any rate, it is portable code -- making a new port consists in writing
a small number of platform-specific functions to interface with the kernel.

> Do we really believe people are going to use wifi links to connect
> devices within their homenet?

Yes.  The ability to simply and cheaply extend one's wireless coverage is
one of the killer features of Homenet, and one that can be easily explained.

-- Juliusz

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 19 Feb 2015 07:39:38 -0800

> >bigger flash and memory requirements anytime soon.

This worry has been raised by others as well.

> future-proof. I see this changing.

What are the indicators you see ?

> manufacturers to make very different devices.

Sure. But does homenet give them the right user benefits to make it
worth their while ?  Especially for gear that would not be OEM'ed by
SPs but in retail. 

How many commercial vendors are really selling boxes with OpenWRT (no
homenet) ? To me, OpenWRT (without homenet) has already a lot of
benefits over those vendor specific SW on home routers, but seemingly
that hasn't helped much to proliferate OpenWRT into commercial offerings.

> router space, and these are very price/performance efficient.

Just having looked for a good router for homenet, i am not really happy
yet with the choices available. Especially because i'd like to also run a bit more than just basic routing. >= 32MByte memory, >= 16MByte flash for
example. Still seems to be a very limited set of choices.

> within their homenet?

Yes. Simply to extend reach. And if i understand it correctly,
If you have the typical setup:

   Internet - router/AP .... wireless1... client/bridge/AP ---- wired
                                                           .... wireless2

AFAIK, this setup is typically sometthing that only works well if
router/AP and client/bridge/AP come from same vendor due to the complexitites
in AP/bridge looking like multiple clients on wireless1. So it
looks like a good opportunity to highlight how homenet would turn
client/bridge/AP into client/router/AP and eliminate those problems,
allowing vendors of such devices to much easier get added - without
expecting that router/AP is upgrade/changed.

Cheers
    Toerless

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 16:29:36 +0100 (CET)

>  Routers      Links   SPF time (ms)

Cisco 12000 with the original GRP (I guess this is what the book is 
referring to) sees to have used a 200Mhz R5000 processor with 512KB L2 
cache. The platform was released back in 1999 if I remember correctly.

From:

http://www.cisco.com/c/en/us/support/docs/routers/12000-series-routers/40060-12000-boot-process-40060.html

cisco 12410/GRP (R5000) processor (revision 0x01) with 524288K bytes of memory.
R5000 CPU at 200Mhz, Implementation 35, Rev 2.1, 512KB L2 Cache

On modern intel based platforms today, SPF calculations typically take a 
few milliseconds, even though the networks have grown larger.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 16:41:43 +0100 (CET)

> routing between the LAN ports.

Really? Then you and me have fundamental difference in opinion what 
topologies we want to see in homenet. I want to see fewer broadcast 
domains. If we're going to keep the large L2 domains, then we don't really 
need homenet at all. The only reason to have homenet is if you're going to 
have a substantial amount of routers in arbitrary topology, that actually 
route. If we want to L2 switch most traffic, then we have made homenet 
unncessarily complex.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 16:50:35 +0100 (CET)

> What are the indicators you see ?

I see people starting to talk about putting sandboxed applications on the 
HGW:s, so you don't need a new box for each vendor application. I also (in 
NDA talks with some HW vendors) see mobile SoCs trickling down into the 
router space and these have very different performance numbers compared to 
the current low-performance devices.

I also see operators wanting to deploy new services quicker than they can 
today when in the current HGW incarnations most of the functonality is set 
at initial procurement time and then not much more happens over the 5+ 
year lifetime of the device.

> that hasn't helped much to proliferate OpenWRT into commercial 

I know commercial offerings that use OpenWRT. Unfortunately they are 
usually tied to the SoC vendor kernel because of device-specific "board 
support packages". I see these vendors putting pressure on the SoC 
manufacturers to stop doing this, because if you want to deploy new 
services but you're locked to a Linux 3.2 kernel because otherwise your 
packet accelerator won't work, this is a pain point.

So just the way it was really hard to buy "linux compatible" hardware in 
the 90ties (been there, felt the pain, had to buy 3c509), this is less of 
a problem today, and what I see is that more and more operators want to 
get "deeper" into their HGWs and control the software development 
themselves over time. See Free.fr for instance. In the project I am, we're 
not going to choose a device were we don't have significant control and 
can roll out own images and put on the HGWs.

> yet with the choices available. Especially because i'd like to also run 
> a bit more than just basic routing. >= 32MByte memory, >= 16MByte flash 
> for example. Still seems to be a very limited set of choices.

Absolutely. Most devices sold today both to operators and end-users are 
"sell and forget". I am saying I am seeing this trend changing, albeit 
slowly.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 16:52:01 +0100

>> that works well when some of the transit networks are wifi
>> networks. This is not an edge case.

> etc, then those requirements need to be brought to the discussion.

I think that the very minimum requirement is that adding a marginal link
to a working network does not impact the performance of the previously
working part of the network: I don't see myself explaining to my users why
plugging in an extra wireless router in the attic causes their network to
become unusable.

Babel solves this particular issue by a combination of three mechanisms:
(1) automatically assigning a high metric to a marginal link, (2) using
hysteresis to avoid churn due to point 1 above, and (3) by ensuring that
routing loops don't occur even if routing data is not propagated in
a timely manner due to packet loss.

I'm pretty sure somebody sufficiently smart could design a similar scheme
for IS-IS, but be aware that it's somewhat more difficult to get right
than you might expect.  It took me a lot of experimentation to get the
hysteresis algorithm just right, and I'm still working on improving the
link quality algorithm.

-- Juliusz

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 10:52:55 -0500

On Feb 19, 2015, at 10:41 AM, Mikael Abrahamsson <[email-masked]> wrote:
> Really? Then you and me have fundamental difference in opinion what topologies we want to see in homenet. I want to see fewer broadcast domains. If we're going to keep the large L2 domains, then we don't really need homenet at all. The only reason to have homenet is if you're going to have a substantial amount of routers in arbitrary topology, that actually route. If we want to L2 switch most traffic, then we have made homenet unncessarily complex.

So you are, I assume, advocating that all in-home switches also have one subnet per port?   I have a 16-port switch, and will probably have to add another, so that's going to consume a lot of prefixes.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 16:57:41 +0100

> domains.

Ah, I see.

I was focusing on avoiding collision domains that contain both Ethernet
and WiFi.  I agree that the natural progression is to have L3 everywhere.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 17:04:59 +0100

> What are the indicators you see ?

I suspect that the two of you may be making different assumptions.  Mikael
is thinking of ISP-provided CPEs (marketed as "boxes" in Europe), which
are slowly becoming fairly powerful devices.  Toerless is thinking of
Plastic Home Routers, which have been stuck at 4MB flash and 8MB RAM for
as long as anyone can remember.

IMHO, both kinds of devices are necessary to the success of Homenet.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 17:20:21 +0100

> cache.

Since SPF makes random memory accesses, that's probably 10-100 times faster
than a MIPS 4KEc at 250 MHz with its 8KB data cache.

So Homenet probably can afford to make one SPF every second, assuming
a carefully crafted, native-code implementation of SPF.  Which is probably
achievable, but would seem to argue in favour of some smart rate limiting.

You're almost tempting me into trying my hand at it ;-)

-- Juliusz

---

From: Steven Barth <[email-masked]>
Date: Thu, 19 Feb 2015 17:27:39 +0100

>
> I was not proposing to use the HNCP TLVs for routing. Actually, I 
> wasn't proposing to change the internal use of TLV movement much at 
> all in HNCP, I was proposing to move them to IS-IS and use that to 
> synchronize them instead of using the current transport. I do not see 
> the fundamental difference in syncronizing TLVs between machines by 
> means of HNCP (trickle based) and by means of a link state protocol 
> (ISIS).

One of the problems here I think is that it runs on L2, thus you cannot 
simply encrypt and / or asymmetrical authenticate it with anything like 
IPsec or (D)TLS. Therefore you would need to actually define, maintain 
and implement your own IS-IS specific TLS or IKE-like protocol which 
would actually add a lot more complexity than DNCP as a transport does.

Actually one might even argue in the opposite direction that unless the 
IGP offers some fancy dynamic metrics based on e.g. bandwidth or 
reliablity etc., what would be the point of combining HNCP with a 
single-area LSP if you might as well do a simple graph traversal on the 
HNCP topology to generate routes?

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 17:40:17 +0100

> reliablity etc., what would be the point of combining HNCP with
> a single-area LSP if you might as well do a simple graph traversal on the

The Commission for Truth in Advertising requests that I mention that we
haven't evaluated HNCP's behaviour in unstable and marginal networks yet.

(But we will.)

-- Juliusz

---

From: Steven Barth <[email-masked]>
Date: Thu, 19 Feb 2015 18:01:27 +0100

> haven't evaluated HNCP's behaviour in unstable and marginal networks yet.
I agree, however this would need to be adressed anyway and irregardless 
of the chosen IGP that runs alongside it. I mean the IGP working but 
HNCP failing wouldn't really help the case.

My point is just that I don't see much use in moving away from DNCP (and 
the current profile) as a transport for HNCP unless there is something 
better (i.e. supporting asymmetrical cryptography and encryption in a 
sane way and is proven to work in maginal networks).

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 18:04:22 +0100 (CET)

> (1) automatically assigning a high metric to a marginal link, (2) using

How is a "marginal link" defined/detected?

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 18:05:29 +0100 (CET)

On Thu, 19 Feb 2015, Ted Lemon wrote:

> So you are, I assume, advocating that all in-home switches also have one 
> subnet per port?  I have a 16-port switch, and will probably have to add 
> another, so that's going to consume a lot of prefixes.

Yes. I don't see the problem with this.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 18:09:24 +0100 (CET)

On Thu, 19 Feb 2015, Juliusz Chroboczek wrote:

> Since SPF makes random memory accesses, that's probably 10-100 times 
> faster than a MIPS 4KEc at 250 MHz with its 8KB data cache.

I don't know if this is true or not. I know ISIS was running on Cisco 2500 
routers (which are motorola 68030 at 20MHz. I've had these kinds of 
devices involved in the "single level" ISIS domain for a fairly large 
network (I believe it contained 400 routers, 1300 ISIS "nodes" if I 
remember correctly).

> achievable, but would seem to argue in favour of some smart rate limiting.

ISIS already has these kinds of "rate-limiting" of how things are 
happening. In modern core routers this is often tuned to a lot more 
agressive levels, because the defaults came from the early 90ties where 
the same CPU was used for both control plane and forwarding.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 18:19:21 +0100

> How is a "marginal link" defined/detected?

The current implementation uses ETX (MobiCom 2003), which is not very good
but relatively easy to implement portably.  See Appendix A2 of RFC 6126
for the gritty details.

(In true truth the it uses something called Z3, which combines ETX with
interference avoidance, and which I haven't published yet.)

We're planning to do something smarter in the future by hooking into
Linux's rate estimator (the idea is due to Dave), but we've got plenty of
more urgent work.  (Such as merging the source-specific code and writing
a paper about Z3.)

At any rate, the current ETX code will remain for non-Linux platforms.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 18:22:47 +0100

>> smart rate limiting.

> happening. In modern core routers this is often tuned

If you need to tune it, it's not smart enough.

-- Juliusz

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 18:33:23 +0100 (CET)

> If you need to tune it, it's not smart enough.

Can babel achieve 200ms convergence times across a network that might span 
continents, out of the box without changing anything? I doubt it.

I don't believe in fit-all default behaviour. I don't even believe in this 
for TCP even. I tune TCP all the time, because TCP has been designed to 
fit everything from 1200 bps links to 100GE links, it doesn't fit each end 
of the extremes very well. This doesn't mean TCP isn't smart (enough) to 
work well in most cases.

Sure, ISIS could be made smarter when it comes to this, but it all comes 
down to what the requirements are. Right now when we started to evaluate 
routing protocols, people started pitching USP for their perticular 
favorite, and we found out we weren't even on the same page when it comes 
to requirements, and even what a homenet looks like.

---

From: "Leddy, John" <[email-masked]>
Date: Thu, 19 Feb 2015 17:41:44 +0000

Trying to understand the ³marginal link² definition as well:

Seems like ³marginal link² is associated with the IGP behavior (static,
DV, link state)

I¹m assuming:
If the L2 domain signals to the IGP that the link is ³down², the link is
down.  Even if it does it a lot.  You need to ³hide² it from the IGP to
create a ³marginal link².
³marginal link² does not say anything about user QOE, throughput or packet
loss directly.  A link that can hold an adjacency, but is dropping user
traffic, would not be a ³marginal link² in this case.

Good Link - Link that can form an IGP adjacency and hold it consistently
in the absence of a L2 domain down state.
Bad Link - Link that cannot form an IGP adjacency.  It can be because of a
L2 domain down state, or other cause.
Marginal Link - Link that can form but not hold an IGP adjacency
consistently in the absence of a L2 domain down state.

So for a ³marginal link²:

Hide L2 UP/DOWN state from the IGP (if desired)
Form an adjacency
Exchange any routing info
Don¹t send any Hello¹s to actively take down the adjacency
Break adjacency only if routing info updates can¹t be sent or DOWN state
received

Is this close to what a ³marginal link² is?

John

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 18:54:38 +0100

> Seems like ³marginal link² is associated with the IGP behavior

No.  A marginal link is simply one that has a measurable amount of packet
loss.

On a properly functioning wired network, you don't have any marginal
links: a link is either up or down, and a link that is up has negligible
amount of packet loss (the link, not the switches).  A marginal link is
one that's neither up or down -- it's somewhere in between.

This is made somewhat more complicated by layer 2 ARQ (for example, in
802.11b a link that is marginal at the physical layer will usually appear
as merely slow for unicast traffic, but clearly marginal for multicast;
weirder things are possible in 802.11a/g/n).

> Is this close to what a ³marginal link² is?

No.  Being marginal is a property of a link, it has nothing to do with the
IGP.

-- Juliusz

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 13:00:47 -0500

> Yes. I don't see the problem with this.

It means that every single device on a wired network is on a different subnet.   Perhaps it doesn't cause any extreme harm, but it certainly makes managing and debugging the network harder, and it means that you can't have a layer two switch anymore.   So the question I would ask is not "is there a problem with this," because obviously there is, but rather "is there a benefit to doing it this way."   I am curious to know what you think the benefit is.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 13:01:56 -0500

On Feb 19, 2015, at 12:09 PM, Mikael Abrahamsson <[email-masked]> wrote:
> I don't know if this is true or not. I know ISIS was running on Cisco 2500 routers (which are motorola 68030 at 20MHz. I've had these kinds of devices involved in the "single level" ISIS domain for a fairly large network (I believe it contained 400 routers, 1300 ISIS "nodes" if I remember correctly).

I think we can stipulate that in principle all of the routing protocols under consideration can be made to be run in a homenet mode on small hardware.   I don't think that's even an interesting basis for discriminating between them.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 19:08:50 +0100 (CET)

On Thu, 19 Feb 2015, Ted Lemon wrote:

> It means that every single device on a wired network is on a different 
> subnet.  Perhaps it doesn't cause any extreme harm, but it certainly 
> makes managing and debugging the network harder, and it means that you 
> can't have a layer two switch anymore.  So the question I would ask is 
> not "is there a problem with this," because obviously there is, but 
> rather "is there a benefit to doing it this way."  I am curious to know 
> what you think the benefit is.

I am not mandating that each and every device is in its own broadcast 
domain, I am however advocating that we leave the model that has been 
prevalent for 10-15 years at least, ie that a home gateway has a "WAN" 
port and 4 "LAN" ports, and these 4 ports are bridged. I'm saying the 
typical device should have 4-5 "L3" ports. You're then free to connect one 
of these to your L2 switch if you so please.

I would like my router-to-router links to not have a lot of hosts in them 
if I can avoid it.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 13:11:43 -0500

On Feb 19, 2015, at 12:33 PM, Mikael Abrahamsson <[email-masked]> wrote:
> Sure, ISIS could be made smarter when it comes to this, but it all comes down to what the requirements are. Right now when we started to evaluate routing protocols, people started pitching USP for their perticular favorite, and we found out we weren't even on the same page when it comes to requirements, and even what a homenet looks like.

I think it is you who are not on the same page, not the working group.   The working group recently published a document describing our goals, RFC 7368.   Many of the points you have raised as supposed points of disagreement are addressed in the document, although I will admit that it is not always obvious that they have been addressed.   The case of routing over Wifi was discussed at length; the text referring to it in the document is here:

   Due to the use of a variety of diverse underlying link technologies,
   path selection in a homenet may benefit from being more refined than
   minimising hop count.

Regarding your criticism about people pitching their favorite routing protocols, you seem to be doing precisely that, so the criticism seems a bit unfair.   The responses you have been hearing seem like legitimate attempts to address points you have raised on a technical level, not mere advocacy, and it is unfortunate that you would suggest otherwise: I haven't actually seen you raise any technical points in favor of IS-IS, which you seem to be advocating.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 13:13:30 -0500

On Feb 19, 2015, at 1:08 PM, Mikael Abrahamsson <[email-masked]> wrote:
> I would like my router-to-router links to not have a lot of hosts in them if I can avoid it.

Why is that?

---

From: Ole Troan <[email-masked]>
Date: Thu, 19 Feb 2015 19:18:20 +0100

> I am not mandating that each and every device is in its own broadcast domain, I am however advocating that we leave the model that has been prevalent for 10-15 years at least, ie that a home gateway has a "WAN" port and 4 "LAN" ports, and these 4 ports are bridged. I'm saying the typical device should have 4-5 "L3" ports. You're then free to connect one of these to your L2 switch if you so please.
> 

+1.

there are very few shared media around anymore. I don't think I've ever been connected to a 10base5.
why should the IP subnet model emulate a shared medium, when the physical topology is a star.

wireless with security is also a star topology, with a unidirectional broadcast channel.

cheers,
Ole

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 19:22:34 +0100 (CET)

>> I would like my router-to-router links to not have a lot of hosts in them if I can avoid it.

If we're going to be routing multicast within the home, we're most likely 
going to have to use some kind of variant of PIM. Asking the L2 switches 
people connect to the router to support both PIM and MLD snooping seem 
like it might be asking too much.

I might be wrong though.

---

From: "Leddy, John" <[email-masked]>
Date: Thu, 19 Feb 2015 18:23:51 +0000

Wasn’t thinking Quantum Networking would pop up this way…

The “marginal link” can’t simply be measurable packet loss.
There are L2 networks, wired/wireless that can have measurable packet loss
“as properly designed”.
Not good, and you could avoid it, but it is expected behavior.
Reasonable sized 10Base5 networks were fun.

Hopefully the whole Homenet effort is to keep the Home; with multiple MAC
types, speeds, wired and wireless links in a large L2 bridge domain from
being a “marginal link”.

---

From: Ralph Droms <[email-masked]>
Date: Thu, 19 Feb 2015 13:29:54 -0500

> On Feb 19, 2015, at 1:18 PM 2/19/15, Ole Troan <[email-masked]> wrote:
> 

If I can extrapolate and oversimplify a bit, now we've gotten to a fundamental problem: how does a random collection of devices, links and ports sort itself by DWIM into a coherent home network?  How does a device with 16 ports decide to group ports (0-5, 7, 10), (6, 8, 12-15) and (9, 11) into separate subnets?

- Ralph

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 19:32:08 +0100 (CET)

On Thu, 19 Feb 2015, Ted Lemon wrote:

> Regarding your criticism about people pitching their favorite routing 
> protocols, you seem to be doing precisely that, so the criticism seems a

I never said I wasn't. People usually have favorites, I am not excluded.

> bit unfair.  The responses you have been hearing seem like legitimate 
> attempts to address points you have raised on a technical level, not 
> mere advocacy, and it is unfortunate that you would suggest otherwise: I 
> haven't actually seen you raise any technical points in favor of IS-IS, 
> which you seem to be advocating.

It wasn't my intention to imply that people didn't have technical reasons 
to suggest what they do. I have reasons to suggest what I do. They have 
theirs.

If people want to choose babel because it works well facing adverse radio 
conditions in a mesh-networking environment (that I know nothing about), I 
however have to question if this is relevant to the discussion we're 
having here. Is that really a requirement for a homenet routing protocol?

If 802.11 is so bad for multicast, do we even want to run IPv6 over it?

I know Dave T is trying to enlighten IEEE about these issues, which is 
great. Do we presume he (and others) succeed in getting 802.11 fixed, or 
do we choose routing protocol based on current 802.11 behavior?

Or do we say we want to avoid multicast altogether in homenet, and include 
an architecture where the ingress homenet router converts multicast TV 
streams to unicast within the home? That would solve some issues...

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 13:42:29 -0500

On Feb 19, 2015, at 1:22 PM, Mikael Abrahamsson <[email-masked]> wrote:
> If we're going to be routing multicast within the home, we're most likely going to have to use some kind of variant of PIM. Asking the L2 switches people connect to the router to support both PIM and MLD snooping seem like it might be asking too much.

Hm, maybe so.   Thanks for the more detailed answer.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 13:45:50 -0500

> If I can extrapolate and oversimplify a bit, now we've gotten to a fundamental problem: how does a random collection of devices, links and ports sort itself by DWIM into a coherent home network?  H…

What Mikael is saying is that on homenet devices (not switches) each port is treated as a separate network, on the assumption that all of the physical ports on the homenet device are likely to connect to routers, and that any switched topology would be done using switches connected to homenet routers.

I am not convinced by this; one of the issues I see here is that until DNSSD comes up with a solution, we are breaking service discovery on the home network by doing this.   Granted, we already do that from Wifi to wired, but this seems gratuitous.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 13:49:25 -0500

> If 802.11 is so bad for multicast, do we even want to run IPv6 over it?

This would be a great objection to using 802.11 for host networks, but it's a lousy argument against using 802.11 for transit networks!

> I know Dave T is trying to enlighten IEEE about these issues, which is great. Do we presume he (and others) succeed in getting 802.11 fixed, or do we choose routing protocol based on current 802.11 behavior?

We have no choice.

> Or do we say we want to avoid multicast altogether in homenet, and include an architecture where the ingress homenet router converts multicast TV streams to unicast within the home? That would solve some issues...

I don't know.   Homenet multicast is an open issue.   But I don't think this use case represents a serious problem, because as far as I can tell streaming video is not done using multicast in practice anyway.

---

From: Steven Barth <[email-masked]>
Date: Thu, 19 Feb 2015 19:55:08 +0100

> I am not convinced by this; one of the issues I see here is that until DNSSD comes up with a solution, we are breaking service discovery on the home network by doing this.   Granted, we already do…
There is draft-ietf-dnssd-hybrid-00 for which we have specified glued to 
HNCP and a reference implementation. I think DNSSD is actually mostly 
covered or are you referring to any specific issue of that solution?

UPNP (SSDP) is a different beast though and would probably depend on 
having proper multicast routing.
This is from memory but I guess Markus could elaborate here.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 19:55:58 +0100 (CET)

On Thu, 19 Feb 2015, Ted Lemon wrote:

> This would be a great objection to using 802.11 for host networks, but 
> it's a lousy argument against using 802.11 for transit networks!

Please elaborate.

> I don't know.  Homenet multicast is an open issue.  But I don't think 
> this use case represents a serious problem, because as far as I can tell 
> streaming video is not done using multicast in practice anyway.

What do you mean by "streaming video"?

We have approximately 50 IPv6 multicast TV channels in the test network 
currently.

My employer has millions of subscribers using IPv4 multicast TV channels 
currently.

We are not sure this is the best way forward, perhaps we just stop doing 
multicast to the home network and just do multicast to the nearest PoP or 
alike.

---

From: Ralph Droms <[email-masked]>
Date: Thu, 19 Feb 2015 13:56:54 -0500

> On Feb 19, 2015, at 1:45 PM 2/19/15, Ted Lemon <[email-masked]> wrote:
> 

But I think one of the important points for homenet is that many people will just buy "internet" devices, not routers and switches.  I've been out of the loop so I should go back and check the architecture before I say too much more ... what is the expectation for grouping ports on homenet devices and is there an expectation that people will buy homenet devices and switches and know where to use them both?

> I am not convinced by this; one of the issues I see here is that until DNSSD comes up with a solution, we are breaking service discovery on the home network by doing this.   Granted, we already do…

..and, of course, if we solve WiFi-wired for DNS-SD, the solution will likely work for some small number of subnets.

- Ralph

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 14:05:56 -0500

On Feb 19, 2015, at 1:55 PM, Steven Barth <[email-masked]> wrote:
> There is draft-ietf-dnssd-hybrid-00 for which we have specified glued to HNCP and a reference implementation. I think DNSSD is actually mostly covered or are you referring to any specific issue of that solution?

Hm, I will have to try it out.   Is it in a distribution?

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 14:06:31 -0500

On Feb 19, 2015, at 1:55 PM, Mikael Abrahamsson <[email-masked]> wrote:
> My employer has millions of subscribers using IPv4 multicast TV channels currently.

How's that getting through the NAT?

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 14:07:53 -0500

> But I think one of the important points for homenet is that many people will just buy "internet" devices, not routers and switches.  I've been out of the loop so I should go back and check the arch…

I don't think the architecture document says anything about this.   It's how CeroWRT currently operates by default.   I find it to be a royal pain, and in fact that's why I run OpenWRT and not CeroWRT.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 20:09:06 +0100 (CET)

On Thu, 19 Feb 2015, Ralph Droms wrote:

> But I think one of the important points for homenet is that many people 
> will just buy "internet" devices, not routers and switches.  I've been 
> out of the loop so I should go back and check the architecture before I 
> say too much more ... what is the expectation for grouping ports on 
> homenet devices and is there an expectation that people will buy homenet 
> devices and switches and know where to use them both?

As far as I know, this hasn't been discussed that much. From a quick look 
in the arch document, there are all kinds of examples. Looking for 
instance at 3.2.2.1 we have all kinds of topologies, from router-router 
links without any hosts on it, to subnets with 2 hosts on them, to shared 
networks used to inerconnect routers. From what I remember and what I can 
see the document stating, the solution is supposed to handle all kinds of 
topologies.

> ..and, of course, if we solve WiFi-wired for DNS-SD, the solution will 
> likely work for some small number of subnets.

What is "small" here? If it works for 2, why wouldn't it work for 50?

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 20:09:38 +0100 (CET)

> How's that getting through the NAT?

IGMP proxy in the HGW.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 20:11:20 +0100 (CET)

On Thu, 19 Feb 2015, Ted Lemon wrote:

> I don't think the architecture document says anything about this.  It's 
> how CeroWRT currently operates by default.  I find it to be a royal 
> pain, and in fact that's why I run OpenWRT and not CeroWRT.

I'd imagine it's easier to do AQM on routed ports instead of switched 
ports as well, that's where I can imagine CeroWRT choosing this approach.

It's good that we're having this discussions since I seem to not be the 
only one thinking that there should be one port per subnet.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 19 Feb 2015 20:17:34 +0100 (CET)

>> How's that getting through the NAT?

Perhaps should add that we're doing the same thing for IPv6, but using MLD 
proxy instead, of course.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 14:18:21 -0500

On Feb 19, 2015, at 2:11 PM, Mikael Abrahamsson <[email-masked]> wrote:
> I'd imagine it's easier to do AQM on routed ports instead of switched ports as well, that's where I can imagine CeroWRT choosing this approach.

I don't think it is easier to do AQM on routed ports.   If you do the easy version of AQM for switched ports, that's easy; if you do per-port AQM I think it's equally hard in both cases. The plumbing for the L2 solution would look different than for the L3 solution, and that might make it trickier, though.

CeroWRT did this, as far as I understand it, because Dave wanted to shake out cross-subnet issues, not because he felt it was technically preferable in the long run, but perhaps I am misrepresenting his position on the topic.

> It's good that we're having this discussions since I seem to not be the only one thinking that there should be one port per subnet.

Indeed, there are two of you!

---

From: Ralph Droms <[email-masked]>
Date: Thu, 19 Feb 2015 14:27:29 -0500

> On Feb 19, 2015, at 2:09 PM 2/19/15, Mikael Abrahamsson <[email-masked]> wrote:
> 
> As far as I know, this hasn't been discussed that much. From a quick look in the arch document, there are all kinds of examples. Looking for instance at 3.2.2.1 we have all kinds of topologies, from router-router links without any hosts on it, to subnets with 2 hosts on them, to shared networks used to inerconnect routers. From what I remember and what I can see the document stating, the solution is supposed to handle all kinds of topologies.

I promise to go back and re-read the architecture document because I'm having trouble matching the architecture we're discussing here with the desired properties and logical architectures we want the homenet devices to self-organize into.

> What is "small" here? If it works for 2, why wouldn't it work for 50?

In theory, sure.  In practice, 25x (or maybe 25^2?) things to break.

- Ralph

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 19 Feb 2015 14:36:32 -0500

I would note that RFC 7368 says that "simple Layer 3 topologies 
involving as few subnets as possible are preferred in home networks".  I 
presume this is reflective of WG agreement.
While it does go on to note that multiple subnets are sometimes needed, 
mandating that each physical port on the access router be a distinct 
subnet (much less extending such a mandate further in the home) seems to 
violate this agreement.

Yours,
Joel M. Halpern

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 20:45:58 +0100

> A marginal link is simply one that has a measurable amount of packet loss.

Ok, re-reading this exchange, it looks like I may have wrongly assumed
that people are aware of background.  I'll need to put that into the
routing comparison document, this is as good a place as any to draft my
text.

Carrier and enterprise routing protocols are optimised for wired networks,
where a link is either down (drops all packets) or up (drops one packet in
10^10).

Home networks usually include link technologies that have an intermediate
"marginal" state: a link that drops a non-negligible fraction of packets.
With 802.11, in particular, marginal links have fairly unpleasant performance
characteristics.  For example, if the BER is 10^-4,

  - multicast packets are dropped at roughly the nominal rate (10% for 120
    byte packets, 90% for 1500 byte packets);
  - unicast packets are dropped at a much lower rate, but the efficiency
    suffers (0.9 for 120 byte packets, 0.1 for 1500 byte packets).
  - unicast packets have a high probability of being duplicated (no
    figures given, I'm too lazy to look up the size of an ACK frame).

This is of course a very naive analysis -- in practice, errors tend to
come in bursts, and furthermore 802.11a/g/n implements some very complex
lower-layer magic.  Don't believe any results from computation or
simulation, with radio, only actual testing in real-world networks is
trustworthy.

This has two consequences for routing protocols:

  (1) the routing protocol must be able to prefer solid links to marginal
      ones when there is enough redundancy;
  (2) control data may fail to be propagated in a timely manner,
      especially if it is sent over multicast.

1. Prefer solid links

This is pretty obvious.  If you can choose between an Ethernet link and
a wireless link with heavy packet loss, choose the Ethernet.  But there
are more subtle cases that are extremely common in practice:

   Internet --- A --- B....C          --- is Ethernet
                 .        .           ... is WiFi
                  ........

A is your CPE, B is the router in your living room, connected by Ethernet
to the CPE, and C is the wireless router in the guest room.  Yeah, it's
a large house.
                  
Suppose now that the wireless link B..C is rock solid, but the longer A..C
link is marginal.  Murphy's law dictates that the link A..C will have huge
packet loss, but it will be just good enough for the routing instances on
A and C to become neighbours.  If the routing protocol performs hop-count
routing, even if it is smart enough to prefer wired to wireless, it will
prefer the highly marginal route A..C to the rock solid route A-B..C.

Note that this is easily fixed by a competent administrator: just disable
the wireless interface on A.  But this is Homenet, and we're assuming
there's no administrator.

In the mesh community, the hop-count metric is known as "worst-path routing".

2. Control data may be delayed

Since multicast has such massive packet drops, the routing data may be
dropped or delayed.  Using a reliable sub-protocol for propagating
critical data, as is done by OSPF, IS-IS and EIGRP might work if the
reliable protocol can cope with the weirdness of the 802.11 MAC.  If it
cannot, it might (or might not, I don't know) suffer from routing
pathologies, such as routing loops in OSPF and IS-IS, inconsistent DR/DER
election, and SIA in EIGRP.

Of course, people have managed to get link-state protocols to work on
wireless networks, even on highly marginal mesh networks, but they had to
design new mechanisms.  OLSR, for example, doesn't use reliable flooding
and DR election, but instead uses unreliable flooding and MPR election,
together with a mechanism that drops marginal links.  I refer you to
RFCs 3626 and 5449.

Babel uses different mechanisms.  Babel uses a feasibility condition
similar to that of EIGRP, which means that it avoids loops even when
control data is delayed.  Unlike EIGRP, however, it doesn't rely on
a reliable subprotocol, and it doesn't have an active phase.  End-to-end
synchronisation is achieved in a softer manner, and a single router that
fails to propagate requests doesn't prevent reconvergence if there are
alternate paths available -- look Ma, no SIA, even with massive packet
loss.  (This is not meant as a critique of EIGRP -- I rather like EIGRP,
and I'm pretty sure it could be adapted to home networks if somebody spent
a couple of years of their life doing that.)

I'd like to finish this slightly lengthy message by stating that the
current implementation of Babel does not work well in networks where all
links are highly marginal; in other words, Babel really wants a solid
backbone to be available.  In particular, Babel tends to aggregate control
data into large packets, which have a higher probability of being dropped,
and tends to drop links that have more than 90% pre-ARQ drop rate for
full-size frames.  There exist specialised, highly tuned protocols that
are able to deal with topologies that rely on such very marginal links,
notably bmx6, BATMAN-adv, and the highly tuned version of OLSR-ETX used by
the Berlin and Vienna meshes.

-- Juliusz

---

From: Pierre Pfister <[email-masked]>
Date: Thu, 19 Feb 2015 21:06:00 +0100

> I would note that RFC 7368 says that "simple Layer 3 topologies involving as few subnets as possible are preferred in home networks".  I presume this is reflective of WG agreement.
> While it does go on to note that multiple subnets are sometimes needed, mandating that each physical port on the access router be a distinct subnet (much less extending such a mandate further in the home) seems to violate this agreement.
> 
> Yours,
> Joel M. Halpern

+1

Why would we even want to separate these 4 switched ports in the first place ? I know we are L3 people but, this is IMHO, going too far. Don’t kill Ethernet switching. 
- It works well when you don’t have drastically different throughputs. 
- It has non-expensive and efficient hardware home-router vendors are used to put into their devices.
- Home routers mostly don’t have L3 hardware. They are not able to route at 1Gbps.

The goal of introducing L3 routing was to isolate very different link types such as Wifi Vs GigabitEthernet.

I know we *did* separate ports in all the homenet demos at bits-n-bites. Maybe this was sending the wrong message. The purpose was to show the principle of introducing routing in the home. I guess next time it will be 2x2 ports instead of 4x1 ports.

- Pierre

---

From: Henning Rogge <[email-masked]>
Date: Thu, 19 Feb 2015 21:12:28 +0100

> wireless with security is also a star topology, with a unidirectional broadcast channel.

Unfortunately on layer-1 both are still broadcast... you send a
unicast, it might interfere with anyone else in range.

Henning Rogge

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Thu, 19 Feb 2015 20:13:09 +0000

It seems, in each case, that the word "RECOMMEND" is far superior to "MUST". If I connect two ports from the same router to the same LAN, I really don't normally want them in different subnets (although from a specific reason I might choose that). MIF tells us that we can enumerate members of the same subnet on different physical media in certain cases. While that is true, to my small mind it's not a great configuration from a routing perspective.

I would indeed RECOMMEND that different ports on the same router be, in the normal case, connected to different subnets, and I would RECOMMEND that a small simple network be kept as small and simple as meets the design requirements of its operator - which in a home network, is the occupants of the home. If the person is trying to (as I do) connect things that don't move, like TVs, using wired media and connect things that do move using wireless, or to route high volume data by different paths than occasional data flows, there exist requirements that probably don't belong in a document like this, but need to be configurable by a clueful operator.

I'd stick to RECOMMEND, and make it clear that the recommendation targets the general case.
________________________________________
From: homenet [[email-masked]] on behalf of Joel M. Halpern [[email-masked]]
Sent: Thursday, February 19, 2015 11:36 AM
To: Ted Lemon; Mikael Abrahamsson
Cc: [email-masked]; Juliusz Chroboczek
Subject: Re: [homenet] Routing protocol comparison document

Yours,
Joel M. Halpern

---

From: Dave Taht <[email-masked]>
Date: Thu, 19 Feb 2015 12:14:09 -0800

On Thu, Feb 19, 2015 at 11:18 AM, Ted Lemon <[email-masked]> wrote:
> I don't think it is easier to do AQM on routed ports.   If you do the easy version of AQM for switched ports, that's easy; if you do per-port AQM I think it's equally hard in both cases.

In either case AQM and FQ has to be per port in the switch hardware
itself. I have been patiently awaiting for that sort of hardware to
arrive.

On most consumer routers the cpu forwarding path isnt capable of gbit
rates in the first place.

If you are doing software bridging (say between wired and wireless),
you apply fq_codel to the underlying interfaces, not the bridge, and
you are golden, except for dealing with multicast...

>The plumbing for the L2 solution would look different than for the L3 solution, and that might make it trickier, though.
>

You are correct, I primarily chose routing between tons of interfaces
precisely to expose what cross-subnet issues existed in the real
world, and to make it be soooo painful as to inspire those writing
code that overused multicast to find unicast solutions!

The core things that break are things we long ago identified - mdns
and local service discovery, notably. (I have generally found that
every android app lets me plug in an ip address for a given service,
so for me it hasnt been a huge problem - except that not having a
dynamic ipv6 to name mapping makes dynamic ipv6 unusable in such
cases)

The net result is a lot of cero people said to hell with that and went
back to bridging everything, instead of working harder on mdns-proxy,
cidr, etc.

This microcosm of induced routing pain was also intended to show how
multicast didnt scale at all into larger wifi networks in the the
small business, or educational campus, that are bridged to wifi, here
the problems induced by multicast are so crippling as to leading to
the vast establishment of things like AP isolation, which disconnects
everyone from everyone else, on the same AP, which to me, is a
terrible solution.

I do note that not bridging ethernet and wifi together in cero has led
to making wifi quite a bit more pleasant and less jittery on apps like
webrtc, at least for me, and there is work in play on improving wifi´s
aggregation handling and multicast queue management coming up soon.

>> It's good that we're having this discussions since I seem to not be the only one thinking that there should be one port per subnet.

I do like occasionally having more than one vlan, but in the general case,
routing is far more expensive than switching.

The no-nat case here shows the impact of a current 44 entry routing
table on downloads, in the present linux 3.18 fib lookup system.

http://snapon.lab.bufferbloat.net/~d/openwrt-3.18.7/openwrt-wndr3800-3.18.7.svg

and also shows the performance difference between ipv6 and ipv4 on
this hardware.

There has been some GREAT work on improving linux´s fib lookup system
of late, but hardware bridging will always outperform software
bridging which will always outperform routing, IMHO.

Anyway I cant imagine a homeowner wanting more than 2-3 vlans at most,
and most, just one. There is no problem scaling an ethernet broadcast
domain to 4096 devices. Using up 16 ports on 16 different /64 subnets
is kind of crazy, especially since at least one provider (comcast) is
only supplying a /60 in the first place.

wifi on the other hand, barely scales to 30 active stations.

-- 
Dave Täht

---

From: Henning Rogge <[email-masked]>
Date: Thu, 19 Feb 2015 21:15:46 +0100

> simulation, with radio, only actual testing in real-world networks is

There is also the really bad case of wifi links without packet loss
which are stable at 6 Mbit/s... stable but incredible slow.

If you have a combination of two 450 Mbit/s links you can use instead,
you want to do it...

Henning Rogge

---

From: Dave Taht <[email-masked]>
Date: Thu, 19 Feb 2015 12:19:10 -0800

On Thu, Feb 19, 2015 at 12:14 PM, Dave Taht <[email-masked]> wrote:
> and local service discovery, notably.

Also firewalling became a log(nports) problem without the special solutions
in cerowrt. That became AMAZINGLY slow when we got past 4 interfaces.
It still is quite slow even with the pattern match we use there.

>(I have generally found that

-- 
Dave Täht

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 21:25:40 +0100

> conditions in a mesh-networking environment (that I know nothing about),

I think this is misrepresenting the argument somewhat.

We want a routing protocol that works well in an unadministered network
that consists of a mixture of wired and wireless links.  In a previous
mail, I presented a topology that I think is realistic in a Homenet
deployment:

Current implementations of Babel are known to work well in such topologies.

As far as I am aware, current implementations of IS-IS are not.  Getting
IS-IS to work well on such topologies is not completely trivial.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 21:38:38 +0100

> If you have a combination of two 450 Mbit/s links you can use instead,

Absolutely.  Just to clarify -- Henning is referring to the modulation
rate that is dynamically chosen by the MAC, and may vary over time and be
different for two neighbours on the same interface.

See my previous note about dropping ETX in favour of hooking (non-portably)
into the MAC layer in some future version of Babel.  I believe that
Henning is working on something similar for OLSRv2
(draft-ietf-manet-olsrv2-dat-metric).

-- Juliusz

---

From: Dave Taht <[email-masked]>
Date: Thu, 19 Feb 2015 13:01:43 -0800

> where a link is either down (drops all packets) or up (drops one packet in

Um, er, ah, an ideal low latency drop or mark rate is about .01-.02%
of big packets on 100mbit links, I dont remember what it is for gigE.
It is much higher at 10mbit. I should just go out and draw the nearly
linear relationship between drop rate and bandwidth for fixed latency
from the data I have...

> "marginal" state: a link that drops a non-negligible fraction of packets.

I have over the past year become a reluctantly enthusiastic user of
homeplug powerline technologies. For stuff rated at 600Mbit () I
typically get throughput in the range of 12-140mbits in a small apt
and latencies in the 30-140ms range under load. They degrade
enormously with more than 2 in use and dont work worth a damn in
larger houses... but are still generally faster and less jittery than
present day wifi, when you have an electrical system that they work
on.

These sort of devices would be ideal to add aqm/fq to but public
firmware is non-existent, as near as I can tell.

And, as layer 2 bridging devices, they supply no indications to layer
3 routing protocols that they are weirder than ethernet.

I do like the idea of using RTT based metrics as an abstraction for
determining link quality, but it would be rather nice to also have a
way to get their actual bandwidth and actual load statistics somehow,
for smarter usage.

> This is pretty obvious.  If you can choose between an Ethernet link and

Well, ethernet runs at different speeds and there is no abstraction
for that in babel...

-- 
Dave Täht

---

From: Teco Boot <[email-masked]>
Date: Thu, 19 Feb 2015 22:19:36 +0100

> Op 19 feb. 2015, om 19:18 heeft Ole Troan <[email-masked]> het volgende geschreven:
> 
> there are very few shared media around anymore.

Still one to phase out: spectrum ;-)

I would say the opposite, I see kids that have never been connected to a wired link. What about wireless repeaters, which forward packets and do not have a network jacket? Wireless backhaul links to ISP?

Teco

> I don't think I've ever been connected to a 10base5.

---

From: Ole Troan <[email-masked]>
Date: Thu, 19 Feb 2015 22:27:28 +0100

> I would say the opposite, I see kids that have never been connected to a wired link. What about wireless repeaters, which forward packets and do not have a network jacket? Wireless backhaul links t…

yes, at L1 but not L2, which was what I was trying to say. citing 802.11 as an example.

cheers,
Ole

---

From: Teco Boot <[email-masked]>
Date: Thu, 19 Feb 2015 22:39:04 +0100

> Op 19 feb. 2015, om 21:25 heeft Juliusz Chroboczek <[email-masked]> het volgende geschreven:
> 

The issue here is link metric feedback into the routing protocol. Can be done for both Babel and ISIS. draft-mrw-homenet-rtg-comparison-01 is correct.

With ISIS, I am not aware of any implementation for dynamic link metric feedback.

And yes, it must be automatic and link metrics for wireless links are dynamic, even for static nodes. Other objects may move.

Teco

---

From: Teco Boot <[email-masked]>
Date: Thu, 19 Feb 2015 23:07:54 +0100

> Op 19 feb. 2015, om 22:27 heeft Ole Troan <[email-masked]> het volgende geschreven:
> 
>>> 

Hmm, I don't know what a L2 shared medium is.

More important: as long as switching IP addresses break connections, subnetting WiFi APs is really bad. So what we do today is setup our multiple home WiFi AP networks in the same subnet, same broadcast domain. Handover isn't great, but it works.

As long as we don't bring in a WLAN controller or alternative, what is our plan?   

Teco

---

From: Teco Boot <[email-masked]>
Date: Thu, 19 Feb 2015 23:23:08 +0100

> Op 19 feb. 2015, om 22:01 heeft Dave Taht <[email-masked]> het volgende geschreven:
>> 
>> 1. Prefer solid links
>> 

Why not? 
I mean the babel protocol, not current code.

After having selected the Homenet Routing Protocol, I expect some discussion on how to set the metric. If box A from vendor A sets cost=100 on 1GE links and another box B from vendor B sets cost=50 on 2Mbit/sec WiFi links, we might have non-optimal paths. No-one around to adjust the config, right?

So what I want to see is a proposal for a routing protocol that specify link metrics for a set of commonly used link types in homes. This spec could be BCP.

Teco

---

From: Ted Lemon <[email-masked]>
Date: Thu, 19 Feb 2015 17:25:48 -0500

> After having selected the Homenet Routing Protocol, I expect some discussion on how to set the metric. If box A from vendor A sets cost=100 on 1GE links and another box B from vendor B sets cost=50…

Bear in mind that this stuff changes depending on what's plugged in, so it's a legitimate point that if you are 1gbps-capable, but a 100mbps thing is plugged into one port and a 10mbps into another, you have a mismatch to deal with.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 19 Feb 2015 23:42:19 +0100

> Why not? 

Switches.

Most home routers have their (on-chip) Ethernet interface behind an
internal switch.  That is how they provide 5 or more Ethernet ports using
a single-NIC SoC.

That has the unfortunate consequence that neither carrier sense nor link
speed sensing is going to work on most home routers without model-specific
code.

That is what I allude to in Section 4.3 of the draft, but I see that
I need to be more explicit in -02.  Your (plural) feedback is being
helpful.

> I mean the babel protocol, not current code.

Oh, if I don't have to implement it, then I'm all in favour ;-)

> So what I want to see is a proposal for a routing protocol that specify
> link metrics for a set of commonly used link types in homes.

That's something the IS-IS community has already done, I believe, although
their values might not map directly to Babel metrics (which are just 16 bits
long).

-- Juliusz

---

From: Michael Richardson <[email-masked]>
Date: Thu, 19 Feb 2015 18:03:07 -0500

Ralph Droms <[email-masked]> wrote:
    >> What Mikael is saying is that on homenet devices (not switches) each
    >> port is treated as a separate network, on the assumption that all of
    >> the physical ports on the homenet device are likely to connect to
    >> routers, and that any switched topology would be done using switches
    >> connected to homenet routers.

    > homenet devices and is there an expectation that people will buy
    > homenet devices and switches and know where to use them both?

My claim is that overtime there will be no "switches" in the home or in
futureshop/bestbuy/newegg.  They will all be homenet.  It's already been a
decade that people buy new "routers" when they should have bought switches,
and then wind up with dualing DHCP servers, or, as we have discussed here
many times: unintended chains of NAT.

But, as for physical ports on homenet devices, in the architecture we say
that those are supposed to be merged as a single l2 domain, and only split up
if there is some (out of band) reason to do that.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Sander Steffann <[email-masked]>
Date: Fri, 20 Feb 2015 00:45:01 +0100

Hi Ted,

> Op 19 feb. 2015, om 19:49 heeft Ted Lemon <[email-masked]> het volgende geschreven:
> 

Sorry, bad assumption. I just finished working on a TV streaming project for an ISP and there is lots of multicast there. All live TV channel streams to STBs are multicast and this is a common setup amongst ISPs.

Cheers,
Sander

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 19 Feb 2015 15:45:55 -0800

> IMHO, both kinds of devices are necessary to the success of Homenet.

Right on all points. So the biggest challenge is really how to motivate
better "in-home" devices which are not SP-OEMed and do not connect to a WAN-link.

I think those loadable apps are the key to getting the right HW into the home.

If there is an evolving SW platform for such apps that is not tied to specific
vendors on the HGW, then the same SW platform should easily proliferate
into the non-HGW homenet devices and raie their HW requirements to what we 
want.

Alas, i am not aware of a widely distributed SW platform for "embedded"
apss. Aka: Android/iOS apps - but for devices without display. Not that
it's technically difficult, its more a question of creating a critical mass
of support/interest. Java would likely be nicely lightweight. Docker could
do more and be faster but is likely  a lot more disk heavy, etc. pp.

Cheers
    Toerless

---

From: Dave Taht <[email-masked]>
Date: Thu, 19 Feb 2015 15:47:14 -0800

> internal switch.  That is how they provide 5 or more Ethernet ports using

It is even worse than that. Many of the more modern devices only have
a single ethernet device, and they use a vlan trick in order to get an
ISP uplink port out of it.

This is one of the cases where applying a hardware rate limited aqm/fq
system doesn´t work at all, you end up with Hol as everything in all
directions vanishes into the switch, and of course things like link
detection go by the wayside. Another painful aspect of this is when a
box is supplying a default gateway but actually has not achieved
internet access. Tools like babel-pinger help, but are not tightly
integrated with dhcp.

Welcome to the race to the bottom.

-- 
Dave Täht

---

From: Dave Taht <[email-masked]>
Date: Thu, 19 Feb 2015 15:49:27 -0800

> Sorry, bad assumption. I just finished working on a TV streaming project for an ISP and there is lots of multicast there. All live TV channel streams to STBs are multicast and this is a common setu…

One of the reasons free.fr can´t do down to the cpe shaping
effectively is because their multicast IPtv code is deeply embedded
into the DSL framing layer and not visible to the overlying OS.

-- 
Dave Täht

---

From: Andrew Mcgregor <[email-masked]>
Date: Fri, 20 Feb 2015 11:04:41 +1100

Why?  PIM and MLD snooping are pretty standard on very low-end enterprise
switches, which will be next year's midrange consumer models.  If the
dumb-as-rocks stuff goes away, that would generally make people happier.

On 20 February 2015 at 05:22, Mikael Abrahamsson <[email-masked]> wrote:

> people connect to the router to support both PIM and MLD snooping seem like
> it might be asking too much.
>
> I might be wrong though.
>
> --

-- 
Andrew McGregor | SRE | [email-masked] | +61 4 1071 2221

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 20 Feb 2015 05:52:27 +0100 (CET)

On Thu, 19 Feb 2015, Joel M. Halpern wrote:

> I would note that RFC 7368 says that "simple Layer 3 topologies involving as 
> few subnets as possible are preferred in home networks".  I presume this is 
> reflective of WG agreement.
> mandating that each physical port on the access router be a distinct subnet 
> (much less extending such a mandate further in the home) seems to violate 
> this agreement.

I never said I wanted this to be mandated. I have just presumed from 
discussions in the WG and for instance fro RFC 7368 3.2.2.1 figure 1 and 
since the issue of "organizing ports" never came up, that this was the way 
it was supposed to be (and the fact that we've been trying to automate 
"edge discovery etc"). What is the physical layout of the CER in 3.2.2.1 
fig 1?

Does it have 2xWAN ports and 2 blocks of 4xLAN ports and the customer is 
supposed to know how to connect what to where?

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 20 Feb 2015 06:11:30 +0100 (CET)

On Thu, 19 Feb 2015, Michael Richardson wrote:

> But, as for physical ports on homenet devices, in the architecture we 
> say that those are supposed to be merged as a single l2 domain, and only 
> split up if there is some (out of band) reason to do that.

Are you referring to 3.3.2 now (it took me quite a while to find it)?

3.3.2.  Largest Practical Subnets

    Today's IPv4 home networks generally have a single subnet, and early
    dual-stack deployments have a single congruent IPv6 subnet, possibly
    with some bridging functionality.  More recently, some vendors have
    started to introduce 'home' and 'guest' functions, which in IPv6
    would be implemented as two subnets.

    Future home networks are highly likely to have one or more internal
    routers and thus need multiple subnets for the reasons described
    earlier.  As part of the self-organisation of the network, the
    homenet should subdivide itself into the largest practical subnets
    that can be constructed within the constraints of link-layer
    mechanisms, bridging, physical connectivity, and policy, and where
    applicable, performance or other criteria.  In such subdivisions, the
    logical topology may not necessarily match the physical topology.
    This text does not, however, make recommendations on how such
    subdivision should occur.  It is expected that subsequent documents
    will address this problem.

    While it may be desirable to maximise the chance of link-local
    protocols operating across a homenet by maximising the size of a
    subnet, multi-subnet home networks are inevitable, so their support
    must be included.

The way this is currently written, it does look like we need some kind of 
protocol and intelligence to self-organize the ports into either L2 
domains or dedicated ports, for instance to avoid L2 loops, depending on 
how the user connects things together. I am not aware of any work done in 
this area. 3.3 says the network should be self-organizing, so the user 
should not be required to know what they're doing.

---

From: Teco Boot <[email-masked]>
Date: Fri, 20 Feb 2015 08:56:14 +0100

> Op 19 feb. 2015, om 23:42 heeft Juliusz Chroboczek <[email-masked]> het volgende geschreven:
> 

Bad luck, I kindly ask you to pay a little more attention to it. Link metrics for wireless links are crucial, but let's not forget wired links.

Some years ago, Thales NLD worked on olsr-lc (link costs, ETT). A plugin probed WiFi link speed with large & small packets, filtered out jitter and used the outcome as link metric (merged with ETX, I think). For static networks and very patient people, it may work. For mobile networks, it is far, far to slow. Convergence is tens of minutes. Speed up some timers increases load on the wireless links to unacceptable levels. So it died.
 
But for wired links at homes, this plug&play mechanism could work out well. No L2 API needed.

http://sourceforge.net/projects/olsr-lc/

Teco

---

From: Henning Rogge <[email-masked]>
Date: Fri, 20 Feb 2015 09:04:39 +0100

> But for wired links at homes, this plug&play mechanism could work out well. No L2 API needed.

There is also the "linklayer database" approach I selected for my
olsrd2 implementation. Instead of hardcoding linklayer specific code
into the metric, I split the codebase into "link layer gathering" code
(which is often OS and linklayer specific), generic routing metric
code... and a generic API in between that stores the values.

This makes it quite easy to adapt the codebase to new linklayer types.

Henning Rogge

---

From: Teco Boot <[email-masked]>
Date: Fri, 20 Feb 2015 09:19:02 +0100

> Op 20 feb. 2015, om 09:04 heeft Henning Rogge <[email-masked]> het volgende geschreven:
> 

So you have the placeholder for an automatic ethernet link speed detection. Great! 

We cannot trust ethernet port L2 feedback. Ports can be connected to switches with multi-rate ports. Could be powerline, wired_over_wireless bridges or other stuff that hides a slow link. In homes, there is no place for protocols that cannot detect and handle such cases.

Teco

---

From: Henning Rogge <[email-masked]>
Date: Fri, 20 Feb 2015 09:22:48 +0100

> We cannot trust ethernet port L2 feedback. Ports can be connected to switches with multi-rate ports. Could be powerline, wired_over_wireless bridges or other stuff that hides a slow link. In homes,…

At the moment I just get the ethernet port link speed... that is not
really good for switched ports, but its better than nothing.
http://olsr.org/git/?p=oonf.git;a=blob;f=src-plugins/generic/eth_listener/eth_listener.c;h=1646a60722d60c54d7ab6b5df2e67fba7d52de22;hb=master

If you have hardware with a built-in switch that can report the
link-speed, it would be easy to add code that integrates this (and
some traffic statistics).

Henning Rogge

---

From: Teco Boot <[email-masked]>
Date: Fri, 20 Feb 2015 09:51:49 +0100

> Op 20 feb. 2015, om 09:22 heeft Henning Rogge <[email-masked]> het volgende geschreven:
> 

Your SW depends on ethtool, right? Not a problem, this is implementation detail. Link speed probes could be added for verification the ethtool provided info.

And yes, I didn't mean we cannot use the 80% solution. What I want to say is that the Homenet Routing Protocol shall be able to use all the link metrics it can get.

I am still worried about loops caused by dynamic link metrics used by a link state routing protocol. For me, this is the major difference between ISIS and Babel. Thats because I don't code the protocol. If I was, I would be worried about the non-IP transport in ISIS. 

Teco

---

From: Henning Rogge <[email-masked]>
Date: Fri, 20 Feb 2015 09:54:17 +0100

> Your SW depends on ethtool, right? Not a problem, this is implementation detail. Link speed probes could be added for verification the ethtool provided info.

No, it does directly call into the operation system. Calling a
different executable and parsing the output is a good source for
subtle bugs.

> I am still worried about loops caused by dynamic link metrics used by a link state routing protocol. For me, this is the major difference between ISIS and Babel. Thats because I don't code the prot…

It is a matter of metric stability... so you need a good hysteresis
and post-processing to make it work.

Henning Rogge

---

From: Teco Boot <[email-masked]>
Date: Fri, 20 Feb 2015 10:11:16 +0100

> Op 20 feb. 2015, om 09:54 heeft Henning Rogge <[email-masked]> het volgende geschreven:
> 

Great.

> and post-processing to make it work.

I wonder. If the change hits the hysteresis threshold, two actions will happen. SPF calculation is triggered and routes may change. This would take place in sub-ms. The other action is sending link state to neighbors or flood over the whole network. That takes time, especially when there is no triggered update, there are some packets queued, there is a shared medium with timer based media access etc. Meanwhile, the topology is not guaranteed to be loop-free. Little you can do on the node itself. OK, we have the enterprise/ISP loop avoidance mechanisms, but to reuse this in homenet? Deploy this year?

I highly appreciate your work on olsrd/olsrd2, of course !!

Teco

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 20 Feb 2015 10:57:24 +0100 (CET)

On Fri, 20 Feb 2015, Andrew Mcgregor wrote:

> Why?  PIM and MLD snooping are pretty standard on very low-end 
> enterprise switches, which will be next year's midrange consumer models. 
> If the dumb-as-rocks stuff goes away, that would generally make people 
> happier.

There are enterprise switches out there currently (pretty expensive ones) 
that still do not have MLD snooping, and the ones having PIM snooping is 
most likely a lot less. I've been in the networking industry for close to 
20 years, the first "bridge" I laid hands on was a pretty advanced thing 
back in the time with 3 AUI ports, and could barely do 10 megabit/s. I've 
then seen the evolution into 100meg hubs, then 10/100 dual speed hubs 
(basically two hubs with a switch in between), to 10/100 switches with all 
ports switched, to gig equivalent etc. During this entire time, switches 
that could do IGMP snooping has always been substantially more expensive, 
mostly (I guess) they couldn't be implemented in pure switch silicon, but 
always required administration interface, operating system etc. Still 
today, these typically cost 100 USD or more, when you can buy a stupid one 
for 30USD. So yes, over time this might change, but I still think there 
will be cost involved. It might be that the homenet "routers" are going to 
look quite different than the typical router we see today when it comes to 
phsyical ports. Or perhaps they're only going to have 2-3 ports and the 
rest is going to be wifi. What do I know.

What I do know is that so far, cable has always been a lot better than 
radio. Lots of support calls to ISPs end up being related to wifi 
problems. I have CAT6 to every room in my apartment, but then again, I am 
not a typical user. However, I often speak to people who have performance 
problems who then end up pulling a physical cable and after that their 
problems are solved.

With 60GHz wifi, you're going to need line-of-sight to every AP from the 
clients, which will probably be located in the ceiling in every room where 
you want good performance. These APs are going to need physical cables for 
uplinks to get any meaningful bump in performance.

I have thought of this as mostly L3 network. I thought the service 
discovery problem between subnets was being solved or had been solved. 
>From the discussion here the past few days it's clear to me now that the 
mind image of what a future homenet is differs a lot between participants 
in this working group.

---

From: Teco Boot <[email-masked]>
Date: Fri, 20 Feb 2015 12:53:01 +0100

> Op 20 feb. 2015, om 10:57 heeft Mikael Abrahamsson <[email-masked]> het volgende geschreven:
> 
>> Why?  PIM and MLD snooping are pretty standard on very low-end enterprise switches, which will be next year's midrange consumer models. If the dumb-as-rocks stuff goes away, that would generally make people happier.
> 
> There are enterprise switches out there currently (pretty expensive ones) that still do not have MLD snooping, and the ones having PIM snooping is most likely a lot less. I've been in the networking industry for close to 20 years, the first "bridge" I laid hands on was a pretty advanced thing back in the time with 3 AUI ports, and could barely do 10 megabit/s. I've then seen the evolution into 100meg hubs, then 10/100 dual speed hubs (basically two hubs with a switch in between), to 10/100 switches with all ports switched, to gig equivalent etc. During this entire time, switches that could do IGMP snooping has always been substantially more expensive, mostly (I guess) they couldn't be implemented in pure switch silicon, but always required administration interface, operating system etc. Still today, these typically cost 100 USD or more, when you can buy a stupid one for 30USD. So yes, over time this might change, but I still think there will be cost involved. It might be that the homenet "routers" are going to look quite different than the typical router we see today when it comes to phsyical ports. Or perhaps they're only going to have 2-3 ports and the rest is going to be wifi. What do I know.
> 
> What I do know is that so far, cable has always been a lot better than radio. Lots of support calls to ISPs end up being related to wifi problems. I have CAT6 to every room in my apartment, but then again, I am not a typical user. However, I often speak to people who have performance problems who then end up pulling a physical cable and after that their problems are solved.
> 
> With 60GHz wifi, you're going to need line-of-sight to every AP from the clients, which will probably be located in the ceiling in every room where you want good performance. These APs are going to need physical cables for uplinks to get any meaningful bump in performance.
> 
> I have thought of this as mostly L3 network.

What I can add: the multicast snooping feature could be somewhat behind development and deployment of the standards and / or buggy. So some administrators switch off the snooping / rate limiting feature. I do.

L3 all the way to switch ports make the network more robust. But this requires L3 forwarding in hardware, multicast routing and adjustments in discovery protocols. Can all multicast forwarding be performed in hardware?

Do you have CAT6 to WiFi APs in every room? Can you share experience with moving WiFi devices?

Teco

> I thought the service discovery problem between subnets was being solved or had been solved. 
> mind image of what a future homenet is differs a lot between participants in this working group.
> 
> --

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 20 Feb 2015 13:17:17 +0100 (CET)

> with moving WiFi devices?

No, my apartment is covered by a single 5GHz AP in the center of the 
apartment.

I mainly use cabled connections for media players and similar devices 
since they work much better over full duplex gige than over wifi. If I 
just could go back to my 1ms RTT Internet access link, things would be a 
lot better because my current DOCSIS3 cable connection is a lot worse (for 
instance when it comes to RTT and PDV) than my previous connection I had 
at the previous apartment which was a lot more consistent.

I tend to use a CAT6 ethernet adapter in my laptop when I'm at my desk, 
even though both my AP and laptop has 802.11ac. Wifi is mostly used to 
handle mobile devices such as tablets and mobile phones.

My experience is that even with state of the art equipment, wifi still is 
not even close in quality of experience compared to cable, apart from very 
light use where it's still sufficient.

My experience with multiple APs (from other places) is that clients don't 
switch APs easily enough so they're seldom connected to the optimal AP.

---

From: Teco Boot <[email-masked]>
Date: Fri, 20 Feb 2015 14:08:11 +0100

Thanks sharing this.

What I can add:

Our house is a little bit bigger. It was formerly a farmhouse. It has a stone firewall between living room and formerly hay storage place. This firewall is quit good in blocking RF.

My "production" network has 2 dual band APs. I have good indoor coverage. In summertime, I use WiFi in my garden to watch live TV (iPhone, iPad, MacBook, Android gear). It is partly testing, partly following Tour de France, news etc. My partner is less a geek and carries the TV, coax and power cables. Still possible with some analogue channels. Soon we have to carry a lot more stuff to decode DTV, this is bad. Or watch over WiFi. I don't want to CAT6 my garden, sorry.

Walking from living room to desk to garden means AP switch-over. 

I used to have 2.4 and 5GHz SSIDs, this doesn't work well. With exact same SSID and keys, handover works. Not well, but it works. With different SSIDs and different IP subnets: no, total failure.

There are WiFi AP kits around that operate in same channel and spoof AP BSSID. Roaming is transparent for clients. Nice idea, not accepted by standards bodies nor by industry. 

Back to my point: I want L3 on each and every homenet box. At the same time, I want high performance wireless links. It must be cheap, I don't want to pay € 1000 for a WLC and €400 for each AP.

As long as homenet enlarges my WiFi problem, it is useless for me. And I thought I was candidate for early adoption, as I have multiple APs, a wired backbone, thinking of dual ISP links etc. And as geek, I want to pay a bit more for high performance and I am able to troubleshoot a bit.

Back to the subject: What are the requirements of a high performance WiFi home network to the homenet routing protocol? I guess we don't know.

Teco

> Op 20 feb. 2015, om 13:17 heeft Mikael Abrahamsson <[email-masked]> het volgende geschreven:
> 
> No, my apartment is covered by a single 5GHz AP in the center of the apartment.
> 
> I mainly use cabled connections for media players and similar devices since they work much better over full duplex gige than over wifi. If I just could go back to my 1ms RTT Internet access link, things would be a lot better because my current DOCSIS3 cable connection is a lot worse (for instance when it comes to RTT and PDV) than my previous connection I had at the previous apartment which was a lot more consistent.
> 
> I tend to use a CAT6 ethernet adapter in my laptop when I'm at my desk, even though both my AP and laptop has 802.11ac. Wifi is mostly used to handle mobile devices such as tablets and mobile phones.
> 
> My experience is that even with state of the art equipment, wifi still is not even close in quality of experience compared to cable, apart from very light use where it's still sufficient.
> 
> My experience with multiple APs (from other places) is that clients don't switch APs easily enough so they're seldom connected to the optimal AP.
> 
> --

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 20 Feb 2015 14:22:11 +0100 (CET)

On Fri, 20 Feb 2015, Teco Boot wrote:

> Back to the subject: What are the requirements of a high performance 
> WiFi home network to the homenet routing protocol? I guess we don't 
> know.

Within the current framework to solve this problem with what exists today 
when it comes to clients, I would say we need either:

1. HNCP helps set up an overlay L2 tunnel infrastructure connecting all 
the APs using the same SSID, so the SSID can have the same L2 domain. This 
would probably mean we want to increase MTU on the physical links to avoid 
fragmentation. Messy. Possibly we could advertise lower MTU on the wifi 
network to minimize fragmentation if we don't raise MTU.

2. We set up some kind of L2 switching domain between the APs. This would 
require VLAN support in the HGWs, and something to set this up with loop 
avoidance etc. Oh oh oh, we could use IEEE 802.1aq that already uses ISIS 
as control plane, that way we could possibly run the same IGP for both L2 
and L3. Interconnecting APs over wifi seems weird though. Oh, and messy 
sounds like an understatement.

Frankly, I don't know how to solve this without a lot of complication.

We need clients to be able to change IPv6 addresses without losing 
existing connections. SHIM6 anyone? MP-TCP? Asking IEEE to make 802.11 
keep two connections at once and inform the application that one address 
is going away soon so it can do its thing to try to handle this?

---

From: Ted Lemon <[email-masked]>
Date: Fri, 20 Feb 2015 08:22:14 -0500

So Teco, to satisfy your use case, which I share, you would actually need the homenet to identify all Wifi access points that are being used to serve hosts, and treat those as a single subnet, correct?

---

From: Ole Troan <[email-masked]>
Date: Fri, 20 Feb 2015 14:49:21 +0100

Mikael,

> Within the current framework to solve this problem with what exists today when it comes to clients, I would say we need either:
> 
> 1. HNCP helps set up an overlay L2 tunnel infrastructure connecting all the APs using the same SSID, so the SSID can have the same L2 domain. This would probably mean we want to increase MTU on the physical links to avoid fragmentation. Messy. Possibly we could advertise lower MTU on the wifi network to minimize fragmentation if we don't raise MTU.
> 
> 2. We set up some kind of L2 switching domain between the APs. This would require VLAN support in the HGWs, and something to set this up with loop avoidance etc. Oh oh oh, we could use IEEE 802.1aq that already uses ISIS as control plane, that way we could possibly run the same IGP for both L2 and L3. Interconnecting APs over wifi seems weird though. Oh, and messy sounds like an understatement.
> 

why do you think this has to be solved at L2?

> We need clients to be able to change IPv6 addresses without losing existing connections. SHIM6 anyone? MP-TCP? Asking IEEE to make 802.11 keep two connections at once and inform the application that one address is going away soon so it can do its thing to try to handle this?

at least you can do:
L3 - route injection (got a routing protocol there already, use it)
L3.5 - SHIM6. not deployable
L4/L5 - MP-TCP
L5/L7  - MOSH

cheers,
Ole

---

From: Teco Boot <[email-masked]>
Date: Fri, 20 Feb 2015 14:50:12 +0100

> Op 20 feb. 2015, om 14:22 heeft Ted Lemon <[email-masked]> het volgende geschreven:
> 

Because I am not optimistic on deployment of mobility protocols in the home, my answer is yes. WiFi APs in same home has same SSID(s). Each SSID provides an IP subnet.

Providing multiple SSIDs should be in scope, with <my_home> and <my_guest> as default SSIDs. Both <my_home> and <my_guest> differ from my neighbors SSIDs.

Al least two directions for a solution: 
 1) the WLAN controller with tunnels between the APs and the WLC. 
 2) distribute the AP configuration for APs, either manually or guided by a homenet protocol. I hope we can circumvent VLANs in homenet. An alternative would be /128 prefix routing.

Whatever we choose, we have to keep in mind that dual stack operation will not fade away soon.

Teco

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 20 Feb 2015 14:57:21 +0100 (CET)

> why do you think this has to be solved at L2?

I don't that's why I wrote the next paragraph outlining L3 solutions.

> L3 - route injection (got a routing protocol there already, use it)

This sounds like it needs at least a coordination protocol between the 
APs?

> L3.5 - SHIM6. not deployable

I would prefer the last two. I use MOSH all the time, it's great.

---

From: Ted Lemon <[email-masked]>
Date: Fri, 20 Feb 2015 09:14:15 -0500

> 2. We set up some kind of L2 switching domain between the APs. This would require VLAN support in the HGWs, and something to set this up with loop avoidance etc. Oh oh oh, we could use IEEE 802.1aq…

Long ago I proposed using Trill to make this work, but nobody listened...

:)

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 20 Feb 2015 06:16:34 -0800

Snooping switches would certainly be a good reason to prefer MLD proxying
over PIM in homenet. There could be other reasons like reducing codeside
(thats Pierre pet reason ;-). But trying to be compatible with
snooping switches also implies that improvements in MLD that we might
wnat then for homenet would break interoperability with that L2 equipment,
because you must assume this L2 equipment will only correctly react to MLD
message and message sequences as they have been defined 8 years ago.

So foremost, it would be good to understand if there really is home
L2 equipment that MUST see MLD to operate correctly. Otherwise i'd
happily ignore the problem and say there is enough bandwidth to just NOT DO
snooping but have multicast be flooded in the L2 segments.

I only remember vaguely one old data point that seemed to indicate that
some powerline equipment was doing snooping and you couldn't switch it
off. can't find the pointers to this data point though. Would love to
know if any powerline standards exist and what they say about MLD...

Worst case, whatever the preferred solution for IP multicast is in
homenet, if we identify that there is L2 equipment that MUST get MLD,
we need to ALSO have homenet routers send out MLDv1/v2 backward
compatible MLD messages to keep that L2 equipment happy. That is also 
pretty much what we do in commercial environments too, for example in
satellite links where we really want to connect router-to-router 
signaling with PIM, but those routers are set up to ALSO send IGMP/MLD
to keep the satellite modems happy.

Toerless

-- 
---
Toerless Eckert, [email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 20 Feb 2015 06:20:41 -0800

> mind image of what a future homenet is differs a lot between participants 

There are good proposal how to do servicce discovery in homenets or
the like with DNS-SD (/mDNS), but i think we should still worry
about compatibility with UPnP. Both of these requirements 
(UPnP and DNS-SD) are IMHO better solved with router-level "proxy"
solutions than with ASM IP multicast.

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 20 Feb 2015 14:37:38 +0000

> There are good proposal how to do servicce discovery in homenets or the
> like with DNS-SD (/mDNS), but i think we should still worry about
> compatibility with UPnP. Both of these requirements (UPnP and DNS-SD) are
> IMHO better solved with router-level "proxy"

FYI. I did a presentation to DLNA (which has a lot of key UPnP people) that warned them to start thinking about service discovery in the multi-router world, and the world where more and more Wi-Fi APs are limiting the multicast that makes it to Wi-Fi devices. I mentioned the DNS-SD proxy work, homenet, and told them:
" - People have started to study the current quantity of multicast traffic and its impact on power consumption by battery-operated Wi-Fi devices.
http://tools.ietf.org/html/draft-desmouceaux-ipv6-mcast-wifi-power-usage-00
http://tools.ietf.org/html/draft-vyncke-6man-mcast-not-efficient-00
 - Concern is growing and proposals are starting to appear for ways to decrease the quantity of multicast messages.
http://tools.ietf.org/html/draft-yourtchenko-colitti-nd-reduce-multicast-00 "

My recommendations were that they had 2 basic options of either also supporting DNS-SD / mDNS for service discovery, or defining a SSDP proxy similar to what I expect will be done for DNS-SD. Anyway, I told them I'd keep them updated on the state of affairs regarding this issue.
Barbara

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 20 Feb 2015 06:41:38 -0800

I have seen more L2 switches that have broken IGMP/MLD snooping than
working ones. I am not aware of real proliferation of PIM snooping.
Snooping in transit LANs with PIM is a bad idea anyhow, and i have
tried to steer any customer who asked me away from it.

Bidir-PIM makes snooping particularily difficult. For once
you have to track DF-winner election messages (difficult) and
secondly you have to then flood all multicast to all DF winners
because you don't know which group-range is served by which RP.

IMHO, there is never enough multicast to justify this snooping complexity.
The only thing to worry about is what packets to send out from a router
to ensure the snooping doesn't screw up the traffic flow.

I would be surprised to find equipemnt that does PIM snooping by default.

So, i'd recommend to a homenet router:

-> even if you don't do PIM, send out PIM Hellos. This should
   effectively make all stupid "enterprise class" IGMP/MLD snooping
   switches (that often have broken IGMP/MLD snooping) send
   you all multicast traffic (inhibits snooping).

-> Still send MLD/IGMP memberships to get traffic through the
   L2 home equipment that has likely never heard about PIM.
   I'd guess that eg: powerline equipment falls into this
   category.

Cheers
    Toerless

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 20 Feb 2015 14:55:46 +0000

> So what I want to see is a proposal for a routing protocol that specify link
> metrics for a set of commonly used link types in homes. This spec could be
> BCP.

Does it need to be a routing protocol? Just to throw another possible protocol into the mix being tossed around (like we don't have enough), I'd like to suggest that a non-routing protocol be considered for topology discovery and sharing of link metrics. Specifically, IEEE 1905.1a could help with this. It can provide L1 and L2 topology info with link metrics, and identify intervening bridges that support LLDP for IEEE 802.1 bridge discovery. I wouldn't want individual devices all doing their own 1905 topology discovery -- that's too chatty. But if the all the homenet routers did, and then advertised a "service" (DNS-SD) that provided a link to a router-provider HTML page with the topology and metric info the router can discover provided in a standardized XML or JSON syntax, then everything who can discover that service (on each router) would have access to this info. Applications could also make use of this info to do intelligent source address selection in a multi-WAN-interface network, because routers could report on the speed of their WAN connections, as well.

Just a thought. I've been working on trying to get myself to write a contribution describing the idea of the router-provided info service for a while now.
Barbara

---

From: Philippe Klein <[email-masked]>
Date: Fri, 20 Feb 2015 15:14:53 +0000

At the end if the day, L3 will need to get services from L2 to set and managed constrained  path based on Link attributes. 802.1Q is specifying multi-path short path bridging for this purpose and will allow to maximize the BW provide by the whole topology of the home network in a loop free  manner.
L3 and L2 for this manner could share the same IS-IS topology DB.
All the elements are there but require to make a joint IEEE 702.1 and IETF effort

/Philippe

Philippe Klein, PhD
Broadcom BTG

Sent from my iPhone

On Feb 20, 2015, at 16:56, STARK, BARBARA H <[email-masked]> wrote:

> Does it need to be a routing protocol? Just to throw another possible protocol into the mix being tossed around (like we don't have enough), I'd like to suggest that a non-routing protocol be considered for topology discovery and sharing of link metrics. Specifically, IEEE 1905.1a could help with this. It can provide L1 and L2 topology info with link metrics, and identify intervening bridges that support LLDP for IEEE 802.1 bridge discovery. I wouldn't want individual devices all doing their own 1905 topology discovery -- that's too chatty. But if the all the homenet routers did, and then advertised a "service" (DNS-SD) that provided a link to a router-provider HTML page with the topology and metric info the router can discover provided in a standardized XML or JSON syntax, then everything who can discover that service (on each router) would have access to this info. Applications could also make use of this info to do intelligent source address selection in a multi-WAN-interface netw
> ork, because routers could report on the speed of their WAN connections, as well.
>

---

From: Teco Boot <[email-masked]>
Date: Fri, 20 Feb 2015 16:26:09 +0100

Were you before me?
http://www.ietf.org/mail-archive/web/homenet/current/msg00971.html
(November 2011)

More than three years later, same discussion. That makes me sad.

Teco

> Op 20 feb. 2015, om 15:14 heeft Ted Lemon <[email-masked]> het volgende geschreven:
>

---

From: Michael Thomas <[email-masked]>
Date: Fri, 20 Feb 2015 07:26:26 -0800

On 02/18/2015 11:54 PM, Mikael Abrahamsson wrote:
> The complaints about the Erlang implementation has so far been on 
> memory and flash footprint, not CPU resources and affecting 
> performance. The above is true for most interpreted languages, such as 
> python etc. When you install the environment, they're going to use 
> noticable memory and disk space. Next application that needs the 
> environment won't add much to the footprint though, it's the 
> environment itself that takes space.
>
> My Python executable on my debian box is 2.7 megabyte. I don't know 
> about the rest. So if one wants to run Python on the home gateway, 
> that would also use significant space.
>
> protocol we use, that'll be done by the kernel. The routing protocol 
> just programs the RIB/FIB.
>

One nice side-effect of putting in, say, a headless android into a 
router is that it would forevermore insure that there's
plentiful memory, just like it did for cell phones. It's not like we 
physically can't get enough ram/flash into those boxen
after all... it's the will to bear the cost that's really at issue.

A better development environment for home routers would *really* help 
development efforts along. From what I've seen
they suck out loud.

Mike

---

From: Markus Stenberg <[email-masked]>
Date: Fri, 20 Feb 2015 17:37:43 +0200

> On 20.2.2015, at 15.22, Mikael Abrahamsson <[email-masked]> wrote:

What if clients do not need to change addresses? In 2 words: Route /128s.

Two ways to do it:
- have clients talk stub RP + stub HNCP, be done with it
- have first-hop router do proxying for them by advertising the /128s only when they are connected to it (may have issues with e.g. clients assuming /64, but nothing a redirect could not fix, hopefully)

This L3(ish) solution actually works for all apps, unlike L4+ solutions some people advocated.

Cheers,

-Markus

---

From: Teco Boot <[email-masked]>
Date: Fri, 20 Feb 2015 16:41:53 +0100

> Op 20 feb. 2015, om 15:55 heeft STARK, BARBARA H <[email-masked]> het volgende geschreven:
> 
> Does it need to be a routing protocol? Just to throw another possible protocol into the mix being tossed around (like we don't have enough),

I don't think current protocols used by ISPs or enterprises fits our plug&play requirements. Also, there is a lot more wireless in homes. That doesn't mean we cannot build on current protocols. 

> I'd like to suggest that a non-routing protocol be considered for topology discovery and sharing of link metrics. Specifically, IEEE 1905.1a could help with this. It can provide L1 and L2 topology info with link metrics, and identify intervening bridges that support LLDP for IEEE 802.1 bridge discovery. I wouldn't want individual devices all doing their own 1905 topology discovery -- that's too chatty. But if the all the homenet routers did, and then advertised a "service" (DNS-SD) that provided a link to a router-provider HTML page with the topology and metric info the router can discover provided in a standardized XML or JSON syntax, then everything who can discover that service (on each router) would have access to this info. Applications could also make use of this info to do intelligent source address selection in a multi-WAN-interface network, because routers could report on the speed of their WAN connections, as well.

My thoughts are as follows: sub-IP mechanisms provide usable metrics. Somewhere, near to L3 routing, this information is translated to a dimensionless cost metric for each link, where a link is a sub-IP path between two routers. Cost is not symmetric, cost A->B can differ from B->A. It is the routing protocol to distribute this information in the routing domain.

Teco

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 20 Feb 2015 15:45:09 +0000

> > I don't know.   Homenet multicast is an open issue.   But I don't think this
> use case represents a serious problem, because as far as I can tell streaming
> video is not done using multicast in practice anyway.
> 
> Sorry, bad assumption. I just finished working on a TV streaming project for
> an ISP and there is lots of multicast there. All live TV channel streams to STBs
> are multicast and this is a common setup amongst ISPs.

OTT video does not use multicast. IPTV deployments do use multicast. Those that I'm aware of require use of the provider-supplied CE router, which has an IGMP proxy (and MLD proxy if IPv6 multicast is supported) for LAN-to-WAN multicast management. Where Wi-Fi distribution of these multicast streams is supported, it is only done on a dedicated (and highly managed and somewhat proprietary) Wi-Fi network, that is distinct from the general-purpose (and not "professionally" managed) Wi-Fi network. The LAN interfaces of the provider-supplied CE router do tend to have IGMP/MLD snooping, to keep from flooding the general-purpose network interfaces with large multicast streams. There may also be a coax networking interface. If there is, this also tends to be dedicated and highly managed. Where Ethernet is used for distribution of multicast, it is done so that the provider STBs are all attached to the provider CE router via the same Ethernet port (and no other devices use that port) and there are no intervening routers. I mentioned at the last IETF that I expect some "home networks" to have managed and unmanaged segments. I don't consider the dedicated, managed segments to be part of the homenet domain. I would hope that the provider-supplied CE router would support homenet on its general purpose network interfaces. 

I'd be curious to learn about multicast IPTV deployments that allow users to supply their own CE routers and send multicast streams on network segments that are also used for general-purpose home networking (especially general-purpose Wi-Fi networks). 

BTW, the dedicated, highly-managed and somewhat proprietary Wi-Fi delivery of IPTV multicast streams has been very successful.
Barbara

---

From: "Leddy, John" <[email-masked]>
Date: Fri, 20 Feb 2015 16:09:31 +0000

Has the group considered a Bier model for multicast in the home?

On 2/20/15, 9:41 AM, "Toerless Eckert" <[email-masked]> wrote:

>>enterprise

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 20 Feb 2015 17:25:29 +0100

> If you have hardware with a built-in switch that can report the link-speed,

The WNDR3700v2 under OpenWRT appears to do so:

    # swconfig dev switch0 show | grep 'link:'
    link: port:0 link:down
    link: port:1 link:down
    link: port:2 link:up speed:100baseT full-duplex 
    link: port:3 link:down
    link: port:4 link:down
    link: port:5 link:up speed:1000baseT full-duplex txflow rxflow auto

but to use this information, you need to know:

  1. that vlan1 on eth0 is connected to ports 0 through 3 of switch0;
  2. that neighbour fe80::dead:beef is connected over port 2.

You could do (1) with a small amount of model-specific code (yuck), but
I have no idea how to do (2).  Consult the ND cache to find out the
neighbour's MAC, then find a way to snoop on the switch's forwarding table?

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 20 Feb 2015 17:33:16 +0100

> Has the group considered a Bier model for multicast in the home?

As in a place where you put dead people?

Please explain.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 20 Feb 2015 17:39:43 +0100

> L3 - route injection (got a routing protocol there already, use it)

Yes, just put a stub router on the host and advertise /128.

As far as I'm aware, current HNCP doesn't provide a way for a host to
request a subnet-independent /128.  What's the thinking on that?  Just
grab a /128 from whichever subnet you happen to be connected to at boot,
and advertise that?

> L3.5 - SHIM6. not deployable

I don't think that's specific to Homenet -- being able to deal with
multiple addresses that change over time is something that our stacks
don't deal with very well.

-- Juliusz

---

From: Ted Lemon <[email-masked]>
Date: Fri, 20 Feb 2015 11:46:18 -0500

> As in a place where you put dead people?

Bier is a new working group in the routing area.

https://datatracker.ietf.org/wg/bier/charter/

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 20 Feb 2015 17:51:02 +0100

> port and 4 "LAN" ports, and these 4 ports are bridged.

You certainly have a point -- that's something we never discussed, and it
appears that there's no consensus.

> I'm saying the typical device should have 4-5 "L3" ports. You're then
> free to connect one of these to your L2 switch if you so please.

I'll just point out that on all the hardware I know this is
configurable -- it's quite possible to set up an OpenWRT router to put
each Ethernet port on a different VLAN.

So perhaps a Homenet router could ship with the LAN ports bridged, and
have a configuratifon option somewhere in the "Beware the tiger" tab of
the "Advanced" menu that allows unbridging selected ports?

(I'm sure somebody will point out that the right configuration could be
chosen automatically, but I'm not sure I'm comfortable with the set of
interfaces changing at runtime on a home router.)

-- Juliusz

---

From: Sander Steffann <[email-masked]>
Date: Fri, 20 Feb 2015 17:58:42 +0100

Hi Barbara,

> I'd be curious to learn about multicast IPTV deployments that allow users to supply their own CE routers and send multicast streams on network segments that are also used for general-purpose home n…

The above is correct. Provider-supplied CPEs are currently necessary and intermediate routers are not supported. The TV streams can be on a separate VLAN. But this is what is currently done because of the difficulty of doing it over the regular home network. It doesn't mean we should design Homenet in the same way. I'd rather see that providers supply Homenet routers to their customers and that the multicast TV streams just work.

Cheers,
Sander

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 20 Feb 2015 17:02:28 +0000

> > Does it need to be a routing protocol? Just to throw another possible
> > protocol into the mix being tossed around (like we don't have enough),
> 
> I don't think current protocols used by ISPs or enterprises fits our plug&play
> requirements. Also, there is a lot more wireless in homes. That doesn't mean
> we cannot build on current protocols.

??? Huh? IEEE 1905.1 is being pushed by the likes of Broadcom, Qualcomm, MStar Semiconductor. They've suggested they want to have it in their Wi-Fi, MoCA, HomePlug, and Ethernet products, that are targeted at the CE industry. The certification is being offered by HomePlug Alliance. While Orange and AT&T have expressed an interest (because helping customers with their home networking issues is a huge customer care cost), I wouldn't call a "protocol used by ISPs". And enterprises?! I don't think so. Enterprises are definitely *not* the target audience.

> > I'd like to suggest that a non-routing protocol be considered for topology
> discovery and sharing of link metrics. Specifically, IEEE 1905.1a could help with
> this. It can provide L1 and L2 topology info with link metrics, and identify
> intervening bridges that support LLDP for IEEE 802.1 bridge discovery. I
> wouldn't want individual devices all doing their own 1905 topology discovery -
> - that's too chatty. But if the all the homenet routers did, and then advertised
> a "service" (DNS-SD) that provided a link to a router-provider HTML page
> with the topology and metric info the router can discover provided in a
> standardized XML or JSON syntax, then everything who can discover that
> service (on each router) would have access to this info. Applications could
> also make use of this info to do intelligent source address selection in a multi-
> WAN-interface network, because routers could report on the speed of their
> WAN connections, as well.
> 
> My thoughts are as follows: sub-IP mechanisms provide usable metrics.
> Somewhere, near to L3 routing, this information is translated to a
> dimensionless cost metric for each link, where a link is a sub-IP path between
> two routers. Cost is not symmetric, cost A->B can differ from B->A. It is the
> routing protocol to distribute this information in the routing domain.

I don't understand your point here, and whether you're trying to calculate the cost of running a L7 (application) protocol, or the relative cost of using different routes? Note that all protocols used to supply info to routing configuration applications are operating at the application layer (L7). HNCP is a L7 protocol. IEEE 1905.1 is a L7 protocol. As are IS-IS, and HTTP. The question is "what information does a router's routing configuration application need to properly configure routing behavior?" Once you figure out the info you need, then you can figure out the various options for getting that info to the router. It's not always necessary to multicast or broadcast all the information all over the place. Sometimes it's enough (and more efficient) to broadcast the existence (and where to get) the information, and then just let routers (or other devices) who want the info to go and ask for the info (unicast). If info changes, you can broadcast a message saying something changed, and then let interested devices query to find out what the change was. You could even stick a "for more info" TLV in HNCP that supplies the URL of a router's info-providing web page.
Barbara

---

From: "'Toerless Eckert'" <[email-masked]>
Date: Fri, 20 Feb 2015 09:09:39 -0800

Thanks, Barbara.

Lets hope UPnP/DLNA are starting to consider to cooperate with IETF on service
discovery.

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 20 Feb 2015 17:24:42 +0000

> Lets hope UPnP/DLNA are starting to consider to cooperate with IETF on
> service discovery.

"Cooperate" is such a strong word. :) They are aware of the issue, they will be kept informed as to how the issue is progressing and what homenet and dnsext are doing to tackle the issue, and they know what their options are.
Barbara

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 20 Feb 2015 19:10:41 +0100 (CET)

On Fri, 20 Feb 2015, Toerless Eckert wrote:

> So foremost, it would be good to understand if there really is home L2 
> equipment that MUST see MLD to operate correctly. Otherwise i'd happily 
> ignore the problem and say there is enough bandwidth to just NOT DO 

I know of ATA (VoIP) devices with 10/half uplinks that fall over HARD when 
they receive two simultaneous IPTV streams when doing channel switching.

So it's not only about bandwidth, but about how much traffic these kinds 
of devices are designed to receive.

Also, remember that we're going to 4k IPTV streams that might be in the 20 
megabit/s range, and we might be doing multiples of them. All devices on 
the subnet will receive this if there is no MLD/PIM snooping precent.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 20 Feb 2015 19:16:56 +0100 (CET)

> - have clients talk stub RP + stub HNCP, be done with it

Oooh, if we want to require changes to clients, I have all kinds of ideas 
how to solve this in other ways than you propose.

---

From: Dave Taht <[email-masked]>
Date: Fri, 20 Feb 2015 10:22:57 -0800

On Fri, Feb 20, 2015 at 10:10 AM, Mikael Abrahamsson <[email-masked]> wrote:
>> ignore the problem and say there is enough bandwidth to just NOT DO snooping
>> but have multicast be flooded in the L2 segments.
>
>
> So it's not only about bandwidth, but about how much traffic these kinds of
> devices are designed to receive.
>
> megabit/s range, and we might be doing multiples of them. All devices on the
> subnet will receive this if there is no MLD/PIM snooping precent.

Is there a formal definition of IPTV? as used here, it seems to imply multicast.

Frankly, until now, I was expecting the world to go to a HAS model for content,
especially big content.

-- 
Dave Täht

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 20 Feb 2015 19:28:50 +0100

> if we want to require changes to clients, I have all kinds of ideas how
> to solve this in other ways than you propose.

Please share.  We might not implement them, but we'll certainly listen.

(Note that we're not *requiring* changes to hosts, we're just proposing
optional host changes that are likely to enhance the user experience.  I'm
pretty sure that's consistent with Homenet's charter.)

-- Juliusz

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 20 Feb 2015 13:50:08 -0500

It seems pretty clear that over time, the bulk of video will be unicast, 
in order to meet on-demand needs.  There will always be a few items that 
folks really want to watch live, and thus where multicast may add value. 
  But making multicast the design driver for home networking 
archtiecture seems backwards to me.

Yours,
Joel

---

From: Steven Barth <[email-masked]>
Date: Fri, 20 Feb 2015 20:34:48 +0100

>Hm, I will have to try it out.   Is it in a distribution?

ohybridproxy in openwrt. It's mainly useful with hnetd (hncp) though. 

Manual configuration without hncp is a bit awkward since you need to name each link manually and on every router configure the resolver (e.g. dnsmasq). I guess we might want to provide a little example for 2 links at some point.

---

From: "Leddy, John" <[email-masked]>
Date: Fri, 20 Feb 2015 19:50:22 +0000

Agree.  I would not architect around multi-hop multicast, subnet OK,
targeted probably.

Not sure everyone has had the pleasure of running large IP multicast
infrastructures running video, it is a wonderful challenge.
It also has the side benefit of encouraging poorly designed applications.

Interoperability on even core routers for IP multicast barely exists.  In
a home environment, although smaller, will be even less wonderful.

Just the race conditions between PIM and the IGP are always challenging.

On 2/20/15, 1:50 PM, "Joel M. Halpern" <[email-masked]> wrote:

>>>>snooping

---

From: Dave Taht <[email-masked]>
Date: Fri, 20 Feb 2015 12:01:54 -0800

> Manual configuration without hncp is a bit awkward since you need to name each link manually and on every router configure the resolver (e.g. dnsmasq). I guess we might want to provide a little exa…

I would like to deploy hncp in my upcoming make-wifi-fast testbed.
However the biggest headache is ensuring that all the routers
inbetween have hncp burned into them, and are only acting as relays as
I generally only obtain a few (/60) real IPv6 prefixes per GW and they
only need to be on the APs.

GW1 - routerA - routerB - routerC - routerD - AP1
                                         |
GW2 - routerE - routerF - routerG - routerH - AP2
                           |
                         AP3

GW3 ...

(the actual topology of the testbed is way more complex than this
(covering ethernet, wifi, and moca) and I am not going into it here)

1) is there a way to configure hncpd as purely a relay, and NOT do any
address assignment at all on routers B,C,D,F,G,H?

2) have you tested that it is indeed possible to get the separate ipv6
prefixes from GW1,GW2,GW3 to AP1,AP2, AP3?

3) Can ULA and "Real" address assignment be distinguished along the
way? I have no problem assigning ULAs to the routers, but dont want to
use up real addresses on them.

4) What happens when someone pulls the plug on GW1, it reboots, and
gets a new Ipv6 subnet (I have seen comcast do this to me
every time that happens with the code I have in place now - no
retraction, and I go through hell manually eliminating every former
prefix from the network. Yes. I have upses. And cerowrt, at least,
stays up for 90+ days without a problem. But it happens and sucks when
it does)

-- 
Dave Täht

---

From: Markus Stenberg <[email-masked]>
Date: Sat, 21 Feb 2015 00:14:15 +0200

> On 20.2.2015, at 22.01, Dave Taht <[email-masked]> wrote:

In theory (=spec), but not in current implementation (I _think_). You do not really need non-linklocal addresses for HNCP, or routing protocol, so as long as there are no hosts on the link.. (traceroute is a bitch though given just linklocals)

As workaround in current implementation, if you set it to assign e.g. /80 per link used for your intermediate links, you would have almost all of address space left; however, in your case, I am not sure you can even afford to split one /64 for that purpose.

> prefixes from GW1,GW2,GW3 to AP1,AP2, AP3?

Yes.

> use up real addresses on them.

In theory (=spec), not in current implementation (ULA is actively discouraged in current impl, I cannot remember if there was option to override).

> stays up for 90+ days without a problem. But it happens and sucks when

Renumbering should just work as usual. I.e. rest of nodes should learn of the new prefix, and old one should disappear.

-Markus

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 20 Feb 2015 14:39:24 -0800

I was going to say host-routes as well but Markus beat me to the punch.

I definitely think we should target a solution without host changes though
as the tactical solution.

I thought LISP had implemented this type of stuff, eg: host address 
mobility for hosts on access-LANs. Can't quite remember how, but it must
be something involving the tracking of a device when it moves from it's
original location, and the first-hop router(s) would then create a host-route
for that host and make sure that the host keeps its address (acting as
DHCP server or the like ?)

Can't be that hard though. Definitely less work than trying to build
an L2 overlay to give stupid L2 APs the ability to do L2 roaming. IN
any case, it should at most be all control-plane, no new forwarding plane.

Of course, host-routes themselves don't solve the possible problem of
link-local multicast after mobility, but i'd very much hope that we
can get rid of apps relying on that anyhow.

Cheers
    Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 20 Feb 2015 14:42:25 -0800

On Fri, Feb 20, 2015 at 02:57:21PM +0100, Mikael Abrahamsson wrote:
> This sounds like it needs at least a coordination protocol between the APs?

NO, just between the first-hop (homenet) routers. Should work with unchanged
of the shelf crap-APs as long as they're attached to a homenet router.

Cheers
    Toerless

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 21 Feb 2015 07:02:44 +0100 (CET)

> of the shelf crap-APs as long as they're attached to a homenet router.

Could someone please explain to me how this is supposed to work? How do 
the first-hop routers figure out where the client is? Do we do /128 route 
injection into the homenet for active IPv6 addresses in that /64, and 
announce the same /64 everywhere, and then we use proxy-nd for all off-L2 
active IPs? I guess we need to handle IPv4 as well, so we use proxy-arp 
for that?

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 21 Feb 2015 15:41:47 +0100

> Could someone please explain to me how this is supposed to work?

The client is running a stub implementation of the routing protocol.  It's
performing the normal routing protocol activities, such as sending
periodic hellos and announcing routes or flooding LSPs.

At any given time, the client has a default route through some
neighbouring Homenet router.  It sends its outgoing traffic through that
router, which presumably knows how to deal with it.

At any given time, the client is announcing a /128 route to itself to all
neighbouring Homenet routers.  This route gets propagated through the
Homenet by the routing protocol, so that all Homenet routers know how to
route to the client.  This assumes that the Homenet is not doing uRPF
internally.

This approach has been extensively tested with Babel, with no link-layer
support, and the outage after roaming has been measured as 2 to 4 Hello
intervals, which is the time needed by the link quality estimator to yield
meaningful data.  (Which, just between the two of us, is pretty annoying,
even if you're using Mosh.)

-- Juliusz

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 21 Feb 2015 15:45:34 +0100 (CET)

On Sat, 21 Feb 2015, Juliusz Chroboczek wrote:

> The client is running a stub implementation of the routing protocol.

I thought we already had decided we didn't require changes to the host?

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 21 Feb 2015 16:06:40 +0100

> I thought we already had decided we didn't require changes to the host?

We don't *require* changes to the host.  We propose optional host
modifications that improve the user experience.

The chairs will correct me if I'm wrong, but I believe this is consistent
with Homenet's charter.

-- Juliusz

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 21 Feb 2015 09:49:39 -0800

> active IPs? I guess we need to handle IPv4 as well, so we use proxy-arp 

I was referring to eg:
https://tools.ietf.org/html/draft-hertoghs-lisp-mobility-use-cases-01
and available commercial implementations, eg: from cisco for this
(google lisp host mobility and try to ignore all the lisp details ;-)).

I haven't tried to figure out all details and whether/how we would
want to adopt a similar scheme to homenet. Something like:

 ->  a first-hop-router would need to be able to detect presence of
     a roaming client. Probably based on MAC-address. And then generate
     the /128 (and /32) prefix for the clients address.
     Thats effectivly what LISP does, but they do not need to only
     create a route but feed into their database.
 
 ->  for seamless mobility, you'd need to have on all subnets for the
     default-router the same virtual MAC address (eg: VRRP).
     Thats also someting set up with LISP. This ensures that traffic
     sent by the client will continue to be forwarded after roamin.
 
 ->  What i couldn't figure out quickly yet is how DHCP updates to the
     client are dealt with especially default-router IP address.

Cheers
     Toerless

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 21 Feb 2015 10:07:18 -0800

I ran RIP on ca. 50 multi-homed clients in the early 90'th wth /32 route injection
for both addresses, which effectively was also used for what we'd call
mobility today. Worked very good. Trying to remember the
reconvergence time after link down... I think i had tuned down the
RIP hello interval. Not quite sure. Stable & slow 10 Mbps
ethernet + fast & flakey 100 Mbps FDDI or 155 Mbps ATM back then was the
use-case. It did keep TCP connections running over link failures.

I would be a big fan of trying to use homenet to bring in a lightweigt
version of something like this into standard OS host stacks. For
real mobility, multi-homing redundancy and to enable anycasting. But
then again, i have also been a fan hostroutes in ISO CLNS and have been
jealous of only L2 switches being privileged to have host-routes for decades now.

I probably would prefer not to use one of the real routing protocols, but
something lightweight. RIP is a stupid routing protocol but just to 
announce aliveness of host interfaces it is pretty ook. And by not using your
production routing protocol, any real router could easily distinguish
those announcement from real routing protocol neighbor updates and accordingly
apply different policies to them as needed.

Of course something defined specific for this purpose (host address aliveness
or the like) would be most clean IMHO. Wouldn't even bother about IPv4 anymore,
so could be done on one of the IPv6 specific signaling protocols we have (ND ?).

Having said this, the proliferation time of host stack improvements is so
slow that i'd certainly not wanted to bet on having this in any IPv6
capable embedded clients in the home i might buy in the next 10 years,
so the hacky way of the default-router proxying the creation of host routes
is likely the necessary stopgap sollution if we want to go this route.

Cheers
   toerless

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 21 Feb 2015 19:52:44 +0100

> I think i had tuned down the RIP hello interval.

Impossible.  It was the Update interval that you had tuned down.

> announce aliveness of host interfaces it is pretty ook.

RIP(ng) has a major flaw: it doesn't have explicit Hellos.  Reachability
is established by receiving Update packets, which have a size that is
proportional to the number of advertised prefixes.

Modern routing protocols (at least IS-IS, OSPF, EIGRP and Babel) have an
explicit reachability sub-protocol, that consists of explicit Hello
exchanges.  This allows sending Hellos much more often than you send
Updates or topology information.  For example, in Babel a packet
containing just a single, unextended Hello has a payload of just 12 bytes,
independent of the size of the network.

(In principle, you could hack a Hello protocol into RIP by interpreting
empty RIP packets as reachability indications.  But that would no longer
be RIP, right?)

> Of course something defined specific for this purpose (host address
> aliveness or the like) would be most clean IMHO.

I believe that BFD (RFCs 5880 and 5881) is the IETF Standard Track
reachability detection protocol.

-- Juliusz

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 21 Feb 2015 11:12:55 -0800

Of course you're right, but let me play devils advocate for a bit:

I am not sure how important it is to separate fast-hello from route
announcements for hosts. We may have <= 10 addresses on a host. Where would
be have a problem ? processing in the receiving router ? 

Something as simple as RIP also makes it very easy to embed it as
a library into apps, which is particularily good if you have an (anycast)
application specific address and you actually want to make sure the
address is only announced when the app is alive.

BFD takes modularily even one step further and if we used that, then
the benefit of hello/route advertisement separation in the routing 
protocols themselves becomes mood. But it adds even more code and
complexity.

Fast Hellos are good. Actually i'd love to have something thats even
better and can also double to recognize lousy links. Oops: marginal links.
But then there is all this talk about low power wifi clients as well and
that makes active probing supposedly prohibitive.

IMHO, the most important question is: 

Whats the option that could likely be proliferated into eg:
MacOS/Windows/Android ? Lets just say with <= 5 sec failover time.

Cheers
    toerless

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 21 Feb 2015 20:48:29 +0100

> be have a problem ?

Radio congestion due to the announcements from the routers, which in RIP
would be advertising the whole network every Update interval.  One
full-size frame every 72 routes, multiply by the number of routers in
range, divide by the Update interval.

(Recall that multicast is 2Mbit/s at the phy.  13ms for a full-size frame,
not counting the cost of collisions.)

> a library into apps

No sensible platform allows unpriviledged code to manipulate the routing
tables.  But if you manage to solve this particular problem, the stub
implementation of Babel is 1000 lines of portable C.

> Lets just say with <= 5 sec failover time.

Babel has consistently been measured as having failover times of 2 to
5 hello intervals over IBSS, with no link-layer support, independent of
update interval (due to triggered updates and reactive route requests).
The protocol supports Hello intervals down to 10ms, and allows varying
Hello intervals dynamically (a feature not used by the current
implementation).

-- Juliusz

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 21 Feb 2015 12:26:11 -0800

> not counting the cost of collisions.)

Ok. But we also need a way to support fast router redundancy IMHO.

Cheers
    Toerless

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sat, 21 Feb 2015 21:29:17 +0100

> Ok. But we also need a way to support fast router redundancy IMHO.

Could you please explain what you mean?

-- Juliusz

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 21 Feb 2015 12:32:41 -0800

> Could you please explain what you mean?

If i unicast fast hellos from the client to the router and that router fails
and the secondary router takes over, how would it know about the aliveness state
of all client. I guess i could have the primary router forward the client aliveness
or something the like...

---

From: joel jaeggli <[email-masked]>
Date: Sun, 22 Feb 2015 10:05:09 -0800

> If you need to tune it, it's not smart enough.

To be fair, network operators have somewhat different priorities, e.g.
rapid convergence at the expense of cpu may be ok, no effective
bandwidth constraints when you're talking about an igp running on a gig
or 10 gig point-to-point link...

---

From: Juliusz Chroboczek <[email-masked]>
Date: Sun, 22 Feb 2015 20:39:13 +0100

> To be fair, network operators have somewhat different priorities,

Yes, that was a cheap shot.  Sorry, I couldn't resist.

-- Juliusz

---

From: Michael Richardson <[email-masked]>
Date: Sun, 22 Feb 2015 22:41:35 -0500

Margaret Wasserman <[email-masked]> wrote:
    >> More generally, I think -01 puts undue stress on scaling and
    >> convergence speed.  Sections 3 and 4 can be summarised as "any
    >> reasonable routing protocol scales sufficiently well and converges
    >> sufficiently fast for the needs of Homenet".

    > quite a bit of discussion of scaling.  In particular, there was a lot
    > of discussion of how these protocols will scale "on Wi-Fi".  I wasn't
    > entirely sure, during the meeting, what special requirements there are
    > for a routing protocol to scale well on Wi-Fi vs. other link types, but
    > perhaps someone on the WG mailing list can clarify?

Counter-intuitively for a broadcast media, multicast on wifi is very
expensive because not all stations can hear each other, and some stations
require higher power lower-speed transmissions.  So each time the AP
transmits a multicast packet, it transmits at the highest power, using the
slowest method.   Many high-end APs (like we use at IETF), will actually
deliver multicast via a series of L2 unicast messages.

What this means is that a protocol that depends upon multicast to achieve
flooding may perform very poorly over wifi compared to a protocol which forms
1:1 links associations.

Now, does this matter in the home: how many homenet capable routers do we
expect to create adjacencies over wifi?  I will claim that over time, if we
get this right, that it will be larger than what we see today in the home
(which is 1 to 3 APs), but it's on the order of
O(p + r):
       p=number of people in home,
       r=number of rooms.

So assuming some decent high-power 802.11ac in the Bradford house
(http://en.wikipedia.org/wiki/Eight_Is_Enough) to link the per-room router to
legacy 802.11b and per-person (phone) router to BTLE/PAN, it means we have
about 30 routers on the wifi.   How much multicast and how much unicast
traffic results?

---

From: Michael Richardson <[email-masked]>
Date: Sun, 22 Feb 2015 23:23:59 -0500

> neighbour's MAC, then find a way to snoop on the switch's forwarding table?

And assuming that you could do this, you still don't know that
fe80::dead:beef isn't connected via some *other* layer-2 switch external to
the WRT.

So, if anyone is basing their convergence time summaries upon loss of link,
vs keep-alive timeouts, then they need to recalculate, because we are only
rarely gonna see link-down.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 22 Feb 2015 23:44:56 -0500

{my comments before reading the ~170 messages on this thread.}

comparison>   [Isn't it also the case that the HOMENET routing protocol will
comparison>   be implemented on lower-end embedded devices, such as nodes in
comparison>   a low-power wireless network?  What is considered to be a
comparison>   reasonable low-end system requirement for a HOMENET router? --
comparison>   mrw]

As specified in the architecture document, section 3.5.1:

rfc7368>   In this homenet architecture, LLNs and other specialised networks
rfc7368> are considered stub areas of the homenet and are thus not expected
rfc7368> to act as a transit for traffic between more traditional media.

So the gateway machine (the 6LBR) will run the HOMENET routing protocol and
the LLN routing protocol (RPL, or some other mesh-under protocol), but the
HOMENET routing protocol will never be run by the sensors.

The section 9, security consideration does indeed miss the mark.
Given that HNCP/DNCP would take care of "higher" identity and authorization
operations, and would produce some set of anchored credentials for the
routing protocols. the security question is not:
        does the protocol support HMAC-FOOBAR.

The questions are rather:
    1) how does the cost and convergence of the protocol change if security
       considerations force the use of (secured 1:1) unicast for all messages
       rather than multicast?

    2) does the protocol include an asymetric method for authentication?

We know that multicast is hard to secure using symetric methods as all
parties involved can impersonate any other party.   To what extent this is a
concern in homenet is not yet known, but on the other hand, in every
risk/cost/benefit analysis, if the cost of securing things is low enough,
it may make no sense not to...

In section 10, it is useful to know if multicast is supported; it is telling
that nobody has implemented multicast routing in ISIS.  It would be
interesting to know how successful multicast routing is in other Link State
and Distance Vector protocols are.  (Can Babel just include DVMRP for
instance?   Why did nobody implement multicast for ISIS?)

A question which I did not see addressed is about debugging.
How easily can one (perhaps more capable, or having better management
interface) router give diagnostics about problems that might be occuring
elsewhere in the homenet?
Can one deal with broken/mis-having peer routers in a unilateral fashion?
(Consider the ability for parents to mitigate a dispute among technically
astute teenagers, while keeping both online)
Can one get diagnostics from just watching the traffic?
As much as we expect the network to be auto-tuning and auto-configuring and
self-healing; if people do get involved, which one is easier to understand?

{my comments after reading the thread}

Important things that I took out of the discussion:

1) Source address specific routing likely doesn't exist in some hardware.
   My response is mostly: we often no idea what hardware can due to NDAs.
   The movement (see netdev01.org conference this past week) is towards
   having hardware which just lives under the linux kernel and its
   configuration is transparent to "userspace"

2) Layer-2 aspect of ISIS.
> We didn't discuss the fact that Babel runs over UDP, while IS-IS runs
> over tunnels (say, GRE or OpenVPN in tun mode).

It also, btw, makes it unable to run over 15.4, since there is no ethernet
type.  6lo also has been defining ways to run IPv6/6lowPAN over other media
that have never seen IPv6.  Probably doesn't matter as long as these are
really *LLNs*, but it could be that some of these technologies (802.15.4g has
much bigger packet sizes...) get used in new and unanticipated ways in the
IPv6 is dominant future.

3) metrics.
   We don't know how we will assign link metrics, so we don't know how important
   X-bit metrics are.

4) convergence over 802.11 MAC...

5) some amount of "my processor is slower than yours and still ran protocol XYZ"
   (http://www.phespirit.info/montypython/four_yorkshiremen.htm)

6) discussion about multicast: do we even need it. With responses from
   "nobody does it" to "we have 10^X customers" right now.

7) one comment was: "we need topology, so babel won't work for us."

8) some conversation about whether or not one can get link state for a wired
   connection.  I don't see that it matters: there could always be another
   $9 layer-2 switch between the two routers. That's the problem with
   layer-2 switching, we can't see it.

9) then there was a discussion of multiple (wifi) APs, how to either L2-mesh them to
   accomplish mobility, or to do L3-roaming by having stub versions of
   homenet routing protocol and/or RIP.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        | network architect  [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Ole Troan <[email-masked]>
Date: Mon, 23 Feb 2015 14:17:01 +0100

> On 21 Feb 2015, at 16:06 , Juliusz Chroboczek <[email-masked]> wrote:
> 

exactly.
you aren't going to have much fun with multi-prefix multi-homing without host changes anyhow.

cheers,
Ole

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 23 Feb 2015 14:20:33 +0100 (CET)

> you aren't going to have much fun with multi-prefix multi-homing without host changes anyhow.

Let me get this straight: So the proposal is that end systems will have to 
change their IPv6 address when it moves between wifi APs if it's a current 
end system, and in order to keep its address, it needs to start to speak 
HNCP?

---

From: Ole Troan <[email-masked]>
Date: Mon, 23 Feb 2015 14:39:34 +0100

> Let me get this straight: So the proposal is that end systems will have to change their IPv6 address when it moves between wifi APs if it's a current end system, and in order to keep its address, it needs to start to speak HNCP?

are you replying to the point I made? cause fully functioning MHMP requires host support (read MP-TCP/session layer) regardless of moving or not.

with regards to a proposal I haven't seen one written up, but I don't think what your are stating is it. ;-)

cheers,
Ole

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 23 Feb 2015 14:41:49 +0100 (CET)

On Mon, 23 Feb 2015, Ole Troan wrote:

> are you replying to the point I made? cause fully functioning MHMP 
> requires host support (read MP-TCP/session layer) regardless of moving 
> or not.

If I extrapolated correctly what Juliusz wrote, that is not what he had in 
mind.

> think what your are stating is it. ;-)

I didn't think so either. :)

---

From: Mark Townsley <[email-masked]>
Date: Mon, 23 Feb 2015 15:58:36 +0100

When a host connects to a different link covered by a different subnet,
indeed it will require a new IP address. That's pretty fundamental to what
a subnet is. Hosts are getting better and better at handling multiple
addresses, of both versions, coming and going. MPTCP should continue to
help in this regard. I'm a big fan of seeing more and more MPTCP, and I've
been impressed at the ability of modern devices to upgrade their host
stacks in the past 4-5 years (vs. the decade or so before). I'll remain
cautiously hopeful here for the long term.

In the mean time, I fully expect that to support seamless wifi roaming from
one AP to another there will have to be some sort of bridging or tunneling
(capwap, et. al) taking place to keep the single ethernet wifi subnet alive
for IPv4 and less modern IPv6 hosts. That doesn't mean there will be no
room for routing in the home, as one of the primary motivations for IP
routing is to support diverse media types. i.e, it's much easier I think if
wi-fi only has to worry about roaming within wi-fi, and not MOCA, EoPL,
802.15.4, Bluetooth LE, etc. as well.

One day if/when hosts and apps finally become fully resilient to IP address
changes, then it could become much easier on APs as they will have the
chance to simply hand out a new IPv6 address and route, avoiding bridging
and/or tunneling tricks. That could be a nice win for wifi scalability, but
to be honest is shrouded in so many operational practicalities and moving
parts that it's best not to try pretend that this will come to pass even if
we are successful in providing the world with zero-touch IP routing
technology in the home. It could be a very nice bonus, and I hope it
happens, but I wouldn't want to set the bar that high in the near term as
something homenet can make happen on its own.

- Mark

On Mon, Feb 23, 2015 at 2:41 PM, Mikael Abrahamsson <[email-masked]>
wrote:

>> requires host support (read MP-TCP/session layer) regardless of moving or
>> not.
>>
>

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 23 Feb 2015 16:37:01 +0100

Hi Michael,

The work on the document is being done on https://github.com/choppsv1
and I try to keep an up-to-date version of the generated files on

  http://www.pps.univ-paris-diderot.fr/~jch/private/draft-mrw-homenet-rtg-comparison-XX.html
  http://www.pps.univ-paris-diderot.fr/~jch/private/draft-mrw-homenet-rtg-comparison-XX.txt

> HOMENET routing protocol will never be run by the sensors.

I've vastly expanded the section about stub networks; please check if it
suits you better or whether you wish anything added.

> It also, btw, makes it unable to run over 15.4,

Noted, will add.

> We don't know how we will assign link metrics, so we don't know how important

I've expanded the section about metrics.  The IS-IS part is still woefully
inadequate.

> 4) convergence over 802.11 MAC...

I've expanded this section.

> (http://www.phespirit.info/montypython/four_yorkshiremen.htm)

Not my choice, sorry.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Mon, 23 Feb 2015 16:46:55 +0100

> about 30 routers on the wifi.

I'm under opposing pressures relating to scaling.  A lot of people, like
you, seem to think it's irrelevant, some others feel very strongly about it.

The current version of the draft has this to say:

    Experts appear to disagree on the expected size of a Homenet: we have
    heard estimates ranging from just one router up to 250 routers. In any
    case, while scaling beyond a few thousand nodes is not likely to be
    relevant to Homenet in the foreseeable future, the Homenet protocols,
    if successful, will be repurposed to larger networks, whether we like
    it or not, and it is desirable to use a protocol that scales well.

The thinking is that if Homenet routers are cheaply and widely available,
then people will use them for larger deployments; no amount of legislating
such uses out of scope will change that fact.  The obvious example would
be a hotel: why pay for a professionally installed network when you can
just plug in a bunch of $40 Homenet routers and be done with your customer
network.  I can well imagine a hotel using 200 routers.

If hotels don't meet your fancy -- think schools, think appartment blocks
in third world countries, think hippy communes.  Do we wish to explicitly
support such use cases?  Hopefully not.  But do we wish to have our
protocols collapse just because we didn't have the foresight to avoid
gratuitious limitations?

-- Juliusz

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 23 Feb 2015 12:42:47 -0800

Some Q/Thoughts:

Q: Did i hear it correctly, there is no standard "show mac-address-table" like
CLI on linux/openwrt for built-in switches ?

Q: Is there ANY code in linux  that propagates L2 switch port status to the attached
L3 interface ? Especially if i do have a 1:1 mapping ? In commercial products
that commonly done.

Thought: In general, my feeling is that L2 link status is widely relied upon in commercial
product/dpeloyments. If homenet feels it can not rely on it due to the non-commercial nature
of its development platforms, thats an interesting aspect, especially because
it could impact an IETF standard that we'd like to see commercially implemented and then
the constraints could be different...

Funny ;-)

Toerless

---

From: Michael Richardson <[email-masked]>
Date: Mon, 23 Feb 2015 15:42:57 -0500

> you, seem to think it's irrelevant, some others feel very strongly about it.

I'm not claiming its irrelevant, I'm claiming that it won't bite us all at once.
I accept that some homenet-type installations will go up to 250 routers
(which is only 1 order of magnitude larger than 30, btw), but they won't go

    > network.  I can well imagine a hotel using 200 routers.

but, what I'm claiming is that the 200 routers won't be in a single wifi
channel.    The network will get a small amount of tuning, if only because
the walls will get in the way and some people will have to run some cables in
places.  That's exactly what most of think: that to scale things we need
wired back hauls, so while we hit 250 routers in the routing table, we don't
have to accomodate 250 routers forming adjacencies on a single wifi channel.

    > protocols collapse just because we didn't have the foresight to avoid

So, there are limitations like:
      struct router all_the_routers[256];

and then there are protocol collapses due to taking the entire channel for
adjacencies as happened with OLPC.  The trickle mechanism of DNCP ought to
deal with that.

What we need to do is to make sure that whatever parameters we pick for
scaling fail in a safe way when exceeded.  I'd rather that the router which
has HomeNet v1 parameters in it just stops when it reaches some limit rather
than cause congestion collapse.  The router will then get replaced (or
better, reflashed), but if not, it won't get in the way of the newer, better
tuned devices.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Tue, 24 Feb 2015 01:37:37 +0100

> adjacencies as happened with OLPC.

We're in full agreement about most of what you say.  Are you happy with
the current wording, or are you suggesting I change something?

-- Juliusz

---

From: Michael Richardson <[email-masked]>
Date: Mon, 23 Feb 2015 19:51:52 -0500

> CLI on linux/openwrt for built-in switches ?

Correct, because there is no standard interface between the Linux kernel and
the switches. That's because switch chip manufacturers refuse to release
details except under very strong NDAs.   The partial exception, as I
understand it, is the switch chip in the OpenWRT supported SOCs are supposed
by "swconfig", but it's not integrated into the kernel.

This isn't a desireable state.

    > L3 interface ? Especially if i do have a 1:1 mapping ? In commercial products

If you unplug the cable on a Linux router that has a supported interface
(i.e. e1000), the L3 status on IPv6 goes immediately.  There are hysterical
rainsons why they don't pull the IPv4 link status immediately.

    > Thought: In general, my feeling is that L2 link status is widely relied
    > upon in commercial product/dpeloyments. If homenet feels it can not
    > rely on it due to the non-commercial nature of its development
    > platforms, thats an interesting aspect, especially because it could
    > impact an IETF standard that we'd like to see commercially implemented
    > and then the constraints could be different...

1) any of the commercial home router vendors (cisco, linksys, belkin, dlink,
   etc.) could read the riot act to one of their higher-end silicon
   suppliers (the people with 1G/s capable L3 forwarding engines).  It only
   takes one to cave: that vendor gets to dictate kernel/switch APIs for the
   next ten years.

2) link status becomes meaningless when the end (home) user chains
   L2-switches together.  Professional operators naturally avoid doing this.

---

From: Gert Doering <[email-masked]>
Date: Tue, 24 Feb 2015 09:29:16 +0100

Hi,

> CLI on linux/openwrt for built-in switches ?

Right.  Usually communication to these built-in switches is limited due
to lack of documentation, lack of developer time, and so on - so if you
get VLAN configuration and link status, it's already a win.

> L3 interface ? Especially if i do have a 1:1 mapping ? In commercial products

Not that I know of.

It does not really matter here, though - because you can't rely on the
L2 link status to convey end-to-end sanity, as there can always be an
intermediate L2 switch that links "to you" but lost the link on the other 
end... (and this is where ethernet OAM etc. comes into play in commercial
implementations)

Gert Doering
        -- NetMaster

---

From: Juliusz Chroboczek <[email-masked]>
Date: Tue, 24 Feb 2015 12:15:00 +0100

> upon in commercial product/dpeloyments. If homenet feels it can not rely
> on it due to the non-commercial nature of its development platforms,
> thats an interesting aspect, especially because it could impact an IETF
> standard that we'd like to see commercially implemented and then the
> constraints could be different...

Are you referring to the routing protocol comparison, or to something else?

I have the impression that Babel and IS-IS behave essentially the same
wrt. L2 status -- they both converge fast enough after link status has
been established, and they essentially provide the same facilities for
application-layer link sensing (IMHO Babel's Hello/IHU subprotocol is
slightly more flexible, but that's probably not a big deal).

As to wireless links -- as far as I'm aware, making efficient use of
wireless L2 information in a routing protocol is an open research problem.

-- Juliusz

---

From: Ray Hunter <[email-masked]>
Date: Wed, 25 Feb 2015 16:47:54 +0100

Mikael Abrahamsson wrote:
> Within the current framework to solve this problem with what exists 
> today when it comes to clients, I would say we need either:
>
> 1. HNCP helps set up an overlay L2 tunnel infrastructure connecting 
> all the APs using the same SSID, so the SSID can have the same L2 
> domain. This would probably mean we want to increase MTU on the 
> physical links to avoid fragmentation. Messy. Possibly we could 
> advertise lower MTU on the wifi network to minimize fragmentation if 
> we don't raise MTU.
>
> 2. We set up some kind of L2 switching domain between the APs. This 
> would require VLAN support in the HGWs, and something to set this up 
> with loop avoidance etc. Oh oh oh, we could use IEEE 802.1aq that 
> already uses ISIS as control plane, that way we could possibly run the 
> same IGP for both L2 and L3. Interconnecting APs over wifi seems weird 
> though. Oh, and messy sounds like an understatement.
>
> keep two connections at once and inform the application that one 
> address is going away soon so it can do its thing to try to handle this?
>
I agree that WiFi roaming is a problem that needs addressing in Homenet.

One suggestion from my side for handling WiFi roaming is for these 
clients to incorporate a software loopback interface that does not 
renumber, and is always up, and these roaming clients also actively take 
part in HNCP, and the Homenet routing protocol as "stub routers."

That way the devices can roam at L3, without all of the nasty side 
effects of re-establishing TPC sessions, or updating dynamic naming 
services, or having to run an L2 overlay network everywhere, or having 
to support protocols that require a specialised partner in crime on the 
server side (mTCP, shim6 et al).

YMMV.

-- 
Regards,
RayH

---

From: Ted Lemon <[email-masked]>
Date: Wed, 25 Feb 2015 10:58:50 -0500

On Feb 25, 2015, at 10:47 AM, Ray Hunter <[email-masked]> wrote:
> One suggestion from my side for handling WiFi roaming is for these clients to incorporate a software loopback interface that does not renumber, and is always up, and these roaming clients also actively take part in HNCP, and the Homenet routing protocol as "stub routers."

Something like this is currently done with Babel, but it's not a complete solution because it requires Babel-specific modifications on the client.   The proposal to have a single bridged WiFi broadcast domain would eliminate this problem, at the cost of some substantial pain in terms of how to set that up automatically and in terms of performance, which clearly would suffer from the expanded broadcast domain.   And there's also the problem of differentiating backbone wifi links from wifi leaf networks.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 25 Feb 2015 17:14:47 +0100 (CET)

> server side (mTCP, shim6 et al).

It's my firm belief that we need to rid clients of IP address dependence 
for its sessions. Asking clients to participate in HNCP only addresses the 
problem where HNCP is used.

Fixing this for real would reap benefits for devices moving between any 
kind of network, multiple providers, mobile/fixed etc.

---

From: V6ops <[email-masked]>
Date: Wed, 25 Feb 2015 17:20:34 +0100

> On 25 Feb 2015, at 16:58, Ted Lemon <[email-masked]> wrote:
> 

Agreed.

For the purposes of this discussion, I think a soft requirement on a stub-only implementation of the chosen Homenet routing protocol on a host OS could be an interesting differentiator.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 26 Feb 2015 08:42:06 +1300

On 26/02/2015 05:14, Mikael Abrahamsson wrote:
>> That way the devices can roam at L3, without all of the nasty side effects of re-establishing TPC sessions, or updating
>> dynamic naming services, or having to run an L2 overlay network everywhere, or having to support protocols that require a
>> specialised partner in crime on the server side (mTCP, shim6 et al).
> 
> It's my firm belief that we need to rid clients of IP address dependence for its sessions. Asking clients to participate in HNCP
> only addresses the problem where HNCP is used.
> 
> Fixing this for real would reap benefits for devices moving between any kind of network, multiple providers, mobile/fixed etc.

Violent agreement. This is not a homenet problem; it's an IP problem.
In fact, it's exactly why IP addresses are considered harmful in
some quarters. Trying to fix it just for homenet seems pointless.
http://www.sigcomm.org/ccr/papers/2014/April/0000000.0000008

   Brian

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 25 Feb 2015 15:36:29 -0500

In message <[email-masked]>
> wireless L2 information in a routing protocol is an open research problem.

Other than signal strength and collision rate, what L2 information is
available?  Per MAC information would be nice for the AP side or any
node in mesh or adhoc mode but that isn't collected anywhere AFAIK.

> -- Juliusz

ISPs mostly use Ethernet as point-to-point (PTP) links.  Including
using 100GbE as PTP.  No switches.  L2 Link down is one fast indicator
of link down.  But for Ethernet over transport gear (ie: OTN) BFD is
almost always used.  For short distances L2 over extended reach optics
can be used, including colored optics and WDM.  This is also PTP and
in this case BFD should not be needed.  To the extent that routers use
SONET or OTN interfaces, these have fast L2 link down indication and
are integrated with L3 link down detection.  In all of these detection
is on the order of 10 msec (geographic distance dependent), failover
using FRR is under 50 msec, and IGP convergence is well under a second
(typical 100-200 msec today AFAIK).  L3 hellos are way too slow.

BFD is not heavy weight.  L3 hellos (OSPF, ISIS) can be set down to 1s
with detection in 3s (too slow).

BFD, Hellos, or any form of probe traffic over wireless has the speed
of detection vs overhead issue.  At nominal 10 Mb/s (low end today for
wireless) a small probe would take about 0.1 msec (for example, 125
bytes including all overhead is about 1000 bits).  Not bad if running
100 probes/sec (30 msec detection) unless there are a very large
number of stations using the AP and doing the same thing.  In that
case 10 probes per second might be better.  A very high collision rate
(typically not due to probes, but to real traffic) might result in a
link down indication.  If that is the case, then moving to another AP
would be a good thing.

Flapping needs to be avoided if an alternate is available (ie: with
20% loss, in .2*.2*.2 = .008 = 0.8% of intervale a down indication
would occur).  If any packet received would bring it back up, then at
100 probes / sec, a change in IGP link state could occur about once a
second on average.  Remembering a link down and holding a down state
for a (longish) while would be a good thing.  If there is no alternate
route, not probing at all and/or holding an up state would be good.
OTOH- 20% loss borders on completely unusable.

Curtis

---

From: Henning Rogge <[email-masked]>
Date: Wed, 25 Feb 2015 21:46:48 +0100

> node in mesh or adhoc mode but that isn't collected anywhere AFAIK.

Raw linkspeed and (on Linux) even Throughput to each neighbor... and a lot more.

Just run "iw wlan0 station dump" on a Linux system with wifi and you
will be surprised how much information you will get.

Henning Rogge

---

From: Michael Richardson <[email-masked]>
Date: Wed, 25 Feb 2015 15:50:47 -0500

> I agree that WiFi roaming is a problem that needs addressing in

Yes, but can we rule it out of scope for now?

Can we agree that it's not strictly a routing problem, and that the set of
solutions that we are considering could be used, and that we could also
explain how to do something like automatically setup capwap using HNCP for
discovery, but that we don't have the head-of-queue block on this item for
now?

---

From: Mark Townsley <[email-masked]>
Date: Wed, 25 Feb 2015 22:00:26 +0100

> On Feb 25, 2015, at 9:50 PM, Michael Richardson <[email-masked]> wrote:
> 
> 

Yes, we can.

I think the WG needs to focus on securing success before taking on wild success at this moment in time. 

- Mark

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 25 Feb 2015 17:39:52 -0500

In message <[email-masked]>
 
> something homenet can make happen on its own.

The way this was solved circa early-mid 1990s (with configuration
required and a bit of code hack) involved putting a fixed address on
the loopback interface.  For incoming TCP connections, only an
ifconfig lo0 was required.

For outbound TCP connections, no MPTCP was needed, just an ability to
favor binding to the loopback address on critical applications where
an outbound connection is made.  This was the code change needed back
then (and was tupically done to klogin, and the other kerberos enabled
r-commands).  Today, with address selection policy (ie: ip6addrctl on
BSD), no code changes are needed for outbound TCP connections as well.

This type of allocation would work if moving from one AP to another
within an administrative domain (within a homenet or from an office to
a conferance room at work) but not when roaming from home to the
coffee shop wifi AP.

Here is one way it might get done.

It might be worth considering a multiple means to hand out addresses
for lookbacks.  For example extensions to DHCP, SLAAC, DHCPv6, would
require only minimal host changes.  Hosts could also run OSPF and be
stub routers (can be dual or multi homed, but no transit).

A router can use OSPF link local addresses as it does today and make
an implicit request for a loopback address.  The implicit request is
made by its existance in the topology with no routable address.
Allocation can be by the CPE at a provider border for the router
itself, mapping OSPF or ISIS router-id to the requested prefix.
Explicit requests can be made using a TLV with a MAC associated with a
DHCP lease.  Together these two TLVs imply a /32 or /128 stub.

If a host is dual homed and gets information from DHCP, then when
making a request on a second interface, its already allocated /32
and/or /128 or its other MAC can be included in the DHCP request to
avoid a second allocation.  The router could then advertise an
alternate way to reach that loopback.  The host would now be a stub
reachable via two routers.

For the benefit of older OSPF or ISIS implementations if there were
such a thing in the home, any /32 or /128 stub can be explicitly
advertised into the LSDB.

I don't expect a homenet to get overrun with /128 addresses in the
OSPF or ISIS LSDB.

Hosts with a /128 on its own loopback would use the default route to
get to other stubs within the same /64 as its loopback.  This would
only be updated hosts, so it would be safe to assume they won't make
the /128 into a /64.

Note that hosts then don't require routeable interface addresses, only
an address on the loopback.

This works within an administative domain and doesn't require that the
other side of a TCP connection to be upgraded (as is the case with
MPTCP).  There are no TCP stack changes required, but also
implementing MPTCP won't hurt when wandering to the coffee shop.

Old hosts works as they did before.  They get two interface addresses.
As long as they send traffic to an interface that is up and gave it a
default route things should work.  Switch over based on lease expire
would be bad with long lease times (ie: AP is a switch but
router/dhcpd behind the AP went down).

Old routers work as before except for some reason they see a lot of
IPV4 /32 and/or IPv6 /128 stubs.  Old dumb routers work as long as the
WAN port is not used since they act as NAT from the WAN side but act
as a switch on the LAN side.  Switches are tolerated.

If wifi APs are not used for transit and the APs are routers rather
than switches, then there is no reason to run OSPF over wifi in
broadcast mode.  If the APs are dumb switches, the WiFi hosts look
like Ethernet hosts to any router attached to the AP with Ethernet,
therefore the WiFi subnet might see OSPF multicast packets intended
for other OSPF routers.

Curtis

---

From: Teco Boot <[email-masked]>
Date: Thu, 26 Feb 2015 13:28:31 +0100

> Op 25 feb. 2015, om 22:00 heeft Mark Townsley <[email-masked]> het volgende geschreven:
> 
> 
> 
> 
> 

When performance in dual stack networks with multiple WiFi AP's in homes suffers from homenet protocols, this WG produces dead protocols. I am more hopeful. I hope we can enable 802.11 fast handover by distributing the required info. Still open question how to route or bridge over the wired backbone.

Teco

---

From: Mark Townsley <[email-masked]>
Date: Thu, 26 Feb 2015 13:56:21 +0100

On Thu, Feb 26, 2015 at 1:28 PM, Teco Boot <[email-masked]> wrote:

> When performance in dual stack networks with multiple WiFi AP's in homes
> suffers from homenet protocols, this WG produces dead protocols.

Why would homenet cause wifi APs to suffer more than they do today? Worst
case, wifi remains a single bridged dual-stack LAN. Best case, routing is
possible and hosts are resilient to IP address changes. Somewhere in
between HNCP helps with auto-configuration in one way or another. I don't
see how the status quo path we are without Homenet can be worse with
Homenet (separating out here whatever overhead of getting homenet itself
deployed).

> I am more hopeful. I hope we can enable 802.11 fast handover by
> distributing the required info. Still open question how to route or bridge
> over the wired backbone.
>

 certainly do not want to inhibit your hope in anyway. Indeed, history
shows that we successful protocols are often deployed in ways the designers
do not expect. Just trying to nudge the group towards a bit more focus in
order to ship sooner rather than later.

- Mark

---

From: Teco Boot <[email-masked]>
Date: Fri, 27 Feb 2015 05:27:20 +0100

> Op 26 feb. 2015, om 13:56 heeft Mark Townsley <[email-masked]> het volgende geschreven:
> 
> 
> 
> When performance in dual stack networks with multiple WiFi AP's in homes suffers from homenet protocols, this WG produces dead protocols.
> 
> Why would homenet cause wifi APs to suffer more than they do today? Worst case, wifi remains a single bridged dual-stack LAN. Best case, routing is possible and hosts are resilient to IP address changes. Somewhere in between HNCP helps with auto-configuration in one way or another. I don't see how the status quo path we are without Homenet can be worse with Homenet (separating out here whatever overhead of getting homenet itself deployed).

Today's WiFi stacks typically prefer staying on connected SSID, until the link is really bad and breaks. Then the alternative SSID is selected, IP address is flushed and a new IP address is requested. This takes time. Connections break.

Staying on the same SSID has advantages: IP address survives handover so connections continue to work. Also smarter handover can take place, this eliminates the bad link.

So my opinion is that the homenet protocols shall be able to support a layer-2 backbone over a wired backbone connecting a set of access points. Or provide an alternative that works well over a layer-3 backbone. I'm all ears.

Teco

>  
> I am more hopeful. I hope we can enable 802.11 fast handover by distributing the required info. Still open question how to route or bridge over the wired backbone.
> 
>  certainly do not want to inhibit your hope in anyway. Indeed, history shows that we successful protocols are often deployed in ways the designers do not expect. Just trying to nudge the group towards a bit more focus in order to ship sooner rather than later. 
> 
> - Mark
> 
> 
> Teco
> 
> 
> >
> > - Mark
> >
> >>

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 27 Feb 2015 12:39:07 +0100

> Why would homenet cause wifi APs to suffer more than they do today?

I think Teco was reacting to the suggestion that we perform wifi-wifi
bridging at a larger scale that is done today.  We'd need to actually try
it out and perform some serious measurements in order to be sure (no, I'm
not volunteering), but I'd expect it to suck, for at least the following
reasons:

1. 802.11 bridging is weird, there are some restrictions on the possible
   topologies (but I don't recall the exact details).

2. I'd expect broadcast/multicast to be fun, especially if the different
   APs are set up to interfere.

3. Things like TRILL aside, bridging performs spanning tree routing, so
   unless you design your topology carefully, you have a good chance of
   pushing all of your traffic through a slow link.  Never mind avoiding
   self-interference.

I've already expressed my opinion (sometimes way too strongly, sorry Ted)
that I'm opposed to reliance on L2 bridging until somebody shows how it
can be made to work with good performance in a hybrid network.  The 802.11s
experience doesn't encourage us to be optimistic.

-- Juliusz

---

From: Teco Boot <[email-masked]>
Date: Fri, 27 Feb 2015 13:26:16 +0100

> Op 27 feb. 2015, om 12:39 heeft Juliusz Chroboczek <[email-masked]> het volgende geschreven:
> 
> bridging at a larger scale that is done today.

I'm not advocating large layer-2 topologies. I prefer to stay away from broadcast storms and other fun. But today's WiFi stack is build on the assumption that an SSID equals an IP subnet. WiFi handover is within an SSID. 

My personal goal is to set up a standards based CAT5 + WiFi homenet that performs well. I can replace all my switches and wireless routers with the monthly budget for cable and mobile connections. But I cannot buy something that works, unless I tenfold the budget and buy an enterprise solution.

My call is: keep going, let's solve it. If it takes TRILL, CAPWAP, CPE/cloud based SDN: so be it. I want to see two- or three-pack high-end wired/wireless homenet router kits in the shop that will replace our current gear.

Teco

>  We'd need to actually try

---

From: Ted Lemon <[email-masked]>
Date: Fri, 27 Feb 2015 10:41:53 -0500

> My call is: keep going, let's solve it. If it takes TRILL, CAPWAP, CPE/cloud based SDN: so be it. I want to see two- or three-pack high-end wired/wireless homenet router kits in the shop that will…

I'd love it if it didn't require that, but in principle I think this is an avenue worth exploring.   Maybe we can have a hackathon in Prague... :)

---

From: Dave Taht <[email-masked]>
Date: Fri, 27 Feb 2015 07:53:45 -0800

I am glad, incidentally, that for the first time, this wg is
considering some of the problems wifi has, and growing towards
understanding them in more detail. I have long been working on finding
answers to these deep, underlying problems - after first identifying
some the major ones:

https://www.youtube.com/watch?v=Wksh2DPHCDI&feature=youtu.be

and proposing some solutions to the IEEE that still worked inside the
standard (well, obsoleting part of 802.11e entirely)

http://snapon.lab.bufferbloat.net/~d/ieee802.11-sept-17-2014/11-14-1265-00-0wng-More-on-Bufferbloat.pdf

and also proposing some deep changes for 802.11ax (the successor to ac).

Work on getting some of that stuff done is proceeding - unfunded, by
volunteers that care, in their spare time...

(one major set of needed patches: "minstrel-blues" - coupled power and
rate control - is out for review on the linux-wireless mailing list
and it could used an acked-by because it is great and desperately
needed. You don't need to transmit at the highest possible rate when
you are right next to the AP, and vice versa)

Finally, a few weeks ago, I convinced a major wifi testing house to
actually start poking at one subset of the problem, which is multiple
stations attempting to transmit at the same time. This test uses a
single TCP flow each, 1 up, 1 down, and measures the latency.

http://www.candelatech.com/downloads/rtt_fair4be-comparison-box-plot.png

This is on the latest and "best" 802.11ac hardware on the retail
market, transmitting at the highest possible rate, to 4 stations,
under lab conditions. Under load, you presently observe latencies of
50-1000ms, and jitter, same. They only achieved ~ 1/3 the rate of the
base mac capability.

They haven't tried lower rates, or added interference, nor mixed in
multicast, nor tried WDS, or 802.11s, or added stations - any one of
which can mess things up by another order or two or *more* of
magnitude, I am awaiting further results from them, testing lower
rates in particular. But I do hope that they eventually manage to
duplicate the kinds of results I have obtained all over the world in
conference centers, hotels, and apartment buildings, where the typical
latencies I observe can be in the 3-6 second range, and bandwidth,
below a few dozen k, at best.

This sort of result should be concerning to the people that would like
to bridge everything, use range extenders, transmit any multicast at
all (and add in new forms of multicast, like nd/hnetd/babel/etc), or
have hope that wifi can continue to work at all - in the face of
adding lots and lots more (IoT) wifi clients - without some major work
on how our APs and client chipsets work at a deep level.

And thus, this is why I am not paying a whole lot of attention to the
ietf anymore, and sinking more of my energies into finding ways to
preserve and repair one of the coolest, most free-ing internet
technologies that has has ever existed. And it is also sort of why I
am running ethernet or homeplugs everywhere I must.

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 27 Feb 2015 16:37:55 -0500

In message <[email-masked]>
 
> will be surprised how much information you will get.

Henning,

How can a router make use of throughput to a mostly idle neighbor?
How do you get packets sent to a neighbor that dropped or packets that
a neighbor sent to you that didn't arrive here?  Raw link speed or
packet and byte counts don't by themselves tell you much.  The
equivalent of PPP LQM or MPLS-TP LM OAM would be the sort of thing
needed is you didn't want to use BFD with a high probe rate.

As I said above (or tried to say), the most useful hints that the link
isn't as good as nominal link speed might indicate might be signal
strength and collision rate.  [What I meant above by "available" was
available and useful for "making efficient use of wireless L2
information in a routing protocol" in the quoted text.  So maybe I was
too terse in saying that.]

Thanks for the response though.  I use FreeBSD and other than rate and
S/N there isn't much, so could you send me sample output from a Linux
host or better yet a Linux AP with a few "neighbors".  We can take
this off list and discuss the sample output but so far "lots of stuff"
(sic) doesn't help.

Curtis

ps - maybe I should stop procrastinating and compile and flash openwrt
and see for myself, but for now ...

---

From: Markus Stenberg <[email-masked]>
Date: Fri, 27 Feb 2015 23:51:22 +0200

On 27.2.2015, at 23.37, Curtis Villamizar <[email-masked]> wrote:
> (sic) doesn’t help.

Random example output of one.. ;) (I rather not spam the list)

# iw dev wlan1 station dump
Station <mac> (on wlan1)
	inactive time:	1090 ms
	rx bytes:	13841464
	rx packets:	99054
	tx bytes:	65086103
	tx packets:	90179
	tx retries:	1023
	tx failed:	0
	signal:  	-59 [-66, -64] dBm
	signal avg:	-58 [-64, -61] dBm
	tx bitrate:	300.0 MBit/s MCS 15 40MHz short GI
	rx bitrate:	24.0 MBit/s
<snip> irrelevant parts (various flags on/off)
#

I see both tx failed and retries nonzero on some nodes, and for all, signal strength, and tx/rx bitrates. I guess all of this could be used for _something_ at least.. 

-Markus

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 27 Feb 2015 17:00:10 -0500

In message <[email-masked]>
 
> http://www.sigcomm.org/ccr/papers/2014/April/0000000.0000008

Brian,

Seriously - your paper may be overstating the problem.  At least if we
discount IPv4 and in doing so eliminate NAT we solve a lot of problems
that never should have existed in the first place.  If we carry NAT
over to IPV6, then shame on us.

If we get rid of NAT a big part of the problem just goes away.  No
alternate spaces, kludgy rendezvous mechanisms, etc.  Using an address
on the loopback gets rid of the multiple interface problem and where
it really matters (ISP router and ISP or DS server reachability) this
has been done with configuration for two decades.  The multihoming
failover or roaming are a bit more difficult but things MPTCP is
supposed to address.

Curtis

---

From: Dave Taht <[email-masked]>
Date: Fri, 27 Feb 2015 14:28:38 -0800

> over to IPV6, then shame on us.

I am sorry, I no longer share this opinion. The pains of renumbering
someones entire home network every time the ISP feels like it, given
the enormous number of devices I have encountered that don't handle it
well, are just too much to bear.

The next version of cerowrt will do translation from the external IPv6
address range to a static internal one (or ones, in the case of
multiple egress gateways), and lacking a standard for such will use
fcxx/8 addressing. I will make it be an option for people to turn off,
but I've had it with being renumbered.

I am sure this will break stuff, and I don't know what all it will do,
and I intend to find out.

Until some far off day where we have stable name to ipv6 address
mapping, and vice versa, it is otherwise impossible to have useful
ipv6 based services inside the home or small business.

-- 
Dave Täht
Let's make wifi fast, less jittery and reliable again!

https://plus.google.com/u/0/107942175615993706558/posts/TVX3o84jjmb

---

From: Dave Taht <[email-masked]>
Date: Fri, 27 Feb 2015 16:26:47 -0800

> number of stations using the AP and doing the same thing.  In that

You are thinking about it wrong. Wireless-g is only capable of roughly
1300 TXOPs total per second. Bandwidth has NOTHING to do with it.

I don't have the figure in my head for n or ac, but it is not a lot, and
in the presence of g, falls back to the above figure.

losing 10% of the bandwidth - per station - to run BFD is not an option.

> case 10 probes per second might be better.  A very high collision rate

Still too much. Next question?

---

From: Dave Taht <[email-masked]>
Date: Fri, 27 Feb 2015 16:28:48 -0800

> and see for myself, but for now ...

You don't have to compile a thing, just download the right nightly
from chaos calmer (preferred as hnetd etc are in it), and flash it.

Everyone here, has been 90 bucks, 5 minutes and one reflash away from
eating it's dogfood for 9+ months now.

http://downloads.openwrt.org/snapshots/trunk/

---

From: Dave Taht <[email-masked]>
Date: Fri, 27 Feb 2015 16:56:45 -0800

One of the things I am testing is the new (so far pretty wonderful)
minstrel-blues patches for linux which couples rate control with
reducing power where it can, and adds a per station rc_stats_csv file
that can be easily parsed by external utilities. This gives you a
snapshot of the actual rate to the station being achieved, what the
fallback rates are, and the probability of success.

Example stats:

http://snapon.lab.bufferbloat.net/~d/minstrel-ht-stats.xls

minstrel-blues info: http://www.linuxplumbersconf.org/2014/ocw/sessions/2439

Minstrel itself is the base wifi rate estimator for, I don't know, at
least 1/4 billion wifi enabled devices. IMHO, most others are worse.

A sad thing is that the (GREAT) paper on how minstrel actually works
has never managed to get publication (it circulates as samizdat), and
I really do wish more people grokked wifi rate control problems and
realized that the wifi rate is never a constant, and only rarely even
close to the rated specification of the device, and multicast and
management frames are treated very differently from unicast.

And very differently compared to the old shared pre-switched models of ethernet.

Another useful set of statistics (only if you have an ath9k chip) are
summaries of basic
errors and aggregation stats, for each built in 802.11e compliant queue.

root@minstrel-ht:/sys/kernel/debug/ieee80211/phy1/ath9k# cat xmit
                            BE         BK        VI        VO

MPDUs Queued:                0          0         0       182
MPDUs Completed:         50453          0         0       193
MPDUs XRetried:              0          0         0         0
Aggregates:                  0          0         0         0
AMPDUs Queued HW:            0          0         0         0
AMPDUs Queued SW:        50462          0         0        11
AMPDUs Completed:            9          0         0         0
AMPDUs Retried:              0          0         0         0
AMPDUs XRetried:             0          0         0         0
TXERR Filtered:              0          0         0         0
FIFO Underrun:               0          0         0         0
TXOP Exceeded:               0          0         0         0
TXTIMER Expiry:              0          0         0         0
DESC CFG Error:              0          0         0         0
DATA Underrun:               0          0         0         0
DELIM Underrun:              0          0         0         0
TX-Pkts-All:             50462          0         0       193
TX-Bytes-All:          8539213          0         0     21175
HW-put-tx-buf:               1          0         0         1
HW-tx-start:             50462          0         0       193
HW-tx-proc-desc:         50462          0         0       193
TX-Failed:                   0          0         0         0

---

From: Henning Rogge <[email-masked]>
Date: Sat, 28 Feb 2015 10:36:27 +0100

> needed is you didn't want to use BFD with a high probe rate.

You need a bit of probing traffic... a few packets per second are
enough to give you an idea about the speed of the link. Of course you
only need to probe neighbors that you did not send normal unicast
anyways since the last probe.

Henning Rogge

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 28 Feb 2015 13:38:44 -0500

In message <[email-masked]>
 
> discovery, but that we don't have the head-of-queue block on this item for

Perhaps consider it two problems, roaming within an administrative
domain and roaming into another administrative doamin.

The latter is harder to solve.

Other than bridging all of the AP within an administrative domain,
there is no way to support the former without at least some host
change.  As I mentioned before, I favor putting a /128 on the
loopback and having the routers/APs forward to the interface address
of the moment to that /128.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 28 Feb 2015 18:40:32 -0500

> well, are just too much to bear.

I renumbered 140.222/16 into 147.225/16 in the mid 1990s (T3-NSFNET to
ANSNET as part of the NSFNET decommissioning).  Not by myself of
course, but there were only a few of us on this.  It wasn't all that
bad and we had about 2,000 things to renumber in hundreds of
locations, many of them not manned.  Shell scripts and ksh (kerberized
rsh, not the Korn shell) made it simpler.  The worst was Cisco routers
and various old DSU/CSU and other commercial stuff since you had to
use "expect" scripts on their CLI rather than just something of the
form "ksh fqdn -l root ifconfig newaddr/mask alias" People with root
on their workstation that didn't give us acess were given notice.  We
used interface aliases and gradually took down the old aliases and
subnet addresses.  Nothing critical was affected.  It took a day or
two, then bake time, then less than a day to remove aliases.

OTOH - Most homes can't run two prefixes for a week or two before
taking the old prefix down.  And lots of consumer devices don't do
aliases.  But now days there is DHCP which didn't exist then (and
ICMPv6 RS/RA and SLAAC).

Are you having problems with the provider not giving you a static IPv6
prefix, but rather changing it on a whim?

I also renumbered my home net multiple times, but again, not much
pain.  Only a few consumer gadgets here have fixed addresses (one
because it never renewed DHCP leases and therefore needed a fixed
address, but that has since been tossed in e-waste recycling).

> but I've had it with being renumbered.

Are you suggesting that we add NAT to IPv6?  [I ask with disbelief.]

I would definitely be turning that off since I have a plenty big and
very static IPv6 prefix.  I also have a (tiny) static IPv4 prefix so I
have no choice but to IPv4 NAT due to its tiny size.

A better option might be to use something that switches over faster
than a DHCP lease times of days.  It seems that ICMPv6 RA can be sent
with prefix prefix information TLV with valid lifetime and/or
preferred valid lifetime of zero.  This is in RFC 4861 on Page 55:

      - If the prefix is already present in the host's Prefix List as
        the result of a previously received advertisement, reset its
        invalidation timer to the Valid Lifetime value in the Prefix
        Information option.  If the new Lifetime value is zero, time-out
        the prefix immediately (see Section 6.3.5).

Would that help?

Also, stateful DHCPv6 can invalidate leases (me thinks)?  Maybe
DHCPv4?  Am I mistaken about that?

> and I intend to find out.

Just breaks the end-to-end principle and requires rendezvous and all
that ugliness.

IMHO it would be better to send an immediate RA with a zero lifetime
on the old prefix and a normal lifetime on the new prefix.  If hosts
don't do the right thing they are in violation of RFC 4861.

OTOH, invalidating a DHCP lease would not do what we want here.

> ipv6 based services inside the home or small business.

Oh yeah.  And then there is DNS.  Maybe it would be better to keep a
short non-zero valid lifetime (longer than DNS TTL) and a preferred
valid lifetime of zero on that old prefix so it sticks around and is
usable locally but not preferred so not used for outgoing
connections.  Good reason to not set DNS TTL to days.  That way hosts
with cached DNS mapping to the old addresses will still work.

> > failover or roaming are a bit more difficult but things MPTCP is

You want to keep NAT for IPV6?  Really?

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 28 Feb 2015 18:52:22 -0500

> Still too much. Next question?

Oops yeah.  You are right.  Forgot about TXOPs.

How about a PPP LQM / MPLS-TP LM OAM equivalent?

Briefly:

  "A" sends pkt/byte sent to "B".

  "B" notes pkt/byte recv and adds that to the packet.

  "B" adds its pkt/byte sent and sends that back to "A".

  "A" notes its pkt/byte recv and sends back to "B".  "A" saves this.

  "B" simply receives and saves this.

  Save prior packet, wait some time, and then repeat:

    Now subtract the two sets of counter (new - old packet).

    Each side compares other side sent to their own received.
    Difference is number of packets dropped.

  Repeat forever using prior result to update drop rate.

If any of the control packets drop, drop a partial result, repeat
later, and compare to the last complete result.

Is one cycle per neighbor too much?  How about one cycle per neighbor
each 5 seconds?  If "B" is the AP it sends only one packet per cycle
but both sides get accurate drop rate for both directions.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Sat, 28 Feb 2015 19:00:52 -0500

> http://downloads.openwrt.org/snapshots/trunk/

Dave,

I do need to compile if I prefer to try some of my own recipes for dog
food.  Not that I don't also want to try yours.

I'm more interested in Ethernet than WiFi for my home use.  WiFi is
sort of like the penalty box here.  It works fine for normal web
browsing and such.  If I want to move more than just a few bits around
I plug the laptop into a GbE port somewhere.

btw- You do have an advantage in any discussion based on running code.

Curtis

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 01 Mar 2015 14:24:08 +1300

Admittedly 6renum was targetted at enterprise networks, partly because
of the
observation that homenets renumber anyway after every power cut. But
we have spent a lot of cycles on this issue.

http://tools.ietf.org/html/rfc4192
http://tools.ietf.org/html/rfc5887
http://tools.ietf.org/html/rfc6866
http://tools.ietf.org/html/rfc6879
http://tools.ietf.org/html/rfc7010
and maybe it's time to revive
https://tools.ietf.org/html/draft-liu-opsarea-ipv6-renumbering-guidelines

   Brian