Subject: MANET & IP Layering

From: Chip Elliott <[email-masked]>
Date: Thu, 05 Apr 2001 10:39:21 -0400

MANET Folks,

I have enjoyed reading the recent thread on exactly
how a MANET "link" or "subnet" should be treated in
the IP architecture. Fred Templin did a masterful
job of clearly laying out the issues, and we all owe
him a round of thanks. His writings would be an excellent
foundation for an Internet Draft on the subject.

In my own view, it is by no means obvious yet whether
a MANET network is better treated as a sub-network with
its own routing and forwarding layers, or whether everything
should be brought up to the IP layer. I know that a
number of people on this list do have a firm belief in
this area, namely, that it should or must be handled at
the IP layer. The arguments in favor of this view have
been well-stated and I won't repeat them here.

Rather, I'd like to bring up what seems to me to be the
main argument in the other direction.

I would argue that the IP worldview has a remarkably limited
set of "links" or "subnets" that it can handle well:

* Point to point links
* Shared broadcast links (eg classic Ethernets)

The assumption that all links fit into one of these two
broad categories permeates the Internet architecture. It
crops up in the notion of "broadcast" and "multicast"
addresses, indeed, but also in the semantics of address
resolution, service discovery, router election & redundancy,
etc etc.

Unfortunately, as Fred has pointed out, the basic RF channel
model doesn't fall into either of these categories. Thus there
are in essence two ways to deal with this fact:

1. Extend the IP semantics
2. Make the MANET "link" or "subnet" appear to fit
   into one of the categories

My own belief is that the chances of success with Option 1
are pretty low, but that Option 2 has been used repeatedly in
the Internet and seems to work pretty well.

Let me explain by a few examples.

A few years back, ATM wrestled with exactly the same problems.
An ATM cloud is neither a point-to-point link nor a shared
broadcast network. The IETF put a fair amount of work into
treating ATM via Option 1 above, eg, by defining a special
form of address resolution for IP / ATM, etc. This was good
work, and done by skillful people. However, in my view, in the
end all this work can to naught, and today people employ ATM
links as classic point-to-point links at the IP layer.
In short, a lot of people tried introducing a new kind of
IP link or subnet, and in the end the IP architecture proved
too resistant.

Ethernet provides a telling contrast. Modern Ethernet is by
no means a "shared broadcast" channel as it was back in the
coax days. Many modern Ethernet "clouds" employ all kinds of
switches & link-layer routing protocols, etc. Yet the Ethernet
community has been very careful to preserve the old-fashioned
Ethernet semantics, so that a modern Ethernet fits very neatly
into the Internet architecture (as either a point to point link,
or as a shared broadcast medium when seen from the IP layer).
The result has been that many generations of Ethernet technology
have slid very neatly into the Internet with nary a ripple.

Although it can seem odd and redundant to have routing running
at two distinct layers (IP and link layer), in fact such schemes
have been employed literally for decades and do work OK in
practice. I've mentioned Ethernet above, and the fact that
Ethernet switches do their own routing and forwarding between
different Ethernet segments. Other obvious examples are SONET,
frame relay, IP-in-IP tunnels, the new IEEE/IETF Resilient Packet
Ring technology, and so forth.

In conclusion, let me remind you that I'm not arguing that
MANET routing and forwarding (etc) *must* be done below the
IP level. Rather, there are fairly strong arguments both for
and against that proposition. Personally I think this is an
area where we, the MANET community, still have not achieved
the necessary level of clarity.

Cheers,

Chip Elliott
Principal Scientist, BBN

---

From: "Fred L. Templin" <[email-masked]>
Date: Thu, 05 Apr 2001 17:32:01 -0700

Chip,

I enjoyed reading your note, and I agree that we are still not
completely converged on all aspects of this subject. I must say
that I've had my eyes opened in a number of areas from the
discussions, but I think that there is still much left to be
explored in more detail.

Fred Templin
[email-masked]

---

From: Kevin Grace <[email-masked]>
Date: Fri, 06 Apr 2001 11:24:01 -0400

Hi Chip,

Thanks for taking the time to express your opinions. I do have
differing views on some points; comments below...

> the Internet and seems to work pretty well.

I'd argue that there is a third option: Work with existing IP
semantics and build the "missing pieces" on top of IP. Some existing
semantics are still applicable in a MANET environment, some are not
and those should therefore be avoided. For example, sending to the
limited broadcast address 255.255.255.255 has traditionally meant that
all directly connected interfaces which hear the transmission should
handle the receive packet but not forward it. This is a very useful
feature that is still very appropriate to RF channels. In fact, some
of the MANET routing protocols rely on this feature.

On the other hand, sending to a subnet directed broadcast address
(eg. 192.168.1.255) has traditionally meant that a transmission will
be received by all interfaces having addresses belonging to the
subnet. Moreover, it has been assumed that such a transmission could
efficiently be delivered by unicasting it from the source (if the
source did not reside on the destination subnet) until it reached a
node with an interface residing on the destination subnet which would
then forward it using a single link layer broadcast. Unfortunately, we
know that in general, this final link layer broadcast is insufficient
to reach all the subnet interfaces...and this is the dilemma that we
have all been discussing. I suggest features like this one which do
not naturally fit for RF channels be avoided.

As Fred has pointed out, some existing protocols/applications rely
upon subnet broadcasts, though I question that the number of such
protocols is so large that the only reasonable alternative is to do
link layer forwarding. For example, you and Fred have both indicated
that router advertisements rely on subnet broadcasts yet this is only
true of "wired" routing protocols...certainly these protocols are not
appropriate for MANETs! Address resolution was also mentioned as a
problem; yet ARP and RARP work fine in MANETs too. 

Certainly there will be some apps/protocols that rely on subnet
broadcast that we'll want to operate in a MANET environment (though
the vast majority do not; consider ftp, telnet, http, ping, etc.), but
its not clear that there are more than a handful. And if that's the
case, then it appears that modifying them for MANET operation may be a
tenable solution.

Also, it does not appear that subnet broadcast are uniquely qualified
for "service discovery" like operations. "Service discovery" like
things can easily be implemented with a simple advertisement protocol
running on a multicast group. This building on top of IP seems a much
more viable option than trying to extend IP's semantics.

> the necessary level of clarity.

Hmmm...let me try to summarize the arguments as I see them:

IP forwarding:
 General solution for *all* media
 Able to utilize all links, regardless of media type, to overcome
partitions
 Readily available Open Source implementations
 Promotes heterogeneity
 Precludes use of subnets...requires mods to a few apps/protocols
 
Below IP forwarding:
 Specific solution for a specific media
 Unable to utilize all links, only those of the same media type, to
overcome partitions
 No readily available Open Source implementations
 Promotes homogeneity
 Enables subnets to be utilized

Does not appear to me to be a close call...of course everyone is
entitled to their own opinion :-)

Kevin

---

From: "Fred L. Templin" <[email-masked]>
Date: Fri, 06 Apr 2001 16:16:50 -0700

Kevin,

Just some follow-up comments on you meesage:

> problem; yet ARP and RARP work fine in MANETs too.

Actually, I have not spoken of subnet-directed broadcasts in any
of my messages, but I do agree they present an interesting set of
challenges for MANETs. What I have spoken of are service discovery
protocols that utilize the all-1's broadcast address and/or the
low-numbered IP multicast addresses in the range from 224.0.0.0
thru 224.0.0.255. Router advertisements rely on this latter form,
as told from the following text in RFC 1256:

=> AdvertisementAddress
=>   The IP destination address to be used for multicast
=>   Router Advertisements sent from the interface.  The
=>   only permissible values are the all-systems multicast
=>    address, 224.0.0.1, or the limited-broadcast address,
=>    255.255.255.255.  (The all-systems address is preferred
=>    wherever possible, i.e., on any link where all
=>    listening hosts support IP multicast.)
=> 
=>    Default: 224.0.0.1 if the router supports IP multicast
=>    on the interface, else 255.255.255.255

The problem with the low-numbered multicasts is that all of
the literature claims they are only supposed to go a single hop
*regardless* of the value in the IP_TTL header field, which means
that a MANET node that is more than one hop away from a sender
won't hear the sender's RA's - unless we introduce special new
legislation for MANETs. What would ideally work for low-numbered
IPv4 multicasts in MANETs would be a combination of setting the
IP_TTL to something other than 1 along with MANET-specific
administrative scoping rules (e.g. "Don't forward beyond the
edge of the MANET"). But, this would require modifications to a
significant number of Internet standards documents as well as
all of the host/router software out there that implement them.   

> more viable option than trying to extend IP's semantics.

Yes, this is basically what I have been saying in my messages.
But still, the problem is the single-hop nature of the all-1's
broadcast address and the low numbered multicast group addresses
used by service discovery protocols. You may not think there are
*lots* of service discovery protocols that require our attention
in MANETs, but there are certainly some *very important* ones,
such as agent advertisements in MobileIPv4. As told in RFC 2002,
MobileIPv4 agent advertisements basically ride on top of RFC 1256
router advertisements, meaning they will only travel 1 hop in a
MANET. But, I think it would be of paramount importance to allow
a node from a foreign MANET that enters the local MANET to detect
the presence of a MobileIP foreign agent with whom to associate;
whether/not the foreign agent is multiple MANET hops away.

You went on to list tradeoffs between the "strictly structured"
vs. "totally unstructured" viewpoints for the IPv4 addresses used
by nodes in a MANET, and perhaps this was with reference to
something Chip said in his message. As I've been saying in my
messages, however, I think the truth lies somewhere in the middle
between the two extremes. To be sure, a MANET *must* present itself
as a coherent aggregatable IPv4 prefix (or collection of IPv4
prefixes) if it is to be connected as a stub network or transit
network to the Internet. And, I think we all agreed that NAT is not
a very attractive alternative to accomplish this. To that end, some
sort of "loosely structured" coordination should probably be done
within the MANET. Whether this means some sort of router affiliation
based method for doleing out IPv4 addresses from one or more common
IPv4 prefixes or whatever is probably open to interpretation.

Fred
[email-masked]

---

From: Kevin Grace <[email-masked]>
Date: Sat, 07 Apr 2001 19:33:16 -0400

"Fred L. Templin" wrote:

> > upon subnet broadcasts...
> 
> challenges for MANETs. 

Apologies for the poor recollection of the conversation...

> all of the host/router software out there that implement them.

There's a broadcast storm problem waiting to bite anyone who blindly
resends a broadcast/multicast, hence the restriction on forwarding. It's
possible to prevent the storm by maintaining a cache at every node which
remembers recently transmitted/forwarded broadcast/multicast packets;
kind of ugly but certainly doable. 

Instead of forwarding broadcast/low-numbered multicasts, it's possible
to utilize a higher numbered multicast group which will be forwarded
using traditional rules. This would preserve traditional limited
broadcast/low-numbered multicast rules and would likely be more
efficient than the alternative of flooding. 

It's clear there is more than one way to approach this problem; whether
we try to preserve apps/protocols and modify IP to allow
broadcast/low-numbered multicasts to be forwarded or whether we instead
try to modify the apps/protocols and preserve IP, is a good question. I
think it might be useful to identify the set of apps/protocols that we
want in a MANET environment which are challenged. That might help bound
the work required by the latter approach.

Kevin

---

From: "Fred L. Templin" <[email-masked]>
Date: Mon, 09 Apr 2001 10:06:21 -0700

Kevin,

Another point about your earlier message:

Kevin Grace wrote:
> Address resolution was also mentioned as a

I think there needs to be an asterisk after the words "work fine".
On the surface, ARP is just a means to bind an IEEE 802.11 MAC
address resolution to an IPv4 address. And, I agree that this
function does work fine in a MANET. But at the core, ARP is an
on-demand neighbor discovery protocol and in that regard it
embodies some fundamental assumptions that don't mesh well with
MANETs.

When node A sends an ARPOP_REQUEST asking to resolve the IPv4
address B into a MAC address, B will assume the bi-directional
link A<->B is up if B receives A's message and as such B will
create an ARP table entry for A. This assumption is fine on a
connected multiple-access broadcast network, but in a MANET
there will be many instances in which B is hearing A's messages
but A is *not* hearing B's messages. In these cases, the link is
really asymmetric from A->B, but B won't detect this at the
ARP level.

This problem is complicated by the fact that ARP cache entries
are aged based on criteria that have nothing to do with the
topology of the MANET. Namely, the ARP entry B created for A
will stay around for a timeout period based on the most recent
message B has sent to A. So, especially in a MANET with a high
degree of mobility, cached ARP entries for asymmetric links can
interfere with communications if there is no synergy between
the MANET routing protocol and the ARP cache.

Fred
[email-masked]