Subject: [rbridge] IS-IS for TRILL interoperability issues

From: James Carlson <[email-masked]>
Date: Wed, 18 Feb 2009 10:02:45 -0500

In testing our TRILL implementation, I've discovered that there's both
ambiguity in the current -11 I-D (separate designers here were able to
interpret the specification in incompatible ways), as well as serious
issues looming further down the road.

I recommend (1) including explicit diagrams of *all* of the packet
formats (currently, only the TRILL header is diagrammed), so that
there can be no confusion about the meaning of the English text, and
(2) reconsidering the recent "IS-IS unencapsulated" decision.

The first issue (specification ambiguity) revolves around the way
IS-IS frames are transmitted, and the interpretation of the word "and"
in this text:

      o  "TRILL" frames are those (1) with a multicast destination
         address allocated to the TRILL protocol (see Section 7.2) and
         (2) non-control frames with the TRILL Ethertype. There are

Assuming Ethernet, my reading of the current specification results in
TRILL IS-IS frames that look like this:

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               All-IS-IS-RBridges MAC Destination              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |     MAC Destination (cont)    |       Source MAC Address      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                      Source MAC Address                       |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   Ethernet 802 Length Field   |   DSAP = FE   |   SSAP = FE   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   CTRL = 03   |   IDRP = 83   |   Length ...
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-...

That is, I read the "and" in the text to mean that any packet with
that special multicast destination *OR* with the appropriate Ethertype
is to be considered a "TRILL frame," and I understood "unencapsulated"
to mean that the *ONLY* thing distinguishing this from a regular IS-IS
frame was the destination MAC address (and woe unto those
implementations that don't filter multicast perfectly).

The other designer here apparently read that "and" literally, because
his frames look like this on the wire:

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               All-IS-IS-RBridges MAC Destination              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |     MAC Destination (cont)    |       Source MAC Address      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                      Source MAC Address                       |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |        TRILL Ethertype        |   IDRP = 83   |   Length ...
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-...

Both seem plausible, and given the lack of information, it's possible
that others may read this specification in other ways, perhaps
including the LLC header with the Ethertype, depending on what those
readers consider to be the IS-IS frame.  Thus, I strongly encourage
the authors to include an appropriate diagram (such as the above) to
make the frame format unambiguous.

However, this observation leads directly into the second problem,
which I view as much less tractable.  Because of the "unencapsulated"
change, the specification now critically depends on Outer.MacDA.
Unfortunately, the presence of Outer.<mumble> itself depends on the
underlying medium.  When TRILL is run over something *other* than
Ethernet, those fields change.  For example, when run over PPP, there
are no Other.anything fields available at all other than a protocol
type.

A TRILL data packet on PPP would look like this (without header
compression):

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   ADDR = FF   |   CTRL = 03   |       TRILL Protocol ID       |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      | V | R |M|Op-Length| Hop Count |   Egress RBridge Nickname     |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  Ingress RBridge Nickname     |  Options ...
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-...

But what does a TRILL IS-IS packet look like on PPP?  There's no way
to set Outer.MacDA to distinguish this message, so the remaining
options appear to be:

  - Allocate two protocol IDs, one for TRILL data and the other for
    TRILL IS-IS.  (As the chair of PPP extensions, I can tell you that
    allocation of multiple protocol IDs for a single purpose would
    likely be viewed dimly by a significant number of participants.
    It just reeks of poor planning.)

  - Cheat on the TRILL header.  As long as the version field "V" is
    never set to '10', you can always tell an IS-IS frame by testing
    the first byte after the protocol ID: it's 83 for IS-IS, and
    anything else for TRILL.

  - Given both this problem and the subtle MTU issues, reconsider the
    "unencapsulated" decision, and put IS-IS frames after a TRILL
    header that includes a flag indicating whether the payload is
    IS-IS or data.

-- 
James Carlson, Solaris Networking              <[email-masked]>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: James Carlson <[email-masked]>
Date: Wed, 18 Feb 2009 13:20:41 -0500

>     header that includes a flag indicating whether the payload is

I know, bad form to follow up my own posting, and I should have
included this in my original message, but I could live with either the
original Inner.MacDA test (needed for both IS-IS and ESADI) to
distinguish these frames or with a discrete flag in the header.  The
latter would chew up a precious bit, but would be far easier to
implement in hardware and embedded systems (much easier than looking
for a giant Ethernet address later in the message at a potentially
*variable* offset due to options).

That latter sort of header would look like this on Ethernet:

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               All-IS-IS-RBridges MAC Destination              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |     MAC Destination (cont)    |       Source MAC Address      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                      Source MAC Address                       |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |        TRILL Ethertype        | V |R|C|M|Op-Length| Hop Count |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   Egress RBridge Nickname     |  Ingress RBridge Nickname     |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  TRILL Options ...
      +-+-+-+-+-+-+-+-+-...
      |   IDRP = 83   |   Length ...
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-...

and this on PPP:

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   ADDR = FF   |   CTRL = 03   |       TRILL Protocol ID       |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      | V |R|C|M|Op-Length| Hop Count |   Egress RBridge Nickname     |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  Ingress RBridge Nickname     |  TRILL Options ...
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-...
      |   IDRP = 83   |   Length ...
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-...

Where the big change is the addition of the "C" (command/data) flag:

  If C == 0, then the header specifies TRILL data, and contains an
  Ethernet MAC header with VLAN tag after the TRILL Options (if any).

  If C == 1, then:

	M must be 0
	Op-Length is zero because no options are yet defined
	Hop Count must be 0
	Egress and Ingress RBridge Nicknames are 0 on transmit
	The IS-IS packet (starting with IDRP) begins after the
	(non-existent) options field.

A form like this guarantees that the key information required to
distinguish control from data is always at fixed offsets in the
packet, which is necessary for hardware design, simple embedded
systems, and some kinds of packet filtering mechanisms.

---

From: Radia Perlman <[email-masked]>
Date: Wed, 18 Feb 2009 16:10:43 -0800

(Note: I like descriptive subject lines, so I'm changing the subject 
line for the thread to discuss
James Carlson's latest noticing of a problem in the spec).

He mentioned that currently the way to tell the difference between an 
IS-IS message and an encapsulated
TRILL message is that the outer destination MAC address is 
"All-IS-IS-RBridges" for IS-IS messages, and
"All-RBridges" for encapsulated data, and ESADI. But on PPP, there is no 
"outer destination MAC addres". So,
how can we differentiate them?

Here are the possibilities that were floated around:

a) get two protocol types for TRILL for PPP (it's frowned on to "waste" 
these)

b) use the top bit after the PPP header, which would be the version 
field if there were
a TRILL header there, which fortuitously today happens to be 0 for 
TRILL-encapsulated, and 1 for
IS-IS. (kind of kludgy, and uses a bit of the already small version 
number field)

c) insert a "sub-protocol type" field after the PPP header, either a 
byte, or more if
people want the header byte-or word aligned. Currently it would only 
have two
values (IS-IS, or encapsulated)

d) always encapsulate with a TRILL header, and differentiate based on 
the "inner"
MAC destination address

************
I think I prefer suggestion c).

Here's James Carlson's original post:

and this on PPP:

  If C == 1, then:

	M must be 0

-- James Carlson, Solaris Networking <[email-masked]> Sun 
Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS 
UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 
_______________________________________________ rbridge mailing list 
[email-masked] http://mailman.postel.org/mailman/listinfo/rbridge

---

From: James Carlson <[email-masked]>
Date: Thu, 19 Feb 2009 08:18:38 -0500

> values (IS-IS, or encapsulated)
[...]
> I think I prefer suggestion c).

I dislike (c) for at least two reasons:

  - This means that TRILL headers have two "formats."  There's one
    format (without this sub-protocol [kludge] field) on media that
    have L2 addresses that we can allocate, and another format (with
    the field) on media that don't.  This is unlike all other L3
    protocols that run on PPP, which all have well-known formats that
    also work on other media.

  - The difference in headers means that the already-subtle MTU
    computation (which is marred by the "unencapsulated" IS-IS running
    in parallel with the highly-encapsulated data) gains another wart.
    I'm already a bit wary that we can get interoperability at all
    (the spec has no real guidance on MTU issues), and this will add
    to the problem.

  - What happens if we simplify the implementation?  Why not have the
    same TRILL header format on both Ethernet and PPP?  To me, the
    sub-protocol idea is just a half-step into using a flag to
    distinguish data from control, so we might as well go all the way.

OK; at least two reasons.  Probably more like three.  ;-}

One more possible solution (peculiar to PPP) would be:

  (e) use the rarely-used 4xxx range of PPP Protocol IDs to carry
      TRILL IS-IS and ESADI.

That's still not so great, as we'd need to distinguish between TRILL
IS-IS and ESADI, and it still doesn't address MTU.

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 19 Feb 2009 09:30:08 -0500

Hi Jim,

>         (2) non-control frames with the TRILL Ethertype. There are

Probably should be re-worded something like:
  "TRILL" frames are those that either (1) have a multicast
destination address allocated to the TRILL protocol (see Section 7.2)
or (2) are non-control frames with the TRILL Ethertype. ...

> TRILL IS-IS frames that look like this:

The above is indeed what was intended.

> his frames look like this on the wire:

Yes, the wording in the definition of a TRILL frame should be improved
and an explicit frame diagram for a TRILL IS-IS frame should also be
included.

> ... [see other messages re PPP problem]

Thanks,
Donald
-- 
=============================
 Donald E. Eastlake 3rd   +1-508-634-2066 (home)
 155 Beaver Street
 Milford, MA 01757 USA
 [email-masked]
_______________________________________________
rbridge mailing list
[email-masked]
http://mailman.postel.org/mailman/listinfo/rbridge

---

From: James Carlson <[email-masked]>
Date: Thu, 19 Feb 2009 10:15:31 -0500

> or (2) are non-control frames with the TRILL Ethertype. ...

Yes, that's clearer, though I still like the idea of pictures better.
There's no arguing with an array of bytes.

> and an explicit frame diagram for a TRILL IS-IS frame should also be

That'll help; thanks.

> > ... [see other messages re PPP problem]

For what it's worth, I had them in one message because the two issues
are tangled together.  The text is unclear in part because of this
change ... and the change itself has also made the text dependent on
features of Ethernet that aren't present on other media.

---

From: Dinesh G Dutt <[email-masked]>
Date: Thu, 19 Feb 2009 13:59:47 -0800

I do not like any solution that in Ethernet doesn't distinguish based on 
the DA or an immediate ethertype. All control protocols so far have 
relied on this property (at least on Ethernet) and is easiest to 
implement in hardware. There is also precedence which makes hardware use 
existing logic.

So, I dislike (d) for sure.

I think (a) maybe simplest. With 64K possible protocol types, using two 
seems in the noise. IEEE has also typically easily granted two protocol 
types for distinguishing control from data frames.

If we go with inserting a "C" bit in the TRILL header, I vote for it to 
be only relevant on PPP and that even if we must set the bit on 
Ethernet, that the DA is the only way to identify IS-IS frames. I don't 
want there to be the possibility of saying that we can use the IS-IS DA 
to carry some form of data that is now distinguished based on the 
presence of the "C" bit.

Dinesh

-- 
We make our world significant by the courage of our questions and by 
the depth of our answers.                               - Carl Sagan

---

From: James Carlson <[email-masked]>
Date: Thu, 19 Feb 2009 17:26:53 -0500

> So, I dislike (d) for sure.

We're talking about an Ethertype, which behaves like a network layer
protocol -- specifically, a tunneling protocol that has a control and
a data portion.

As a former (reformed?) hardware implementor, it seems to me that if
you are parsing the TRILL header at all (which you must if you're
forwarding or doing encaps/decaps in hardware), then checking for a
single bit at a fixed location to determine whether what you're
looking at is control (kick to software) or data (forward normally) is
quite trivial.

Compared with the other issues that running unencapsulated causes --
the ambiguity with other L2 media and the MTU difficulties -- it seems
hard to say that this is a burden.

> types for distinguishing control from data frames.

Doing (e) (where we allocate from the 4xxx range) is much more likely
to fly through the working group, though it does still smell pretty
funny to me, and it leaves the MTU issue unresolved.

> to carry some form of data that is now distinguished based on the 

In this case (which is (d) from Radia's list), the DA is no longer
significant.  We can allocate a single multicast MAC address on
Ethernet, because the "C" bit unambiguously tells us whether we're
looking at control or data for that fixed TRILL Ethertype, and we
needn't bother with address checks.