Subject: [Int-area] SEAL - draft-templin-intearea-seal-05.txt

From: Ralph Droms <[email-masked]>
Date: Sat, 19 Sep 2009 07:16:24 -0400

The IESG has received draft-templin-intearea-seal-05.txt, "The  
Subnetwork Encapsulation andAdaptation Layer (SEAL)" as an Individual  
Submission.  I need intarea review and comment on the document as part  
of the review process.  Please read and respond with comments to the [email-masked] 
  mailing list.  Thanks.

- Ralph

---

From: Margaret Wasserman <[email-masked]>
Date: Tue, 22 Sep 2009 00:37:41 -0400

Hi Ralph,

On Sep 19, 2009, at 7:16 AM, Ralph Droms wrote:

> Subnetwork Encapsulation andAdaptation Layer (SEAL)" as an  
> Individual Submission.  I need intarea review and comment on the  
> document as part of the review process.  Please read and respond  
> with comments to the [email-masked] mailing list.  Thanks.

The SEAL draft (draft-templin-intarea-seal-05.txt) is a reasonably
well-written document with some interesting technical content.
However, I don't believe that it is suitable for publication as an
IETF Proposed Standard RFC for several reasons:

(1) This is fairly substantial document that has not been produced by
an IETF WG, and I do not believe that it has received wide-enough
visibility or review to determine if there is IETF consensus to
publish this document as a standards-track document.  If
standards-track RFC publication is the desired goal, I think it would
be better to hold a BOF to see if there is interest in forming a WG to
pursue this work.

(2) There are a number of significant technical issues with the
document that would (IMO) need to be addressed before it is ready for
publication.  While this document might be a good starting point for
IETF work (if there are enough people who want to work on it), I do
not think it is even close to ready for standards-track RFC
publication in its current form.

I have included my review notes below, which raise several
significant issues with this document.

Margaret

 >Abstract
 >
 >   For the purpose of this document, subnetworks are defined as  
virtual
 >   topologies that span connected network regions bounded by
 >   encapsulating border nodes.  These virtual topologies may span
 >   multiple IP and/or sub-IP layer forwarding hops, and can introduce
 >   failure modes due to packet duplication and/or links with diverse
 >   Maximum Transmission Units (MTUs).  This document specifies a
 >   Subnetwork Encapsulation and Adaptation Layer (SEAL) that
 >   accommodates such virtual topologies over diverse underlying link
 >   technologies.

This abstract (and the title) refer to SEAL as a "layer".  However, it
is unclear to me that SEAL acts as a "layer" in the typical sense that
the term is used in the IETF...

 >   First, the IPv4 header Identification field is only 16 bits in
 >   length, meaning that at most 2^16 packets pertaining to the same
 >   (source, destination, protocol, Identification)-tuple may be active

This doesn't follow...  I think, perhaps, what this document intends to
say is that there may only be 2^16 packets with the same source IP,
destination IP and protocol three-tuple that all have a unique IP
ID value?  This is better stated in one of the appendices.

 >   in the Internet at a given time.  Due to the escalating  
deployment of
 >   high-speed links (e.g., 1Gbps Ethernet), however, this number may
 >   soon become too small by several orders of magnitude.

This claim should be substantiated with a reference of some sort of
data.  Also, there should be some explanation included for why this is
an operational (or even theoretical) problem...

 >  Furthermore,
 >   there are many well-known limitations pertaining to IPv4
 >   fragmentation and reassembly - even to the point that it has been
 >   deemed "harmful" in both classic and modern-day studies (cited
 >   above).  In particular, IPv4 fragmentation raises issues ranging  
from
 >   minor annoyances (e.g., in-the-network router fragmentation) to the
 >   potential for major integrity issues (e.g., mis-association of the
 >   fragments of multiple IP packets during reassembly).

Reference?

 >   As a result of these perceived limitations, a fragmentation- 
avoiding
 >   technique for discovering the MTU of the forward path from a source
 >   to a destination node was devised through the deliberations of the
 >   Path MTU Discovery Working Group (PMTUDWG) during the late 1980's
 >   through early 1990's (see Appendix D).  In this method, the source
 >   node provides explicit instructions to routers in the path to  
discard
 >   the packet and return an ICMP error message if an MTU restriction  
is
 >   encountered.  However, this approach has several serious  
shortcomings
 >   that lead to an overall "brittleness".

Reference?  Also, there was more recent work on PMTUD that doesn't
seem to be reflected in this paragraph.  The transport area should be
asked to review this document, particularly the PMTUD WG if that
remains in some form.

 >   In particular, site border routers in the Internet are being
 >   configured more and more to discard ICMP error messages coming from
 >   the outside world.  This is due in large part to the fact that
 >   malicious spoofing of error messages in the Internet is made simple
 >   since there is no way to authenticate the source of the messages.

Reference?

 >   Furthermore, when a source node that requires ICMP error message
 >   feedback when a packet is dropped due to an MTU restriction does  
not
 >   receive the messages, a path MTU-related black hole occurs.  This
 >   means that the source will continue to send packets that are too
 >   large and never receive an indication from the network that they  
are
 >   being discarded.

My (incomplete) understanding is that this problem has been addressed
by new PMTUD work.

 >   The issues with both IPv4 fragmentation and this "classical" method
 >   of path MTU discovery are exacerbated further when IP-in-IP  
tunneling
 >   is used.  For example, site border routers that are configured as
 >   ingress tunnel endpoints may be required to forward packets into  
the
 >   subnetwork on behalf of hundreds, thousands, or even more original
 >   sources located within the site.

This wording is awkward in a number of ways:

1) There is not typically one site border router in cases where tunnel
decapsulation is happening at the edge of a network, because typically
the tunnel encapsulation/decapsulation happens (at least logically)
outside of the network perimeter (i.e. outside the firewall).

2) I am not sure what it means to say that the ingress tunnel endpoint
forwards packet into the subnetwork on behalf of local sources... It
forwards those packet out to the Internet, not onto the subnetwork (at
least as I would use that term).  It seems like this document is using
the term "subnetwork" to mean "tunnel"?  If so, that is a bit
confusing given that subnet already has a well-defined meaning.

 > If IPv4 fragmentation were used,
 >   this would quickly wrap the 16-bit Identification field and could
 >   lead to undetected data corruption.

Ummm... Why and how?  I think I understand what this is driving at,
but it is not clearly explained.  If the ingress tunnel endpoint is
using a single source adddress/destination address/protocol three-tuple
for a very large amount of traffic generated from a site, and if that
tunnel endpoint is performing fragmentation on _external_ packets to
make them fit in the tunnel, I can see how you might have accidental
ID overlaps.

However, it is more typical for tunnels to fragment the _inner_ IP
packet, causing reassembly to happen at the end nodes, and thus
avoiding the need to collapse many ID numbering spaces into one.  For
precisely this reason.

 >If classical IPv4 path MTU
 >   discovery were used instead, the site border router may be
 >   inconvenienced by excessive ICMP error messages coming from the
 >   subnetwork that may be either untrustworthy or insufficiently
 >   provisioned to allow translation into error messages to be returned
 >   to the original sources.

The passive voice hurts here, as it isn't clear who would be doing the
PMTUD...  The original sending nodes?  Or the ingress tunnel endpoint
itself?

The "subnetwork may ... be untrustworthy?  What does that mean?  What
would be insufficiently provisioned?

I am not sure what problem is being described here.

 >   The situation is exacerbated further still by IPsec tunnels, since
 >   only the first IPv4 fragment of a fragmented packet contains the
 >   transport protocol selectors (e.g., the source and destination  
ports)
 >   required for identifying the correct security association rendering
 >   fragmentation useless under certain circumstances.  Even worse,  
there
 >   may be no way for a site border router that configures an IPsec
 >   tunnel to transcribe the encrypted packet fragment contained in an
 >   ICMP error message into a suitable ICMP error message to return to
 >   the original source.

This is one of the classic reasons to avoid intermediate fragmentation.

 >   Due to these many limitations, a new approach to accommodate links
 >   with diverse MTUs is necessary.

I do not know that this is true...  There are several tunneling
protocols (GRE, L2TP, etc.) that already handle these issues, and this
document has not discussed how they handle them nor has it made any
case that they are being handled inadequately in currently, widely-
deployed tunneling solutions.

 >   This document introduces a Subnetwork Encapsulation and Adaptation
 >   Layer (SEAL) for tunnel-mode operation of IP over subnetworks that
 >   connect Ingress and Egress Tunnel Endpoints (ITEs/ETEs) of border
 >   nodes.  It provides a standalone specification designed to be
 >   tailored to specific associated tunneling protocols such as VET
 >   [I-D.templin-intarea-vet], the Locator-Identifier Split Protocol
 >   (LISP) [I-D.ietf-lisp] and others.

I do not believe that the LISP WG has explicitly been asked to review
this document.  However, this solution has been proposed on the LISP
WG mailing list...  It did not receive wide-spread support, was
rejected by at least some LISP participants, and was ruled
out-of-scope for LISP discussion.  So, it is a bit odd to see an AD-
sponsored submission that claims that this is tailored to LISP...

 >   SEAL encapsulation additionally includes a 2-bit version number.
 >   This document specifies SEAL protocol versions 0 and 1.

A two-bit version number where two of the versions are already assigned?

 >      ITE - Ingress Tunnel Endpoint
 >
 >      ETE - Egress Tunnel Endpoint

These terms need references or actual definitions.  There are several
terms in this section that just expand acronyms with no actual
definition.

 >      DF - the IPv4 header "Don't Fragment" flag

Reference?

 >   The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD,
 >   SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL, when they appear in  
this
 >   document, are to be interpreted as described in [RFC2119].

This document makes frequent use of lower-case requirements keywords
(such as "must") in cases where they appear to be (or could at least
be interpreted as) requirements.  They should be uppercased, switched
to other words, or clarified.

 >   SEAL was motivated by the specific case of subnetwork abstraction  
for
 >   Mobile Ad hoc Networks (MANETs);

And, yet the MANET group did not adopt SEAL?  Why?

 > however, the domain of applicability
 >   also extends to subnetwork abstractions of enterprise networks, ISP
 >   networks, SOHO networks, the interdomain routing core, and many
 >   others.  In particular, SEAL is a natural complement to the
 >   enterprise network abstraction manifested through the VET mechanism
 >   [I-D.templin-intarea-vet], the RANGER architecture
 >   [I-D.templin-ranger][I-D.russert-rangers] and the LISP protocol
 >   [I-D.ietf-lisp].  The term "subnetwork" within this document is  
used
 >   synonymously with the term "enterprise" that appears in these
 >   references.

Again, LISP is mentioned despite the fact that there is no clear
cosensus that this work complements LISP.

So, the term "enterprise" is yet another term for "tunnel"?

 >   SEAL introduces a minimal new sublayer for IPvX in IPvY  
encapsulation
 >   (e.g., as IPv4/SEAL/IPv6), and appears as a subnetwork  
encapsulation
 >   as seen by the inner IP layer.  SEAL can also be used as a sublayer
 >   for encapsulating inner IPvX packets within outer IPvY/UDP headers
 >   (e.g., as IPv4/UDP/SEAL/IPv6) such as for the Teredo domain of
 >   applicability [RFC4380].  When it appears immediately after the  
outer
 >   IPv4 header, the SEAL header is processed exactly as for IPv6
 >   extension headers.

What does it mean that the SEAL header is processed "exactly as for IPv6
extension headers"?  IPv4 does have an IP header extension mechanism
(IP options), why wasn't that used?

 >   network.  SEAL-enabled ITEs insert a SEAL header during the
 >   encapsulation of inner IP packets in mid-layer and outer
 >   encapsulating headers and trailers.  For example, an inner IPv6
 >   packet would appear as IPv4/*/SEAL/**/IPv6/**/* after the mid-layer
 >   '**' encapsulations, the SEAL header, and outer IPv4 and '*'
 >   encapsulations are added.

It is still quite unclear to me, after reading the document twice (and
up to here the third time), what the ** and * are supposed to
signify. Could these things be given names or explained more clearly
in the introduction, or something?

 >   o  For tunnel-mode IPsec encapsulations, [RFC4301], the SEAL header
 >      is inserted between the {AH,ESP} header and outer IP headers as:
 >      IPvX/SEAL/{AH,ESP}/*/IPvY.

Ummm...  It is my understanding that if the SEAL header comes after
the ESP header, it needs to be encrypted.  But, the middle box that
is inserting the SEAL header doesn't have the credentials needed to
encrypt it.  So, I think this example is invalid.

 >   For all other packets, the ITE admits the packet if it is no larger
 >   than the tunnel interface MTU; otherwise, it drops the packet and
 >   sends a PTB error message to the source with the MTU value set to  
the
 >   tunnel interface MTU.  The message must contain as much of the
 >   invoking packet as possible without the entire message exceeding  
the
 >   minimum IP MTU (i.e., 576 bytes for IPv4 and 1280 bytes for IPv6).

I am not sure that all IPv4 stacks will handle a PTB message correctly
if it is returned in response to a packet that did not have the DF bit
set.  It is likely that those nodes do not implement PMTUD and/or
won't have the newer code needed to read the MTU length from the ICMP
error.  IMO, a survey of how existing stacks will reply to this message
in this situation should be conducted before we standardize a protocol
that depends on specific behavior in this case.

 >   Note that this SEAL segmentation ignores the fact that the mid- 
layer
 >   packet may be unfragmentable outside of the subnetwork.  This
 >   segmentation process is a mid-layer (not an IP layer) operation
 >   employed by the ITE to adapt the mid-layer packet to the subnetwork
 >   path characteristics, and the ETE will restore the packet to its
 >   original form during reassembly.  Therefore, the fact that the  
packet
 >   may have been segmented within the subnetwork is not observable
 >   outside of the subnetwork.

I think there may be some significant problems with this approach.
While it true that the exact same bits will be received on the other
end in normal circumstances, this has most of the same problems that
are present with any in-the-network fragmentation.  In particular,
what I think of as the "unit of loss" is not maintained end-to-end.  A
single packet from the source is broken into several pieces that may
be lost separately.  In order to make up for one last piece, the full
original packet needs to be retransmitted.  This has very bad
properties for congestion, especially when large packets are used.

 >   The ITE next encapsulates each segment in the requisite IP/* outer
 >   headers according to the specific encapsulation format (e.g.,
 >   [RFC2003], [RFC2473], [RFC4213], [RFC4380], etc.), except that it
 >   writes 'SEAL_PROTO' in the protocol field of the outer IP header
 >   (when simple IP encapsulation is used) or writes 'SEAL_DPORT' in  
the
 >   outer destination service port field (e.g., when IP/UDP  
encapsulation
 >   is used).  The ITE finally sets the A bit as specified in Section
 >   4.3.5 (if necessary), sets the packet identification values as
 >   specified in Section 4.3.6 and sends the packets as specified in
 >   Section 4.3.7.

It does not appear that SEAL includes any mechanism to allow per-flow
load balancing (for LAG/ECMP systems) when UDP is used in the
encapsulation.  Since that is, as I understand it, the main reason
_why_ someone would want to use UDP encapsulation for a tunneling
protocol, this seems like a significant omission.  If the IETF chooses
to go through with this proposal, the author shoudl look at the UDP
source port mechanism in the LISP encapsulation for an example of how
to support LAG/ECMP per-flow load balancing by using different UDP
source ports for different flows.

 >   Note that when IPv6 is used as the outer IP encapsulation layer,  
the
 >   ITE must insert an IPv6 fragment header with an Identification  
value
 >   set as described in Section 4.3.6.

Middle boxes are not supposed to fragment IPv6 packet en-route.  There
should be no reason to do so, as all IPv6 nodes support PMTUD.

I am somewhat concerned about the descriptions of SEAL probes and
acknowlegments.  Has any analysis or experimentation been done to
determine how much management (or other non-data) traffic will result
in a "typical" SEAL deployment?

 >   For tunnels specifically designed for the traversal of Network
 >   Address Translators (NATs) (e.g., Teredo [RFC4380]) and other
 >   middleboxes that may rewrite the outer IP ID field, the ITE instead
 >   writes least significant bits of the SEAL_ID in the ID field of the
 >   SEAL header and writes a random value in the Identification field  
in
 >   the outer IP header.

How is SEAL expected to detect if one of these boxes exists in the path?

 > Since the ID field in the SEAL header is only
 >   16 bits, however, the ITE must limit the rate at which it sends
 >   packets to avoid wrapping the ID field.  Alternatively, the ITE and
 >   ETE can use SEAL-FS to obtain a larger ID field in the SEAL header
 >   (see Section 5.3.6).  In either case, both the ITE and ETE must be
 >   aware of the manner in which the SEAL_ID is inserted.

How are they aware of this?  Through manual configuration of both ends?

 >   The ITE should specifically process raw ICMPv4 Protocol Unreachable
 >   messages and ICMPv6 Parameter Problem messages with Code
 >   "Unrecognized Next Header type encountered" as a hint that the ETE
 >   does not implement the SEAL protocol.

And do what?

 >4.4.1.  Reassembly Buffer Requirements
 >
 >   ETEs must be capable of performing IP-layer reassembly for SEAL
 >   protocol IP packets up to 2KB in length, and must also be capable  
of
 >   performing SEAL-layer reassembly for mid-layer packets up to (2KB -
 >   OHLEN).  Hence, ETEs:
 >
 >   o  MUST configure a reassembly buffer size (i.e., a SEAL Maximum
 >      Reassembly Unit (S_MRU)) of at least 2KB

This seems small to me...  Most host stacks I am familiar with can
reassembly a 4K packet, and such packets are frequently used for
block-based file sharing protocols, such as NFS (or at least they were
a few years ago).

It is not clear to me why the SEAL spec essentially reinvents ICMP
over UDP.  The motivation for this should be better explained.

The IANA Considerations section is missing a registry for new SEAL
version numbers.

 >10.  Security Considerations
 >
 >   Unlike IPv4 fragmentation, overlapping fragment attacks are not
 >   possible due to the requirement that SEAL segments be non-
 >   overlapping.

Include some further explanation of how this is enforced.

 >   The SEAL header is sent in-the-clear (outside of any IPsec/ESP
 >   encapsulations) the same as for the outer IPv4/* headers.  As for
 >   IPv6 extension headers, the SEAL header is protected only by L2
 >   integrity checks and is not covered under any L3 integrity checks.

Analysis is needed to explain why the SEAL header is not vulnerable
due to this lack of protection.

The security section needs to analyze the SEAL control protocol and
explore the possible security issues with attackers modifying or
fabricating SEAL control messages.

Also, the security section is missing coverage of the general
security issues with tunnels including the ability to inject packets
into a remote network, etc.

 > 2.  Classical path MTU discovery requires that
 > routers generate a PTB message for *all* packets lost due to an MTU
 > restriction; this situation is exacerbated at high data rates and
 > becomes severe for in-the-network tunnels that service many
 > communicating end systems.

Reference?  It is my understanding that ICMP error messages are
always rate limited.

 >   5.  Using SEAL, ETEs encapsulate ICMP error messages in an outer
 >       UDP/IP header such that packet-filtering network middleboxes  
will
 >       not filter them the same as for"raw" ICMP messages that may be
 >       generated by an attacker.

Most firewalls and filtering middle boxes will drop inbound messages
that don't correspond to recent outbound packets and/or all UDP
traffic that is not to an allowed port.  So using UDP isn't a magic
bullet to get these messages through firewalls and NATs.

 >   When the SEAL-TE ITE has knowledge that it wil traverse a  
subnetwork
 >   with non-negligible loss due to, e.g., interference, link errors,
 >   congestion, etc. ,it can solicit Reassembly Reports from the ETE
 >   periodically to discover missing segments for retransmission  
within a
 >   single round-trip time.  However, retransmission of missing  
segments
 >   may require the ITE to maintain considerable state and may also
 >   result in considerable delay variance and packet reordering.

Given that the source will also be retransmitting, it seems like
having SEAL endpoints retransmit fragments could be a bad idea,
especially in the presence of congestion.

 >   Each link in the path over which a SEAL-TE tunnel is configured is
 >   responsible for veryfying the integrity of packets that traverse  
the
 >   link.  Typical links employ strong integrity checks for packet  
sizes
 >   that are no larger than the link MTU.  Therefore, as long as the
 >   packet sizes supported by the underlying link are not violated, the
 >   SEAL-TE tunnel can reasonably expect that each SEAL segment will be
 >   correctly verified by the underlying link integrity checks.
 >
 >   The SEAL-TE tunnel therfore need only concern itself with packet-
 >   splicing errors that can occur due to reassembly misassociations,
 >   i.e., when a segment from packet X is reassembled with segments  
from
 >   packet Y. The primary sources of such errors include software bugs
 >   and wrapping IP ID fields.  Given that IP fragmentation and
 >   reassembly implementations are well-tested, and that SEAL
 >   segmentation and reassembly entails a very simplistic procedure,  
only
 >   the latter scenario bears further mention.

This isn't actually correct.  According to research I've read (e.g. the
Stone-Partridge paper on this subject), there is a sinificant amount of
end-to-end packet corruption that occurs _between_ the links, in the
routers themselves.  Because the link integrity check is performed on
the inbound interface and regenerated on the outbound interface, errors
that occur inside the router (from bus errors, or whatever) are not
detected and pass to the remote end.  Research and recent discussion on
the LISP list would tend to indicate that these errors happen at a
rate on the order of 10^3 to 10^4 when traffic transits a multi-link
network.

I have a general question about the appendices.  Are they intended
to normative or merely informative?  This should be clarified in the
specification.

 >Appendix D.  Historic Evolution of PMTUD
 >
 >   (Taken from "Neighbor Affiliation Protocol for IPv6-over-(foo)- 
over-
 >   IPv4"; written 10/30/2002):

Why not just reference this document instea of quoting it?

Templin                  Expires January 3, 2010               [Page 42]

---

From: Eliot Lear <[email-masked]>
Date: Tue, 22 Sep 2009 16:41:52 +0200

Ralph,

Here are a few comments.  This document deserves more time than I've
given it.

Summary

This protocol specification provides a new and different tunneling
mechanism (actually, really two - one over UDP and one within in a new
protocol).  It provides back-signaling in an attempt to cause tunnel
ingresses to reduce their MTU.

Comments

To start with, this document could do with a heavy dose of editing for
brevity and clarity.  Substantial time is spent on the case for SEAL.  I
think we can all accept that there are MTU problems on the network.  We
needn't revisit each and every one of them.  Or if we must, let's do so
in a different document.  So for starters, start the doc at §1.2.

Second, while I realize that some might have grown attached to the name,
and that this is picky, the term "subnetwork" is so overloaded that it
is confusing to the casual reader such as myself.

There are a number of imprecise terms, such as "reasonable reliability"
that should be better qualified.  There are also some sentences that
just don't make sense.  For instance:

   While it is well known that the addressing
   properties of IPv4 are limited (hence the larger address space
   provided by IPv6), there is a lesser-known but growing consensus that
   other limitations may be unable to sustain continued growth.

???

This document normatively makes references to several other drafts,
including [draft-templin-intarea-vet], [draft-templin-ranger] and
[draft-russert-rangers] for a singular notion of the "enterprise
abstraction".  From a matter of readibility, and general approach to
standards, for a single definition can we reference a single document?

Section 4.3.6 dicusses the need for per-ETE state.  Security
Considerations do not adequately address mitigating attacks based on,
say, large pings or some other approach that can force a large packet to
return.

In addition, any mechanism where we would expect wide deployment
probably deserves substantial review.  Experimental in this case would
seem to me to be more appropriate than Proposed Standard.

Again, I wish I had more time for more detailed review.

Eliot

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 22 Sep 2009 14:14:55 -0700

Margaret,

Thank you for taking the time to review the document. In
a first pass over your comments, I believe that many could
be addressed in a new document version while others may
need further discussion here on the list. I will work
through the comments and let you know when I am ready
to discuss further.

Again, thanks for your time.

Fred
[email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Margaret Wasserman
> Sent: Monday, September 21, 2009 9:38 PM
> To: Ralph Droms
> Cc: [email-masked]
> Subject: Re: [Int-area] SEAL - draft-templin-intearea-seal-05.txt
>
>
> Hi Ralph,
>
> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 22 Sep 2009 14:27:05 -0700

Eliot,

Thanks for taking a look; I believe I can address most of
your concerns in the next document version. I will work
through the comments and let you know when I am ready to
discuss further.

About publication category, there is already an earlier
version of SEAL awaiting publication as an experimental
RFC via the independent submissions stream. Like many
others in the independent stream, that publication has
been held up for some time now so it was necessary to
move forward with the new version of SEAL we are
discussing here.

Fred

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Eliot Lear
> Sent: Tuesday, September 22, 2009 7:42 AM
> To: Ralph Droms
> Cc: [email-masked]

---

From: Joe Touch <[email-masked]>
Date: Wed, 23 Sep 2009 07:34:53 -0700

Hi, Margaret,

I'll help with a few answers, since I've been tracking SEAL fairly
closely. I hope this helps the list and Fred in his update...

Joe

Margaret Wasserman wrote:
...
>>   feedback when a packet is dropped due to an MTU restriction does not
>>   large and never receive an indication from the network that they are
>>   being discarded.
>

That work requires upper layer protocol participation in MTU discovery;
for protocols that do not already implement this, or for largely
unidirectional protocols (e.g., streaming), this can be difficult. SEAL
supports PMTUD in its tunnel mechanism over the path of the tunnel,
which can be used either on a sub-path or end-to-end to augment the
ability of an application to traverse paths that do not support the
expected MTU.

...
> avoiding the need to collapse many ID numbering spaces into one.  For

The primary reason for fragmenting the inner IP is to avoid the need to
reassemble a single stream at the tunnel egress, i.e., to distribute the
load of reassembly, or, more directly, to simplify (and reduce the
expense) of tunnel egress implementations. I like to call this "writing
a check you don't have to cash", and I think we should avoid that sort
of thing wherever possible.

In particular, this can subject less capable endpoints (e.g., PDAs,
cellphones) to reassembly overheads that they are challenged to support,
both computationally and memory space.

However, this also perpetuates intermediate fragmentation, which you
note below has problems.

...
>>   transport protocol selectors (e.g., the source and destination ports)
>>   fragmentation useless under certain circumstances.  Even worse, there

...
> sponsored submission that claims that this is tailored to LISP...

LISP encapsulates from ingress to egress, but the WG is focused on how
to determine the encapsulation information. LISP writes off the MTU
issue (draft-ietf-lisp-04.txt, section 5 - basically drops all packets
that can't fit), and proposes two mechanisms in sec 5.4 that have a very
thin explanation. LISP docs do not discuss ID collapse issues, partly
because they just drop packets that have DF=1 and won't fit.

I don't think these are tenable positions. IMO, SEAL is as much in-scope
as these sections are to the primary LISP document, and even moreso LISP
requires a solution that can support DF=1 that won't fit the MTU or
it'll be relegated to an experimental curiosity.

...
>>   SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL, when they appear in this

FWIW, I have inserted language in I-Ds that clarifies as follows:

	When used in lower case (e.g., must, must not, etc.), these
	words MUST NOT be interpreted as described in [RFC2119], but
	are rather interpreted as they would be in common English.

>>   SEAL introduces a minimal new sublayer for IPvX in IPvY encapsulation
>>   (e.g., as IPv4/SEAL/IPv6), and appears as a subnetwork encapsulation
>>   applicability [RFC4380].  When it appears immediately after the outer

IPv6 options are handled as successive encapsulations, where the outer
header's protocol field indicates the next layer of option, not the
protocol of the payload.

I.e., IPv6 options indicate the next option type in the outermost (i.e.,
first) protocol field and indicate the payload protocol in the innermost
(i.e., last) protocol field in the options chain. IPv4 options indicate
the payload protocol in the outermost protocol field. SEAL is
implemented as an IP encapsulation sub-layer, creating an arbitrarily
extensible chain in the same way that IPv6 does, and in the same way
that IPsec does for IPv4.

...
>>   Note that this SEAL segmentation ignores the fact that the mid-layer
>>   original form during reassembly.  Therefore, the fact that the packet

This is the way IP fragmentation interacts with TCP congestion control
currently for both IPv4 and IPv6. Retransmission of lost fragments at
lower layers could result in variable delays and feedback loop
interactions with TCP which are undesirable.

...
>>   Note that when IPv6 is used as the outer IP encapsulation layer, the
>>   ITE must insert an IPv6 fragment header with an Identification value

An ITE adds the outer header. As such, it is an *origin* of those
packets, and so if fragmentation is required, it MUST happen at that
location according to IPv6 requirements.

It's required here because IPv6 is transmitting packets whose size it
cannot control. The ITE can find out the MTU of the path, but it can't
necessarily communicate it back to the origin of the packets it receives
(they may not be IPv6, e.g.). So it has no choice but to either drop all
packets larger than the known MTU, or to issue IPv6 fragments.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 24 Sep 2009 09:19:47 -0700

Margaret,

Thanks again for taking the time to contribute this
thorough review. Please see below for my responses to

> the term is used in the IETF...

Well, from the perspective of the inner IP protocol SEAL
is seen as "the layer below IP".

> ID value?  This is better stated in one of the appendices.

You are right; this will be addressed in the next document
version.

> an operational (or even theoretical) problem...

RFC4963 provides such data, and can be cited here.

>  >   fragments of multiple IP packets during reassembly).

I could cite RFC4963 here also.

>  >   that lead to an overall "brittleness".

RFC2923, RFC4459 and RFC4963 would be the references to
cite here.

> Also, there was more recent work on PMTUD that doesn't
> seem to be reflected in this paragraph.

I agree that text explaining the relation of this work to
RFC4821 would be helpful in this section and will work on
something for the next draft version.

> The transport area should be

Will cite 'draft-ietf-tcpm-icmp-attacks'.

> My (incomplete) understanding is that this problem has been addressed

First, the tunnel has no way of knowing whether the original
source uses RFC4821 PMTUD. Secondly, end systems that use
RFC4821 PMTUD require all links on the path to be consistent
about the MTU they will support, where tunnels on the path
are seen by end systems as an ordinary "link". SEAL allows
tunnels to traverse a diverse set of links while still
presenting a consistent MTU.

> outside of the network perimeter (i.e. outside the firewall).

Perhaps it would be sufficient for me to simply remove
the phrase "site border routers that are configured as"?

> least as I would use that term).

SEAL applies to not just the Internet but to any subnetwork
over which inner IP packets can be tunneled; in that sense
SEAL sees the global Internet DFZ as simply another subnetwork
for tunneling. SEAL subnetworks can be concatenated together
in an end-to-end path, where each subnetwork is traversed by
tunnels bounded by ITEs/ETEs. For a graphical depection of
what I am referring to, please see:

  http://osprey67.com/seal/catenet_seal.pdf

> It seems like this document is using
> the term "subnetwork" to mean "tunnel"?

The abstract and introduction both say, "subnetworks are
defined as virtual topologies that span connected network
regions bounded by encapsulating border nodes". The virtual
topology is manifested by tunneling, but includes all
ITEs/ETEs that connect to the subnetwork and not just a
single tunnel between a single ITE/ETE pair. In that sense,
the subnetwork appears as an NBMA link.

> If so, that is a bit

Subnet and subnetwork in my opinion are quite different
terms. Subnet implies a set of nodes connected to the same
link and that configure addresses from a shared IP subnet
prefix. A subnetwork on the other hand can be an arbitrary
collection of L2 devices (segments, switches, bridges, etc.)
joined together to present the impression of a unified shared
link that connects hosts and routers. In particular,
"subnetwork" makes no statement about a shared IP subnet prefix.

> make them fit in the tunnel, I can see how you might have accidental

For IPv4, IP fragmentation on the external packet can occur
either at the ITE or at some router along the path to the
ETE. A major function of SEAL is to detect external IP
fragmentation and tune it out asap.

> avoiding the need to collapse many ID numbering spaces into one.  For

Inner fragmentation is fine, but only available for inner
IPv4 packets with DF=0. I know Joe Touch expressed concern
about allowing gratuitous inner fragmentation, but if inner
fragmentation occurs, and the final destination is hurt by
it, then IMHO it is the original source and not the ITE
(nor any other aspect of the network) that is to blame.
(Because the original source is the one sending large
packets with DF=0).

> PMTUD...  The original sending nodes?  Or the ingress tunnel endpoint

The latter. I can change this to: "If the ITE uses classical IPv4
path MTU discovery, it may be inconvenienced by excessive..."

> The "subnetwork may ... be untrustworthy? What does that mean?

"Untrustworthy" is admittedly not a great choice of terms.
Perhaps a better word would be "suspect"?

> What would be insufficiently provisioned?

An ICMP message that encapsulates only a small portion
of the packet-in-error, e.g., only the minimum 8 bytes
required by RFC0792. I can change this to something like
"or may contain insufficient information for translation...".

> This is one of the classic reasons to avoid intermediate fragmentation.

OK.

> deployed tunneling solutions.

Picking just GRE for example, RFC2784 "Appendix - Known Issues"
lists MTU interactions as the first issue because GRE _in itself_
does nothing to ameliorate path MTU issues.

It is true that existing tunneling protocols can often pick
a lowest common denominator MTU value 'M' that is known to
work across all paths in the subnetwork. For example, the
LISP assertion is that the vast majority of paths across the
Internet DFZ should support an MTU value of at least M = ~4KB.
The problems with this approach are a) in some subnetworks,
there may be no way to determine an LCD value for M, b) in other
subnetworks, the LCD value for M may be too small for practical
use, and c) once a value for M is chosen, it immediately becomes
ossified as there is no way to know when it can be increased to
larger values at a later date.

SEAL supports subnetworks in which M is either non-deterministic
or too small, and also supports true path MTU diversity by
measuring based on what links are *actually* in the path as
opposed to what links are *presumed* to be in the path.

> sponsored submission that claims that this is tailored to LISP...

Joe Touch has pointed out the shortcomings of the LISP MTU
handling, and I concur with his observations. But, the LISP
group's attitude toward me on this has been one of "don't
call us; we'll call you". I'm also not sure about rejection
as you phrased it, since a rejection of the SEAL approach
would amount to a rejection of true path MTU diversity for
tunnels and migration to larger MTUs in the future.

> A two-bit version number where two of the versions are already assigned?

In retrospect, this may have better been called "mode"
instead of "version". SEAL-FS and SEAL-TE are different
"modes" of SEAL, which may be a more proper way to express
it than by calling them "versions". I am looking into a
third mode at the present time, and the fourth mode can
be reserved for future use.

> terms in this section that just expand acronyms with no actual

I can fix that.

>  >      DF - the IPv4 header "Don't Fragment" flag

Will cite RFC0791.

> to other words, or clarified.

Joe Touch offered what seemed to me to be a reasonable
method for qualifying lower-case requirements keywords. I
think for now I may take Joe's example, but leave the door
open to changing to upper case later.

> And, yet the MANET group did not adopt SEAL?  Why?

The MANET group really didn't have this in their charter,
and I didn't push hard to get it added. From a routing
area perspective, the interest in SEAL for MANET is the
well-behaved ip-id which can help in duplicate packet
detection, but this seems to be a back-burner item for
MANET. The MTU on the other hand is more of an intarea
concern, and I just don't think this is very much on
the MANET group's radar screen at the current time.

> So, the term "enterprise" is yet another term for "tunnel"?

No - enterprise is another term for subnetwork. Neither
term equates to "tunnel", but rather they provide a
substrate over which tunnels are formed.

> (IP options), why wasn't that used?

I agree with what Joe had to say about this, and I don't
think I could explain it better than him.

> in the introduction, or something?

I'm sure I can come up with something in the next document
version that will make this clearer.

> encrypt it.  So, I think this example is invalid.

These examples are read from left to right; not from right to left.
So, in this example, the outermost header is 'IPvX', followed by
the SEAL header, followed by the {AH,ESP} header, followed by etc.

> that depends on specific behavior in this case.

This paragraph is specific to DF=1 packets only; the immediately
preceding paragraph talked about DF=0 packets.

> properties for congestion, especially when large packets are used.

Joe again had some helpful things to say about this. But,
consider also that in many environments we are only trying
to account for the fact that encapsulation makes an otherwise
"right-sized" packet just a little bit too big. So, we would
see mostly 2-fragment fragmented packets such that the loss
unit is only a little bit smaller than the retransmission unit.

> protocol,

Actually, NAT traversal was my main reason in looking at
the UDP, e.g., such as for RFC4380. The LAG systems really
haven't been very prominent in my thought process, only to
the extent that they seem to me to be contributing to the
ossification of the Internet and might better be updated
to understand protocols other than TCP and UDP.

> this seems like a significant omission.  If the IETF chooses

But, are LAGs a major consideration for domains other than
the global Internet? SEAL was designed for operation in
subnetworks both large and small, and LAGs may not be as
major of a consideration in some domains than they are
in others.

Back to LISP for a moment, LISP could adopt the fragmentation
sensing aspect of SEAL-FS to support MTU diversity in the global
Internet domain *without* having to make the LISP header format
conform to the SEAL header format. I have been trying to get
this point across, but I'm not sure that it has been heard.

> should be no reason to do so, as all IPv6 nodes support PMTUD.

I think Joe had a good answer for this one as well.

> in a "typical" SEAL deployment?

SEAL, like ICMP, uses rate limiting except for the case
in which the ITE asks for an explicit acknowledgement from
the ETE. The ETE in turn sends an explicit acknowledgement.
The management traffic can therefore grow linearly with
the data traffic in the extreme case that an explicit
ack is requested on all data packets.

This should not be a problem in subnetworks in which no
spoofing is possible, since a misbehaving ITR would only
stand to harm itself. In subnetworks in which spoofing
is possible, the ETR can check the SEAL_ID to make sure
packets are not coming from off-path attackers.

> How is SEAL expected to detect if one of these boxes exists in the path?

I was thinking mostly about NAT-specific tunneling
protocols like Teredo for this. Teredo is specifically
designed for traversing NATs so there is no need to
detect whether a NAT is in the path, i.e., the alternate
SEAL_ID writing approach will be used unconditionally.

> How are they aware of this?  Through manual configuration of both ends?

Manual configuration as the first-order approach, but if
the ITE chooses a SEAL mode that is not supported by the
ETE, the ETE will let the ITE know.

>  >   does not implement the SEAL protocol.

I think that is out of scope for this document. The point
is that the ITE can detect if the ETE does not support
SEAL - but, then it is up to the ITE to pursue some other
course of action.

> block-based file sharing protocols, such as NFS (or at least they were

Well, it does say "MUST configure at least 2KB" and "MAY
configure a larger value". The 2KB minimum is there to
ensure that the current "Internet cell size" of 1500
bytes will be well supported, but perhaps the "MAY"
here should be made stronger?

> over UDP.  The motivation for this should be better explained.

We will be wanting control messages that are specific
to SEAL and are distinct from either inner-IP ICMPs
or outer-IP ICMPs. SEAL is a mid-layer encapsulation,
so its error messages are neither fish nor fowl - they
are something in between.

That said, the alternative pursued in the earlier experimental
version of SEAL was to wrap the SEAL error messages in an
outer IPvX/SEAL header instead of an outer IPvX/UDP header;
but that puts the control messages into the data plane.
Do you see advantages/disadvantages for either approach?

> The IANA Considerations section is missing a registry for new SEAL

Would we still need a new registry if we started calling
the field "mode" instead of "version", and if the values
for "mode" were fully populated?

> Include some further explanation of how this is enforced.

OK.

> due to this lack of protection.

AFAICT, this is no different than for the IPv6 fragmentation
header. Surely there must have been sufficient analysis when
the IPv6 fragmentation header was designed in order to support
its standardization?

> fabricating SEAL control messages.

OK, but SEAL control messages already carry a 32-bit nonce
that the ITE can use for authentication purposes. This was
specifically designed to prevent off path attacks.

> into a remote network, etc.

OK; I think I can find references that I can cite here.

> Reference?  It is my understanding that ICMP error messages are

I will check this.

> bullet to get these messages through firewalls and NATs.

If the UDPs don't get through, then the ETE will simply
appear to be unreachable and the ITE will try another.
But, the same state of affairs occurs for other UDP
protocols such as LISP, so I don't see this as being
a consideration that would be unique to SEAL.

> especially in the presence of congestion.

Not always. For example, certain L2s like 802.11 perform
limited ARQ to cope with high BERs. There is a balance
between link persistency (how long the link will retry)
and end-to-end reliability, however.

> rate on the order of 10^3 to 10^4 when traffic transits a multi-link

I have seen the Stone-Partridge works. Undetected errors at
rates of 10^-{3,4} need to be detected at some level above
the link-layer, which would entail the network, transport and
application layer integrity checks; else, there would be rampant
data corruption in the Internet which does not seem to be the
case. The point that can be made here however is that the data
corruption profile is no better or worse than for IPv6
fragmentation and reassembly. I'll see if I can rework
this text.

> to normative or merely informative?  This should be clarified in the

I will clarify.

> Why not just reference this document instea of quoting it?

This document expired many years ago and there are no plans
of reviving it. The "(Taken from ..." sentence was put there
simply to show that the SEAL work has been going on for about
7 years now. Would it be satisfactory for me to simply remove
this line and leave the rest of the historic text in place?

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 24 Sep 2009 10:26:42 -0700

Hi Eliot,

Thanks for taking the time to post this review, and see
my responses below:

> protocol).

I would prefer to call it a new and different "encapsulation"
rather than a new and different tunneling mechanism. The
intent is that this new encapsulation would be used in
conjunction with new and existing tunneling protocols,
e.g., VET.
 
> It provides back-signaling in an attempt to cause tunnel

That is part of it, yes, but I would prefer to say that the
goal is to cause the tunnel ingresses to "adjust their packet
sizing parameters", which does not always equate to reducing
their MTU.
 
> in a different document.  So for starters, start the doc at §1.2.

The document actually started out as you say, but over
the course of several version updates the motivational
text was added at the suggestion of others who felt the
reader would appreciate a broader discussion of the issues. 
> is confusing to the casual reader such as myself.

The term subnetwork is defined inline in the abstract and
then again in the Introduction, but in concurrence with
Margaret's point the document should probably include a
Terminology section. Would it be satisfactory for me to 
nclude a formal definition for subnetwork under a new
terminology section?
 
> that should be better qualified.

I will work on this, as well as any others you could
point out.

> There are also some sentences that

I will re-work this sentence in the next document version
such that it will hopefully flow better into the following
paragraphs, which is what it was intended to do.
 
> standards, for a single definition can we reference a single document?

I was wanting these documents to be viewed as informative,
but I think by "normative", you are meaning that since
the term "enterprise" is not defined in this document
then the reader has to go off and read the other documents
to see what is meant? Would it be satisfactory for me to
simply include a definition for enterprise in a new
Terminology section?

> say, large pings or some other approach that can force a large packet to

I'm not sure I see that scenario. SEAL is concerned with
unidirectional MTU determination (i.e., from ITE->ETE),
and the ITE would not take any actions on behalf of large
packets coming back. Did I miss your meaning?

> seem to me to be more appropriate than Proposed Standard.

We already have an earlier version of SEAL in the
publication queue as experimental category, and there
has been substantial review of the SEAL approach over
the years both on and off list that has helped shape
the current document version. Proposed Standard seemed
like the next logical step.

> Again, I wish I had more time for more detailed review.

Thanks for taking the time to make these comments,

Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 29 Sep 2009 08:30:25 -0700

A new version of the SEAL draft is available that addresses
the comments raised on the list; please review and send
comments:

http://tools.ietf.org/html/draft-templin-intarea-seal-06

Fred