Subject: [v6ops] draft-taylor-v6ops-fragdrop WGLC

From: Fred Baker <[email-masked]>
Date: Sun, 18 Aug 2013 11:00:06 -0700 (PDT)

This is to initiate a two week working group last call of
http://tools.ietf.org/html/draft-taylor-v6ops-fragdrop.  Please read it now. If you find nits
(spelling errors, minor suggested wording changes, etc), comment to the
authors; if you find greater issues, such as disagreeing with a
statement or finding additional issues that need to be addressed,
please post your comments to the list.

We are looking specifically for comments on the importance of the
document as well as its content. If you have read the document and
believe it to be of operational utility, that is also an important
comment to make.

---

From: Jason Fesler <[email-masked]>
Date: Sun, 18 Aug 2013 20:11:42 -0700

> believe it to be of operational utility, that is also an important

I do believe there is a value in this document being published. The
contents very much reflect the operational  reality of $employer; both
on the technical issues at hand, and the related practices we now take
as a result.  And I believe this will be a useful document for the
context of other papers relating to fragments.

---

From: Sheng Jiang <[email-masked]>
Date: Mon, 19 Aug 2013 03:43:18 +0000

Reading this document made me believe that fragment should NOT be deprecated and the support for fragment would be better in long-term. I personally agree on such observations and conclusions. I support the publication of this document. Thanks authors for gather these value information.

Sheng

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On Behalf
>Of Fred Baker
>Sent: Monday, August 19, 2013 2:00 AM
>To: [email-masked]
>Subject: [v6ops] draft-taylor-v6ops-fragdrop WGLC
>
>_______________________________________________
>v6ops mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/v6ops

---

From: George Michaelson <[email-masked]>
Date: Mon, 19 Aug 2013 14:27:03 +1000

I think this is a useful review of the issues. I think it makes
operationally pertinant observations.

I think it stands reasonably neutrally in the 'frag or not' debate, because
both sides can inform themselves from it. I doubt its going to change many
people's minds on the fundamental question.

I think it helps inform the question(s) being asked. It made me think
harder about the potential for changes to fragments to include upper-layer
headers, and to what extent that kind of change might break things, or help
with the considerations of risk and load and capability in deployed
equipment.

I am a little worried the document makes unattributed statements about ASIC
and FPGA costs. I think they need to be fleshed out, because this kind of
unsubstantiated and unattributed comment can lead to people continuing to
assume things which applied once, to some vendors, but doesn't apply now
(consider the whole thing about secure disk wiping which was predicated on
a very specific generation of HDD hardware. Or, the "BGP is dying we need
locator+id" debate, which arguably stems from a false premise)

Having said that, if google employees say that they use systems on their
'edge' which do this, thats certainly attributed.

-G

---

From: Havard Eidnes <[email-masked]>
Date: Mon, 19 Aug 2013 10:47:04 +0200 (CEST)

Some perhaps non-nit comments (I sent the nits privately):

Under section 2.1, Possible Causes:

   Some filtering and dropping of fragments is known to be done for
   hardware, performance, or topological considerations.

That's quite vague, I think it could stand with some more details.

The entire section 2.1.1. "Stateful inspection" is about problems with
resource exhaustion in middleboxes or end-system firewalls.
Implementations which willingly devote unlimited resources to fragment
reassembly display a severe lack robustness, and IMHO should not be
used as justification for why we can't use fragments.  Yes, I'm
suggesting that one can forego the specified 60s fragmentation
reassembly timeout in the case of resource shortage of a fixed pool.

Under section 2.1.2. Stateless ACLs:

   Stateless load balancing schemes may
   hash fragmented datagrams from the same flow to different paths
   because the 5-tuple may be available on only the initial fragment.
   While rehashing has the possibility of reordering packets in ISP
   cores it is not disastrous.  However, in front of a stateful
   inspection device, load balancer tier, or anycast service instance,
   where headers other than the L3 header -- for example, the L4 header,
   interface index (for traffic already rehashed onto different paths),
   DS fields -- are considered as part of the hash, rehashing may result
   in the fragments being delivered to different end-systems

AFAIK the use of a 5-tuple which includes the L4 header to compute a
hash to make a forwarding decisions among otherwise equal paths is an
implementation optimization only, it is not something which is
explicitly prescribed by our standards.  Therefore, I'll claim that
applying this logic to fragments, causing packet delivery to different
hosts for initial and subsequent fragments (e.g. in the case of a load
balancer) is an incorrectly applied optimization, also known as a bug.

Under section 2.1.3. Performance considerations

   Leaving aside these incentives towards fragment dropping, other
   considerations may weigh on the operator's mind.  One example cited
   on the NANOG list was that of a router where fragment processing was
   done by the control plane processor rather than in the forwarding
   plane hardware, with a consequent hit on performance.

Is this about traffic directed at the one of the local addresses of
the router?  "Of course", one needs to be careful about letting
fragments from anywhere from passing into the data-plane / control-
plane bottleneck (but you could conceivably allow fragments from a
small controllable subset, e.g. your iBGP neighbors etc.)  The impact
of having limited ability to send fragments from any source to the
routers should be minimal, if noticeable at all.

Or is this about traffic passing through the router?  If so, doesn't
that discussion belong under section 2.1.2?

In section 2.1.4. Other considerations you say:

   It is common practice [RFC6192] to recommend that control-plane
   ACLs protecting routers and network devices be configured to drop
   all fragments.

Well, there exists an informational RFC [6192] which contains an
example policy and also some detailed discussion about handling of
fragments.  I don't think it in sum says what is said here, though.

Best regards,

- Håvard

---

From: joel jaeggli <[email-masked]>
Date: Mon, 19 Aug 2013 07:40:47 -0700

On 8/18/13 9:27 PM, George Michaelson wrote:
> I think it stands reasonably neutrally in the 'frag or not' debate,
> because both sides can inform themselves from it. I doubt its going to
> change many people's minds on the fundamental question.
>
> harder about the potential for changes to fragments to include
> upper-layer headers, and to what extent that kind of change might
> break things, or help with the considerations of risk and load and
> capability in deployed equipment.
>
> I am a little worried the document makes unattributed statements about
> ASIC and FPGA costs. I think they need to be fleshed out, because this
> kind of unsubstantiated and unattributed comment can lead to people
> continuing to assume things which applied once, to some vendors, but
> doesn't apply now (consider the whole thing about secure disk wiping
> which was predicated on a very specific generation of HDD hardware.
> Or, the "BGP is dying we need locator+id" debate, which arguably stems
> from a false premise)
>
Actually I think we pretty carefully skirted that in fragdrop, it's

http://tools.ietf.org/html/draft-wkumari-long-headers-01

where there hardware is front and center.

here it's fast vs slow (potentially no path) / no feasible way to do
reassembly.

2.1 mentions hardware a consideration but stops there.
> Having said that, if google employees say that they use systems on
> their 'edge' which do this, thats certainly attributed.
>
> -G
>
>
> On Mon, Aug 19, 2013 at 4:00 AM, Fred Baker <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     [email-masked] <mailto:[email-masked]>

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 Aug 2013 11:28:35 -0700

Hi, all,

I don't quite understand a WGLC on an individual submission. IMO, if 
it's a WG doc it needs to be opened up for substantial revision.

As an individual submission, I think it strays too far into the purview 
of this WG to be permitted.

On the specific point of content, the paragraph at the end of the intro 
is disingenuous; Section 2.2 is far too brief to be considered anything 
but dismissive of the cons of dropping fragments, and the rest of the 
document reads like justification of that behavior.

Joe

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Mon, 19 Aug 2013 19:13:44 +0000

On Aug 19, 2013, at 11:28 AM, Joe Touch <[email-masked]>
 wrote:

> Hi, all,
> 
> I don't quite understand a WGLC on an individual submission. IMO, if it's a WG doc it needs to be opened up for substantial revision.
> 
> As an individual submission, I think it strays too far into the purview of this WG to be permitted.

I can see this both ways. v6ops has in the past sent individual documents this way when it agreed to them. My perspective is that the draft is largely agreed to and is close to being ready to move on. If we need to rework it in some way, we can have the reworked draft as a working group document. If we agree to it as it stands, I'm not sure I see the mechanical point.

> On the specific point of content, the paragraph at the end of the intro is disingenuous; Section 2.2 is far too brief to be considered anything but dismissive of the cons of dropping fragments, and the rest of the document reads like justification of that behavior.

Ack. Thanks.

>> believe it to be of operational utility, that is also an important

If at first the idea is not absurd, then there is no hope for it.  
Albert Einstein

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 20 Aug 2013 08:36:32 +1200

> I can see this both ways. v6ops has in the past sent individual documents this way when it agreed to them. My perspective is that the draft is largely agreed to and is close to being ready to move…

The fact that the draft doesn't happen to match the draft-ietf-v6ops naming
convention is beside the point, and there is no formal stage called "WG
adoption" in the IETF standards process. It's perfectly "legal" to bypass
these two conventional steps.

I think the document is valuable and should be published, but...

>> On the specific point of content, the paragraph at the end of the intro is disingenuous; Section 2.2 is far too brief to be considered anything but dismissive of the cons of dropping fragments, an…

I agree that section 2.2 needs more work. Dropping fragments breaks stuff
that is supposed to work, and we should explain this thoroughly. I agree that
work-arounds are out of scope - this draft should focus on current reality.

At the minimum, the last sentence of 2.2 needs to be turned into English.
I can't understand it at all.

   Brian

> ------------------------------------------------------------------------
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 Aug 2013 13:50:32 -0700

> these two conventional steps.

Like some of the content of the doc, IMO that is misleading at best. 
There is a well-understood convention for WG docs:

http://www.ietf.org/id-info/guidelines.html#naming

I did assume that this isn't a WG doc because of its naming. IMO, a WG 
shouldn't endorse non-WG docs that are within its purview.

Joe

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Mon, 19 Aug 2013 21:25:37 +0000

On Aug 19, 2013, at 1:50 PM, Joe Touch <[email-masked]>
 wrote:

> Like some of the content of the doc, IMO that is misleading at best. There is a well-understood convention for WG docs:
> 
> I did assume that this isn't a WG doc because of its naming. IMO, a WG shouldn't endorse non-WG docs that are within its purview.

As I note, this isn't the first time it has been done in this working group. That said, I'm not sure what a legal discussion of the naming of the draft has to do with the charter of the working group. Here's my proposed resolution:

(1) Joe, please go to whatever list it would be that naming guidelines are discussed ([email-masked]?) and get me some guidance. If I'm doing it wrong, let's have that discussion in the place that is chartered to have it, and correct me. With my blessing.

(2) I have heard several comments to the effect that, specific issues with the draft notwithstanding, they support the document itself. I can interpret that as support for adoption as a working group draft. General request to the working group: if you would like it adopted, please feel free to say so; if not, please say so. It would be a draft describing operational practice of some operators, and the logic behind that practice, but not recommending operational practice one way or the other.

(3) Given that, at the end of the WGLC (which turns out to be a "please comment on the document" call), we know that we have some updates coming. We will ask the authors to update it per the commentary and resubmit as draft-ietf-v6ops-fragdrop.

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 Aug 2013 14:33:18 -0700

Hi, all,

On 8/19/2013 2:25 PM, Fred Baker (fred) wrote:
> As I note, this isn't the first time it has been done in this
> working group. That said, I'm not sure what a legal discussion of the
> naming of the draft has to do with the charter of the working group.

It doesn't. Is this a WG doc or not? If it isn't a WG doc, then I fail 
to see the rational of a WGLC (even though "legally" there's no such 
thing as a WGLC anyway).

> Here's my proposed resolution:
>
> (1) Joe, please go to whatever list it would be that naming
> guidelines are discussed ([email-masked]?) and get me some guidance.
> If I'm doing it wrong, let's have that discussion in the place that
> is chartered to have it, and correct me. With my blessing.

The naming conventions are here:

http://www.ietf.org/ietf-ftp/1id-guidelines.txt

If you don't think that they're useful, you should take to whatever list 
is relevant.

Note that I'm not claiming whether it's 'legal' to have a non-WG doc 
without using those conventions; it's clearly misleading given this is 
the widely-accepted convention.

> (2) I have heard several comments to the effect that, specific
> issues  with the draft notwithstanding, they support the document itself. I can
> interpret that as support for adoption as a working group draft. General
> request to the working group: if you would like it adopted, please feel
> free to say so; if not, please say so. It would be a draft describing
> operational practice of some operators, and the logic behind that
> practice, but not recommending operational practice one way or the other.

I support having this be a WG doc.

Joe

> (3) Given that, at the end of the WGLC (which turns out to be a
> "please comment on the document" call), we know that we have some
> updates coming. We will ask the authors to update it per the commentary
> and resubmit as draft-ietf-v6ops-fragdrop.

---

From: George Michaelson <[email-masked]>
Date: Tue, 20 Aug 2013 11:06:18 +1000

> here it's fast vs slow (potentially no path) / no feasible way to do
I think a citation at this point in the document would help. Really thats
all my point is: If you repeat text inline which says "...is a problem..."
then without a backchain to where you explore it fully, it becomes
unsubstantiated, and untestable against changes in the h/w horizon.

Why skirt it?

Not a big deal anyway. I like the document and think its worth publishing.

-G

---

From: "George, Wes" <[email-masked]>
Date: Wed, 21 Aug 2013 09:19:24 -0400

As to whether this is ready for IETF LC or should be adopted by the WG, no it's not ready, and I don't know if it should be adopted. Specifically, I think that guidance for implementers about use of fragments is useful, especially in documenting current behavior on the network and the justifications for it. However, we're starting to get a lot of overlap between the drafts dealing with fragmentation, whether it should be deprecated, whether we should simply provide a caveat implementer statement on using fragments but leave the standard alone, etc. It makes matters worse when one considers fragmented packets together with large headers, since at least some of the technical considerations overlap. To avoid confusion, I think we need to decide which direction we want to go before advancing drafts on the matter. If indeed we want implementers to read and heed the advice in this draft (or other related drafts), we should make sure we have a cohesive message. Right now we have justifications and operational observations spread across at least this draft and draft-bonica-6man-frag-deprecate (which I'd argue discusses the operational considerations much more robustly than this document), and possibly even draft-generic-6man-tunfrag.

Some specific comments on the draft, if indeed I'm in the minority and this advances:

General grammar pedantry nit - there's a good bit of passive voice in this draft, most of which could be eliminated through rewording.

Intro:
"The filtering of IPv6 datagrams with fragmentation headers is
   presumed to be a non-issue in the core of the Internet, where
   fragments are routed just like any other IPv6 datagram. here
   fragments are routed just like any other IPv6 datagram.  However,
   fragmentation can creates operational issues at the edges of the
   Internet"
This is unclear. Use fewer words. I think what you're trying to say is that "the core doesn't commonly filter fragments, but the edge does, and that creates operational issues".

2.1 - as others have said, the last sentence needs to be explained. This is probably the place to add a reference to one or more of the drafts I mentioned above rather than rehash it. (assuming that you don't end up incorporating much of draft-bonica section 2 into this draft, such that draft-bonica can simply recommend deprecation of fragmentation using this draft as a foundation)

2.1.3 - if this document wishes to remain neutral, I'd recommend that you drop the transition/intro text "Leaving aside these incentives towards fragment dropping". It's not really adding anything, and "these incentives" is somewhat unclear anyway.

2.1.4 - control plane ACLs only affect traffic destined *to* the router, not through it. It would be better to make that clearer. "drop all fragments" is misleading as currently written.

2.2 - folks have raised this point in discussion of the other fragmentation drafts, but there are also legitimate uses for fragmentation when dealing with data links that cannot support IPv6's minimum MTU of 1280. Some data links will do their own fragmentation to compensate, others expect the IP layer to do it.

A reference to RFC 6980 might also be useful, though I'm not certain where it would best fit.

Thanks,

Wes George

Anything below this line has been added by my company's mail server, I have no control over it.
-----------------

This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

---

From: joel jaeggli <[email-masked]>
Date: Wed, 28 Aug 2013 11:27:43 -0700

On 8/21/13 6:19 AM, George, Wes wrote:
> As to whether this is ready for IETF LC or should be adopted by the WG, no it's not ready, and I don't know if it should be adopted. Specifically, I think that guidance for implementers about use of fragments is useful, especially in documenting current behavior on the network and the justifications for it. However, we're starting to get a lot of overlap between the drafts dealing with fragmentation, whether it should be deprecated, whether we should simply provide a caveat implementer statement on using fragments but leave the standard alone, etc. It makes matters worse when one considers fragmented packets together with large headers, since at least some of the technical considerations overlap.
Imho this is the first of these documents in recent times. As an author
I don't see the goal as guidance to implementors, I see it as guidance
from operators.
>  To avoid confusion, I think we need to decide which direction we want to go before advancing drafts on the matter. 
I agree that the plan should be cohernent.
> If indeed we want implementers to read and heed the advice in this draft (or other related drafts), we should make sure we have a cohesive message. Right now we have jus
>  tifications and operational observations spread across at least this draft and draft-bonica-6man-frag-deprecate (which I'd argue discusses the operational considerations much more robustly than this document), and possibly even draft-generic-6man-tunfrag.
long-headers  is also in this space for relatively similar reasons.

---

From: Mark Andrews <[email-masked]>
Date: Thu, 29 Aug 2013 08:07:55 +1000

Please, please, please.  Use line breaks.  Use blank lines.  I really
can't tell without going into a editor and finding the end of lines
who said what.

In message <[email-masked]>, joel jaeggli writes:
> > As to whether this is ready for IETF LC or should be adopted by the WG, no 
> it's not ready, and I don't know if it should be adopted. Specifically, I thi
> nk that guidance for implementers about use of fragments is useful, especiall
> y in documenting current behavior on the network and the justifications for i
> t. However, we're starting to get a lot of overlap between the drafts dealing
>  with fragmentation, whether it should be deprecated, whether we should simpl
> y provide a caveat implementer statement on using fragments but leave the sta
> ndard alone, etc. It makes matters worse when one considers fragmented packet
> s together with large headers, since at least some of the technical considera
> tions overlap.
> > If indeed we want implementers to read and heed the advice in this draft (o
> r other related drafts), we should make sure we have a cohesive message. Righ
> t now we have jus
> >  tifications and operational observations spread across at least this draft
>  and draft-bonica-6man-frag-deprecate (which I'd argue discusses the operatio
> nal considerations much more robustly than this document), and possibly even 
> draft-generic-6man-tunfrag.
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 29 Aug 2013 21:56:40 -0700 (PDT)

On Wed, 28 Aug 2013, joel jaeggli wrote:
> Imho this is the first of these documents in recent times. As an 
> author I don't see the goal as guidance to implementors, I see it 
> as guidance from operators.

OK, but that guidance is most helpful if finds its way into atual 
advice to implementors ("here are some things you need to improve in 
order to meet our operational needs") and to maintainers of the IPv6 
specifications ("here are some places where the protocol, as 
specified, does not meet our operational needs").

> reassembly timeout in the case of resource shortage of a fixed pool.

This is true, but the other side of the coin (as is mentioned 
elsewhere in the draft) is that operators often have to deal, at 
least in the short term, with implementations that do have these 
sorts of deficiencies.  Maybe the points that should be made in this 
section are:

- If fragmented datagrams are allowed in at all, there is no way to 
  prevent a malicious party from sending an incomplete series of 
  fragments to a vctim host (or stateful middlebox).

- Such an attach can cause resource exhaustion issues on 
  unsophisticated implementations, owing in part to the 60 sec 
  timeout specified in RFC 2460.

It now becomes easy to provide actionable advice to implentors ("you 
may want to consider doing a better job of managing your resources") 
and possibly to the 6man wg ("you may want to consider revising the 
60 sec timeout, or clarifying that it is an upper bound")

In section 2.15 the draft states:

   The IETF and operators can help this effort by identifying specific
   classes of fragments that do not represent legitimate use cases and
   hence should always be dropped.

That's good, actionable advice.  However, I get great heartburn from 
this:

   However, some cases will remain where legitimate fragments are 
   discarded for legitimate reasons.

If the fragments are legitimate, it seems to me that the operator 
who discards them is shooting himself in the foot.

May I susggest that a more productive thing to say would be that the 
IETF can also help by:

- identifying applications that can be modified to avoid the use of 
  fragmentation

- identifying applications that can't live without it

The hope is that this would lead to specification changes or 
appropriate implementation advice in cases where fragmentation is 
not really needed, and where it is needed to improved middlebox 
implementations that provide smore precisely targeted filtering 
tools than the blunt instrument of indiscriminately discarding all 
fragments (for instance, a middlebox capable of parsing the entire 
header chain if it's available and sufficiently short, and having 
the abiity to discard stuff that does not conform to 
ietf-6man-oversized-header-chain).

> balancer) is an incorrectly applied optimization, also known as a bug.

+1 to that last sentence.

Note that the problems faced by stateless load balancers are 
discussed in Section 3 of the flow label specification (RFC 6437).

> fragments.  I don't think it in sum says what is said here, though.

Agreed, and I think that the document would be improved by leaving 
out the last sentence in 2.1.4.  (The points about disproptionate 
numbers of software errors in fragment processing are on target, 
though.)

Mike Heard

---

From: Randy Bush <[email-masked]>
Date: Sat, 31 Aug 2013 20:13:03 +0900

http://mailman.nanog.org/pipermail/nanog/2013-August/060709.html