Subject: Re: I-D Action:draft-krishnan-6man-header-reserved-bits-00.txt

From: Brian E Carpenter <[email-masked]>
Date: Sat, 23 Oct 2010 11:21:14 +1300

Hi,

>    This document updates Section 3 of [RFC2460] to reduce the length of
>    the flow label field from 20 bits to 16 bits, and in the process
>    creating a 4 bit reserved field.  

OK, that's clear enough...

1. I do think that the justification in the draft for such a major change,
after 12 years work based on RFC 2460, is weak. Before even knowing whether
or not I like the idea, I'd expect to see much more discussion of the
rationale and the implications. It's proved remarkably difficult to widely
propagate basic changes in IP (diffserv is little used, ECN is still largely
invisible, and Re-ECN???). We all know the story on IPv4 options, especially
router alert, and we are seeing similar issues with IPv6 extension headers.
Not to mention the flow label itself.

So the chances that we actually see significant use of such reserved bits
seem to be very low.

2. The downside of reducing the flow label to 16 bits is that it becomes
clearly useless as a nonce. It can still presumably provide enough
pseudo-randomness for load balancing, and it's unlikely that a single
pair of hosts will need more than 65535 unique flow labels, but 65k
is clearly too small a space to evade random-guess attacks.

3. By the way, we do have some spare bits, i.e. the first four bits
in every packet, where only the values 4 and 6 are used. We probably
need to reserve the value 0 anyway, and the value 5 is theoretically
needed for RFC 1819. But even so, that leaves 12 spare code points...

Regards
   Brian Carpenter

---

From: Randy Bush <[email-masked]>
Date: Sat, 23 Oct 2010 07:09:20 -0700

> 1. I do think that the justification in the draft for such a major
> change, after 12 years work based on RFC 2460, is weak.

how much do you want for hacking on an unused field, another glorious
pile of second system syndrome, for which dozens of people have tried to
find a use for over a decade?

> So the chances that we actually see significant use of such reserved
> bits seem to be very low.

lower or higher than the use of flow label?

not that i am impressed with this hack.  but objecting on the basis that
flow label is a sacred cow seems somewhat specious.

randy

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 24 Oct 2010 11:23:14 +1300

> lower or higher than the use of flow label?

Lower, at the moment, since we have people very interested in the
flow label for ECMP/LAG.

> flow label is a sacred cow seems somewhat specious.

Yeah, well I did say that I haven't made my mind up, I want to
see stronger arguments. The flow label isn't a sacred cow; I just
want some discussion on the fact that reducing it to 16 bits
has a significant effect on its guessability. We need to decide
whether we care about that.

   Brian

---

From: Fred Baker <[email-masked]>
Date: Sat, 23 Oct 2010 16:01:29 -0700

On Oct 23, 2010, at 3:23 PM, Brian E Carpenter wrote:

> Lower, at the moment, since we have people very interested in the flow label for ECMP/LAG.

well, yes, but 6man does everything in its power to prevent it from being useful for that. You said that after 15-20 years of history with IPv6 things were pretty well set in stone. Well, I can show you 15 years use of the source and destination addresses, the next header field, the traffic class aka DSCP/ECN, hop count, and so on. Can you show me 15 years actual use of the flow label?

If not, the rules for taking a protocol to draft standard would suggest the use be deprecated, no?

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 24 Oct 2010 12:41:08 +1300

> If not, the rules for taking a protocol to draft standard would suggest the use be deprecated, no?

Well, if you're talking about 2460, it basically says: set the flow label to zero,
anything else is experimental. That's fine for DS, imho.

Indeed, the Proposed Standard 3697 is on the table. My opinion
is that it needs to be fixed, but deprecation would be an alternative.

   Brian

---

From: Rémi Després <[email-masked]>
Date: Sun, 24 Oct 2010 17:49:28 +0200

> lower or higher than the use of flow label?

Before IPv6 traffic becomes really high, the need for efficient IPv6 load balancing remained low.
That's why the need to fix loose ends of flow labels hasn't been pressing so far. 

Fixing flow-label details, so that their intended use becomes practical, makes IMHO more sense than throwing the baby out with the bath water. 

> flow label is a sacred cow seems somewhat specious.

Deprecating FLs, instead of fixing them for their intended purpose, requires IMHO more solid reasons than just their not being a "sacred cow".
(They indeed aren't a "sacred cow", but they still are the IETF endorsed purpose of a 20-bit field present in IPv6 headers.)

Regards,
RD

> 
> randy
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Christian Huitema <[email-masked]>
Date: Sun, 24 Oct 2010 23:26:01 +0000

> Fixing flow-label details, so that their intended use becomes practical, makes IMHO more sense 
> than throwing the baby out with the bath water. 

Maybe. But then, waiting for the traffic to increase and the load balancing needs to materialize also makes a lot of sense. I am pretty sure that 5 or 10 years from now someone will be glad that there are unused bits in the header...

-- Christian Huitema

---

From: Randy Bush <[email-masked]>
Date: Sun, 24 Oct 2010 16:28:05 -0700

> Maybe. But then, waiting for the traffic to increase and the load
> balancing needs to materialize also makes a lot of sense. I am pretty
> sure that 5 or 10 years from now someone will be glad that there are
> unused bits in the header...

folk have been trying to agree/find a use for flow labels for 15.  not
to worry, the unused bits will be there in another ten.  :)

randy

---

From: "George, Wes E IV [NTK]" <[email-masked]>
Date: Mon, 25 Oct 2010 15:01:49 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Christian Huitema
> Subject: RE: I-D Action:draft-krishnan-6man-header-reserved-bits-00.txt
>
>But then, waiting for the traffic to increase and the load
> balancing needs to materialize also makes a lot of sense.

[WES] No, actually it doesn't, especially not as a justification for this draft. We as operators are telling you that we need something now, because we've seen this load-balancing movie already in IPv4 and we'd like to get ahead of it in IPv6. If you are proposing waiting to implement a better solution for load-balancing because IPv6 traffic isn't there yet, you are simply being short-sighted, which is funny given that this draft is an attempt to future proof the IPv6 header. All it takes is a few key traffic sources/sinks moving to IPv6 on your average network... People see their IPv6 traffic increase dramatically *overnight* simply by enabling whitelisting for the Googtubemail sites. 3 years ago, I might have bought that tired old "wait and see" argument, but not when we're this close to IPv4 exhaust and starting to see real IPv6 deployments happening in many major ISPs.

>I am pretty

[WES] Call me crazy, but I'm pretty sure that's why Extension Headers exist. Someone would be directed to RFC 2460 section 4 and asked to explain why that wouldn't work for their pet idea. This draft's authors have the same burden of proof as far as I am concerned.
And no,
   "...extension headers are not examined or processed
   by any node along a packet's delivery path..."
is not a good enough reason in and of itself, because one would still have to convince people (specifically router vendors and SPs) that this set of bits is so important that routers in the middle of the path would need to slow down their processing or build silicon to look at it (and take some action) at linerate. Yes, I realize that the same issue exists for any existing use of FL that is on the table, but at least there's a reasonable justification for that as being integral to high-scale packet forwarding.

See also RFC 3330/5735 for a great example of "future use" gone awry.

Wes George

________________________________

This e-mail may contain Sprint Nextel proprietary information intended for the sole use of the recipient(s). Any use by others is prohibited. If you are not the intended recipient, please contact the sender and delete all copies of the message.

---

From: Suresh Krishnan <[email-masked]>
Date: Mon, 25 Oct 2010 13:10:36 -0400

Hi Brian,
   Thanks for your comments. Please see responses inline.

> So the chances that we actually see significant use of such reserved bits

One thing I want to clarify that this draft is not intended to exist in 
isolation. If we are not changing the current definition of the flow 
label, this draft does not come into play at all.

> is clearly too small a space to evade random-guess attacks.

Fully agree. For this reason, I think it makes sense to decide on the 
intended use of the flow label before deciding on this proposal. The 
ECMP use of the flow label is personally very important, but I am unsure 
about the other uses. If 20 bits are required by the new targeted use of 
the flow label, so be it. But if not, putting aside a few bits can be 
considered.

> needed for RFC 1819. But even so, that leaves 12 spare code points...

:-). Those bits will not be of use if we need to perform markings on 
IPv6 packets (by definition Version=6).

Thanks
Suresh

---

From: Suresh Krishnan <[email-masked]>
Date: Mon, 25 Oct 2010 13:57:19 -0400

Hi Wes,
   Thanks for your comments. Please find responses inline.

> [WES] No, actually it doesn't, especially not as a justification for this draft. We as operators are telling you that we need something now, because we've seen this load-balancing movie already in…

I strongly support the load-balancing use of the flow label. The 
question is how many bits are really required. I am not an operator and 
hence cannot speak authoritatively, but talking to our customers leads 
me to believe that 16 bits is sufficient for this purpose. I would love 
to see some more data on this.

> is not a good enough reason in and of itself, because one would still have to convince people (specifically router vendors and SPs) that this set of bits is so important that routers in the middle…

It is hard for me to guess all the future uses, but I will take one 
probable use case: re-ECN. This mechanism requires certain characteristics

a) All nodes in the path must be able to see and modify markings
b) The packet needs to go through nodes on the path that do not support 
this mechanism (re-ECN), without getting dropped.
c) The presence of the markings should not significantly affect the 
processing of the packets containing them.

Extension headers may not be a good choice because they do not fulfil a) 
and a new kind of extension header that fulfils a) will not fulfil b).

And as you rightly said it would be difficult to convince all SPs on the 
path and hence incremental deployment becomes important, thus 
necessitating b).

Thanks
Suresh

---

From: "George, Wes E IV [NTK]" <[email-masked]>
Date: Mon, 25 Oct 2010 18:37:22 +0000

> From: Suresh Krishnan [mailto:[email-masked]]
> Sent: Monday, October 25, 2010 1:57 PM
> Subject: Re: I-D Action:draft-krishnan-6man-header-reserved-bits-00.txt
>
>

[WES] I guess the question as it regards this draft is, what if the actual answer is 17 or 18 bits? Is 2-3 bits still enough to reserve for an arbitrary future use? For that matter, is 4? Why or why not?
Might be worth anonymizing and including some of what you've been told by your customers as rationale for the decision for 16 bits in your draft, and then we can debate whether that's accurate, or a good enough compromise because we aren't as concerned about the random-guess attacks Brian references.
> probable use case: re-ECN.
[WES] ok, maybe - I'd have to do more research on that, both its implementation and the likelihood of significant deployment before agreeing/disagreeing with you. My point was: Document in draft along with hopefully other examples so that we can decide if we care more about that than having another 4 bits for FL loadbalancing :-)

Thanks
Wes George

---

From: Suresh Krishnan <[email-masked]>
Date: Mon, 25 Oct 2010 17:55:16 -0400

Hi Wes,

> Might be worth anonymizing and including some of what you've been told by your customers as rationale for the decision for 16 bits in your draft, and then we can debate whether that's accurate, or…

OK.

> [WES] ok, maybe - I'd have to do more research on that, both its implementation and the likelihood of significant deployment before agreeing/disagreeing with you. My point was: Document in draft al…

Sounds good. We cannot get this done before the draft cutoff in 2 hrs 
:-) . If you are going to be in Beijing, we can have a short discussion.

Cheers
Suresh

---

From: Rémi Després <[email-masked]>
Date: Tue, 26 Oct 2010 18:24:07 +0200

Hi, Suresh,

Le 25 oct. 2010 à 19:57, Suresh Krishnan a écrit :
> ...
> I strongly support the load-balancing use of the flow label.

Isn't it better, then, to quickly improve what we have for this, than inventing new uses that would prevent backward compatibility?

> The question is how many bits are really required. I am not an operator and hence cannot speak authoritatively, but talking to our customers leads me to believe that 16 bits is sufficient for this purpose. I would love to see some more data on this.

What is clear is that:
- 20 bits is always better than 16 for any hash function.
- 20 it the number we have today.

King regards,
RD

---

From: Suresh Krishnan <[email-masked]>
Date: Wed, 27 Oct 2010 00:26:47 -0400

Hi Remi,

> Isn't it better, then, to quickly improve what we have for this, than inventing new uses that would prevent backward compatibility?

I doubt that any new use of the flow label will be backward compatible. 
Do you have any text about this proposal that I can read up on?

> - 20 bits is always better than 16 for any hash function.

Absolutely :-). But the question is whether more bits are **needed**.

> - 20 it the number we have today.

Yes. And if we are not thinking about changing the current usage of the 
flow label, it will certainly stay that way.

Thanks
Suresh

---

From: Randy Bush <[email-masked]>
Date: Wed, 27 Oct 2010 13:41:10 +0900

> I doubt that any new use of the flow label will be backward

ok.  i give.  backward compatible to what?

randy

---

From: Suresh Krishnan <[email-masked]>
Date: Wed, 27 Oct 2010 00:48:21 -0400

Hi Randy, 

> ok.  i give.  backward compatible to what?

With the usage of the flow label as defined by RFC3697. 
Sorry for not being specific.

Thanks
Suresh

---

From: Randy Bush <[email-masked]>
Date: Wed, 27 Oct 2010 14:45:51 +0900

> With the usage of the flow label as defined by RFC3697. 

i have never seen anyone move a packet over an rfc or powerpoint

randy

---

From: Rémi Després <[email-masked]>
Date: Wed, 27 Oct 2010 09:26:47 +0200

Le 27 oct. 2010 à 06:26, Suresh Krishnan a écrit :

> I doubt that any new use of the flow label will be backward compatible. Do you have any text about this proposal that I can read up on?

In my understanding:
- draft-carpenter-flow-ecmp-03 details a promising particular use of 20-bit FLs that wasn't detailed before (Flow Label for tunnel ECMP/LAG)
- it can work with the current rules about flow-label settings, but too few hosts, if any, set FLs to non-zero values today
- this is, at least in part, due to the fact that these rules are more complex that needed: stateful processing is required, or "seems" to be required (there is a debate), while algorithmic hashes would be easier to implement in hosts and sufficient for load balancing purposes. 
- draft-zhou-softwire-ds-lite-p2p-02 describes a new use of 20-bit FLs (Deployment DS-lite in Point-to-Point Access Network)
- it remains backward compatible with current rules because it is only between point-to-point tunnel endpoints, i.e. without relationship with host FL settings.

If one of these proposals would fail with hosts that support current rules for FL settings, it couldn't be claimed that they are backward compatible, but I have personally identified no such incompatibility.

> Yes. And if we are not thinking about changing the current usage of the flow label, it will certainly stay that way.

The point, IMHO, is more more about making the intended really practical (ecmp proposal), and to extend its scope (DS-lite application), than to create new instability in the meaning of IPv6-header fields.

Regards,
RD

PS: Copies also addressed to authors of draft-zhou-softwire-ds-lite-p2p-02

---

From: Jari Arkko <[email-masked]>
Date: Wed, 27 Oct 2010 15:01:57 +0300

I think that the only real world problem that I see today is the load 
balancing. We should fix that.

Its hard to imagine that we'd really need full 20 bits for that. We 
could even say use the last 16 bits for load balancing regardless of 
what we decide to do about formal reserved/not reserved status for the bits.

I think the rest of the debate is rather academic. In practice the flow 
label is today reserved, no matter what the RFCs say. Nothing will 
happen with the flow labels unless the IETF makes a decision that we 
should do <some new flow label scheme> or <some other cool thing>. And 
I'm not particularly optimistic about inventing that cool thing that 
would be a good match for using the remaining bits. I can see some 
danger of adopting some kludges or support for something relatively 
unimportant if we have the free bits, however. That being said, I 
support the idea of documenting that the bits are reserved because that 
does seem to match reality.

Jari

P.S. I'm not sure I understand why draft-carpenter-flow-ecmp requires 
the use of the flow label to happen only in routers that have 
participated in setting up the flow state... that seems like a pretty 
big non-starter to me.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 28 Oct 2010 10:25:40 +1300

> Sorry for not being specific.

I believe it is the case that any usage that is compatible with
RFC 3697 would remain compatible with draft-carpenter-6man-flow-update-04
(but *not* with earlier versions of that draft), except for
one fact of life - the current draft recognises the reality
that flow label might in the real world be changed in transit,
whereas RFC 3697 asserted that it must not be changed and
assumed everyone would obey.

In answer to Randy, the motivation for this flurry of activity
is to respond to operators who have said that they really want
to use the flow label for ECMP or LAG. We'd like to do what we
can to make that happen.

   Brian

---

From: Suresh Krishnan <[email-masked]>
Date: Thu, 28 Oct 2010 01:18:45 -0400

Hi Remi,

> If one of these proposals would fail with hosts that support current rules for FL settings, it couldn't be claimed that they are backward compatible, but I have personally identified no such incomp…

The proposals need consenting hosts on both ends to be useful, don't 
they? What would be the point otherwise? Can you think of an example 
involving an unmodified host in either case?

> The point, IMHO, is more more about making the intended really practical (ecmp proposal), and to extend its scope (DS-lite application), than to create new instability in the meaning of IPv6-header…

My point was, if we need to change the nodes at both ends to implement a 
new use of the flow label, the backward compatibility argument is moot.

Thanks
Suresh

---

From: Rémi Després <[email-masked]>
Date: Thu, 28 Oct 2010 08:50:26 +0200

Le 28 oct. 2010 à 07:18, Suresh Krishnan a écrit :

>> - this is, at least in part, due to the fact that these rules are more complex that needed: stateful processing is required, or "seems" to be required (there is a debate), while algorithmic hashes would be easier to implement in hosts and sufficient for load balancing purposes. - draft-zhou-softwire-ds-lite-p2p-02 describes a new use of 20-bit FLs (Deployment DS-lite in Point-to-Point Access Network)
> The proposals need consenting hosts on both ends to be useful, don't they?

They don't!
(Load balancing is a one-way function. If a host sets FLs, its flows to all destinations can be load balanced with ecmp, independently of what other ends do.)

>> ....
> 
> My point was, if we need to change the nodes at both ends to implement a new use of the flow label, the backward compatibility argument is moot.

Right "if we need to change the nodes at both ends", but we don't.

Regards,
RD

---

From: Suresh Krishnan <[email-masked]>
Date: Thu, 28 Oct 2010 12:01:05 -0400

Hi Remi,

> (Load balancing is a one-way function. If a host sets FLs, its flows to all destinations can be load balanced with ecmp, independently of what other ends do.)

For one-way functions, it should not matter if one side thinks of these 
20 bits as two different fields (4+16) and the other as a single 20 bit 
field, right? I agree with the goals you have stated in this thread, but 
I am trying to figure out *what exactly breaks* because of 
shortening/splitting out the field.

And this one-way quality is certainly not true of the other work 
(draft-zhou) that you mentioned that requires that the Flow Label be be 
mated to the NAT binding on one side and to the point-to-point link id 
on the other side.

> Right "if we need to change the nodes at both ends", but we don't.

As I said before, for draft-zhou, we need to.

Thanks
Suresh

---

From: Rémi Després <[email-masked]>
Date: Thu, 28 Oct 2010 19:35:51 +0200

Le 28 oct. 2010 à 18:01, Suresh Krishnan a écrit :

> For one-way functions, it should not matter if one side thinks of these 20 bits as two different fields (4+16) and the other as a single 20 bit field, right?

For load balancing between two hosts, anything ensuring that FL's are in general different between two unidirectional flows having the same source does work.

What I don't follow at this point is what you wish to have:
- statu quo
- a completion, at last, of the FL specification (i.e. in my understanding without changing the 20-bits field we have for it) 
- doing something else, but then what, with which target time frame?

> I agree with the goals you have stated in this thread, but I am trying to figure out *what exactly breaks* because of shortening/splitting out the field.
> 
> And this one-way quality is certainly not true of the other work (draft-zhou) that you mentioned that requires that the Flow Label be be mated to the NAT binding on one side and to the point-to-point link id on the other side.
> 
>>>> ....

I would rather say that, for draft-zhou application, we "want" both ends of point-to-point tunnels to have the same FL values for both directions of each bidirectional flow, and that, because both ends identify flows in the same way, this can easily be ensured.

I see this as a worthwhile application of flow labels (simple, efficient, and perfectly compatible with a possible load balancing between the two ends of each tunnel).
Quickly finishing the specification so that this application becomes formally compatible with, without a risk to become incompatible later on, makes sense to me.  

Regards,
RD

---

From: Suresh Krishnan <[email-masked]>
Date: Fri, 29 Oct 2010 18:58:20 -0400

Hi Remi,

> - doing something else, but then what, with which target time frame?

Certainly not option 1. It is not clear to me what your option 2 means 
(What do you actually mean by "a completion at last"). If it means using 
it for load balancing, yes, I am supportive of that. I am also willing 
to explore option 3.

> Quickly finishing the specification so that this application becomes formally compatible with, without a risk to become incompatible later on, makes sense to me.  

I am still waiting to find out *what exactly* is the incompatibility 
risk for a 16 bit flow label these two use cases you mentioned. As a 
hypothesis, consider two nodes A and B one. B is a legacy 3697 node and 
A is a new 16 bit flow label node performing ECMP or vice versa. What 
exactly would break.

NOTE: This is not really applicable to draft-zhou since both the nodes 
need to be updated for the proposal to work. They can easily support a 
shorter flow label.

Given all that, as Brian said, one class of potential applications that 
will not work well with a shorterflow label are the ones where "longer 
is better". e.g. The flow label as nonce proposal. This is where we need 
to evaluate how many bits are really required.

Thanks
Suresh

---

From: Steven Blake <[email-masked]>
Date: Fri, 29 Oct 2010 20:28:18 -0400

> to evaluate how many bits are really required.

For nonce usage, obviously more bits are better, though its not clear
whether 20 is significantly better than say 16.  However, without iron-clad
consensus that the FL must be preserved e2e, the nonce proposal won't fly. 
I don't see such consensus.

I would like to see the WG move forward with updated spec language that
accommodates the ECMP/LAG usage of the FL.

Personally, I think that that a FL = 5 bits is plenty sufficient for
ECMP/LAG usage (all that adding the FL to the load-balancing hash achieves
is path diversity amongst flows between a particular source/destination
pair).  We could be generous and allocate 8-10 bits for load balancing. 
That would leave the other 10-12 bits for the devil's playground.  If we
went this route, we should then rename the load balancing field to
something other than "Flow Label", because it would be too small to
uniquely enumerate all of the potential flows between a source/destination
pair (an existing property of the RFC 3697 definition of non-zero FLs).

Bottom line: if you are *only* using the FL (in combination with the
address pair) for ECMP/LAG load balancing, 20 bits is overkill.  If you
want to carve out bits from the FL to set aside for future use, 4 bits is
probably too small to bother with.

Regards,

// Steve

---

From: Rémi Després <[email-masked]>
Date: Sat, 30 Oct 2010 09:48:01 +0200

Le 30 oct. 2010 à 00:58, Suresh Krishnan a écrit :

>> - a completion, at last, of the FL specification (i.e. in my understanding without changing the 20-bits field we have for it) - doing something else, but then what, with which target time frame?
> 
> Certainly not option 1.

> It is not clear to me what your option 2 means (What do you actually mean by "a completion at last").

It refers to the fact that, as long as the spec of FL's is under a menace of being deprecated or changed, no one can reliably plan to use it, including for load balancing.
I find it unfortunate.
Getting out of this, at last, would IMHO be a step forward.

I think I have nothing more than to say on this subject, and therefore should probably better quit this discussion, and just let others exchange their technical arguments.
If the community prefers to endlessly defer a reasonable spec of FLs, I doubt I can avoid it, and there is nothing vital for me.

Kind regards,
RD

---

From: Randy Bush <[email-masked]>
Date: Sat, 30 Oct 2010 19:15:50 +0900

> It refers to the fact that, as long as the spec of FL's is under a
> menace of being deprecated or changed, no one can reliably plan to use
> it, including for load balancing.

we've gotten used to this being the state for about 15 years.  some
people are trying to unstick that.  as FL is utterly undeployed, that
they wish to hack its currently irrelevant spec is more admirable than
despicable.

randy