Subject: RE: Higher level question about flow label

From: [email-masked]
Date: Fri, 17 Aug 2001 08:14:01 +0300

Hi Brian,
> I think the WG needs to decide once and for all whether the 
> flow label is
>    a) a CATNIP or MPLS-like routing handle 
> or b) a QOS hint for intserv only
> or c) a QOS hint for intserv and diffserv
> or d) a waste of bits
> 
> We can get back to the details once we have a consensus on this.

I prefer c, but wouldn't mind it being simplified down
to "A QoS hint."

John
--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: [email-masked]
Date: Mon, 20 Aug 2001 11:38:16 +0300

Out of your choices I vote for (b). I also vote for John's notion of "just a
QoS hint".

Also, relax the pseudo-random number requirement of the flow label. Routers
can't really know how good the PRF used by the host is, so no router should
assume the flow label to be a pseudo random number! Also, if the flow label
does not need to be a result of a PRF, usage of "well-known values" becomes
possible, if anyone ever would find a need for that (currently I really
don't see the need for this as the Traffic Class should provide more than
enough tags for aggregates needing specific per-hop-behavior).

Additionally, the flow label should not be mutable in transit, except within
specific "QoS domains", where the flow-label value could temporarily be
changed. I mean that the value seen at the flow label field on "QoS domain"
entry will be restored at the "QoS domain" exit. Whatever signaling etc.
might be needed to effect this is the problem of the "QoS domain".
End-to-end the value should not be changed, except, perhaps as Francis
suggested to allow it to be changed once from all-zero to a non-zero value.

As for routing, the addresses themselves should be sufficient. And isn't
MPLS supposed to live beneath the IP-layer?

	Jarno

> -----Original Message-----
> From: ext Brian E Carpenter [mailto:[email-masked]]
> Sent: 16. elokuuta 2001 16:45
> To: ipng
> Subject: Higher level question about flow label
> 
> 
>   Brian

---

From: Thomas Eklund <[email-masked]>
Date: Mon, 20 Aug 2001 11:31:50 +0200

Hi Jim,
Of course it can, a CAM is programmable and that's not he problem, in fact
the current with the current CAM you could do a 576 bit wide lookup if you
are smart (and those bits does not need to be consecutive in the packet) in
OC-192 which is sufficient for IPv6 multicast.

It can but I dont belive in a model where the intserv bits will be carried
end-to-end (over some transit provider) and can be trusted on the other end.
I would put my two cents on the model where the intserv flow label only is
going to be used within the enterprise or within an administrative domain
(like an Radio Access Network in 3G)

What I also think that b will be a good model where for instance you could
signal the flow end to end over a VPN.

The a model could really benfit when it comes to native  TE for IPv6. You
also really simplify the mpls model and get a the same approach for IPv6
where you could do TE over your IPv6 flows. That mean that you would make
IPv6 "link-layer ware". I see many benefits of such an approach besides the
obvius one the intserv which I also support.

-- thomas

>From: Jim Bound [mailto:[email-masked]]
>Sent: den 20 augusti 2001 05:37
>To: Thomas Eklund
>Cc: 'Brian E Carpenter '; ''Francis Dupont ' '; ''ipng ' '
>Subject: RE: Higher level question about flow label
>
>
>thomas,
>
>the flowlabel with src global addr can be used by the CAM or SRAM for
>CATNIP or MPLS lookup.
>
>Why differentiate types of use?
>
>With b the Intserv model one gets a is my belief?
>
>thanks
>
>
>/jim
>
>
>On Mon, 20 Aug 2001, Thomas Eklund wrote:
>
>> Dear Brian,
>> The intention is not to combine those at the same time the 
>idea I support is
>> to let people have both and let the user decide.
>> 
>> Baically I see two different scenarios:
>> 1) service provider, hop-by-hop semantic and wants the 
>flowlabel to resemble
>> mpls flow label
>> 2) enterprise, end-to-end (within the admin doman i.e. the 
>enerprise) where
>> you probably would like a intserv model.
>> 
>> I dont see the need for defining a diffserv model since you 
>have a diffserv
>> label today, but if there is many ISP that want more DSCP 
>then I would
>> support it (eventhough the ISP Í talk is not looking for 
>that in a near
>> future).
>> regards 
>>   Thomas
>> 
>> 
>> 
>> From: Brian E Carpenter
>> To: Thomas Eklund
>> Cc: 'Francis Dupont '; 'ipng '
>> Sent: 2001-08-19 21:30
>> Subject: Re: Higher level question about flow label
>> 
>> Thomas,
>> 
>> How can you combine a routing handle usage with intserv usage?
>> These usages are totally disjoint. It's one or the other.
>> 
>> The historical fact (thanks to Frank Solensky for pointing 
>this out in
>> private mail) is that we rejected the routing handle usage right at
>> the beginning of IPv6, even though some people disagreed and still
>> regret
>> it. But we also moved the language about the flow label to 
>an appendix
>> in RFC 2460, which means that many implementors seem to have 
>ignored it.
>> I believe it is time to clean up that ambiguity.
>> 
>> BTW it's the presence of an ESP header that tells you if the 
>packet is
>> encrypted. I don't see any need for a bit.
>> 
>> 
>>    Brian
>> 
>> Thomas Eklund wrote:
>> > 
>> > I vote for a semantics where you have MPLS or intserv 
>style semantics
>> and I
>> > would also like a bit telling if the packet is encrypted or not.
>> > 
>> > [email-masked]
>> > -PS: Intserv is not 100% dead because there is an environment
>> (wireless)
>> > -where to get more bandwidth is really expensive (in 
>Europe at least
>> :-).
>> > -PPS: there is another usage: flow-based switching for fast routing
>> > -but I don't believe this really helps (petabit router vendors?)
>> > 
>> > --> No there is no need in term of a flow based switching 
>fast routing
>> since
>> > the memory where you store the routing tables are so huge 
>today. 512 K
>> IPv4
>> > prefixes could be stored in one CAM memory and usually you can use
>> several
>> > to obtain larger tables - all the CAM vendors support 
>between 1 - 4 M
>> IPv4
>> > prefixes to be stored in their CAM and do one look up in a clock
>> cycle.
>> > 
>> > But many people are using flow to do traffic engneering and that's
>> where I
>> > belive the "mpls" op by hop semantic is needed. for the 
>flow label. It
>> would
>> > also be easier to signal up to the routing protocls like 
>IS-IS, OSPF.
>> BGP
>> > etc when a change occured.
>> > 
>> > -- thomas

---

From: [email-masked]
Date: Wed, 22 Aug 2001 23:19:16 +0300

Derek Fawcus wrote:
> 
> > 2) If there is an alternative format, that is not pseudo-random (as
being
> > proposed), the routers will have to implement look-ups that function
> > efficiently with that format. Since we do not know what portion of the
> > traffic in the future would use this non-pseudo-random format, the
> > implementations have to assume that all traffic can be labeled with this
> > alternative format.
> 
> Lookups only occur when a QoS type funtion is applied to the flow label,
> say we've been told about a flow by RSVP.  In this case a PRN still helps
> since we can reduce the cost of lookups by hashing on the flow label to
> pick a subset of flows before doing a search/comparison.
> 
> But then if the flow lable was a PRN,  we'd simply skip the hash and use
> some of it's bits as a hash bucket index,  before jumping to the search
> and comparision phase.
> 

There is specific proposal on the table
<http://search.ietf.org/internet-drafts/draft-conta-ipv6-flow-label-02.txt>,
that proposes changing the current IPv6 flow label specification to add a
non-pseudo-random-number format in addition to the existing format. In my
message I noted that if this is done, implementations (HW or SW) will need
to be able to look up on the basis of both formats, which led me to think
that a single format fulfilling all the requirements might be better.

Also, as currently defined, most packets are sent out with flow label value
0x00000, which makes it particularly bad field for a generic route
distribution hash.

Now, if some portion of the non-zero flow label values are going to be
non-random, what is the value of having the rest of them to be
pseudo-random, especially if it is unknown what the distribution between the
random and non-random formats is?

If the pseudo-random nature of the current flow label specification is very
important and cannot be changed because of router (HW) implementation issues
(of which I do not really know much), then the proposal in the draft cannot
be supported at all. On the other hand, if the pseudo-random number nature
of the flow label is not that important, why keep it?

If it is deemed that the flow label shall support the route distribution for
load balancing then each flow should be assigned a flow label value
independent of the flow QoS requirement (i.e. the default flow label value
should not be used at all). Even then the pseudo-random number requirement
seems too strong, as usage of a simple incremental value (+1 for each new
flow) would seem to result in balanced distribution as well.

	Jarno

---

From: "Christian Huitema" <[email-masked]>
Date: Wed, 22 Aug 2001 17:49:03 -0700

> On Thu, Aug 16, 2001 at 08:44:59AM -0500, Brian E Carpenter wrote:
> > I think the WG needs to decide once and for all whether the flow
label
> is

I think that the most useful use is probably (c). Option (a) is not
terribly interesting: if you want to do MPLS, just insert an MPLS
header; MP in MPLS stands for multi-protocol, which means that MPLS is
always going to have a "layer of indirection" for v4, v6, and whatever
else the ISP wants to run on it. Option (b) really means "a set of
random bits" and is not terribly useful either: it would require that
the routers trusts the random generation in the hosts, which is not
going to happen (the trust, I mean.) 

Option (c) really means "tell me something useful about the content of
the packet so I can use it for classification." Suppose I place here the
port number that identifies the service, e.g. 25 for mail, 80 for web,
etc., plus an indication of whether this is UDP or TCP. This plus the
addresses would enable some easy filtering into intserv or diffserv
classifications, without requiring that the router go deep into the
header chain. It would enable easy traffic statistics. It would be easy
to implement for the host (pick the lowest of dest or source port.) It
would work even if the packet is encrypted. We could reserve values for
specific traffics, e.g. RTP-audio, RTP-video. It would work even if we
use ESP. From an information theory point of view, this looks much more
useful than either random bits or null values.

Indeed, the bits will have to be set by the host. But then, so are the
port numbers. Nothing prevent cooperating hosts from running NFS over
port 80... In fact, if the router is willing to inspect the full header
chain, it could rewrite the bits to a "trusted" value. Also, some hosts
may not be willing to provide the information, e.g. when they actually
want to hide the nature of the encrypted traffic; these hosts could use
a null field, which would be classified to some default rule by the
diffserv filters (you cannot have it both ways.)

Count my vote for (c).

-- Christian Huitema

---

From: Thomas Eklund <[email-masked]>
Date: Thu, 23 Aug 2001 17:25:24 +0200

Dear Steve,
But thats very strange way of hashing since the bulk of for instance
enterprise traffic is probably on port 80 and would take the same path. Its
ok as long it is the same session that take the same path but you would like
to load balance the traffic for the same TCP port but for different src IP
adresses over your composite links. 

In the report you refer to their conclusion was that the CRC 16 on the five
tuple (src IP, dst, IP src port, dst port, prot id) shows the best load
balancing distribution performance.

In this case it could be a 20-bit CRC on the fivetuple that builds up the
flow label.

-- thomas

>From: Steve Blake [mailto:[email-masked]]
>Sent: den 23 augusti 2001 16:55
>To: Thomas Eklund
>Cc: ipng
>> I also saw Steve Blakes comment that is not recommended to 
>use TCP/UDP ports
>> in the hash key for ECMP but the one of the biggest problem with the
>> bundling is to send one TCP flow over different path since 
>it might end up
>> in packet re-ordering due the different delays etc and 
>trigger TCP slow
>> start. To me it makes a lot of sense then to include the 
>TCP/UDP ports when
>> you have your "hash key" since you would like that the same 
>TCP flow take
>> the same path to not experience these problems.
>
>If you only hash on <srcaddr, dstaddr>, then all of the flows 
>between the
>two end-points will follow the same path.
>
>There is a good paper by Cao, Wang, and Zegura analyzing the 
>performance
>of various hashing schemes at 
>http://www.ieee->infocom.org/2000/papers/650.ps.
>
>
>Regards,
>
>
>
>=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
>Steven L. Blake               <[email-masked]>
>Ericsson IP Infrastructure                   919-472-9913
>
>

---

From: Thomas Eklund <[email-masked]>
Date: Sat, 25 Aug 2001 00:44:10 +0200

Frank I agree,
And in fact we do hardware for speeds up to OC-768 which are fully
programmable so for us it does not matter, but it matters for guys that do
fixed function ASICs.

But you right in the sense that it is not difficult to classify the 20 bits
of the flow label and store it in CAM. The problems sits in the action steps
associated with that flow label lookup, where in fact people would like to
do different things. Other issues are of course how to update yor CAM's with
new "flows" and write them in to the CAM. In fact this is where I think that
there could be more things that needs to be standardized for updating the
CAM(SRAM)tables with new flow which could be signalled with "hop-by-hop
option flow setup messages" which could trigger the HW to update the CAM
wirespeed in the forwarding plane instead of letting the control plane deal
with it.

-- thomas

-----Original Message-----
From: Kastenholz, Frank
To: [email-masked]
Sent: 2001-08-24 19:27
Subject: Re: Higher level question about flow label

All this talk about the format of the flow-id field
is rather interesting. But one thing has been missing -- 
input from someone who will actually _look_ at the
field, at very very very high speeds, in hardware,
and make forwarding decisions based on it.

I am fairly confident in saying that there will not
be many sqmm of silicon devoted to determining
if the flow id is 'intserv-format' or 'diff-serv
format' or 'tennis-serv format'. The edit/compile
/debug loop of silicon is such that we can not 
react soon enough to the next format that is
developed. This means that the flowid looker-
upper will be very very general. That means
that we don't care what the format is. We just
take the whole 20-bits and look 'em up (e.g.,
throw them all at a CAM, and see what comes out)

The only thing that would make the looker-upper's
job easier is to know that the flow-id is going
going to be shorter than 20 bits (maybe via the
protocol specification, maybe via some software
magic ala MPLS label negotiations). A looker-upper
that is limited to 4 bits is a different beast
than a looker-upper that does 20. 

So, you folks can argue all you want about the format
of the flow-id and those format differences may matter
at some level in the software. But to the people who
are the actual customers of the bits -- the silicon
forwarders -- it just doesn't matter. 

Frank Kastenholz

---

From: [email-masked]
Date: Thu, 30 Aug 2001 01:24:45 +0300

Steve Blake wrote:
> 
> Jarno Rajahalme wrote:
> 
> > Steve Blake wrote:
> > > RFC2475 was built on the assumption of bilateral agreements between
> > > peering providers, because that was the only model that had a hope
> > > of being deployed.
> > 
> > Has this changed? Would there be hope for non-locally-mappable DSCP
> > deployment NOW? I've understood the standardized values already exist
(the
> > PHB definition recommended DSCP values).
> 
> Not to my knowledge.  I fail to see much use for privately defining the
> DSCP->PHB mapping for the standardized PHBs, but I also don't see any
reason
> why someone who wanted that feature would be willing to give it up, and I
> don't see any mechanism by which they could be forced.
> 

I have in earlier messages proposed "giving up just a little": map
intra-domain DSCP back to a standard recommended DSCP at the domain egress.
Next ingress would understand the standard values, and could then again map
to it's own intra-domain DSCP->PHB mappings.

Brian's answer to this was that this cannot be forced on the operators.

> > > The Diffserv flow-label proposal is trying to
> > > invent an end-to-end, in-band QoS "signaling" mechanism to operate in
> > > parallel with the hop-by-hop DSCP "signaling".
> > 
> > I can see this to be useful, IF DSCP cannot be made non-mappable, and
the
> > proposed flow label usage would be mutable.
> 
> Why would it be useful?

Intra-domain egress would have an idea of what PHB the packet needs (that
the next ingress understands).

> 
> > > The only additional
> > > in-band information that would be remotely useful for Diffserv would 
> > > be a credit card account number.
> > 
> > Assuming that the flow label usage would be immutable. The first
operator
> > that doesn't see the transitive out-of-band credit card should re-mark
the
> > flow label to '00000'.
> 
> I don't get your point.  The flow label is in-band.  A 
> mutable flow label
> is just a 20-bit DSCP.  Why would we want that?
> 

Well, maybe 16 or just 6 bits depending on the allocation, but non-locally
mappable. An alternative would be to get the diffserv WG mandate use of
standard recommended DSCP->PHB mappings only. "Intserv" flow label would get
to keep 19 bits.

What I meant with the remark above is that if the ingress can't associate
the "diffserv" flow label with a "payment" should value in the flow label to
indicate "best effort" (if not drop the packet). There would be no sense to
give preferential treatment for a packet that is not being paid for and then
have the operator pay the next operator for preferential treatment. As soon
as the money (originating from the packet originator) stops flowing, any
preferential treatment should stop.

	Jarno