Subject: Re: [PCN] PCN encoding options and tunnelling

From: "Kwok-Ho Chan" <[email-masked]>
Date: Mon, 10 Mar 2008 08:56:37 -0400

Bob, all:
Why can't we consider the use of PCN as establishing a PCN VPN?
This view goes well with the use of DSCP to differentiate PCN 
packets/traffic from all other
packets/traffic.  With the definition of the PCN PHB as the behavior 
for such a PCN VPN.
In essences, the PCN packets/traffic travels in a VPN "tunnel" defined for it.
This view allows clear definitions of:
1. PCN VPN support on a per domain basis.
2. SLAs for interfacing the PCN VPN between different domains.
3. How such a PCN VPN may be extended through multiple domains, 
possibly end to end.

With the PCN VPN thinking, the other non PCN packets/traffic can be 
viewed as being packets
in VPNs/tunnels different from the PCN VPN/tunnel.  Then the PCN 
domain is the world inside
the PCN VPN.

With such a PCN VPN in mind, I would like to ask:
1. How does the VPNs in use today need to consider the ECN bits?
2. How does the VPNs in use today need to worry about mixing of each 
VPNs' packets/traffic
     together and separating them?
3. Does the VPNs in use today have to worry about other VPNs/tunnels:
     a. Starting the ingress end of one (or more) of the other 
VPN/tunnel inside its VPN?
     b. Endng the egress end of one (or more) of the other VPN/tunnel 
inside its VPN?
     c. Have both the ingress and egress endw of one (or more) of the 
other VPN/tunnel inside its VPN?
     If yes to any of the above, how are they being solved and can 
PCN adopt the same solution?
4. Can SLAs be written to handle both the DSCP and ECN bits so they 
are clearly defined
     in the PCN VPN ingress and egress points (PCN domain edges)?

I have the PCN VPN thinking in mind from the beginning and this is 
the reason I support the
use of DiffServ and DSCP to separate the PCN packets/traffic from the 
other packets/traffic.
Hopefully allowing many of the problems to be understood, solved, or 
worked arounded.

Section 2.2.7 on page 11 of
http://www.ietf.org/internet-drafts/draft-chan-pcn-encoding-comparison-03.txt
have that thinking in mind and started putting some of the packet 
"leakage" issues
as exceptions rather then the norm.

Thanks!
-- Kwok --

At 07:53 AM 3/10/2008, Bob Briscoe wrote:
>Kwok, George, Toby, Michael,
>
>I believe tunneling represents the greatest constraint on the choice 
>of codepoints in the ECN field, so it nearly decides this issue for us...
>
>First I believe we have to allow for the likely possibility of a 
>tunnel starting and ending within a PCN region (as in S.5.7 of the 
>PCN arch I-D). For example, a multi-site VPN that is one big PCN 
>region under one operator's control joined together over the public 
>Internet by IPSec VPNs. If the customer-sites are part of the PCN 
>region they could have typical paths across the VPN like this:
>Sender > PCN_ingress > IPSec_encap > IPSec_decap > PCN_egress > Receiver
>
>Then I suggest the following requirement sentence in S.2.2.7 of PCN 
>encoding Comparision about tunnels:
>=============================================================================
>"Tunnel endpoints, whether encapsulators or decapsulators, should 
>not require any new behaviour to handle PCN-capable packets. 
>Expecting vendors of all IPSec and all IP in IP tunnel endpoints to 
>cater specifically for PCN would be unrealistic, and require complex 
>backward compatibility rules for tunnels prior to PCN. If tunnels 
>did require PCN-specific behaviour, the resulting patchy and 
>error-prone deployment would probably cause PCN to suffer byzantine 
>feature interactions with poorly implemented tunnels.
>
>Tunnel endpoints should not have to detect that a packet's DSCP is 
>PCN and implement special PCN-specific encapsulating or 
>decapsulating behaviour. The rules that already exist for tunnel 
>endpoints to handle both the Diffserv field and the ECN field should 
>'just work' when handling packets with a PCN Diffserv codepoint."
>=============================================================================
>
>That sounds reasonable, but it puts strong further constraints on 
>our PCN encoding options. Decapsulation in both RFC3168 'full 
>functionality [tunnelling] mode' and the old and new RFC4301 IPSec 
>architecture discards all ECN codepoints in the outer header except 
>11, forwarding the inner header unchanged.
>
>If we find this leaves us with no PCN encoding options, we might 
>have to require all tunnel decapsulators within a PCN region to be 
>changed (as currently implied in S.5.7 of the PCN architecture), but 
>let's see whether we have to first...
>
>With this constraint, it seems that none of 00, 01 or 10 of the ECN 
>field can be used for encoding AM (admission marked) or TM 
>(termination marked), without risking losing markings on 
>decapsulation that were added to the outer header due to congestion 
>in the middle of the tunnel.
>
>Further, 00 cannot be used for the unmarked (UM) state, because a 
>tunnel decapsulator will drop a packet if the inner header is marked 
>00 and the outer anything but 11. That leaves only 01 or 10 for UM.
>
>So far, if we use the ECN field at all, we're left with 01 or 10 for 
>the unmarked state and only 11 for anything else.
>
>Further, if either of 01 or 10 were used for UM, there would be a 
>risk of losing the UM meaning if a tunnel straddled the PCN ingress 
>but not the PCN egress, because the inner header is forwarded and 
>the outer header discarded if the outer is 01 or 10. I think it 
>would be reasonable for PCN standards to mandate that tunnel 
>decapsulators take special account of PCN encodings if the 
>encapsulator is not also in the region.
>
>We also now have to work around RFC5129 recently standardising ECN 
>in MPLS (with discussion of PCN in an appendix). I don't think that 
>constrains us further, but I'll check. I'll also check I can still 
>do the anti-cheating trick I'm proposing if we re-charter for 
>multiple domains (I suspect this will be hard now).
>
>Diffserv code points
>--------------------
>For PCN to work at all, we would have to mandate that all tunnel 
>decapsulators within a PCN region used the uniform model of RFC2983 
>(copying outer DSCP to inner).
>
>The disadvantage of using /only/ DSCPs to encode PCN are as follows:
>
>a) If an operator wants to have multiple PCN PHBs (e.g. for data 
>plane as well as signalled precedence for emergency traffic), it has 
>to use 1 more DSCP to /each/ PHB in order to add termination and 1 
>more /each/ to add admission marking. Whereas using 11 in the ECN 
>field for AM or TM requires only 1 DSCP per PHB. If an operator 
>wants AM and TM, and data plane precedence it seems to have no 
>choice but to use up multiple DSCPs, but at least it can use 11 in 
>the ECN field for one of the markings.
>
>b) If the operator uses equal cost multipath (ECMP) routing within 
>the PCN region, and the vendor's ECMP algorithm depends on the DSCP 
>in each packet, ideally we don't want to admission mark by changing 
>DSCPs because this will throw some packets of each flow onto 
>different routes, causing re-ordering. Ideally we don't want 
>termination marking to change DSCPs either, but given all the 
>constraints we might have to live with some packet re-ordering in 
>all flows until some have been terminated, as this is becoming a bit 
>of a corner case.
>
>In summary
>----------
>It looks like we have two main option families left:
>1/ use only the DSCP field for all PCN marking and the PCN ingress 
>would have to set the ECN field to 01 or 10. No PCN interior nodes 
>would change the ECN field.
>2/ Use 01 or 10 in the ECN field for UM. Use 11 in the ECN field for 
>AM. Use an extra DSCP for TM.
>
>Option 2/ seems a slightly better compromise because of a) & b) above.
>
>In either case, use another DSCP for affected marking if it is 
>required at all (but see next email).
>
>If there's one lesson from this,...
>--------------------------------
>...it's that encoding into a tightly constrained field should be 
>standardised very very carefully. Most of our problems come from all 
>the changes of mind concerning ECN that have happened over the 
>years, leaving us no space to manoeuvre between all the backward 
>compatibility constraints.
>
>That doesn't mean we should standardise slowly. But it does mean we 
>should get a design team of experts to focus on this one issue, all 
>locked in a room until they are all happy. That means now. On this issue.
>
>For instance, I believe Option 1/ would be better if we want to 
>allow for end-to-end ECN as well. But I haven't thought that all 
>through. And I also strongly believe that we shouldn't allow 
>adaptive codecs to use PCN unless they explicitly signal that they 
>are going to. Otherwise all PCN's admission control assurances 
>break. If endpoints explicitly signal they want to be adaptive, I 
>think we can manage forwarding ECN markings properly.
>
>Further reading?
>----------------
>All the background to why ECN tunneling is like it is, and how it 
>will need to change in the future, is explained in the draft I did 
>for tsvwg on tunneling congestion notification, which I highlighted 
>to the PCN list at the time. Bruce Davie & David Black have been 
>helping with it and may become co-authors.
>
>TSVWG and the area directors agreed to put that I-D on hold until 
>the PCN w-g had got closer to a decision on encodings, but it was 
>written to help PCN in that task. It recently expired, because we 
>didn't think PCN encodings would take this long! You can still 
>access an archived copy from my publications page:
><http://www.cs.ucl.ac.uk/staff/B.Briscoe/pubs.html#ecn-tunnel>
>
>To save time you can cut to S.5 and onwards, if you just want to 
>know the encap/decap behaviours and the variants that you have to be 
>backward compatible with. Earlier sections explain why. Appendix A 
>specifically discusses PCN tunneling, but I hadn't thought the 
>encoding constrints through like I just did above.
>
>Essentially, discarding 00, 01 & 10 from the outer header was 
>unfortunately necessary in RFC3168 and RFC4301 for backward 
>compatibility with the first experimental RFC2481 ECN, which 
>satisfied possible concerns at the time about the ECN field being 
>abused as a covert channel from the public Internet into an IPSec 
>tunnel. Since then the security area has decided these concerns were 
>unwarranted and the risk can be managed.
>
>In the PCN sessions this week, I can repeat some of the slides from 
>the summary presentation of this I gave to the tsvwg last Jun:
><http://www.cs.ucl.ac.uk/staff/B.Briscoe/present.html#0707ecn-tunnel>
>but I believe Toby will now be covering this in his slides anyway.
>
>
>Bob
>
>
>____________________________________________________________________________
>Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
>B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196

_______________________________________________
PCN mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pcn

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 10 Mar 2008 11:53:42 +0000

Then I suggest the following requirement sentence in S.2.2.7 of PCN 
encoding Comparision about tunnels:
=============================================================================
"Tunnel endpoints, whether encapsulators or decapsulators, should not 
require any new behaviour to handle PCN-capable packets. Expecting 
vendors of all IPSec and all IP in IP tunnel endpoints to cater 
specifically for PCN would be unrealistic, and require complex 
backward compatibility rules for tunnels prior to PCN. If tunnels did 
require PCN-specific behaviour, the resulting patchy and error-prone 
deployment would probably cause PCN to suffer byzantine feature 
interactions with poorly implemented tunnels.

Tunnel endpoints should not have to detect that a packet's DSCP is 
PCN and implement special PCN-specific encapsulating or decapsulating 
behaviour. The rules that already exist for tunnel endpoints to 
handle both the Diffserv field and the ECN field should 'just work' 
when handling packets with a PCN Diffserv codepoint."
=============================================================================

That sounds reasonable, but it puts strong further constraints on our 
PCN encoding options. Decapsulation in both RFC3168 'full 
functionality [tunnelling] mode' and the old and new RFC4301 IPSec 
architecture discards all ECN codepoints in the outer header except 
11, forwarding the inner header unchanged.

If we find this leaves us with no PCN encoding options, we might have 
to require all tunnel decapsulators within a PCN region to be changed 
(as currently implied in S.5.7 of the PCN architecture), but let's 
see whether we have to first...

Further, if either of 01 or 10 were used for UM, there would be a 
risk of losing the UM meaning if a tunnel straddled the PCN ingress 
but not the PCN egress, because the inner header is forwarded and the 
outer header discarded if the outer is 01 or 10. I think it would be 
reasonable for PCN standards to mandate that tunnel decapsulators 
take special account of PCN encodings if the encapsulator is not also 
in the region.

We also now have to work around RFC5129 recently standardising ECN in 
MPLS (with discussion of PCN in an appendix). I don't think that 
constrains us further, but I'll check. I'll also check I can still do 
the anti-cheating trick I'm proposing if we re-charter for multiple 
domains (I suspect this will be hard now).

Diffserv code points
--------------------

a) If an operator wants to have multiple PCN PHBs (e.g. for data 
plane as well as signalled precedence for emergency traffic), it has 
to use 1 more DSCP to /each/ PHB in order to add termination and 1 
more /each/ to add admission marking. Whereas using 11 in the ECN 
field for AM or TM requires only 1 DSCP per PHB. If an operator wants 
AM and TM, and data plane precedence it seems to have no choice but 
to use up multiple DSCPs, but at least it can use 11 in the ECN field 
for one of the markings.

b) If the operator uses equal cost multipath (ECMP) routing within 
the PCN region, and the vendor's ECMP algorithm depends on the DSCP 
in each packet, ideally we don't want to admission mark by changing 
DSCPs because this will throw some packets of each flow onto 
different routes, causing re-ordering. Ideally we don't want 
termination marking to change DSCPs either, but given all the 
constraints we might have to live with some packet re-ordering in all 
flows until some have been terminated, as this is becoming a bit of a 
corner case.

In summary
----------

If there's one lesson from this,...
--------------------------------
...it's that encoding into a tightly constrained field should be 
standardised very very carefully. Most of our problems come from all 
the changes of mind concerning ECN that have happened over the years, 
leaving us no space to manoeuvre between all the backward 
compatibility constraints.

For instance, I believe Option 1/ would be better if we want to allow 
for end-to-end ECN as well. But I haven't thought that all through. 
And I also strongly believe that we shouldn't allow adaptive codecs 
to use PCN unless they explicitly signal that they are going to. 
Otherwise all PCN's admission control assurances break. If endpoints 
explicitly signal they want to be adaptive, I think we can manage 
forwarding ECN markings properly.

Further reading?
----------------

TSVWG and the area directors agreed to put that I-D on hold until the 
PCN w-g had got closer to a decision on encodings, but it was written 
to help PCN in that task. It recently expired, because we didn't 
think PCN encodings would take this long! You can still access an 
archived copy from my publications page:
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/pubs.html#ecn-tunnel>

To save time you can cut to S.5 and onwards, if you just want to know 
the encap/decap behaviours and the variants that you have to be 
backward compatible with. Earlier sections explain why. Appendix A 
specifically discusses PCN tunneling, but I hadn't thought the 
encoding constrints through like I just did above.

Bob

---

From: Steven Blake <[email-masked]>
Date: Mon, 10 Mar 2008 13:29:26 -0400

A few comments on this note:

> Diffserv code points

Dealing with emergency traffic is strictly out of scope for PCN at this
time.  With that said, wouldn't the flow precedence be included in the
flow signaling?  I confess to being really ignorant on all of the work
that went on in IEPREP.

In general, excluding the possibility of conveying flow
admission/termination precedence in the DSCP bits, there are only so
many managed-to-almost-never-be-congested BAs that are needed in a
network.

> flows until some have been terminated, as this is becoming a bit of a 

Using DSCP (or any other router-mutable field) for ECMP hashing is
broken on so many levels.  Do we have evidence that any vendor has
actually implemented this?

Regards,

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Steven Blake                <[email-masked]>
Ericsson/Redback Networks               +1 919-472-9913

---

From: <[email-masked]>
Date: Mon, 10 Mar 2008 17:37:10 -0000

different routes, causing re-ordering. 
 
[phil] to me this is not an "ideally" but a "critical". question is whether any ecmp algos actually do this, or prepared to run the risk that they might do in the future.
 
Ideally we don't want

[phil] happy to call this "ideally". termination marking means things have gone wrong, so a penalty like re-ordering is less unacceptable.

---

From: ken carlberg <[email-masked]>
Date: Mon, 10 Mar 2008 13:47:17 -0400

>> plane as well as signalled precedence for emergency traffic), it has

perhaps it would be a bit more agnostic to swap the above phrase  
"signalled precedence for emergency traffic" with "signaled priority  
traffic".  Emergency is really just a motivation loosely associated  
with target level(s) of precedence, so I've always been a bit  
perplexed as to why it gets explicitly singled out in this group.  On  
the other hand, prioritization is something more tangible that can be  
play a complementary role, imho, to what PCN is trying to achieve.

-ken

---

From: "Kwok-Ho Chan" <[email-masked]>
Date: Mon, 10 Mar 2008 14:01:28 -0400

Hi all:
With the thinking indicated below, the notion of DiffServ tunnels 
indicated by RFC 2983 comes
into play.  As the encoding proposals we have uses DiffServ (with 
different numbers of DSCPs).

IMHO, we should start with the simple case, the use of the "uniform 
model" indicated by
section 3.1 on page 3 of RFC 2983, where the DSCP value of the inner 
header is copied
to the outer header at the tunnel ingress, and the DSCP value of the 
outer header is copied
to the inner header at the tunnel egress.  Taking this simple view 
means the tunnel we are
working with are Not PCN capable, the outer header does not receive 
any PCN modification.
I think most tunnels we first encounter will not be PCN capable.

Can we also indicate the tunnels we encounter be Not ECN 
capable?  Will this be a show stopper?

With PCN requiring DiffServ (as part of the charter and 
architecture).  Can we safely say
the complex case of either tunnel ingress or tunnel egress not 
supporting DiffServ not an
issue with PCN and tunneling?

I am just thinking we are adding too much complexity into the problem 
and want to bring it
down to a more simple view.
-- Kwok --

At 08:56 AM 3/10/2008, Kwok Ho Chan wrote:
>     b. Endng the egress end of one (or more) of the other 
>     c. Have both the ingress and egress endw of one (or more) of 
> the other VPN/tunnel inside its VPN?
>use of DiffServ and DSCP to separate the PCN packets/traffic from 
>the other packets/traffic.
>>I believe tunneling represents the greatest constraint on the 
>>choice of codepoints in the ECN field, so it nearly decides this 
>>issue for us...
>>
>>cater specifically for PCN would be unrealistic, and require 
>>complex backward compatibility rules for tunnels prior to PCN. If 
>>tunnels did require PCN-specific behaviour, the resulting patchy 
>>and error-prone deployment would probably cause PCN to suffer 
>>byzantine feature interactions with poorly implemented tunnels.
>>
>>endpoints to handle both the Diffserv field and the ECN field 
>>should 'just work' when handling packets with a PCN Diffserv codepoint."
>>changed (as currently implied in S.5.7 of the PCN architecture), 
>>but let's see whether we have to first...
>>
>>tunnel decapsulator will drop a packet if the inner header is 
>>marked 00 and the outer anything but 11. That leaves only 01 or 10 for UM.
>>
>>So far, if we use the ECN field at all, we're left with 01 or 10 
>>for the unmarked state and only 11 for anything else.
>>
>>plane as well as signalled precedence for emergency traffic), it 
>>has to use 1 more DSCP to /each/ PHB in order to add termination 
>>and 1 more /each/ to add admission marking. Whereas using 11 in the 
>>ECN field for AM or TM requires only 1 DSCP per PHB. If an operator 
>>all flows until some have been terminated, as this is becoming a 
>>bit of a corner case.
>>
>>In summary
>>----------
>>2/ Use 01 or 10 in the ECN field for UM. Use 11 in the ECN field 
>>for AM. Use an extra DSCP for TM.
>>
>>standardised very very carefully. Most of our problems come from 
>>all the changes of mind concerning ECN that have happened over the 
>>know the encap/decap behaviours and the variants that you have to 
>>be backward compatible with. Earlier sections explain why. Appendix 
>>A specifically discusses PCN tunneling, but I hadn't thought the 
>>tunnel. Since then the security area has decided these concerns 
>>were unwarranted and the risk can be managed.
>>

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Mon, 10 Mar 2008 19:04:35 +0000

Hi all

Regarding the second issue on this point, i.e., when the PCN-domain is
operating ECMP that uses DSCP to select the routes, a risk of
mis-ordering of packets within a flow might occur.  As mentioned already
in the 03 version of the encoding draft, the impact of this drawback
depends on the following:

      1.  the level of deployment of ECMP algorithms that use DSCP for
          route selection;

      2.  mis-ordering of packets within a flow when there is
          termination marking may be acceptable;

      3.  the possibility of configuring the ECMP algorithms that use
          DSCP for route selection in the PCN-domain, that the used PCN
          aware DSCPs are belonging to the same PHB and therefore, all
          these DSCP values should be converted to one preconfigured
          DSCP value before applying it in the ECMP routing algorithm.
          Note that all the additional experimental DSCPs that are used
          within PCN are belonging to the same PHB.

In other words, if the DSCp based ECMP routing algorithm uses one DSCP
hash value for all DSCPs that are belonging to the same PHB, then no
misordering of packets will occur.

Best regards,
Georgios

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 10 Mar 2008 18:30:14 +0000

Steven

At 17:29 10/03/2008, Steven Blake wrote:
>time.

I know - I should have said that - but we should also take into 
account what we might prevent by making choices.

>With that said, wouldn't the flow precedence be included in the

Me too. We have (informally) spoken to an emergency calls agency that 
wants data plane precedence as well as signalling plane, but I don't 
know whether that would turn into a formal requirement. The issue is 
where a disaster is causing continuous disruptions, so that 
signalling precedence doesn't do enough to protect the data from 
continual changes in capacity as routes flap around.

I take this requirement as one of those things it would be nice to 
enable, but if we can't do it, I suspect operators and agencies will 
live without it.

>actually implemented this?

Nope - I'll check around. Anyone on the list know?

Bob

---

From: "Geib, Ruediger" <[email-masked]>
Date: Tue, 11 Mar 2008 09:32:28 +0100

Hi Georgios, all

just to repeat:

To the best of my knowledge, the configuration options on ECMP are
- To activate it.
- To define a balance other that "the same load to all links".

As far as I'm aware of it
- Nothing else is configurable.
- Nothing else is documented.
- Every implementation may differ with other hardware 
  or operating system.
- Different protocol and tunnel stacks may use 
  different fields.

Now PCN isn't the only WG noticing that ECMP is an important feature 
to many carriers and that these carriers may regard the implementation 
and documentation of ECMP as it is now as a show stopper for advanced 
innovative service deployments, network optimisation and so on.

Rather then having the PCN WG just to construct another fragile but 
complex (or costly) workaround to deal with ECMP, it may be worth 
thinking about IETF to take up the issue and come to a more general
solution to the problem. Make ECMP functionality a standard feature 
with standard documentation (where possible). There are limits to 
that, the hashing function will stay vendor specific. The fields 
across which to hash should however be standardised.

Do the IETF officials on our list have a point of view on this?

Regards,

Rudiger

| -----Original Message-----
| From: [email-masked] [mailto:[email-masked]] On 
| Behalf Of Georgios Karagiannis
| Sent: Monday, March 10, 2008 8:05 PM
| To: Steven Blake; Bob Briscoe
| Cc: PCN IETF list
| Subject: Re: [PCN] PCN encoding options and tunnelling
| 
| Hi all
| 
| Regarding the second issue on this point, i.e., when the 
| PCN-domain is operating ECMP that uses DSCP to select the 
| routes, a risk of mis-ordering of packets within a flow might 
| occur.  As mentioned already in the 03 version of the 
| encoding draft, the impact of this drawback depends on the following:
| 
|       1.  the level of deployment of ECMP algorithms that use DSCP for
|           route selection;
| 
| 
|       2.  mis-ordering of packets within a flow when there is
|           termination marking may be acceptable;
| 
|       3.  the possibility of configuring the ECMP algorithms that use
|           DSCP for route selection in the PCN-domain, that 
| the used PCN
|           aware DSCPs are belonging to the same PHB and therefore, all
|           these DSCP values should be converted to one preconfigured
|           DSCP value before applying it in the ECMP routing algorithm.
|           Note that all the additional experimental DSCPs 
| that are used
|           within PCN are belonging to the same PHB.
| 
| In other words, if the DSCp based ECMP routing algorithm uses 
| one DSCP hash value for all DSCPs that are belonging to the 
| same PHB, then no misordering of packets will occur.
| 
| Best regards,
| Georgios
| 
| On 3/10/2008, "Steven Blake" <[email-masked]> wrote:
| 
| >A few comments on this note:
| >
| >On Mon, 2008-03-10 at 11:53 +0000, Bob Briscoe wrote:
| >
| >
| >> Diffserv code points
| >> --------------------
| >> For PCN to work at all, we would have to mandate that all tunnel 
| >> decapsulators within a PCN region used the uniform model 
| of RFC2983 
| >> (copying outer DSCP to inner).
| >>
| >> The disadvantage of using /only/ DSCPs to encode PCN are 
| as follows:
| >>
| >> a) If an operator wants to have multiple PCN PHBs (e.g. for data 
| >> plane as well as signalled precedence for emergency 
| traffic), it has 
| >> to use 1 more DSCP to /each/ PHB in order to add termination and 1 
| >> more /each/ to add admission marking. Whereas using 11 in the ECN 
| >> field for AM or TM requires only 1 DSCP per PHB. If an 
| operator wants 
| >> AM and TM, and data plane precedence it seems to have no 
| choice but 
| >> to use up multiple DSCPs, but at least it can use 11 in 
| the ECN field 
| >> for one of the markings.
| >
| >Dealing with emergency traffic is strictly out of scope for 
| PCN at this 
| >time.  With that said, wouldn't the flow precedence be 
| included in the 
| >flow signaling?  I confess to being really ignorant on all 
| of the work 
| >that went on in IEPREP.
| >
| >In general, excluding the possibility of conveying flow 
| >admission/termination precedence in the DSCP bits, there are only so 
| >many managed-to-almost-never-be-congested BAs that are needed in a 
| >network.
| >
| >> b) If the operator uses equal cost multipath (ECMP) routing within 
| >> the PCN region, and the vendor's ECMP algorithm depends on 
| the DSCP 
| >> in each packet, ideally we don't want to admission mark by 
| changing 
| >> DSCPs because this will throw some packets of each flow onto 
| >> different routes, causing re-ordering. Ideally we don't want 
| >> termination marking to change DSCPs either, but given all the 
| >> constraints we might have to live with some packet 
| re-ordering in all 
| >> flows until some have been terminated, as this is becoming 
| a bit of a 
| >> corner case.
| >
| >Using DSCP (or any other router-mutable field) for ECMP hashing is 
| >broken on so many levels.  Do we have evidence that any vendor has 
| >actually implemented this?
| >
| >
| >Regards,
| >
| >=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
| >Steven Blake                <[email-masked]>
| >Ericsson/Redback Networks               +1 919-472-9913
| >
| >_______________________________________________
| >PCN mailing list
| >[email-masked]
| >https://www.ietf.org/mailman/listinfo/pcn
| _______________________________________________
| PCN mailing list
| [email-masked]
| https://www.ietf.org/mailman/listinfo/pcn
| 
_______________________________________________
PCN mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pcn

---

From: Michael Menth <[email-masked]>
Date: Tue, 11 Mar 2008 12:30:53 +0100

Hi Rüdiger,

thanks again Rüdiger for this input, it'll go right away in today's 
session. We give very high priority to the problem that ECMP might 
perform unintentioally rerouting as soon as the DSCP of a packet is 
changed. Therefore, we pursue a solution that minimizes changing DSCPs.

Regards,

    Michael

-- 
Dr. Michael Menth, Assistant Professor
University of Wuerzburg, Institute of Computer Science
Am Hubland, D-97074 Wuerzburg, Germany, room B206
phone: (+49)-931/888-6644, fax: (+49)-931/888-6632
mailto:[email-masked]
http://www3.informatik.uni-wuerzburg.de/research/ngn

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Tue, 11 Mar 2008 12:57:07 +0000

Hi Ruediger, Hi Michael

As I mentioned in the email of yesterday, I think that the DSCP based
ECMP routing problem can be solved very easily.

if the DSCp based ECMP routing algorithm uses one DSCP
hash value for all DSCPs that are belonging to the same PHB, then no
misordering of packets will occur.

See text sent yesterday:

Best regards,
Georgios

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 11 Mar 2008 13:28:50 +0000

Georgios,

On point 1, after speaking with vendor friends I'm inclined to agree 
with Steve Blake that it is unlikely any vendors do ECMP by hashing 
based on the (mutable) Diffserv field.

On 2, misordering because of termination marking may be acceptable, 
but it's not ideal - it could break up the QoS of flows that are not 
going going to be terminated, which wouldn't be a good result. But if 
no-one does ECMP on the DS field, we're OK anyway.

On 3, I can't imagine any vendor altering their ECMP algo just for 
PCN - again they're more likely not to do ECMP on the DS field in the 
first place.

Bob

---

From: "Geib, Ruediger" <[email-masked]>
Date: Tue, 11 Mar 2008 14:42:23 +0100

Hi Georgios,

I'm not sure, whether any of Deutsche Telekom's vendors uses the DSCP field 
for ECMP. If one of these vendor did so, the easy solution proposed by you 
is, that once Deutsche Telekom figured out which DSCP scheme we are going 
to operate for PCN, we talk to this vendor, which will then adapt his 
ECMP implementation. I'm not an expert on fast forwarding, so I can't judge 
whether ECMP is implemented in hard- or software. "Easy" means, you are 
sure it is software? Did I understand you right?

Please also keep in mind, that larger carriers may have hierachical network 
architectures. ECMP hops may be concatenated and the ECMP hashes need to 
be different per router to make ECMP work correctly. To the best of my 
knowledge, there's no signaling protocol and no ECMP management set up 
exchanging or steering ECMP hashing within backbone routers.

Regards,

Rudiger

| -----Original Message-----
| From: [email-masked] [mailto:[email-masked]] On 
| Behalf Of Georgios Karagiannis
| Sent: Tuesday, March 11, 2008 1:57 PM
| To: [email-masked]; Geib, Rüdiger
| Cc: [email-masked]
| Subject: Re: [PCN] PCN encoding options and tunnelling
| 
| Hi Ruediger, Hi Michael
| 
| As I mentioned in the email of yesterday, I think that the 
| DSCP based ECMP routing problem can be solved very easily.
| 
| if the DSCp based ECMP routing algorithm uses one DSCP hash 
| value for all DSCPs that are belonging to the same PHB, then 
| no misordering of packets will occur.
| 
| See text sent yesterday:
| 
| Regarding the second issue on this point, i.e., when the 
| PCN-domain is operating ECMP that uses DSCP to select the 
| routes, a risk of mis-ordering of packets within a flow might 
| occur. As mentioned already in the 03 version of the encoding 
| draft, the impact of this drawback depends on the following:
| 
|       1. the level of deployment of ECMP algorithms that use DSCP for
|           route selection;
| 
| 
|       2. mis-ordering of packets within a flow when there is
|           termination marking may be acceptable;
| 
|       3. the possibility of configuring the ECMP algorithms that use
|           DSCP for route selection in the PCN-domain, that the used PCN
|           aware DSCPs are belonging to the same PHB and therefore, all
|           these DSCP values should be converted to one preconfigured
|           DSCP value before applying it in the ECMP routing algorithm.
|           Note that all the additional experimental DSCPs that are used
|           within PCN are belonging to the same PHB.
| 
| In other words, if the DSCp based ECMP routing algorithm uses 
| one DSCP hash value for all DSCPs that are belonging to the 
| same PHB, then no misordering of packets will occur.
| 
| Best regards,
| Georgios
| 
| 
| 
| On 3/11/2008, "Michael Menth" 
| <[email-masked]> wrote:
| 
| >Hi Rüdiger,
| >
| >thanks again Rüdiger for this input, it'll go right away in today's 
| >session. We give very high priority to the problem that ECMP might 
| >perform unintentioally rerouting as soon as the DSCP of a packet is 
| >changed. Therefore, we pursue a solution that minimizes 
| changing DSCPs.
| >
| >Regards,
| >
| >    Michael
| >
| >Geib, Ruediger wrote:
| >> Hi Georgios, all
| >>
| >> just to repeat:
| >>
| >> To the best of my knowledge, the configuration options on ECMP are
| >> - To activate it.
| >> - To define a balance other that "the same load to all links".
| >>
| >> As far as I'm aware of it
| >> - Nothing else is configurable.
| >> - Nothing else is documented.
| >> - Every implementation may differ with other hardware
| >>   or operating system.
| >> - Different protocol and tunnel stacks may use
| >>   different fields.
| >>
| >> Now PCN isn't the only WG noticing that ECMP is an 
| important feature 
| >> to many carriers and that these carriers may regard the 
| >> implementation and documentation of ECMP as it is now as a show 
| >> stopper for advanced innovative service deployments, 
| network optimisation and so on.
| >>
| >> Rather then having the PCN WG just to construct another 
| fragile but 
| >> complex (or costly) workaround to deal with ECMP, it may be worth 
| >> thinking about IETF to take up the issue and come to a 
| more general 
| >> solution to the problem. Make ECMP functionality a 
| standard feature 
| >> with standard documentation (where possible). There are limits to 
| >> that, the hashing function will stay vendor specific. The fields 
| >> across which to hash should however be standardised.
| >>
| >> Do the IETF officials on our list have a point of view on this?
| >>
| >> Regards,
| >>
| >> Rudiger
| >>
| >>
| >> | -----Original Message-----
| >> | From: [email-masked] [mailto:[email-masked]] 
| On Behalf 
| >> | Of Georgios Karagiannis
| >> | Sent: Monday, March 10, 2008 8:05 PM
| >> | To: Steven Blake; Bob Briscoe
| >> | Cc: PCN IETF list
| >> | Subject: Re: [PCN] PCN encoding options and tunnelling
| >> |
| >> | Hi all
| >> |
| >> | Regarding the second issue on this point, i.e., when the 
| PCN-domain 
| >> | is operating ECMP that uses DSCP to select the routes, a risk of 
| >> | mis-ordering of packets within a flow might occur.  As mentioned 
| >> | already in the 03 version of the encoding draft, the 
| impact of this 
| >> | drawback depends on the following:
| >> |
| >> |       1.  the level of deployment of ECMP algorithms 
| that use DSCP for
| >> |           route selection;
| >> |
| >> |
| >> |       2.  mis-ordering of packets within a flow when there is
| >> |           termination marking may be acceptable;
| >> |
| >> |       3.  the possibility of configuring the ECMP 
| algorithms that use
| >> |           DSCP for route selection in the PCN-domain, 
| that the used 
| >> | PCN
| >> |           aware DSCPs are belonging to the same PHB and 
| therefore, all
| >> |           these DSCP values should be converted to one 
| preconfigured
| >> |           DSCP value before applying it in the ECMP 
| routing algorithm.
| >> |           Note that all the additional experimental 
| DSCPs that are 
| >> | used
| >> |           within PCN are belonging to the same PHB.
| >> |
| >> | In other words, if the DSCp based ECMP routing algorithm 
| uses one 
| >> | DSCP hash value for all DSCPs that are belonging to the 
| same PHB, 
| >> | then no misordering of packets will occur.
| >> |
| >> | Best regards,
| >> | Georgios
| >> |
| >> | On 3/10/2008, "Steven Blake" <[email-masked]> wrote:
| >> |
| >> | >A few comments on this note:
| >> | >
| >> | >On Mon, 2008-03-10 at 11:53 +0000, Bob Briscoe wrote:
| >> | >
| >> | >
| >> | >> Diffserv code points
| >> | >> --------------------
| >> | >> For PCN to work at all, we would have to mandate that 
| all tunnel 
| >> | >> decapsulators within a PCN region used the uniform model
| >> | of RFC2983
| >> | >> (copying outer DSCP to inner).
| >> | >>
| >> | >> The disadvantage of using /only/ DSCPs to encode PCN are
| >> | as follows:
| >> | >>
| >> | >> a) If an operator wants to have multiple PCN PHBs 
| (e.g. for data 
| >> | >> plane as well as signalled precedence for emergency
| >> | traffic), it has
| >> | >> to use 1 more DSCP to /each/ PHB in order to add 
| termination and 
| >> | >> 1 more /each/ to add admission marking. Whereas using 
| 11 in the 
| >> | >> ECN field for AM or TM requires only 1 DSCP per PHB. If an
| >> | operator wants
| >> | >> AM and TM, and data plane precedence it seems to have no
| >> | choice but
| >> | >> to use up multiple DSCPs, but at least it can use 11 in
| >> | the ECN field
| >> | >> for one of the markings.
| >> | >
| >> | >Dealing with emergency traffic is strictly out of scope for
| >> | PCN at this
| >> | >time.  With that said, wouldn't the flow precedence be
| >> | included in the
| >> | >flow signaling?  I confess to being really ignorant on all
| >> | of the work
| >> | >that went on in IEPREP.
| >> | >
| >> | >In general, excluding the possibility of conveying flow 
| >> | >admission/termination precedence in the DSCP bits, 
| there are only 
| >> | >so many managed-to-almost-never-be-congested BAs that 
| are needed 
| >> | >in a network.
| >> | >
| >> | >> b) If the operator uses equal cost multipath (ECMP) routing 
| >> | >> within the PCN region, and the vendor's ECMP 
| algorithm depends 
| >> | >> on
| >> | the DSCP
| >> | >> in each packet, ideally we don't want to admission mark by
| >> | changing
| >> | >> DSCPs because this will throw some packets of each flow onto 
| >> | >> different routes, causing re-ordering. Ideally we don't want 
| >> | >> termination marking to change DSCPs either, but given all the 
| >> | >> constraints we might have to live with some packet
| >> | re-ordering in all
| >> | >> flows until some have been terminated, as this is becoming
| >> | a bit of a
| >> | >> corner case.
| >> | >
| >> | >Using DSCP (or any other router-mutable field) for ECMP 
| hashing is 
| >> | >broken on so many levels.  Do we have evidence that any 
| vendor has 
| >> | >actually implemented this?
| >> | >
| >> | >
| >> | >Regards,
| >> | >
| >> | >=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
| >> | >Steven Blake                <[email-masked]>
| >> | >Ericsson/Redback Networks               +1 919-472-9913
| >> | >
| >> | >_______________________________________________
| >> | >PCN mailing list
| >> | >[email-masked]
| >> | >https://www.ietf.org/mailman/listinfo/pcn
| >> | _______________________________________________
| >> | PCN mailing list
| >> | [email-masked]
| >> | https://www.ietf.org/mailman/listinfo/pcn
| >> |
| >> _______________________________________________
| >> PCN mailing list
| >> [email-masked]
| >> https://www.ietf.org/mailman/listinfo/pcn
| >>
| >
| >--
| >Dr. Michael Menth, Assistant Professor
| >University of Wuerzburg, Institute of Computer Science Am Hubland, 
| >D-97074 Wuerzburg, Germany, room B206
| >phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 
| >mailto:[email-masked]
| >http://www3.informatik.uni-wuerzburg.de/research/ngn
| _______________________________________________
| PCN mailing list
| [email-masked]
| https://www.ietf.org/mailman/listinfo/pcn
| 
_______________________________________________
PCN mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pcn