Subject: RE: [Tsvwg] why use edge-to-edge ECN re-marking when DSCP re-markingcan do it

From: [email-masked]
Date: Thu, 24 Nov 2005 00:23:55 -0000

hit send too early last time,  
 
>>What is the technical reason of using ECN for edge-toedge congestion control?
The use of Ecn seems natural to me, since we wnat the network to indicate incipient congestion and that's what ECN is supposed to do. Note that the semantics are the same as rfc3168 (ECN) - ie same codepoints of ECN field and CE codepoint indicates incipient congestion. I also think it can help with migration to a fully ecn world. As operators get experience of using ecn, whilst not requiring all nws / end hosts to be using it. 

Second, using DSCP re-marking seems to need a lot of DSCPs. According to your draft, admission control implies 2 per traffic class (one for re-marked pkts, one for other pkts passing thru the same router). Plus another for the severe congestion indication. In addition, in the core network an interesting future extension is to use the mechanisms in a network with MPLS-TE (see our framework draft for brief discussion) - there isnt sufficient space in the label for these extra dscps. 

Third, (as bob has pointed out) that it's easy to overcome the end-to-end vs edge-to-edge marking issue (if that's really needed, which I'm inclined to think it isnt)

________________________________

From: [email-masked] on behalf of [email-masked]
Sent: Wed 23/11/2005 23:48
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: [Tsvwg] why use edge-to-edge ECN re-marking when DSCP re-markingcan do it

some comments below.

best wishes, phil

________________________________

*       To: <rbriscoe at jungle.bt.co.uk <mailto:[email-masked]> >
*       Subject: [Tsvwg] why use edge-to-edge ECN re-marking when DSCP re-marking can do it
*       From: "Georgios Karagiannis" <karagian at cs.utwente.nl <mailto:[email-masked]> >
*       Date: Mon, 21 Nov 2005 10:52:22 +0100
*       Cc: tsvwg at ietf.org <mailto:[email-masked]> , nsis at ietf.org <mailto:[email-masked]> 

Hi Bob

In addition to the previous discussion I would like to
discuss and ask you the following.

[chop]
What is the technical reason of using ECN for edge-toedge congestion control,
when the same solution (on providing admission control
based on congestion notification and severe
congestion [pre-emption]) can be provided by using DSCP re-marking?

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Thu, 24 Nov 2005 10:28:09 +0100

Hi Phil

Please see in line!

----- Original Message ----- 
From: <[email-masked]>
To: <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Thursday, November 24, 2005 1:23 AM
Subject: RE: [Tsvwg] why use edge-to-edge ECN re-marking when DSCP 
re-markingcan do it

> The use of Ecn seems natural to me, since we wnat the network to indicate 
> incipient congestion and that's what ECN is supposed to do. Note that the 
> semantics are the same as rfc3168 (ECN) - ie same codepoints of ECN field 
> and CE codepoint indicates incipient congestion. I also think it can help 
> with migration to a fully ecn world. As operators get experience of using 
> ecn, whilst not requiring all nws / end hosts to be using it.

Georgios: yes I understand but the ECN semantics apply end-to-end, not 
edge-toedge,
please see the previous discussions on this issue!

>
> Second, using DSCP re-marking seems to need a lot of DSCPs. According to 
> your draft, admission control implies 2 per traffic class (one for 
> re-marked pkts, one for other pkts passing thru the same router). Plus 
> another for the severe congestion indication. In addition, in the core 
> network an interesting future extension is to use the mechanisms in a 
> network with MPLS-TE (see our framework draft for brief discussion) - 
> there isnt sufficient space in the label for these extra dscps.

Georgios: regarding the number of DSCP's you are right, but there are ways
of reducing this number, please see the RMD draft.
Please also note that the streaming aplication example that has been 
discussed in
the tsvwg and nsis mailing lists shows that it is usefull to use
end-to-end ECN based congestion control for reserved traffic.
This streaming application example can very efficiently use RSVP and the
DCCP protocol (which uses the end-to-end ECN concept).

Please also note that the support of this type of this type of streaming 
application that
combines the end to end ECN semantics
with reserved minimum bandwidth is needed not only
in the backbone network but also in the wireless network as well.
Therefore the rate-adaptation function of ECN (DCCP functionality)
combined with minimum bandwidth reservation can be very important.

> Third, (as bob has pointed out) that it's easy to overcome the end-to-end 
> vs edge-to-edge
marking issue (if that's really needed, which I'm inclined to think it isnt)

Georgios: regarding this point I am not sure if it is easy! It depends on 
the
imposed additional complexity.

Best regards,
Georgios

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 24 Nov 2005 17:40:38 +0000

Georgios,

(long e-mail)

We thought long and hard about which re-marking to use. My primary concern 
was to keep the Internet architecture consistent, which leads to the 
arguments below.

1/ Intended semantics of the Internet Architecture
==================================================

1.1/ Intended re-marking semantics of Diffserv & ECN
--------------------------------------------------
Diffserv re-marking was defined for two reasons:
     i) To allow the ingress of a Diffserv domain to prevent traffic in 
excess of the contracted rate/burstiness from being given favourable 
treatment at downstream routers if they became congested themselves.
     ii) To allow a border router to map the DSCP used for a per-hop 
behaviour in the upstream domain to that used for the same PHB in the 
downstream domain.

ECN re-marking is used by a node to indicate that the node itself is 
approaching congestion. The intent is to reduce the load from the source, 
but marking is forwarded downstream in the expectation that it will be fed 
back to the source.

     Thus, the intent of Diffserv re-marking is to alter the downstream per 
hop forwarding behaviour if necessary. Diffserv re-marking was not intended 
to be fed back to the source.
     The intent of ECN re-marking is to reduce the load at source through 
feedback. ECN re-marking was specifically not intended to alter the 
downstream forwarding behaviour of re-marked packets.

That's the idealistic view. In practice, it can be convenient to break 
architecture. But architecture is there to warn you to think long and hard 
when you depart from it, so you are aware of what you might break in the 
future.

1.2/ Was ECN at the IP layer intended to have end-to-end semantics?
-------------------------------------------------------------------
RFC3168 was written so that the ECN semantics in IP could be separated from 
the end-to-end ECN semantics of TCP when other ECN transports were defined.

[According to Sally Floyd at the time, the main reason for specifying ECN 
for TCP & IP together in the same RFC was to encourage the first 
experiments with ECN to be in TCP implementations which tend to be written 
with a lot more care by a few kernel experts.]

Indeed, Jon Crowcroft and I wrote an I-D reviewing the ECN I-D before it 
became RFC3168. In section 12, we proposed text on ECN semantics for 
proxies (edge-to-edge), asking that it be added to the ECN RFC:
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/pubs.html#ECN-IP> (Expired)

Sally Floyd's response (off-line) at the time is quoted here:

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
At 11:35 21/02/01 -0800, Sally Floyd wrote:
 >Feedback on Section 12, proposing rules for allowing proxies in the
 >network to change ECN codepoints:
 >
 >I think that this is complicated, and has to be dealt with in a
 >separate document, in context.  That is, when people have specific
 >proposals for intermediate nodes to modify the ECN field,
 >then I think they have to be addressed at that time.  My own opinion
 >is that making general rules ahead of time, in the abstract,
 >such as "we will now propose rules for changing the ECN
 >capability of a packet at intermediate nodes", never gets it right.
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

<draft-briscoe-tsvwg-cl-architecture-01.txt> is exactly that. A separate 
document about the semantics of ECN proxies, in this case with a flow 
signalling transport like RSVP (or NSIS).

The idea of RMD and of <draft-briscoe-tsvwg-cl-architecture-01.txt> is for 
the node at the end of a long signalling 'hop' (a whole Diffserv region) to 
see how much congestion has accumulated since the start of the region. 
Potentially there may be other signalling 'hops' further downstream using 
the same technique again. So the egress of one signalling hop must tidy 
things up by folding any re-marked packets back in to the same marking as 
un-re-marked packets, to avoid confusing further region(s) downstream if 
they are using the same technique.

[For instance, you are thinking of RMD being used for admission control 
across a radio access network. But there may be another RAN at the other 
end of the network using the same technique. So you must ensure any 
re-marked DSCPs are folded back into the same DSCP as non-re-marked packets 
at the RMD egress. ]

1.3/ Summary of architectural concerns
--------------------------------------
To indicate the need for admission control, it is certainly more convenient 
to do Diffserv re-marking as RMD does, but I believe it is more 
architecturally consistent to use ECN re-marking, as we do.

However, architecture is like religion. You can break it, if it becomes too 
inconvenient, and if you can convince enough people you have a good excuse.

2. Practical issues
-------------------
I don't think we need to break the architecture in this case (which is why 
we use ECN), but using ECN re-marking edge-edge does raise the following 
issue...

2.1/ Edge-to-edge ECN conflicts with end-to-end ECN
---------------------------------------------------
We fully understand that use of ECN re-marking to trigger admission control 
within Diffserv regions would conflict with end-to-end ECN congestion 
control for the *same* flow. But we deliberately chose to specify the end 
to end service we were providing as "controlled load [RFC2211]" to try to 
preclude such cases.

RFC 2211 says:

"  ...
2. End-to-End Behavior
    ...                              Should the client's traffic
    generation properties fall outside of the region described by the
    TSpec parameters, the QoS provided to the client may exhibit
    characteristics indicative of overload, including large numbers of
    delayed or dropped packets.

    ...

7. Policing
    ...                                  the network element MUST attempt
      to forward the excess traffic on a best-effort basis if sufficient
      resources are available

    ...

    Beyond requirements 2 and 3 above, the controlled-load service does
    not define the QoS behavior delivered to flows with non-conformant
    arriving traffic.  Specifically, it is permissible either to degrade
    the service delivered to all of the flow's packets equally, or to
    sort the flow's packets into a conformant set and a nonconformant set
    and deliver different levels of service to the two sets.

    ...

9. Guidelines for Implementors

    ...
If most controlled-load traffic arises from rate-adaptive
    realtime or elastic applications attempting to establish a bounded
    minimum level of service, the use of separate resource classes might
    be preferable.

"
I won't paste everything RF2211 says on with advice for implementors on 
scheduling elastic flows with a bounded minimum, as there's rather a lot. 
But basically, you get unavoidably bad re-ordering and other complications 
that make it hard. Even though this seems a desirable service, it's 
unlikely to work well in practice.

2.2/ End-to-end ECN conflicts with edge-to-edge admission control
-----------------------------------------------------------------
The congestion-based admission control mechanism used in RMD and in our 
<draft-briscoe-tsvwg-cl-architecture-01.txt> would break if a significant 
proportion of admitted traffic were adaptive. If congestion on an interior 
path were close to the threshold where admission control was needed, 
adaptive traffic might temporarily drop well below its minimum bound. More 
flows might then be accepted, then the adaptive traffic could grow again, 
destroying the controlled load service for all flows through that path.

[Note: Variable bit rate (VBR) like video traffic would superfically seem 
to break a measurement-based admission control system in the same way as 
adaptive traffic. But unlike with adaptive traffic, troughs in VBR load are 
not correlated with congestion. Our analyses of how the effective bandwidth 
of VBR (and silence-suppressed voice) aggregates has shown that our 
solution works OK for VBR, given specific conditions on the amount of 
aggregation and provided VBR traffic sources are not correlated. We are 
currently preparing this analysis for publication.]

2.3/ Summary of end/edge conflicts
----------------------------------
So, we prefer to say that a different service would have to be requested to 
cater for your hybrid requirement. Hence my preference for explicitly 
requesting "adaptive rate control with a bounded minimum" in the 
signalling, as a different QoS service. In NSIS terms, this is would be in 
the QSPEC, I believe.

2.4/ Codepoint space
--------------------
Real-estate in the IP header is at a premium. Treating the 6-bit Diffserv 
field and the 2-bit ECN field separately leads to 2^6 + 2^2 = 68 
codepoints. Treating them together leads to 2^8 = 256 codepoints. In the 
MPLS EXP field, real-estate is a far greater headache.

If ECN marking is orthogonal to Diffserv marking for most Diffserv 
codepoints, it makes sense to treate the field as 6-bit and 2-bit 
sub-fields. But I don't believe we should be averse to bending the rules to 
save codepoint space across both fields.

For instance both our approach and RMD need another marking state for a 
router in the Diffserv interior to indicate the need for flow pre-emption 
(eg. after re-routes). Pre-emption represents severe congestion and needs 
to be fed back upstream. So by my arguments above pre-emption should be 
re-marked using the ECN field. But in our current pre-emption solution, if 
two or more nodes in a path require some flows to be pre-empted, packets 
that have already been re-marked once to indicate pre-emption need to be 
treated differently by downstream nodes. So by my arguments above 
pre-emption should also be re-marked using the DSCP.

Well, it would be silly to waste two codepoints for one state. And there 
aren't enough ECN codepoints anyway. So, IMHO, you have to be pragmatic and 
be willing to re-mark the Diffserv codepoint for pre-emption (CAVEAT: the 
authors of <draft-briscoe-tsvwg-cl-architecture-01.txt> still haven't 
reached consensus on marking - that's why 
<draft-briscoe-tsvwg-cl-phb-00.txt> hasn't been updated yet).

7/ Consistent meaning of an ECN marking
---------------------------------------
I hope I have answered your question about why we use ECN not Diffserv 
re-marking. We are not completely inflexible about using ECN, but I do have 
further reasons for preferring ECN re-marking---to do with the economic 
interpretation of congestion marking across different classes. It would 
take too long to go into that here, but you can read a recent paper I 
published on the subject if interested.

Commercial Models for IP Quality of Service Interconnect, Bob Briscoe & 
Steve Rudkin (BT), in BTTJ Special Edition on IP Quality of Service, 23(2) 
(Apr 2005)
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/pubs.html#ixqos>

I have also given responses inline...

At 09:28 24/11/2005, Georgios Karagiannis wrote:
>----- Original Message ----- From: <[email-masked]>
>>>>[GK] What is the technical reason of using ECN for edge-toedge 
>>>>congestion control?
>>[PE] The use of Ecn seems natural to me, since we wnat the network to 
>>indicate incipient congestion and that's what ECN is supposed to do. Note 
>>that the semantics are the same as rfc3168 (ECN) - ie same codepoints of 
>>ECN field and CE codepoint indicates incipient congestion. I also think 
>>it can help with migration to a fully ecn world. As operators get 
>>experience of using ecn, whilst not requiring all nws / end hosts to be 
>>using it.
>

[BB] ECN semantics were originally defined for an end-to-end protocol 
(TCP). That does not imply they were only intended to ever be end-to-end. 
RFC3168 did not preclude edge-to-edge. And at least one of the authors 
believed edge-to-edge semantics could be a valid thing to define later (see 
quote from Sally Floyd above).

>>[PE] Second, using DSCP re-marking seems to need a lot of DSCPs. 
>>According to your draft, admission control implies 2 per traffic class 
>>(one for re-marked pkts, one for other pkts passing thru the same 
>>router). Plus another for the severe congestion indication. In addition, 
>>in the core network an interesting future extension is to use the 
>>mechanisms in a network with MPLS-TE (see our framework draft for brief 
>>discussion) - there isnt sufficient space in the label for these extra dscps.
>

[BB] No-one on the list has said it would be efficient. In fact the 
opposite has been said (by Louise Burness) and no-one has disagreed. I 
believe this hybrid service should be set aside for someone else to define 
a mechanism for later. Intserv couldn't even do it very well, even though 
it had a per-flow scheduler.

>[GK] Please also note that the support of this type of this type of 
>streaming application that

[BB] Yes, an adaptive application with a minimum rate would be a desirable 
service. But it's not the one we are defining. And you will find it doesn't 
work with RMD's congestion-based admission scheme either.

[BB] We are trying to do Controlled Load to scale to the Internet. We don't 
know how to implement this hybrid at Internet scale. We have had to limit 
our scope to avoid this hybrid problem. And you will find that such a 
hybrid service wouldn't work with the mechanism you have defined for RMD 
either.

[BB] So I need to turn your question back on you: how would you prevent 
adaptive applications using your RMD congestion mechanism, and breaking the 
admission control system?

[BB] The solution I have proposed is to require the application to signal 
this hybrid requirement and if it is not signalled, to drop any excess 
traffic at the region ingress in order to disincentivise any adaptive 
application trying to use the wrong service for its needs.

>Georgios: regarding this point I am not sure if it is easy! It depends on the

[BB] If a request for this hybrid were signalled (and an operator chose to 
support the hybrid service), with the current state of knowledge, I would 
say that this hybrid traffic would have to be dealt with by a less scalable 
means, such as Intserv per-flow schedulers on every node, or your hop by 
hop reservation method in RMD.

Bob

>nsis mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/nsis

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/130 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.   +44 1473 645196

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Fri, 25 Nov 2005 11:15:26 +0100

Hi Bob

>From your long e-mail I can derive the folowing conclusions:
You agree that:

* To indicate the need for admission control, it is certainly more 
convenient
to do Diffserv re-marking as RMD does, but in your opinion
it is more architecturally consistent to use ECN.

* Furthermore in your opinion you agree that you have to be pragmatic and
be willing to re-mark the Diffserv codepoint for pre-emption

Thus here is a proposal on helping going forward:

* Get a consensus on the fact that ECN re-marking and DSCP re-marking are
both useful (maybe applied in different settings) and therefore it should be 
possible
that both types of  re-marking techniques could be used for edge-to-edge
admission control and severe congestion [pre-emption] in an edge-to-edge 
domain.

Do you agree?

Best Regards,
Georgios

----- Original Message ----- 
From: "Bob Briscoe" <[email-masked]>
To: "Georgios Karagiannis" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Thursday, November 24, 2005 6:40 PM
Subject: Re: [NSIS] Re: [Tsvwg] why use edge-to-edge ECN re-marking when 
DSCP re-markingcan do it

> hop forwarding behaviour if necessary. Diffserv re-marking was not 
> intended to be fed back to the source.
> RFC3168 was written so that the ECN semantics in IP could be separated 
> from the end-to-end ECN semantics of TCP when other ECN transports were 
> defined.
>
> the node at the end of a long signalling 'hop' (a whole Diffserv region) 
> to see how much congestion has accumulated since the start of the region. 
> re-marked DSCPs are folded back into the same DSCP as non-re-marked 
> packets at the RMD egress. ]
>
> To indicate the need for admission control, it is certainly more 
> convenient to do Diffserv re-marking as RMD does, but I believe it is more 
> However, architecture is like religion. You can break it, if it becomes 
> too inconvenient, and if you can convince enough people you have a good 
> excuse.
>
> 2. Practical issues
> -------------------
> We fully understand that use of ECN re-marking to trigger admission 
> control within Diffserv regions would conflict with end-to-end ECN 
> congestion control for the *same* flow. But we deliberately chose to 
> specify the end to end service we were providing as "controlled load 
> [RFC2211]" to try to preclude such cases.
>
> RFC 2211 says:
>
> "  ...
> 2. End-to-End Behavior
> adaptive traffic. But unlike with adaptive traffic, troughs in VBR load 
> are not correlated with congestion. Our analyses of how the effective 
> bandwidth of VBR (and silence-suppressed voice) aggregates has shown that 
> our solution works OK for VBR, given specific conditions on the amount of 
> So, we prefer to say that a different service would have to be requested 
> to cater for your hybrid requirement. Hence my preference for explicitly 
> sub-fields. But I don't believe we should be averse to bending the rules 
> to save codepoint space across both fields.
>
> aren't enough ECN codepoints anyway. So, IMHO, you have to be pragmatic 
> and be willing to re-mark the Diffserv codepoint for pre-emption (CAVEAT: 
> the authors of <draft-briscoe-tsvwg-cl-architecture-01.txt> still haven't 
> re-marking. We are not completely inflexible about using ECN, but I do 
> have further reasons for preferring ECN re-marking---to do with the 
> economic interpretation of congestion marking across different classes. It 
> would take too long to go into that here, but you can read a recent paper 
> I published on the subject if interested.
>
> believed edge-to-edge semantics could be a valid thing to define later 
> (see quote from Sally Floyd above).
>
>
>
> service. But it's not the one we are defining. And you will find it 
> doesn't work with RMD's congestion-based admission scheme either.
>
> [BB] We are trying to do Controlled Load to scale to the Internet. We 
> don't know how to implement this hybrid at Internet scale. We have had to 
> limit our scope to avoid this hybrid problem. And you will find that such 
> a hybrid service wouldn't work with the mechanism you have defined for RMD 
> either.
>
> adaptive applications using your RMD congestion mechanism, and breaking 
> the admission control system?
>
> say that this hybrid traffic would have to be dealt with by a less 
> scalable means, such as Intserv per-flow schedulers on every node, or your 
> hop by hop reservation method in RMD.
>
>
> Bob
>
>>Best regards,
>>Georgios
>>
>>
>>>
>>>*       To: <rbriscoe at jungle.bt.co.uk <mailto:[email-masked]> 
>>> >