Subject: RE: [NSIS] Re: [Tsvwg] why use edge-to-edge ECN re-marking when DSCP re-marking can do it

From: Bob Briscoe <[email-masked]>
Date: Wed, 15 Mar 2006 18:27:20 +0000

John,

Some time ago you asked how we would resolve the disagreement between 
re-marking Diffserv or ECN fields for admission control and pre-emption 
(old thread below).

We have included a number of candidate options for re-marking in the 
appendices to 
<http://www.ietf.org/internet-drafts/draft-briscoe-tsvwg-cl-phb-01.pdf>. 
With pros and cons.

This is a prompt for the people involved in that discussion (and others) to 
read the appendices to the above I-D and comment on the list. The aim is 
that the eventual consensus can be recorded in a future version of that 
draft, with the aim of eventually moving to standards track. Phil Eardley's 
presentation in Dallas intends to re-open debate from the mic as well.

Also, I have produced an extended argument for one of the choices 
(Alternative 5) as part of 
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/projects/2020comms/refb/draft-briscoe-tsvwg-re-ecn-border-cheat-01d.txt>, 
mainly on security grounds.

[This is a corrected version of 
draft-briscoe-tsvwg-re-ecn-border-cheat-00.txt  inserting a missing "NOT" 
in a key sentence. It will be posted once the I-D gates open again.]

Bob

At 12:35 12/12/2005, [email-masked] wrote:
>Bob (& by extension the TSVWG WG),
>
>I'm still left feeling like we don't have consensus around the issues 
>out-lined
>in your mail in the tsvwg community, even outside of RMD issues.
>
>I'm wondering if we should start trying to find somethings that we could get
>consensus on and then see what to do about the other issues.  Many of the 
>points
>are still experimental (in the TSVWG sense of experimental), so I'm wondering
>what the next steps should be.
>
>thanks,
>John
>
> >-----Original Message-----
> >From: [email-masked] [mailto:[email-masked]] On
> >Behalf Of ext Attila Báder (IJ/ETH)
> >Sent: 08 December, 2005 16:02
> >To: Bob Briscoe
> >Cc: [email-masked]; Georgios Karagiannis;
> >[email-masked]; [email-masked]
> >Subject: RE: [NSIS] Re: [Tsvwg] why use edge-to-edge ECN
> >re-markingwhen DSCPre-markingcan do it
> >
> >Hi Bob, Thank you for your answer, please see my comments below.
> >
> >Best regards, Attila
> >
> >> -----Original Message-----
> >> From: [email-masked] [mailto:[email-masked]]
> >> Sent: Thursday, November 24, 2005 6:41 PM
> >> To: Georgios Karagiannis
> >> Cc: [email-masked]; [email-masked]; [email-masked]
> >> Subject: Re: [NSIS] Re: [Tsvwg] why use edge-to-edge ECN
> >> re-markingwhen DSCP re-markingcan do it
> >>
> >>
> >> Georgios,
> >>
> >> (long e-mail)
> >>
> >> We thought long and hard about which re-marking to use. My primary
> >> concern was to keep the Internet architecture consistent,
> >which leads
> >> to the arguments below.
> >>
> >> 1/ Intended semantics of the Internet Architecture
> >> ==================================================
> >>
> >> 1.1/ Intended re-marking semantics of Diffserv & ECN
> >> --------------------------------------------------
> >> Diffserv re-marking was defined for two reasons:
> >>      i) To allow the ingress of a Diffserv domain to prevent traffic
> >> in excess of the contracted rate/burstiness from being given
> >> favourable treatment at downstream routers if they became congested
> >> themselves.
> >>      ii) To allow a border router to map the DSCP used for a per-hop
> >> behaviour in the upstream domain to that used for the same
> >PHB in the
> >> downstream domain.
> >>
> >> ECN re-marking is used by a node to indicate that the node itself is
> >> approaching congestion. The intent is to reduce the load from the
> >> source, but marking is forwarded downstream in the
> >expectation that it
> >> will be fed back to the source.
> >>
> >>      Thus, the intent of Diffserv re-marking is to alter the
> >> downstream per hop forwarding behaviour if necessary. Diffserv
> >> re-marking was not intended to be fed back to the source.
> >>      The intent of ECN re-marking is to reduce the load at source
> >> through feedback. ECN re-marking was specifically not intended to
> >> alter the downstream forwarding behaviour of re-marked packets.
> >>
> >> That's the idealistic view. In practice, it can be convenient to break
> >> architecture. But architecture is there to warn you to think long and
> >> hard when you depart from it, so you are aware of what you
> >might break
> >> in the future.
> >>
> >
> >I agree that this would be an idealistic picture. Note that
> >you would like to use ECN for Admission Control. Diffserv is
> >used also for admission control so there is an overlap between
> >ECN and Diffserv. In the meantime ECN was defined for
> >end-to-end congestion control of elastic traffic type that
> >makes things complicate. While there is no such limitation to
> >use remaining Diffserv codepoints for congestion notification
> >edge-to-edge.
> >
> >> 1.2/ Was ECN at the IP layer intended to have end-to-end semantics?
> >> -------------------------------------------------------------------
> >> RFC3168 was written so that the ECN semantics in IP could be separated
> >> from the end-to-end ECN semantics of TCP when other ECN transports
> >> were defined.
> >>
> >> [According to Sally Floyd at the time, the main reason for specifying
> >> ECN for TCP & IP together in the same RFC was to encourage the first
> >> experiments with ECN to be in TCP implementations which tend to be
> >> written with a lot more care by a few kernel experts.]
> >>
> >> Indeed, Jon Crowcroft and I wrote an I-D reviewing the ECN I-D before
> >> it became RFC3168. In section 12, we proposed text on ECN semantics
> >> for proxies (edge-to-edge), asking that it be added to the ECN RFC:
> >> <http://www.cs.ucl.ac.uk/staff/B.Briscoe/pubs.html#ECN-IP> (Expired)
> >>
> >> Sally Floyd's response (off-line) at the time is quoted here:
> >>
> >> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
> >> /\/\/\/\/\/\/\/
> >> At 11:35 21/02/01 -0800, Sally Floyd wrote:
> >>  >Feedback on Section 12, proposing rules for allowing proxies in the
> >> >network to change ECN codepoints:
> >>  >
> >>  >I think that this is complicated, and has to be dealt with in a
> >> >separate document, in context.  That is, when people have specific
> >> >proposals for intermediate nodes to modify the ECN field,  >then I
> >> think they have to be addressed at that time.  My own opinion  >is
> >> that making general rules ahead of time, in the abstract,  >such as
> >> "we will now propose rules for changing the ECN  >capability of a
> >> packet at intermediate nodes", never gets it right.
> >> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
> >> /\/\/\/\/\/\/\/
> >>
> >> <draft-briscoe-tsvwg-cl-architecture-01.txt> is exactly that. A separate
> >> document about the semantics of ECN proxies, in this case with a flow
> >> signalling transport like RSVP (or NSIS).
> >>
> >> The idea of RMD and of
> >> <draft-briscoe-tsvwg-cl-architecture-01.txt> is for the node at the
> >> end of a long signalling 'hop' (a whole Diffserv region) to see how
> >> much congestion has accumulated since the start of the region.
> >> Potentially there may be other signalling 'hops' further downstream
> >> using the same technique again. So the egress of one signalling hop
> >> must tidy things up by folding any re-marked packets back in to the
> >> same marking as un-re-marked packets, to avoid confusing further
> >> region(s) downstream if they are using the same technique.
> >>
> >> [For instance, you are thinking of RMD being used for admission
> >> control across a radio access network. But there may be another RAN at
> >> the other end of the network using the same technique. So you must
> >> ensure any re-marked DSCPs are folded back into the same DSCP as
> >> non-re-marked packets at the RMD egress. ]
> >>
> >
> >Yes, I agree that Egress edge should restore the orig. DSCP,
> >otherwise there can be multiple actions for the same event. It
> >may not be restored if there is agreement between domains that
> >a downsteram domain handles the problem.
> >
> >> 1.3/ Summary of architectural concerns
> >> --------------------------------------
> >> To indicate the need for admission control, it is certainly more
> >> convenient to do Diffserv re-marking as RMD does, but I believe it is
> >> more architecturally consistent to use ECN re-marking, as we do.
> >>
> >> However, architecture is like religion. You can break it, if it
> >> becomes too inconvenient, and if you can convince enough people you
> >> have a good excuse.
> >>
> >> 2. Practical issues
> >> -------------------
> >> I don't think we need to break the architecture in this case
> >(which is
> >> why we use ECN), but using ECN re-marking edge-edge does raise the
> >> following issue...
> >>
> >> 2.1/ Edge-to-edge ECN conflicts with end-to-end ECN
> >> ---------------------------------------------------
> >> We fully understand that use of ECN re-marking to trigger admission
> >> control within Diffserv regions would conflict with end-to-end ECN
> >> congestion control for the *same* flow. But we deliberately chose to
> >> specify the end to end service we were providing as "controlled load
> >> [RFC2211]" to try to preclude such cases.
> >>
> >> RFC 2211 says:
> >>
> >> "  ...
> >> 2. End-to-End Behavior
> >>     ...                              Should the client's traffic
> >>     generation properties fall outside of the region described by the
> >>     TSpec parameters, the QoS provided to the client may exhibit
> >>     characteristics indicative of overload, including large
> >numbers of
> >>     delayed or dropped packets.
> >>
> >>     ...
> >>
> >> 7. Policing
> >>     ...                                  the network element
> >> MUST attempt
> >>       to forward the excess traffic on a best-effort basis if
> >> sufficient
> >>       resources are available
> >>
> >>     ...
> >>
> >>     Beyond requirements 2 and 3 above, the controlled-load service
> >> does
> >>     not define the QoS behavior delivered to flows with
> >non-conformant
> >>     arriving traffic.  Specifically, it is permissible either to
> >> degrade
> >>     the service delivered to all of the flow's packets equally, or to
> >>     sort the flow's packets into a conformant set and a
> >nonconformant
> >> set
> >>     and deliver different levels of service to the two sets.
> >>
> >>     ...
> >>
> >> 9. Guidelines for Implementors
> >>
> >>     ...
> >> If most controlled-load traffic arises from rate-adaptive
> >>     realtime or elastic applications attempting to establish
> >a bounded
> >>     minimum level of service, the use of separate resource classes
> >> might
> >>     be preferable.
> >>
> >> "
> >> I won't paste everything RF2211 says on with advice for implementors
> >> on scheduling elastic flows with a bounded minimum, as
> >there's rather
> >> a lot.
> >> But basically, you get unavoidably bad re-ordering and other
> >> complications that make it hard. Even though this seems a desirable
> >> service, it's unlikely to work well in practice.
> >>
> >> 2.2/ End-to-end ECN conflicts with edge-to-edge admission control
> >> -----------------------------------------------------------------
> >> The congestion-based admission control mechanism used in RMD and in
> >> our <draft-briscoe-tsvwg-cl-architecture-01.txt> would break if a
> >> significant proportion of admitted traffic were adaptive. If
> >> congestion on an interior path were close to the threshold where
> >> admission control was needed, adaptive traffic might
> >temporarily drop
> >> well below its minimum bound. More flows might then be
> >accepted, then
> >> the adaptive traffic could grow again, destroying the
> >controlled load
> >> service for all flows through that path.
> >>
> >> [Note: Variable bit rate (VBR) like video traffic would superfically
> >> seem to break a measurement-based admission control system
> >in the same
> >> way as adaptive traffic. But unlike with adaptive traffic,
> >troughs in
> >> VBR load are not correlated with congestion. Our analyses of how the
> >> effective bandwidth of VBR (and silence-suppressed voice) aggregates
> >> has shown that our solution works OK for VBR, given specific
> >> conditions on the amount of aggregation and provided VBR traffic
> >> sources are not correlated. We are currently preparing this analysis
> >> for publication.]
> >>
> >
> >That study sounds interesting. I do not think that the
> >congestion-based admission control is good enough for all
> >networks. Meas. based methods certainly have limitations,
> >especially in access network, as you also mention in your
> >draft. In RMD edge nodes are statefull nodes. I think that
> >unit-based aggregated reservation in core combined with
> >per-flow schedulers in edges can provide an Intserv-like
> >service in the domain as well.
> >
> >> 2.3/ Summary of end/edge conflicts
> >> ----------------------------------
> >> So, we prefer to say that a different service would have to be
> >> requested to cater for your hybrid requirement. Hence my preference
> >> for explicitly requesting "adaptive rate control with a bounded
> >> minimum" in the signalling, as a different QoS service. In
> >NSIS terms,
> >> this is would be in the QSPEC, I believe.
> >>
> >
> >I think this traffic should use controlled load service. Or
> >define a new QoS model.
> >
> >> 2.4/ Codepoint space
> >> --------------------
> >> Real-estate in the IP header is at a premium. Treating the 6-bit
> >> Diffserv field and the 2-bit ECN field separately leads to
> >2^6 + 2^2 =
> >> 68 codepoints. Treating them together leads to 2^8 = 256 codepoints.
> >> In the MPLS EXP field, real-estate is a far greater headache.
> >>
> >> If ECN marking is orthogonal to Diffserv marking for most Diffserv
> >> codepoints, it makes sense to treate the field as 6-bit and 2-bit
> >> sub-fields. But I don't believe we should be averse to bending the
> >> rules to save codepoint space across both fields.
> >>
> >> For instance both our approach and RMD need another marking
> >state for
> >> a router in the Diffserv interior to indicate the need for flow
> >> pre-emption (eg. after re-routes). Pre-emption represents severe
> >> congestion and needs to be fed back upstream. So by my
> >arguments above
> >> pre-emption should be re-marked using the ECN field. But in our
> >> current pre-emption solution, if two or more nodes in a path require
> >> some flows to be pre-empted, packets that have already been
> >re-marked
> >> once to indicate pre-emption need to be treated differently by
> >> downstream nodes. So by my arguments above pre-emption
> >should also be
> >> re-marked using the DSCP.
> >>
> >> Well, it would be silly to waste two codepoints for one state. And
> >> there aren't enough ECN codepoints anyway. So, IMHO, you have to be
> >> pragmatic and be willing to re-mark the Diffserv codepoint for
> >> pre-emption
> >> (CAVEAT: the
> >> authors of <draft-briscoe-tsvwg-cl-architecture-01.txt>
> >still haven't
> >> reached consensus on marking - that's why
> >> <draft-briscoe-tsvwg-cl-phb-00.txt> hasn't been updated yet).
> >
> >We think that both DSCP and ECN remarking could be used for
> >congestion notification admission control and severe
> >congestion [pre-emption] with limitation.
> >
> >>
> >> 7/ Consistent meaning of an ECN marking
> >> ---------------------------------------
> >> I hope I have answered your question about why we use ECN
> >not Diffserv
> >> re-marking. We are not completely inflexible about using
> >ECN, but I do
> >> have further reasons for preferring ECN re-marking---to do with the
> >> economic interpretation of congestion marking across different
> >> classes. It would take too long to go into that here, but
> >you can read
> >> a recent paper I published on the subject if interested.
> >>
> >> Commercial Models for IP Quality of Service Interconnect,
> >Bob Briscoe
> >> & Steve Rudkin (BT), in BTTJ Special Edition on IP Quality
> >of Service,
> >> 23(2) (Apr 2005)
> >> <http://www.cs.ucl.ac.uk/staff/B.Briscoe/pubs.html#ixqos>
> >>
> >> I have also given responses inline...
> >>
> >> At 09:28 24/11/2005, Georgios Karagiannis wrote:
> >> >----- Original Message ----- From: <[email-masked]>
> >> >To: <[email-masked]>; <[email-masked]>
> >> >Cc: <[email-masked]>; <[email-masked]>
> >> >Sent: Thursday, November 24, 2005 1:23 AM
> >> >Subject: RE: [Tsvwg] why use edge-to-edge ECN re-marking when DSCP
> >> >re-markingcan do it
> >> >
> >> >
> >> >>>>[GK] What is the technical reason of using ECN for edge-toedge
> >> >>>>congestion control?
> >> >>[PE] The use of Ecn seems natural to me, since we wnat the
> >> network to
> >> >>indicate incipient congestion and that's what ECN is
> >> supposed to do. Note
> >> >>that the semantics are the same as rfc3168 (ECN) - ie same
> >> codepoints of
> >> >>ECN field and CE codepoint indicates incipient congestion.
> >> I also think
> >> >>it can help with migration to a fully ecn world. As operators get
> >> >>experience of using ecn, whilst not requiring all nws / end
> >> hosts to be
> >> >>using it.
> >> >
> >> >Georgios: yes I understand but the ECN semantics apply
> >> end-to-end, not
> >> >edge-toedge,
> >> >please see the previous discussions on this issue!
> >>
> >> [BB] ECN semantics were originally defined for an end-to-end
> >protocol
> >> (TCP). That does not imply they were only intended to ever be
> >> end-to-end.
> >> RFC3168 did not preclude edge-to-edge. And at least one of
> >the authors
> >> believed edge-to-edge semantics could be a valid thing to
> >define later
> >> (see quote from Sally Floyd above).
> >>
> >>
> >>
> >> >>[PE] Second, using DSCP re-marking seems to need a lot of DSCPs.
> >> >>According to your draft, admission control implies 2 per
> >> traffic class
> >> >>(one for re-marked pkts, one for other pkts passing thru the same
> >> >>router). Plus another for the severe congestion indication.
> >> In addition,
> >> >>in the core network an interesting future extension is to use the
> >> >>mechanisms in a network with MPLS-TE (see our framework
> >> draft for brief
> >> >>discussion) - there isnt sufficient space in the label for
> >> these extra dscps.
> >> >
> >> >Georgios: regarding the number of DSCP's you are right, but
> >> there are ways
> >> >of reducing this number, please see the RMD draft.
> >> >Please also note that the streaming aplication example that
> >has been
> >> >discussed in the tsvwg and nsis mailing lists shows that it is
> >> >usefull to use end-to-end ECN based congestion control for reserved
> >> >traffic.
> >> >This streaming application example can very efficiently use
> >> RSVP and the
> >> >DCCP protocol (which uses the end-to-end ECN concept).
> >>
> >> [BB] No-one on the list has said it would be efficient. In fact the
> >> opposite has been said (by Louise Burness) and no-one has
> >disagreed. I
> >> believe this hybrid service should be set aside for someone else to
> >> define a mechanism for later. Intserv couldn't even do it very well,
> >> even though it had a per-flow scheduler.
> >>
> >>
> >> >[GK] Please also note that the support of this type of this type of
> >> >streaming application that combines the end to end ECN
> >semantics with
> >> >reserved minimum bandwidth is needed not only in the
> >backbone network
> >> >but also in the wireless network as well.
> >> >Therefore the rate-adaptation function of ECN (DCCP functionality)
> >> >combined with minimum bandwidth reservation can be very important.
> >>
> >> [BB] Yes, an adaptive application with a minimum rate would be a
> >> desirable service. But it's not the one we are defining. And
> >you will
> >> find it doesn't work with RMD's congestion-based admission scheme
> >> either.
> >>
> >> [BB] We are trying to do Controlled Load to scale to the
> >Internet. We
> >> don't know how to implement this hybrid at Internet scale.
> >We have had
> >> to limit our scope to avoid this hybrid problem. And you will find
> >> that such a hybrid service wouldn't work with the mechanism you have
> >> defined for RMD either.
> >>
> >> [BB] So I need to turn your question back on you: how would you
> >> prevent adaptive applications using your RMD congestion
> >mechanism, and
> >> breaking the admission control system?
> >>
> >> [BB] The solution I have proposed is to require the application to
> >> signal this hybrid requirement and if it is not signalled,
> >to drop any
> >> excess traffic at the region ingress in order to disincentivise any
> >> adaptive application trying to use the wrong service for its needs.
> >>
> >>
> >> >>Third, (as bob has pointed out) that it's easy to overcome
> >> the end-to-end
> >> >>vs edge-to-edge
> >> >marking issue (if that's really needed, which I'm inclined
> >> to think it isnt)
> >> >
> >> >Georgios: regarding this point I am not sure if it is easy!
> >> It depends on the
> >> >imposed additional complexity.
> >>
> >> [BB] If a request for this hybrid were signalled (and an operator
> >> chose to support the hybrid service), with the current state of
> >> knowledge, I would say that this hybrid traffic would have
> >to be dealt
> >> with by a less scalable means, such as Intserv per-flow
> >schedulers on
> >> every node, or your hop by hop reservation method in RMD.
> >>
> >>
> >> Bob
> >>
> >> >Best regards,
> >> >Georgios
> >> >
> >> >
> >> >>
> >> >>________________________________
> >> >>
> >> >>From: [email-masked] on behalf of [email-masked]
> >> >>Sent: Wed 23/11/2005 23:48
> >> >>To: [email-masked]
> >> >>Cc: [email-masked]; [email-masked]
> >> >>Subject: Re: [Tsvwg] why use edge-to-edge ECN re-marking when DSCP
> >> >>re-markingcan do it
> >> >>
> >> >>
> >> >>
> >> >>some comments below.
> >> >>
> >> >>best wishes, phil
> >> >>
> >> >>________________________________
> >> >>
> >> >>*       To: <rbriscoe at jungle.bt.co.uk
> ><mailto:[email-masked]> >
> >>>*       Subject: [Tsvwg] why use edge-to-edge ECN re-marking
> >when DSCP
> >>>re-marking can do it
> >>>*       From: "Georgios Karagiannis" <karagian at cs.utwente.nl
> >>><mailto:[email-masked]> >
> >>>*       Date: Mon, 21 Nov 2005 10:52:22 +0100
> >>>*       Cc: tsvwg at ietf.org <mailto:[email-masked]> , nsis at
> >>>ietf.org <mailto:[email-masked]>
> >>>
> >>>Hi Bob
> >>>
> >>>In addition to the previous discussion I would like to
> >discuss and ask
> >>>you the following.
> >>>
> >>>[chop]
> >>>What is the technical reason of using ECN for edge-toedge congestion
> >>>control, when the same solution (on providing admission
> >control based
> >>>on congestion notification and severe congestion
> >[pre-emption]) can be
> >>>provided by using DSCP re-marking?
> >>>
> >>>
> >>>
> >>>
> >>>_______________________________________________
> >>>tsvwg mailing list
> >>>[email-masked]
> >>>https://www1.ietf.org/mailman/listinfo/tsvwg
> >>>
> >>
> >>
> >>_______________________________________________
> >>nsis mailing list
> >>[email-masked]
> >>https://www1.ietf.org/mailman/listinfo/nsis
> >
> >_______________________________________________________________
> >_____________
> >Bob Briscoe, <[email-masked]>      Networks Research
> >Centre, BT Research
> >B54/130 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.
> >+44 1473 645196
> >
> >
> >
> >_______________________________________________
> >nsis mailing list
> >[email-masked]
> >https://www1.ietf.org/mailman/listinfo/nsis
> >
> >_______________________________________________
> >nsis mailing list
> >[email-masked]
> >https://www1.ietf.org/mailman/listinfo/nsis
> >

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196  

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg