Subject: RE: [Sipping] draft-hilt-sipping-overload-03

From: "Asveren, Tolga" <[email-masked]>
Date: Fri, 2 Nov 2007 09:07:56 -0400

I guess it would be hard to define an overload mechanism for SIP, which
is suitable for all deployment scenarios. 

I agree that subscriptions to large number of endpoints are a problem
but relying on responses for overload control does not provide a
satisfactory answer for this situation either. 

Overall, I *guess* it could be useful to handle subscriber/edge proxy
and core network cases separately (as it is also suggested by this
draft).

I don't know whether this has been discussed among the authors at the
early stages of this draft but one possible method to convey overload
status could be to define congestion levels with associated semantics,
e.g. 0 "no congestion", 1 "if there is an alternate node, prefer it", 2
"if there is an alternate node, use it", 3 "don't send me any messages".
This may be handier/more useful than communicating the percentage of
messages which need to be redirected/rejected (at least it is worthwhile
to include this type of congestion notification in the simulations
IMHO). Using an explicit congestion notification message, e.g. NOTIFYs
in a subscription, would have an advantage if multiple congestion levels
are used because it would allow congestion level change notifications
without a need for a request.

Although not always possible/practical, I guess conveying congestion
information beyond the next hop could be useful for certain scenarios to
take action as early as possible. A subscription based mechanism seems
more suitable for this purpose as its contents could be modified along
the path, could be initiated by intermediaries etc...

A few more comments/question below....

   Thanks,
   Tolga

> -----Original Message-----
> From: Hadriel Kaplan [mailto:[email-masked]]
> Sent: Wednesday, October 31, 2007 5:08 PM
> To: Volker Hilt; Sipping
> Subject: RE: [Sipping] draft-hilt-sipping-overload-03
> 
> > -----Original Message-----
> > From: Volker Hilt [mailto:[email-masked]]
> > Sent: Tuesday, October 30, 2007 2:26 PM
> > To: Sipping
> > Subject: [Sipping] draft-hilt-sipping-overload-03
> >
> > A new version of the overload control draft is available at:
> >
http://www.ietf.org/internet-drafts/draft-hilt-sipping-overload-03.txt
> >
> > The main changes to the -02 version are:
> > - Since this came up multiple times, I've added a more detailed
> > specification of an event package for overload control. This should
help
> > to understand the pros and cons of an event package vs. a new
header.
> > The discussion of a SIP mechanism for overload control is expanded
and
> > now focuses on a header vs. an event package.
> 
> To me, the idea of using a subscribe/notify for overload-control is
too
> problem-ridden to consider.
> 
> Pros:
> 1) You can tell a neighbor you're no longer overloaded, without
waiting
> for it to send you a request.  That to me is a real potential
advantage,
> since that neighbor may not send a request for a long time, e.g. if
you
> fully throttled it.
[TOLGA]Yes.
> 2) You can tell a neighbor you're overloaded before it sends a
request.
> Of course this requires sending *it* a request, and handling the
> retransmission timers for such, subscription state changes with
failure
> codes, and processing a response.  So I don't even see this as an
> advantage.
[TOLGA]For nodes exchanging messages at high rates this could be a real
benefit as well. If a neighbor sends a burst of messages after a silent
period, this could increase congestion level on a node from manageable
to collapse. Relying just on responses would make the sender not to take
action till the first response is received.
> 3) You can tell a neighbor loading info in general without getting
> requests.  But that's really just (1) and (2) above, since the only
time
> you really care about it is in those cases.  Unless we're saying this
> notification of load state is about load-balancing vs. throttling,
which
> this draft no longer does.
> 4) An event package can contain lots of info, should we want to do add
> such in the future.  Of course processing such info is itself a
burden.
> 5) You can tell non-neighbors your load - in other words it's a
mechanism
> useful for multi-hop or end-end overload-control, if we could figure
out
> how such a concept were to actually work from a SIP routing
perspective.
> (which I don't think we have)
> 6) You can challenge subscriptions and notifies, reject them, etc.
I'm
> not sure what benefits that provides for an overload control
mechanism,
> but it's there.
> 
> Cons:
> 1) You have to know who your neighbors are a priori. (ie, they're
> provisioned or register)
[TOLGA]If we exclude UA/Edge Proxy case, this probably would be more
manageable.
> 2) Subscriptions add state and processing.  If a proxy is overloaded
at
> boot up time (not unlikely), this is just more work it needs to do.
[TOLGA]I agree that this is a potential drawback but certain workarounds
could be possible, e.g. first performing all subscriptions and then
starting to accept traffic.
> 3) Notifies add processing, because you need to match it to a
subscription
> dialog, usage, and event type. (vs. a response matching at the
transaction
> layer)
[TOLGA]I would think in a reasonable engineered network the ratio of
NOTIFYs to actual messages should be very small.
> 4) If your system is architected with a distributed processing model,
for
> example one which follows the SIP layer model, you can handle sending
> responses at a very early/low stage.  You can't do so cleanly for
sending
> Notifies.  In other words the mechanism is at a higher layer than it
could
> be.
> 5) Notifying all neighbors is in effect the worst-case of the response
> proposal, every time.  With the response proposal, if a neighbor sends
a
> request, you tell it; if it doesn't, you don't.  There's a statistical
> benefit you gain by not having to assume all neighbors will send
requests
> all at the same overload time.  Even if they did, the net effect would
be
> the same as if you had told them all to not send you requests.
Imagine an
> outbound proxy with hundreds of thousands of endpoints.  Do all of
them
> subscribe, and the proxy tells them all not to send something when
it's
> getting overloaded?  Ouch.
[TOLGA]For a proxy dealing with lots of endpoints I agree that this
won't scale. OTOH, for other scenarios I don't think this would be a
significant problem and if necessary sending NOTIFYs can be throttled as
well. 
> 6) Proxies don't generally need to subscribe for anything, and aren't
> subscribed to.  They can, as a UA, but they didn't have to before this
> mechanism.  Every SIP device type handles responses, afaik.
> 7) There is a non-zero probability you won't have an active
subscription
> from a neighbor - for example, if it just booted up and is sending you
the
> subscribe and that request itself cannot be processed due to overload.
So
> now we need an overload mechanism for the overload mechanism.
> 8) Unless we're saying this mechanism only applies to proxies and not
> UA's, doing this subscribe from UA's exacerbates the Registration
> avalanche problem.  Reg-event (RFC3680) has proven this in the field
(not
> that it needed proving).
> 
> -hadriel
> 
> 
> _______________________________________________
> Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> This list is for NEW development of the application of SIP
> Use [email-masked] for questions on current sip
> Use [email-masked] for new developments of core SIP

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: Hadriel Kaplan <[email-masked]>
Date: Fri, 2 Nov 2007 20:39:19 -0400

Inline...

> From: Asveren, Tolga [mailto:[email-masked]]
> Sent: Friday, November 02, 2007 9:08 AM
> To: Hadriel Kaplan; Volker Hilt; Sipping

Why?  If you only have a few neighbors, the odds are they send you SIP requests quite often. (at least if overload could occur that should be the case)  Essentially my assumption is there's in inverse relationship between the number of neighbors and the traffic volume per neighbor.  If the traffic volume is higher, thereby creating the overload, by definition you will be getting requests with which to respond with overload control info.  Even if some neighbors are fairly quiet on a regular basis, then them being quiet means they don't contribute much to the overload.

The real advantage of the notify is to be able to clear the throttle, which I'm not actually sure is needed or useful - but if it is, it doesn't need to be done as a subscribe/notify.  If the throttle rate is set in the Via for responses, as I suggested in another email, we could also specify it could be set in the inserted Via for requests.  So all you'd need to do is send a request to the neighbor... any request.  Even a request you generate simply to notify the next-hop of this overload info.

> and core network cases separately (as it is also suggested by this

I think that's kinda hokey.  It would get hard to know which is the right mechanism automatically, so would require the admin to configure it.  And there are borderline cases - for example an edge proxy handling a few vs. hundreds/thousands of enterprise PBXs/IADs.

> without a need for a request.

You need that because you would be defining an on/off flow-control, with no timed decay, right?  Whereas the nice thing about percentages with decay is it should dampen the swings from being too busy to too quiet.  Otherwise you'd be flapping a lot.

> the path, could be initiated by intermediaries etc...

The thing is no one's explained how that model would really work.  If you could tell some device two hops away that you're busy, what good would it do that device?  Only if it knew a specific request was going to come to you would it be able to take action; and if it knew the request was going to come to you specifically in advance, why did it send it to an intermediary proxy?  What if the intermediary proxy would have been capable of routing it an alternate way?

-hadriel

---

From: "Asveren, Tolga" <[email-masked]>
Date: Mon, 5 Nov 2007 13:22:37 -0500

Hadriel,

Please see below for some comments/questions.

  Thanks,
  Tolga

> Sent: Friday, November 02, 2007 8:39 PM
> To: Asveren, Tolga; Volker Hilt; Sipping
> Why?  If you only have a few neighbors, the odds are they send you SIP
> requests quite often. (at least if overload could occur that should be
the
> case)  Essentially my assumption is there's in inverse relationship
> between the number of neighbors and the traffic volume per neighbor.
If
> the traffic volume is higher, thereby creating the overload, by
definition
> you will be getting requests with which to respond with overload
control
> info.  Even if some neighbors are fairly quiet on a regular basis,
then
> them being quiet means they don't contribute much to the overload.
[TOLGA]I was referring to the scenario of large number of SIP endpoints,
which rarely send requests. Congestion happens due to aggregate message
rate, not because a few endpoints send requests in a more than usual
rate but because more than usual number of endpoints send requests in a
short window of time.  
> 
> The real advantage of the notify is to be able to clear the throttle,
> which I'm not actually sure is needed or useful - but if it is, it
doesn't
> need to be done as a subscribe/notify.  If the throttle rate is set in
the
> Via for responses, as I suggested in another email, we could also
specify
> it could be set in the inserted Via for requests.  So all you'd need
to do
> is send a request to the neighbor... any request.  Even a request you
> generate simply to notify the next-hop of this overload info.
[TOLGA]In general I agree that simplicity is an important parameter so
that the solution is implemented/used in practice. And I also agree that
subscribe/notify is the less simple solution. OTOH simplicity usually
implies lack of extensibility as well but I am not sure how much
flexibility we would need down the road. 
> I think that's kinda hokey.  It would get hard to know which is the
right
> mechanism automatically, so would require the admin to configure it.
And
> there are borderline cases - for example an edge proxy handling a few
vs.
> hundreds/thousands of enterprise PBXs/IADs.
[TOLGA]Agreed that it is less than ideal and may require some network
engineering but if the defined mechanism doesn't work for large number
of endpoints there is nothing to do.
> You need that because you would be defining an on/off flow-control,
with
> no timed decay, right?  Whereas the nice thing about percentages with
> decay is it should dampen the swings from being too busy to too quiet.
> Otherwise you'd be flapping a lot.
[TOLGA]Oscillation could be limited/prevented by providing appropriate
hysteresis, and defining a few (rather than just two) congestion levels
with well-defined semantics would prevent swinging from too busy to too
quiet and could be more suitable to loadbalance/prevent overload to
occur. I tend to disagree that load balancing is orthogonal to
congestion control. Loadbalancing could be a useful tool to prevent
congestion to happen. 

I would prefer to see some simulation results before deciding how well
percentage based overload mechanism works. Crucial part is how a node
determines these percentage levels. Let's say I want to use the load on
the CPU as overload criteria. CPU is 70% busy. To me, the only safe
value to use as drop rate is 0%. Why should any messages be dropped, if
there are 30% idle CPU resources? So, this could translate easily to
something like "if cpu_idle < 10 drop_rate=100% else drop_rate=0%". 

The percentage rate is dependent on the unknown future message rate.
Deciding for the decay value is also a guess. 

I also don't know how buffering at the sender could be achieved with
percentage rate mechanism. Any message buffered would add to the future
message rate (making it to be different than the message rate used by
the receiver when deciding for the drop rate, even if we assume a steady
message rate on the sender).

With so many unknowns, relying on simulations makes sense to me (and
hopefully simulation software source code and detailed description of
all scenarios will be documented somewhere).

> The thing is no one's explained how that model would really work.  If
you
> could tell some device two hops away that you're busy, what good would
it
> do that device?  Only if it knew a specific request was going to come
to
> you would it be able to take action; and if it knew the request was
going
> to come to you specifically in advance, why did it send it to an
> intermediary proxy?  What if the intermediary proxy would have been
> capable of routing it an alternate way?
[TOLGA]Just from routing perspective I agree with what you say, but
there may be intermediaries necessary for reasons other than routing,
e.g. to implement certain controls/checks, policing functionality, to
interface with other administrative domains, protocol reparation etc...
IMHO, it is preferred to stop traffic as early as possible if it is
known that it will be dropped eventually.

---

From: Volker Hilt <[email-masked]>
Date: Wed, 21 Nov 2007 17:41:15 -0500

Hadriel,

thanks for your comments. More inline.
> 
> The question I have is on the hop-by-hop SIP response feedback
> mechanism.  You define the overload info to be in a new header in the
> response.  Because this can then incorrectly be passed back more than
> one hop, you have to add an address in it, and rules around deletion.
> What I don't get is why you don't just put the info in the Via
> header, as a param.
> 
> So for example, if a proxy supports this mechanism, it inserts an
> "oc" param in the Via it inserts in requests it sends, which tells
> the next-hop that it supports this mechanism.  And when the next-hop
> wants to put the control info in the upstream responses, it sets a
> value for the param in the top-most Via it received (ie, now
> "oc=20"), assuming the Via had the param.  So in theory the next-hop
> will never send oc info unless the previous-hop supported it, and
> even if it did the previous-hop would always remove it since it
> strips its own Via.
> 
I think having a parameter in the Via header that indicates support for 
overload control is certainly useful. It would, for example, enable a 
server to send overload control information only to senders that support 
it and possibly use other techniques for senders that don't.

Inserting the overload control status itself as a param in the Via 
header would indeed avoid the problem of address matching, which I agree 
is very desirable. A concern might be an overloading of the Via header.

> Thus this would provide explicit support indication, no new headers,
> no guessing of addresses, no verifying of addresses, etc.
> 
Yes, I agree.

Volker

>> -----Original Message----- From: Volker Hilt
>> [mailto:[email-masked]] Sent: Tuesday, October 30, 2007 2:26
>> PM To: Sipping Subject: [Sipping] draft-hilt-sipping-overload-03
>> 
>> The main changes to the -02 version are: - Since this came up
>> multiple times, I've added a more detailed specification of an
>> event package for overload control. This should help to understand
>> the pros and cons of an event package vs. a new header. The
>> discussion of a SIP mechanism for overload control is expanded and 
>> now focuses on a header vs. an event package. - Changed the header
>> specification from a 'Load' header to an 'Overload-Control' header
>> since the purpose of the header is to convey overload control
>> feedback and not current load. - Removed the option to report load
>> from the header. - Added a reference to Resource-Priority for
>> message prioritization. - Added a section on overload control
>> algorithms (this is an open issue). - Moved discussion of local
>> overload control. - Added on open issue on the need of a fading
>> mechanism for the control value. - Removed sections on calculating
>> load levels and operations and management. - Lots of other
>> clarifications and editorial changes.
>> 
>> Thanks,
>> 
>> Volker
>> 
>> 
>> 
>> _______________________________________________ Sipping mailing
>> list  https://www1.ietf.org/mailman/listinfo/sipping This list is
>> for NEW development of the application of SIP Use
>> [email-masked] for questions on current sip Use
>> [email-masked] for new developments of core SIP
> 
> 
> _______________________________________________ Sipping mailing list
> https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW
> development of the application of SIP Use
> [email-masked] for questions on current sip Use
> [email-masked] for new developments of core SIP

---

From: Volker Hilt <[email-masked]>
Date: Thu, 22 Nov 2007 15:09:50 -0500

Asveren, Tolga wrote:
>>> IMHO). 

The semantics of these congestion levels is really not well defined. 
What does "if there is an alternate node, prefer it" it mean? Am I still 
allowed sending traffic to this node? If so, how much traffic should I 
send - can I still send everything? For the level "if there is an 
alternate node, use it", the net result depends on the network topology. 
If there is an alternate node, it will turn traffic completely off, if 
not it will do nothing. For an overload control algorithm to produce 
reasonable results, a well-defined feedback loop which has predictable 
effects seems important. This feedback could be a percentage, a request 
rate or a window size, as discussed in the draft. But it has to have a 
well-defined meaning.

> Using an explicit congestion notification message, e.g.
> NOTIFYs

As discussed, this begs the question if you would want to update the 
congestion level when there are no requests.

> determines these percentage levels. 

Yes, determining the feedback is crucial. In fact, there is a large 
amount of existing work on overload control algorithms and, as a 
consequence, there are a number of overload control algorithms (and new 
ones might be invented). For this reason, I think it is important to 
have a well-defined feedback mechanism, with feedback that leads to a 
predictable effect. With this it is possible to start off with a 
baseline algorithm that works well and to plug in new algorithms if 
needed/desired. I agree that we need to have simulation results that 
demonstrate the performance of the mechanism and an algorithm.

> Let's say I want to use the load on
I don't think it is that clear cut and there are different algorithms 
that, for example, factor in other input parameters such as the queue 
length.

> all scenarios will be documented somewhere).
I agree.

>>> the path, could be initiated by intermediaries etc...

Although sub/not would enable feedback to be transparently passed though 
proxies that do not support overload, I think feedback in a response can 
be passed along as well if this is desired in a certain topology.

> known that it will be dropped eventually.
I also don't see how multi-hop overload control is going to work except 
maybe in a few specific cases in which there is a tightly controlled 
environment where the topology and all routing policies in this topology 
are exactly known a priory. In these specific cases, a device can still 
choose to pass overload feedback it has received from a downstream 
neighbor on upwards (it should know what it is doing though).

Thanks,

Volker