Subject: [Sipping] Overload/Congestion mechanisms - design choices and issues

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 27 Jun 2006 16:50:58 -0400

I've submitted an update to the overload requirements document. I've 
made some changes based on list feedback since the last meeting. Until 
the document appears in the archives, you can pick it up at:

http://www.jdrosen.net/papers/draft-rosenberg-sipping-overload-reqs-01.txt

I didnt submit as a WG -00 since, AFAIK, we have no charter item for 
this work and there can't be a work item for it until that happens.

We also are fortunate to have had two documents produced that propose 
solutions to meet the requirements:

http://www.ietf.org/internet-drafts/draft-malas-sipping-congestion-header-01.txt
http://www.ietf.org/internet-drafts/draft-hilt-sipping-hopbyhop-overload-00.txt

I think these two drafts are a good start. However, there are many 
issues to be addressed that are not covered by the drafts. Here are some 
thoughts on some of the issues and design choices involved here:

ISSUE 1: inband with SIP vs. parallel protocol

One approach to solving the overload mechanism is to extend SIP with new 
information to report overload, as both drafts have done. Another 
approach is to design a parallel protocol that runs between elements 
which you could apply to any other protocol, SIP or otherwise. This is 
the approach proposed in GOCAP, which Steve posted a reference to.

My personal opinion is that I prefer a SIP extension. There are several 
reasons for this. The main one is that a parallel protocol is inherently 
more complicated, as it requires its own syntaxes, state machines, 
messages, and so on, and also adds complexity due to the need to define 
interfaces into the elements whose load is being managed. As such, I 
think it raises the bar much higher for implementation, and I think this 
protocol should be simple as possible to drive implementation. We 
already have some overload management in SIP; it just doesn't work well. 
Its totally appropriate to extend SIP to improve whats there.

ISSUE 2: how to make sure that overload information passed in SIP 
messages is only processed by the upstream server

This issue is largely a SIP machinery problem, and there are several 
solutions. A few are proposed in Volker's draft. I personally prefer the 
approach of including congestion information in a response, and using an 
IP address in that header field to indicate the server for which 
overload is being reported.

ISSUE 3: what is the nature of the overload information

Volker proposes a value between 0 and 100, and Daryl proposes a 
multidimensional value from 0 to 5. You really can't answer this design 
issue until you answer the next, which is one of the two BIG design 
issues I think.

ISSUE 4: how does an element respond to overload information

When an upstream element has a set of load indicators from downstream 
elements, how does it use that information to decide which server to 
send it to, or whether it should reject the request. How does it do this 
in a way that keeps the network throughput up? How does it make sure 
that it still distributes load in a reasonable way across the downstream 
elements?

This is big isssue #1.

ISSUE 5: how does a downstream element determine which incoming traffic 
to process based on upstream server?

There are many aspects to this issue. One of the harder ones is a 
policing function. Think of a server has having input queus, one for 
each upstream element. The server processes requests from an upstream 
element by taking a request off that queue. What is the queuing 
discipline implemented by this server? If you just do something basic 
like round-robin, you end up in an interesting situation of rewarding 
upstream elements which DONT implement the mechanism! There needs to be 
a way of verifying that elements are sending at a reduced rate based on 
the congestion indications, and of determining a rate to allocate to 
upstream elements which don't support it.

This is big issue #2.

ISSUE 6: What is the temporal scope of reported congestion information?

When an upstream element receives congestion information, how long does 
it consider it valid for? What does it do when it receives no 
information from a downstream element? What does it do if it cannot 
reach a downstream element?

ISSUE 7: How does the system work for upstream elements which are not 
"servers"

I tend to think of the overload mechanism as having two modes. One mode 
is where an element knows of all of the upstream elements sending it 
traffic. It uses this knowledge to figure out what kind of rate to 
proportionally allocate to each upstream element.

In the other mode, we have somethign like an edge proxy, which has many 
many clients that connect to it. How does that edge element implement 
overload handling? Certainly it does; if the network as a whole is 
overloaded, you need to push back to the actual source so that the 
overall load being sent into the network is reduced. That source are the 
endpoints themselves. In this mode, the existing 503 mechanism works 
better. But, you want the edge proxy to push back gradually, sending 503 
(or something else) to some fraction of the clients, so that you reduce 
the input load proportionally.

I'm sure there are many more issues and many other ways to look at them. 
Hopefully this helps get discussions going on what a mechanism document 
needs to cover.

Thanks,
Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.                   600 Lanidex Plaza
Cisco Fellow                                   Parsippany, NJ 07054-2711
Cisco Systems
[email-masked]                              FAX:   (973) 952-5050
http://www.jdrosen.net                         PHONE: (973) 952-5000
http://www.cisco.com

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: Volker Hilt <[email-masked]>
Date: Wed, 28 Jun 2006 00:30:57 -0400

Jonathan,

thanks for the feedback on this topic. Some thoughts on the issues 1 - 5 
are inline.
> Its totally appropriate to extend SIP to improve whats there.
I fully agree (obviously).

> overload is being reported.
I agree. Using a new header field seems preferable since it clearly 
separates overload control from other mechanisms.

> issue until you answer the next, which is one of the two BIG design 
I think fundamentally there are two distinct pieces of information a 
proxy can report (the draft may have been a bit overly simplistic on this):
- its current resource usage level AND / OR
- the target load the proxy wants to receive (e.g. in messages per 
second) or the fraction of traffic that should be discarded/redirected.

The first piece of information enables an upstream neighbor to make load 
balancing/overload control decisions at its discretion. The second piece 
specifically tells the upstream neighbor *when* to react and to which 
extend.

I believe that both pieces of information are useful. The draft defines 
an implicit relation between both by setting thresholds but it might 
make sense to explicitly report both. This would decouple the tasks of 
load reporting and throttling.

> that it still distributes load in a reasonable way across the downstream 
There is yet another type of information that can be reported: a proxy 
could explicitly tell the upstream neighbor *what* to do. For example, 
the proxy could tell the upstream neighbor to redirect a certain 
fraction of traffic to a specific proxy (e.g. a standby proxy that helps 
to resolve overload) or that a fraction of traffic should be rejected 
(if the proxy knows that no other paths will succeed).

This option would only make sense if the overloaded proxy has 
information about how to resolve the overload condition the upstream 
neighbor can't access. In addition it requires the already overloaded 
proxy to generate these directions to upstream proxies, which may differ 
based on the request targets.

The load status reports a proxy receives from its downstream neighbors 
should enable it to distributed load across them. This and other 
information used for routing should provide a basis for making 
reasonable decisions about which server to route to or whether to reject 
a request.

> upstream elements which don't support it.
Yes, this is an interesting problem. A proxy that implements overload 
control should certainly be rewarded and not punished. An obvious 
solution might be to send 503s to upstream neighbors that do not react 
to overload reports. However, this assumes that a proxy can identify 
these proxies and it will bring back the known problems of 503. On the 
other hand, since these proxies don't support overload control, this 
should not come as a total surprise.

Thanks,

Volker

---

From: "Jeroen van Bemmel" <[email-masked]>
Date: Wed, 28 Jun 2006 09:11:12 +0200

Volker, Jonathan,

"load balancing" and "overload control" are two related but different 
issues. I get the feeling that we need to get the distinction clear before 
plunging into this discussion. I don't pretend to know the answers, but let 
me try and give it a go

"Load balancing" is about resource optimization and scaleability. What makes 
load balancing for SIP (or proxy systems in general) difficult, is that 
behind a SIP element there is typically a cloud of other systems. An 
upstream SIP peer typically has no knowledge of the structure of this cloud, 
especially when it is in another domain. My hunch is that to enable a peer 
to make good load balancing decisions, it will need load information that is 
not only based on its direct next hop but also on the systems behind that. 
So a major issue (for load balancing) is the propagation and aggregation of 
this information (and the underlying model).

"Overload/congestion control" is about reducing the load on a particular 
system that is getting more than it is capable of handling. The two 
mechanisms that can be applied here are traffic diversion and throttling. To 
decide that overload control is needed, information from a single element 
(ie the system that is experiencing overload) suffices. However, to enable 
good diversion/throttling decisions, you may need the same information that 
is used for load balancing.
I wager that the upstream system does not need to know exactly which part of 
the peer system (disk, network, etc) is experiencing overload. This 
information would be useful for diagnosis, but not to decide what (ie divert 
and/or throttle) to do. Since the system under overload would know best what 
effect certain decisions would have on its overload status (and perhaps also 
on the rest of its network), I reckon that the system under overload should 
make a suggestion to the upstream peer on what to do. In other words, not 
"help, I am 98% loaded, and it is because my disks are failing" but concrete 
suggestions / parameters on what to do (see next).

Regarding traffic diversion: what we currently have is forwarding or 3xx 
responses. What we might need here is what eg also Diameter offers, what I 
would call "selective temporary redirection". In Diameter, you can say "for 
the next 10 minutes, send all traffic for {this session, this domain, this 
user, ...} to this and that node". You could do the same for SIP, the 
selection criteria could e.g. be pre-established, standardized profiles or 
fully dynamic.
A solution would probably need to support both implicit and explicit 
alternative hosts, ie the system in overload must be able to express "send 
your traffic to these nodes instead" but also "to any node other than me"

For throttling we have 503, which only applies to a single request. What we 
might need here is a mechanism that allows a node to respond with 
classification criteria (for example "all dialog creating requests", or 
something more dynamic like IMS initial filter criteria) and something like 
shaper settings, eg minimum intervals between requests, max requests per 
time interval, etc. Again, such a filter would probably have a suggested 
time period

My 2 cents,

Jeroen

---

From: Indra Widjaja <[email-masked]>
Date: Wed, 28 Jun 2006 10:25:02 -0400

Jeroen:
Please see below.
Indra

Jeroen van Bemmel wrote:

> issues. I get the feeling that we need to get the distinction clear 
> before plunging into this discussion. I don't pretend to know the 
> answers, but let me try and give it a go

I couldn't agree more with your statement that load balancing and 
overload control are related but different issues.
I'd say that in general they are not related, but in this context, they 
can be argued to be somewhat related.

>
> "Load balancing" is about resource optimization and scaleability. What 
> makes load balancing for SIP (or proxy systems in general) difficult, 
> is that behind a SIP element there is typically a cloud of other 
> systems. An upstream SIP peer typically has no knowledge of the 
> structure of this cloud, especially when it is in another domain. My 
> hunch is that to enable a peer to make good load balancing decisions, 
> it will need load information that is not only based on its direct 
> next hop but also on the systems behind that. So a major issue (for 
> load balancing) is the propagation and aggregation of this information 
> (and the underlying model).

Your hunch is also correct that load balancing based on the information 
(e.g., load) behind the next hop will lead to better performance IF this 
information
could be obtained. We have verified this in our simulation experiments. 
The problem is how to get this "behind the cloud" information.

>
> "Overload/congestion control" is about reducing the load on a 
> particular system that is getting more than it is capable of handling. 
> The two mechanisms that can be applied here are traffic diversion and 
> throttling. To decide that overload control is needed, information 
> from a single element (ie the system that is experiencing overload) 
> suffices. However, to enable good diversion/throttling decisions, you 
> may need the same information that is used for load balancing.
> I wager that the upstream system does not need to know exactly which 
> part of the peer system (disk, network, etc) is experiencing overload. 
> This information would be useful for diagnosis, but not to decide what 
> (ie divert and/or throttle) to do. Since the system under overload 
> would know best what effect certain decisions would have on its 
> overload status (and perhaps also on the rest of its network), I 
> reckon that the system under overload should make a suggestion to the 
> upstream peer on what to do. In other words, not "help, I am 98% 
> loaded, and it is because my disks are failing" but concrete 

Traffic diversion done in a sensible manner is equivalent to load 
balancing, thus confirming the relatedness, right?
I agree with you that the system under overload should make a suggestion 
to its upstream peer to perform the overload control. In fact, if the 
system under overload
performs its own overload control, the overall system may likely 
experience congestion collapse.
To take it to the next level, it is even better, IF it can done, for the 
system under overload to make suggestion to its ingress proxy (this is 
probably pushing too much).

>
> Regarding traffic diversion: what we currently have is forwarding or 
> 3xx responses. What we might need here is what eg also Diameter 
> offers, what I would call "selective temporary redirection". In 
> Diameter, you can say "for the next 10 minutes, send all traffic for 
> {this session, this domain, this user, ...} to this and that node". 
> You could do the same for SIP, the selection criteria could e.g. be 
> pre-established, standardized profiles or fully dynamic.
> alternative hosts, ie the system in overload must be able to express 
> "send your traffic to these nodes instead" but also "to any node other 
> than me"
>
> For throttling we have 503, which only applies to a single request. 
> What we might need here is a mechanism that allows a node to respond 
> with classification criteria (for example "all dialog creating 
> requests", or something more dynamic like IMS initial filter criteria) 
> and something like shaper settings, eg minimum intervals between 
> requests, max requests per time interval, etc. Again, such a filter 
> would probably have a suggested time period
>
> My 2 cents,
>
> Jeroen
>
>
>

---

From: Volker Hilt <[email-masked]>
Date: Wed, 28 Jun 2006 16:36:49 -0400

Jeroen,

> (ie divert and/or throttle) to do. 

I agree. The upstream neighbor of a proxy does not need to know which
part or resource of the proxy is overloaded. It just needs to know that
the proxy is overloaded and not able to process all messages.

> Since the system under overload would 
> know best what effect certain decisions would have on its overload 
> status (and perhaps also on the rest of its network), 

I don't understand this assumption. The upstream neighbor of a proxy
makes routing decisions in the non-overloaded case. Thus, it knows about
the next hop proxies it can route a message to (e.g. via DNS) and can
make routing decisions based on this and possibly other information or
local policy. Since the upstream neighbor of a proxy under load makes
routing decisions in the regular case, it seems natural to have this
proxy make the routing decision if the downstream proxy is overloaded.

This also has the advantage that the overloaded proxy does not have to
worry about finding a good alternative target. It can leave this
decision to the less loaded upstream neighbor (which would do the same
decision in the non-overload case). This reduces the processing required
for the overloaded proxy, which will help it to recover.

> I reckon that the 
> system under overload should make a suggestion to the upstream peer on 
> what to do. In other words, not "help, I am 98% loaded, and it is 
> because my disks are failing" but concrete suggestions / parameters on 
> what to do (see next).
> 
> responses. What we might need here is what eg also Diameter offers, what 
> I would call "selective temporary redirection". In Diameter, you can say 
> "for the next 10 minutes, send all traffic for {this session, this 
> domain, this user, ...} to this and that node". You could do the same 
> for SIP, the selection criteria could e.g. be pre-established, 
> standardized profiles or fully dynamic.
I think there are a number of issues with an explicit redirection:
- the overloaded proxy would need to monitor the proxy it is redirecting
traffic to to avoid that it is redirecting traffic to a proxy that is
overloaded as well. This seems to be a lot of work for the proxy that is
already overloaded.
- even if the proxy does this monitoring, the explicit redirection can
cause oscillation (something we wanted to avoid in the first place).
Assume a proxy redirects traffic to another proxy that still has
capacities left. This would cause this proxy to go into overload state.
Meanwhile the previously overloaded proxy is freed up so that other
proxy (which is now overloaded) can redirect traffic back again. Thus
the traffic would hit the first proxy again and we would be back at
square one.
- as you mentioned, the redirection may be per-session/domain/user,
which by itself seems to be a complex task for an overloaded proxy.

> For throttling we have 503, which only applies to a single request. What 
> we might need here is a mechanism that allows a node to respond with 
> something more dynamic like IMS initial filter criteria) and something 
> like shaper settings, eg minimum intervals between requests, max 
> requests per time interval, etc. Again, such a filter would probably 
> have a suggested time period
> 
The problems of 503 are well documented. The header field we have
proposed enables a proxy to ask the upstream neighbor to lower the
amount of traffic that is forwarded downstream.

Thanks,

Volker