Subject: RE: [Sip] PUBLISH message rates

From: [email-masked]
Date: Fri, 14 Nov 2003 05:50:04 +0200

I agree with Henning. If there is ever 1 subscriber and 1 publisher, then this rate limiting makes sense. But in many cases, especially presence, there are n publishers.

If there are 5 publishers for a package that limits the rate to 5 per second. There will be 20 NOTIFYs and 100 PUBLISHs sent in one minute . How does that help?

The only way to do this right is to do what RTCP does with report interval, and that is to adjust the rate of PUBLISH using some formula that takes into account the number of publishers at a given time. The more publishers that join, the less the rate at which PUBLISH is sent at. The compositors need to communicate that calculated rate to the publishers somehow. Do we want to make it that complicated? If not, then I suggest adding text the says if the compositor is overloaded, it can send a 5xx response with retry-after.

Regards,
Hisham

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of ext
> Henning Schulzrinne
> Sent: Thursday, November 13, 2003 7:01 PM
> To: Niemi Aki (NMP/Helsinki)
> Cc: Jonathan Rosenberg; Adam Roach; [email-masked]'
> Subject: Re: [Sip] PUBLISH message rates
> 
> 
> > Even this isn't solved using the default rate limit in 
> notifications, 
> > because it will only apply to a single notifier. Typically, 
> an endpoint 
> > has many active subscriptions, so the aggregate rate may still be 
> > considerable. The event-throttle work attempts to address this.
> 
> True (and somewhat beyond the topic of this discussion). One could 
> defend the rate limit in the NOTIFY case in arguing that at least the 
> poor resource-constrained UA knows that it is subscribing to N events 
> and thus can upperbound the event rate by summing them up. 
> The PA can't, 
> since it has generally no way of knowing how many entities 
> will publish 
> for a single event.
> 
> 
> > 
> >> However, in most cases of interest, the PA and the publisher are 
> >> closely related. In addition, since the end system has complete 
> >> control over the rate of sending PUBLISH, the 'protect low-end end 
> >> systems' arguments doesn't apply.
> > 
> > 
> > Right, it would mainly be about protecting the SIP routing 
> fabric and 
> > the composer.
> 
> Since it doesn't in any way limit the aggregate rate, this is a very 
> loose way to protect anything. As noted, we don't limit the 
> individual 
> rate of sending email in the Internet by protocol spec to protect the 
> aggregate MTA population. (Obviously, individual ISPs do 
> sometimes apply 
> rate limits for email submission to keep spammers out of 
> their network, 
> but that's a local policy decision, not a protocol decision.)
> 
> 
> 
> > I guess these recommendations would anyway be 
> "hand-waiving" with or 
> > without a hard rate limit recommendations. It's not like 
> the rate could 
> > ever really be enforced in a meaningful way.
> 
> We have Congress to pass meaningless declarations of good 
> intent, so no 
> need for the IETF to get into that business :-)
> 
> > 
> > I think we can agree that documenting this issue is a good 
> thing. Also, 
> > I would say explaining how a composer can use the SIP level 
> push-back 
> > should be in the draft. Whether to have a concrete rate 
> limit or not, 
> > I'm ok either way. I can suggest text that has a SHOULD strength 
> > recommendation that publishing EPAs pay attention to the 
> rate at which 
> > they send updates.
> 
> My basic test for any protocol spec normative statement is 
> whether I can 
> tell if somebody is violating the spec by observing external 
> behavior. 
> I'm unclear how I can measure 'paying attention' - I have a 
> hard enough 
> time with telling whether my students are ...
> 
> 
> 
> _______________________________________________
> Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
> This list is for NEW development of the core SIP Protocol
> Use [email-masked] for questions on current sip
> Use [email-masked] for new developments on the application of sip
> 

_______________________________________________
Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [email-masked] for questions on current sip
Use [email-masked] for new developments on the application of sip

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 13 Nov 2003 23:09:07 -0500

[email-masked] wrote:

> The only way to do this right is to do what RTCP does with report
> interval, and that is to adjust the rate of PUBLISH using some
> formula that takes into account the number of publishers at a given
> time. The more publishers that join, the less the rate at which
> PUBLISH is sent at. The compositors need to communicate that
> calculated rate to the publishers somehow. Do we want to make it that
> complicated? If not, then I suggest adding text the says if the
> compositor is overloaded, it can send a 5xx response with
> retry-after.

And that probably assumes some reasonably uniform generation pattern for 
the events. The whole point of events is that they are unpredictable, so 
that rate smoothing would have to be applied over longish time 
intervals, further complicating the whole system. Unlike for filters 
that are subscriber-specific, the publishers would also have to guess 
what notifications are droppable - which they can only do if they know 
what kind of things the subscribers care about most, even assuming that 
they have uniform tastes.

I can see that for some continuous-change events, this is at least 
plausible. For geospatial notifications that have change thresholds, I 
can easily compute the minimum displacement threshold across subscribers 
and adjust the publication rate accordingly. This would not be a simple 
rate limit, as the rate would depend on my movement pattern (slow if 
circling the block, fast if driving at highway speeds). Thus, you'd 
almost want to feed back some kind of 'distilled' subscription to the 
publisher - I would find that far more useful than some hard 
'messages/second' limit.

---

From: "Christian Stredicke" <[email-masked]>
Date: Fri, 14 Nov 2003 06:07:35 -0600

> If there are 5 publishers for a package that limits the rate to 5 per
> second. There will be 20 NOTIFYs and 100 PUBLISHs sent in one minute .
How
> does that help?

Limiting the rate is about statistics. There is definitely a correlation
between the rate limitation and the actual rate. In other words: it has
an effect. Ok you can't make any guarantees on the actual performance,
but it helps to reduce the network load.

In the above example, we are talking about say 1000 messages without
limitation and 100 with limitation. I would call that a success.

Anyway I would not be too religious about this. I think practically it's
not really relevant as implementers should keep an eye on rate anyway
even without explicit rate negotiation. 

Christian

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 14 Nov 2003 09:29:38 -0500

>> compositor is overloaded, it can send a 5xx response with

Of course use of 5xx with retry-after wastes the effort the compositor 
has already expended in processing the publish up to that point.

It might be interesting to make a change permitting use of the 
Retry-After header with a 2xx response. Then the compositor could accept 
  a publish and at the same time ensure that the next one doesn't come 
too soon. This wouldn't be useful in all cases, but maybe in some.

	Paul