Subject: RE: [NSIS] Any Router Alert Option influence on router performanc e?

From: "Hancock, Robert" <[email-masked]>
Date: Thu, 20 Nov 2003 03:42:28 -0000

Michael, Dave,

these inputs are extremely interesting, thanks.

some perspectives:

i think we are at least as interested in the effect of the RAO
on router processing time as on e2e delay. the second is the cost
for the signalling; the first is the cost for the router. if
widespread adoption of NSIS (e.g. for middlebox signalling) 
causes core router CPUs to collapse processing messages they don't
care about, we will have failed.

i think it's clear that we don't want to introduce a new option
that needs to be processed in software (for the reasons Dave
gives). we need to use an existing interception mechanism - either
based on protocol number or RAO - to do selective interception.

3175 uses protocol number re-writing to avoid RAO processing for
the microflow messages. my impression is that we won't be able
to get enough protocol numbers to achieve the granularity of
selection that we will probably need (and indeed, 3175 suggested
RAO values to go further).

my impression of the reasoning in 2113 is that the RAO is well-
enough designed that you *should* be able to include support for
processing it (and filtering some set of RAO values) in any 
hardware you build. it has the right degree of simplicity 
(fixed 32 bit format) and flexibility (16 bits of value space).

the question is, is that what has actually happened? or do we 
need to rely on using some set of protocol numbers, and if so,
how many?

cheers,

robert h.

> -----Original Message-----
> From: Michael Welzl [mailto:[email-masked]]
> Sent: Thursday, November 20, 2003 12:25
> To: David R. Oran; Geib, Ruediger; [email-masked]
> Subject: AW: [NSIS] Any Router Alert Option influence on router
> performance?
> 
> 
> Hi all,
> 
> > > should anybody of you be aware of measurements
> > > comparing the influence of processing "router
> > > alert option" packets on router processing as
> > > compared to complete absence of these packets,
> > > I'd appreciate information or a URL. The
> > > background isn't to prevent such messages. I'm
> > > interested in learning more about the level
> > > (if any) where they start to hurt.
> 
> This lack of knowledge has been annoying me for a long
> time - so we did measurements. The idea was not to test
> specific equipment and not to check its behavior under
> stress (that would depend on the brand, I suppose) but
> rather to get a general idea of the additional delay
> you get by adding ANY option (we used NOP) to a ping in
> today's Internet.
> 
> 
> > I don't have specific measurements, but without 
> special-case hardware 
> > support for the specific function being requested by RA, 
> the performance 
> > penalty is at least 100x (if you do nothing other than decide to 
> > ignore the 
> > RA) to much more depending on how heavy-weight the actual 
> processing in 
> > software would be.
> 
> At least 100x to much more??? I wonder where these data come from.
> 
> We did two measurements, one in 2001 and one in 2003, and
> we traversed 7something000 and approx. 80000 different router
> addresses (100 pings each with and without options, a break
> of a second in between). Then we did some statistics (applied
> weights based on the frequency of occurence of certain router
> addresses, variance etc. ... the total result is close to 1%
> of additional delay per router (I think it was 0.8% in 2001
> and 1% in 2003 or vice versa).
> 
> Now this is not representative for "the Internet", of course,
> but it's a measurement, so it's at least SOMETHING. I'm working
> on the documentation, and I'll make it available as a technical
> report via the web as soon as possible (maybe in the next 1 or
> 2 weeks). For those of you who are greedy for diagrams   :)
> here's a VERY VERY VERY VERY preliminary link:
> http://www.welzl.at/tmp/option.html
> (BTW - it says "2002" on this page, but that's wrong, it was 2001)
> 
> The tech-rep will be available from:
> http://www.welzl.at/publications/
> 
> I believe that there's a tendency to exaggerate about the impact
> of Slow Path processing, and this prevents any idea based on
> IP options to move forward. On the other hand, judging from your
> email address, you should know more details abour option processing
> than I do - so I really wonder where these data come from.
> 
> Perhaps it's:
> difference in delay when looking at processing in a router
> vs.
> difference in end-to-end delay
> 
> 
> The latter - how the overhead relates to the RTT - is more
> interesting to me.
> 
> 
> Cheers,
> Michael
> 
> 
> _______________________________________________
> nsis mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/nsis
> 

_______________________________________________
nsis mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nsis

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 20 Nov 2003 09:36:25 -0500

To add to this: I think we need to distinguish at least three issues:

(1) Delay added by RAO processing. This only matters if the delay is 
large (milliseconds) and if we include RAO in data packets, YESSIR-style.

(2) CPU load caused by processing RAO where the packet is then ignored.

(3) CPU load caused by processing RAO where the packet is processed. In 
that case, I suspect that the RAO processing is trivial compared to the 
rest of the packet processing and the important consideration is the 
total throughput of the router CPU in handling signaling messages. As 
long as the per-message overhead is reasonable, router vendors and 
Moore's law will make sure that routers have enough CPU horsepower. 
(Indications [see Darmstadt RSVP work] are that even old CPUs can handle 
tens of thousands, if not 100,000, messages a second, translating into 
approximately 3 million concurrent sessions at 30 second refresh. Even 
at OC-192 and 1 million sessions, every single bit of the channel would 
have to be taken up by 10 kb/s voice connections to reach that level.) 
Aggregation obviously avoids this problem, as well.

Thus, only (2) matters in practice, as long as we keep the processing 
overhead for NSIS messages reasonable. Matching on RAO and the flag and 
detecting that there is nobody home for that combination should be 
doable at high speed, as Robert indicates below.

Henning

---

From: Xiaoming Fu <[email-masked]>
Date: Sun, 23 Nov 2003 20:59:32 +0100

I also think (2) is something of importance in the sense of layering: If 
we assume every NSIS node needs to worry about control plane (relatively 
slow) functionality for any incoming NSIS packet, this can introduce 
additional RTT e.g., for firewalling NSLP, since it might only manage 
control state in a limited number of node(s) while traversing several 
(possibly tens of) other NSIS nodes along the path.

The question is, can/should we engineer something only in forwarding 
plane - or a "faster" control plane - with NTLP packets by default 
(without intercepting into the content), and when the encoded NSLP type 
is recognized in a node, the NSIS packet is processed in control plane? 
If this is a desireable feature RAO plus filtering flag(s) can be a way.

Regards,
Xiaoming

Henning Schulzrinne wrote:
>> widespread adoption of NSIS (e.g. for middlebox signalling) causes 
>> core router CPUs to collapse processing messages they don't
>> processing it (and filtering some set of RAO values) in any hardware 
>> you build. it has the right degree of simplicity (fixed 32 bit format) 
>> and flexibility (16 bits of value space).
>>
>> the question is, is that what has actually happened? or do we need to 
>> rely on using some set of protocol numbers, and if so,
>> how many?
>>
>> cheers,
>>
>> robert h.
>>
>>
>>>> ignore the RA) to much more depending on how heavy-weight the actual 
>>>
>>>
>>> processing in
>>>
>>>> software would be.
>>>
>>>