Subject: RE: [bmwg] Security section standard text for BMWG documents

From: "Chip Popoviciu \(cpopovic\)" <[email-masked]>
Date: Wed, 4 Oct 2006 11:46:10 -0400

Hi David,

The alert is very valid and dully noted :) You are right, my earlier
attempt to put in such recommendations were countered based on the same
arguments. It seems that this topic was has been extensively discussed
and I agree with the points you mentioned. My thinking was to simply
draw tester's attention to the fact that monitoring some of these
parameters is useful. The idea is that the evaluator will know what
parameters are relevant for what box (type, vendor, architecture, etc.)
so she/he can select them. This additional information might be useful
just to evaluator's organization but I think it is useful from an
operational stand point. In my experience, network operators request
from vendors the benchmarking data (based on RFC 2544) and the
corresponding values of certain parameters (such as CPU values).

My proposal is not to recommend specific measurements or parameters, I
agree that they might be different from one device to another, from one
vendor to another, but rather to draw attention to the fact that DUT
resource measurements might be very useful in addition to the pure
black-box recommendations made.

Best Regards,
Chip 

-----Original Message-----
From: David Newman [mailto:[email-masked]] 
Sent: Wednesday, October 04, 2006 11:16 AM
To: Chip Popoviciu (cpopovic)
Cc: Al Morton; [email-masked]
Subject: Re: [bmwg] Security section standard text for BMWG documents

Chip Popoviciu (cpopovic) wrote:
> Al,
>  
> I hope we can revive this conversation and close on this point.
>  
> Here is where we stand based on the text you proposed and the feedback
> received:
>  
>> Benchmarking activities as described in this memo are limited to 
>> technology characterization using controlled stimuli in a laboratory 
>> environment, with dedicated address space and the constraints 
>> specified in the sections above.
>>
>> The benchmarking network
>> topology will be an independent test setup and MUST NOT be connected 
>> to devices that may forward the test traffic into a production 
>> network, or misroute traffic to the test management network.
>>
>> Further, benchmarking is performed on a "black-box" basis, relying 
>> solely on measurements observable external to the DUT/SUT.
>>
>> Special capabilities SHOULD NOT exit in the DUT/SUT specifically for 
>> benchmarking purposes. Any implications for network security arising 
>> from the DUT/SUT SHOULD be identical in the lab and in production
networks.
>  
> Here are my comments:
> 1. In the last paragraph I propose we substitute "exist" with "be 
> enabled". The Special Capabilities exit, it is just a matter of 
> enabling them for benchmarking testing.
>  
> 2. I am having some difficulties with the third paragraph and in a 
> way, David Newman touch on this when he mentioned turning on debugs 
> etc. I am a full supporter of the black-box principle of benchmarking 
> but I believe that probes should be available inside the DUT/SUT to 
> monitor certain parameters. Here is the scenario that I have in mind: 
> Lets say I measure the throughput for Router X with 1024 bytes frames.

> The result is that the router forwards at line rate. Is this 
> information by itself useful to a network operations engineer? What if

> the resources of the router are completely used to achieve this 
> forwarding rate? On the other hand, this data along with measurements 
> of router resources can tell the evaluator that yes, the router will 
> achieve line rate NDR but nothing else. Moreover, for some 
> architectures, under certain benchmarking conditions the forwarding
might change from Hardware assist to Software.

rathole alert

What if you're comparing different vendors' implementations and 100
percent CPU utilization on box A means something totally different than
on box B (eg, one can still process routing updates and the other one
has smoke coming out the back)?

What if different implementations from the same vendor differ depending
on architecture?

Dunno all the history, but I believe bmwg has stayed away from white-box
and gray-box approaches because of issues like these.

dn

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg