Subject: RE: [bmwg] Security section standard text for BMWG documents

From: "Chip Popoviciu \(cpopovic\)" <[email-masked]>
Date: Wed, 4 Oct 2006 00:12:06 -0400

Al,
 
I hope we can revive this conversation and close on this point.
 
Here is where we stand based on the text you proposed and the feedback
received:
 
> Benchmarking activities as described in this memo are limited to
> technology characterization using controlled stimuli in a laboratory
> environment, with dedicated address space and the constraints
> specified in the sections above.
> 
> The benchmarking network
> topology will be an independent test setup and MUST NOT be connected
to
> devices that may forward the test traffic into a production network,
> or misroute traffic to the test management network.
> 
> Further, benchmarking is performed on a "black-box" basis,
> relying solely on measurements observable external to the DUT/SUT.
> 
> Special capabilities SHOULD NOT exit in the DUT/SUT specifically for
> benchmarking purposes. Any implications for network security arising
> from the DUT/SUT SHOULD be identical in the lab and in production
networks.
 
Here are my comments:
1. In the last paragraph I propose we substitute "exist" with "be
enabled". The Special Capabilities exit, it is just a matter of enabling
them for benchmarking testing.
 
2. I am having some difficulties with the third paragraph and in a way,
David Newman touch on this when he mentioned turning on debugs etc. I am
a full supporter of the black-box principle of benchmarking but I
believe that probes should be available inside the DUT/SUT to monitor
certain parameters. Here is the scenario that I have in mind: Lets say I
measure the throughput for Router X with 1024 bytes frames. The result
is that the router forwards at line rate. Is this information by itself
useful to a network operations engineer? What if the resources of the
router are completely used to achieve this forwarding rate? On the other
hand, this data along with measurements of router resources can tell the
evaluator that yes, the router will achieve line rate NDR but nothing
else. Moreover, for some architectures, under certain benchmarking
conditions the forwarding might change from Hardware assist to Software.
The processor might be just powerful enough to still handle the traffic
however, the evaluator should know what happens. I fully understand that
"device resources" is a loose term that depends from vendor to vendor.
This is why I would not suggest we indicate what these are but we just
recommend that evaluators should consider monitoring resources during
testing. In this context, I would suggest changing the third paragraph
to:
 
> Further, benchmarking is performed on a "black-box" basis,
> relying primarily on measurements observable external to the DUT/SUT.

With this occasion, I would also want to know WG's thoughts on
introducing in our drafts the recommendation mentioned above: "During
testing, relevant device resources should be monitored in order to
understand the operational state of the DUT/SUT."
 
3. I tend to agree that this text seems more appropriate at the
beginning of our documents. Particularly if we say "The benchmarking
network topology WILL ... ". Nevertheless, we need to agree if the
current text in the Security section is sufficient:
 
> There are no security issues that are or need to be addressed in this
> document.
 
By comparison with Security sections of RFCs and drafts in other WGs,
this seems to be sufficient even though it does not reflect the
specifics of this WG. Along with the above text at the beginning of the
document we could consider a summary for the Security section such as: 
 
> The isolated nature of the benchmarking environments and the fact
> that no special features or capabilities, other than those 
> used in operational networks, are enabled on the DUT/SUT require no
> security considerations specific to the benchmarking process.
 
Looking forward to comments from the WG and hopefully we can close the
topic prior to San Diego.
 
Best Regards,
Chip

________________________________

From: Al Morton [mailto:[email-masked]] 
Sent: Tuesday, October 03, 2006 6:24 PM
To: [email-masked]
Subject: Re: [bmwg] Security section standard text for BMWG documents

At 04:02 PM 10/3/2006, Chip Popoviciu \(cpopovic\) wrote:

	We are finalizing the new version of the IPv6 benchmarking draft
and wanted to include the right text for the Security section.
	 
	Best Regards,
	Chip

Chip,

Folks moved on to other things and the discussion 
came to a resting place, so I don't think we reached
a final consensus on the wording.  But I can say that:

David Newman responded to my attempt at a summary 
(I think that's the text you reproduced.)
Unfortunately, David's response doesn't appear in the 
archives!  But it is on my hard drive, so I've attached
his message with edits below.

I agreed with David's suggestions, and then we discussed
making the paragraph a standard one for the Introduction,
as opposed to introducing all this background info in the 
back (where the Security Section resides). David, Aamer Akhter,
and I all agreed with this.

Dan Romascanu raised the possibility that David's SHOULD NOT
might be better as a MUST NOT, in his proposed paragraph:

	Special capabilities SHOULD NOT exist in the DUT/SUT
specifically for
	benchmarking purposes. Any implications for network security
arising
	from the DUT/SUT SHOULD be identical in the lab and in
production networks.

Also, Dan made a comment about debugging mode that tended to 
agree that we were raising more than security-related points
in the text.

So, if you implement the edits in David's message below,
and give some thought to the SHOULD NOT/MUST NOT point
AND decide *where* you'd like to see this paragraph in your memo,
then you'll be contribution to the discussion by providing
the IPv6 author team's view of the "standard" paragraph.

thanks and regards,
Al

====== David's message follows =====================================

Date: Sat, 22 Jul 2006 13:04:06 -0700 
From: David Newman <[email-masked]> 
Organization: Network Test Inc. 
To: [email-masked] 
Subject: Re: [bmwg] "Standard" Security Section Paragraph 
<https://www1.ietf.org/mailman/listinfo/bmwg> >, 
         < mailto:[email-masked]?subject=unsubscribe
<mailto:[email-masked]%3Fsubject=unsubscribe> > 
List-Post: < mailto:[email-masked] <mailto:[email-masked]> > 
List-Help: < mailto:[email-masked]?subject=help
<mailto:[email-masked]%3Fsubject=help> > 
List-Subscribe: < https://www1.ietf.org/mailman/listinfo/bmwg
<https://www1.ietf.org/mailman/listinfo/bmwg> >, 
         < mailto:[email-masked]?subject=subscribe
<mailto:[email-masked]%3Fsubject=subscribe> > 

minor copy-editing nits and two comments:

> Benchmarking activities as described in this memo are limited to
> technology characterization using simulated stimuli in a laboratory
> environment, using a dedicated address space and the constraints
> specified in the sections above.

s/simulated/controlled/

(stimuli are real, they just occur in a lab)

s/using a dedicated/with dedicated/

("with" avoids repetition of "using")

(deletion of article "a" covers use of multiple noncontiguous address
spaces)

> 
> The benchmarking network
> topology will be an independent test setup and MUST NOT be connected
to
> devices that may forward the test traffic into a production network,
> or miss-route traffic to the test management network.

s/miss-route/misroute/

> 
> Further, benchmarking is performed on a "black-box" basis,
> relying solely on measurements observable external to the DUT/SUT.
> 
> No special benchmarking capabilities should exist in the DUT/SUT,
> therefore disabling such capabilities when the device is used in
> a production network will not be necessary, and there are no
> corresponding implications for network security.

Sorry, still two issues here:

1. Logic problem in the first two clauses. If special capabilities do
not exist, then by definition they cannot be disabled in production or
anywhere else.

Perhaps instead:

"Special capabilities SHOULD NOT exist in the DUT/SUT specifically for
benchmarking purposes. Any implications for network security arising
from the DUT/SUT SHOULD be identical in the lab and in production
networks."

2. In fact, we frequently use "special capabilities" in testing. We
enable debug mode, we use special checked builds, we put interfaces into
promiscuous mode, and so on.

There's an important point being made here, but fundamentally it's not a
security issue. While I agree 100 percent with this point, I'm not sure
it belongs as a security consideration. The only security-specific bit
is the part saying that lab and production security implications should
not differ.

dn

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: David Newman <[email-masked]>
Date: Wed, 04 Oct 2006 08:16:25 -0700

Chip Popoviciu (cpopovic) wrote:
>> topology will be an independent test setup and MUST NOT be connected to
>> from the DUT/SUT SHOULD be identical in the lab and in production networks.
>  
> Here are my comments:

rathole alert

What if you're comparing different vendors' implementations and 100
percent CPU utilization on box A means something totally different than
on box B (eg, one can still process routing updates and the other one
has smoke coming out the back)?

What if different implementations from the same vendor differ depending
on architecture?

Dunno all the history, but I believe bmwg has stayed away from white-box
and gray-box approaches because of issues like these.

dn

---

From: "Bill Cerveny" <[email-masked]>
Date: Wed, 04 Oct 2006 13:28:58 -0400

In my opinion, looking at the phrase:

If it is thought that having "packet blaster" capabilities (or something
similar) in a production DUT/SUT could present a DOS attack
vulnerability, I would think this code shouldn't be available on
theDUT/SUT at all.  I'm not a security type, but if the code for
launching
DOS attack from a DUT/SUT is installed on the device, someone will
figure out a way to access it.  

Changing "exist" to "be enabled" greatly waters down the intent of this
statement. The statement "be enabled" suggests that the code is
installed but not turned on.

Regarding the "black box" statement, three thoughts: 1) How is this a
security concern (unless the thought is that the device is potentially
blatently lying about what it is observing internally) and 2) Being able
to monitor things such as processor loads on a DUT/SUT can be
interesting and relevant, even if not reported in the final report.  If
one was a test methodology purist, I'm guessing one wouldn't trust what 
the device is saying at all. It would however be interesting to have the
device 
indicate that it is about to fail, at which point more load could be put 
on the device to see if it actually does fail. 3) Wouldn't/couldn't part
of the 
benchmarking process include verifying that the device counters match
what the 
tester sent across its interfaces?

I finally agree with Chip's proposed text, changing "enabled" to "exist"
and "require" to "requires" (nature ... requires):

> > used in operational networks, exist on the DUT/SUT requires no

Bill

On Wed, 4 Oct 2006 00:12:06 -0400, "Chip Popoviciu (cpopovic)"
<[email-masked]> said:
> User-Agent: Thunderbird 1.5.0.4 (Macintosh/20060530) 
> To: [email-masked] 
> X-Enigmail-Version: 0.94.0.0 
> X-Spam-Score: 0.0 (/) 
> X-Scan-Signature: f60d0f7806b0c40781eee6b9cd0b2135 
> X-BeenThere: [email-masked] 
> X-Mailman-Version: 2.1.5 
> List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org> 
> List-Unsubscribe: < https://www1.ietf.org/mailman/listinfo/bmwg
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (Darwin)
> 
> iD8DBQFEwoS2yPxGVjntI4IRAg4RAJwKKJwtvlfLVVhQK3K4gDQPCk5c/gCg6Ejh
> qb0ZhFBzh8VoLopDBh315BM=
> =Si1z
> -----END PGP SIGNATURE-----
>