Subject: Re: [bmwg] Power Benchmarking

From: "Sangjin Jeong" <[email-masked]>
Date: Wed, 13 Mar 2013 09:19:37 -0400

Hello Vishwas,

I quickly read the document and here is a few comments from me.

Section 3.2.  In this section, you list device factors affecting power
consumption of a networking device.

However, as I mentioned bmwg session yesterday, the operating conditions of
linecards e.g., active, idle, sleep and 

off may affect the power consumption of the device. Thus, I think it might
be useful to consider the 

operating conditions for more accurate power benchmarking.

Also, there is an increasing demand to benchmark the energy efficiency of
networking devices in terms of 

productivity or work versus power(energy). So I am wondering if you consider
to include these metric in 

this document.

Best regards,

Sangjin

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Vishwas Manral
Sent: Tuesday, January 29, 2013 6:48 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: [bmwg] Power Benchmarking

Hi folks,

We have received some new interest and have recently refreshed a long
dormant draft on Power Benchmarking.

https://tools.ietf.org/html/draft-manral-bmwg-power-usage-03

We would love to hear any comments and queries on the same.

Thanks,

Authors

---

From: "Lucien Avramov (lavramov)" <[email-masked]>
Date: Thu, 14 Mar 2013 11:17:12 -0400

H Vishwas and BMWG,

Here are our notes from the BMWG meeting we had mentioned regarding 
draft-manral-bmwg-power-usage-03

-the idea of power benchmarking is great and Jacob and myself are 
supportive and would like to see this adopted.

-section 3.2 : why is there a focus on TCAM? The TCAM size, should not 
matter. In the end each switches / network devices are different and the 
power used for them is irrelevant of the TCAM size. For example, a 
switch with 18MB of TCAM that we have uses less power than a switch 
which offers 9MB. Overtime the sillicon and technology used improves and 
we can have bigger TCAM sizes [among many other things in the hardware 
chip] while using less power. By starting with mentioning TCAM, every 
other architectural functionality should be mentioned (crossbar switch 
with port ASIC and fabric, switch on chip, forwarding features in 
hardware etc...). Our recommendation is to abandon this specification.

-section 3.3: 'time between successive packets may affect power
    usage but we do not measure the effects in detail.'

The inter packet delay is the % of line rate, so this should be measured 
in detail actually. Please see the typical operating power metric 
definition we suggest bellow for further detail.

-section 5: NEPI looks quite complicated. The hypothesis made on this as 
I understand is that there is energy proportionality to traffic load.

In reality when this is measured on today's equipment it is usually not 
proportional. Why should the power be proportional to load ideally? 
There are small variations in the scheduler and other components of the 
switch architecture which will be active from an ASIC prospective but 
it's not that meaningful compared to load proportion. My suggestion here 
would be to use NEPI to provide a comparison to the typical operating 
power metric, and this could be in a form of a graph showing the 
variance of power for example.

Hence, the suggestion is as we discussed last Tuesday at the BMWG 
meeting to see a new metric, 'typical operating power' defined, besides 
the NECR and NEPI in this draft. This will actually have the default 
behavior for the test used. Then, NECR/NEPI can finally be applied by 
measuring the delta / power difference compared to the typical operating 
power.

The typical operating power would define a realistic use of the network 
equipment while measured such as:

-all power supplies plugged (for switches / routers with more than 1 
power supply)
-all ports up.  For example on a switch with 48 SFP+ ports and 4x QSFP 
port, we connect all the ports.
-traffic load at 100%. if there are drops and the device is not capable 
of achieving linerate, then the highest load in % achievable with no 
drop should be used and in this case it should be stated what percentage 
it is when measured. The RFC measured should be mentioned. For example, 
RFC 2544.
-type of cables used and transceivers for the test. 3 cases here to 
define, the third would be left optional for the measure:
  a) measure with all ports using SR optics and a fixed cable lengtgh 
specified. No guidance at this point to provide in terms of length.
  b) measure with all ports using CX-1 cables of the same length
  c) (optional) measure with a mix between CX-1 and SR, in this case, 
specify numbers used for each, for example 48 CX-1 and 4 QSFP+.

This is important as we actually see a 25% difference of power use 
between using all ports with optical SFP+ transceiver and fiber cables, 
versus a CX-1 twinax type of cable plugged in the SFP+ port.

In the Data Center benchmarking draft we are about to submit with Jacob 
and we presented, there is a section where we define the physical cable 
calibration for performance. this could be used as a reference to 
provide guidance in the a) b) c) cases above.

Hope this helps,

Lucien Avramov
Jacob Rapp

On 3/13/13 9:19 AM, Sangjin Jeong wrote:
> However, as I mentioned bmwg session yesterday, the operating conditions
> of linecards e.g., active, idle, sleep and
>
> off may affect the power consumption of the device. Thus, I think it
> might be useful to consider the
>
> Also, there is an increasing demand to benchmark the energy efficiency
> of networking devices in terms of
>
> productivity or work versus power(energy). So I am wondering if you
> consider to include these metric in
>
> this document.
>
> Best regards,
>
> Sangjin
>
> *From:[email-masked] [mailto:[email-masked]] *On Behalf
> Of *Vishwas Manral
> *Sent:* Tuesday, January 29, 2013 6:48 PM
> *To:* [email-masked]
> *Cc:* [email-masked]; [email-masked]
> *Subject:* [bmwg] Power Benchmarking
>
> Hi folks,
>
> _______________________________________________
> bmwg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/bmwg
>

---

From: Vishwas Manral <[email-masked]>
Date: Mon, 18 Mar 2013 15:29:18 -0700

Hi Lucien/ Jacob/ Sangjin,

I thank you for your detailed comments and acknowledge them. I understand
your concerns you raise around NEPI and the points you raise around Typical
operating power.

We will get back to you after discussing this among the co-authors.

Thanks,
Vishwas
On Thu, Mar 14, 2013 at 8:17 AM, Lucien Avramov (lavramov) <
[email-masked]> wrote:

> draft-manral-bmwg-power-usage-**03
>
> power used for them is irrelevant of the TCAM size. For example, a switch
> with 18MB of TCAM that we have uses less power than a switch which offers
> 9MB. Overtime the sillicon and technology used improves and we can have
> bigger TCAM sizes [among many other things in the hardware chip] while
> using less power. By starting with mentioning TCAM, every other
> architectural functionality should be mentioned (crossbar switch with port
> ASIC and fabric, switch on chip, forwarding features in hardware etc...).
> Our recommendation is to abandon this specification.
>
> -section 5: NEPI looks quite complicated. The hypothesis made on this as I
> understand is that there is energy proportionality to traffic load.
>
> proportional. Why should the power be proportional to load ideally? There
> are small variations in the scheduler and other components of the switch
> architecture which will be active from an ASIC prospective but it's not
> that meaningful compared to load proportion. My suggestion here would be to
> use NEPI to provide a comparison to the typical operating power metric, and
> this could be in a form of a graph showing the variance of power for
> example.
>
> Hence, the suggestion is as we discussed last Tuesday at the BMWG meeting
> to see a new metric, 'typical operating power' defined, besides the NECR
> and NEPI in this draft. This will actually have the default behavior for
> the test used. Then, NECR/NEPI can finally be applied by measuring the
> delta / power difference compared to the typical operating power.
>
> -all power supplies plugged (for switches / routers with more than 1 power
> supply)
> -traffic load at 100%. if there are drops and the device is not capable of
> achieving linerate, then the highest load in % achievable with no drop
> should be used and in this case it should be stated what percentage it is
> when measured. The RFC measured should be mentioned. For example, RFC 2544.
> This is important as we actually see a 25% difference of power use between
> using all ports with optical SFP+ transceiver and fiber cables, versus a
> CX-1 twinax type of cable plugged in the SFP+ port.
>
> calibration for performance. this could be used as a reference to provide
> guidance in the a) b) c) cases above.
>
> Hope this helps,
>
> Lucien Avramov
> Jacob Rapp
>
>
>
>
>
>> https://tools.ietf.org/html/**draft-manral-bmwg-power-usage-**03<https://tools.ietf.org/html/draft-manral-bmwg-power-usage-03>
>>
>> ______________________________**_________________
>> bmwg mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/**listinfo/bmwg<https://www.ietf.org/mailman/listinfo/bmwg>
>>
>>
>