Subject: Re: [eman] Entity identification method

From: "Chris Verges" <[email-masked]>
Date: Mon, 1 Nov 2010 05:39:23 -0700

Hi Christian,

Are you looking to allocate actual energy usage by a virtual entity,
like the amount of energy used by a virtual machine running on a shared
server?  Certainly an interesting problem ... gives a lot of room for
innovation for different vendors to find solutions.

>From a MIB standpoint, it seems like the requirement is to have a
flexible power-related MIB that can either map to physical or logical
entities.  One question is, "where do you want those entities
enumerated?"

Are virtual entities that need to be mapped to power usage normally
enumerated in ENTITY-MIB in the "logical" table?

If so, perhaps something like this would fit the bill:

POWER-ENTITY-MIB:

  pwrEntityTable

   pwrEntityEntry

      pwrEntityIndex

      pwrEntityName

      pwrEntityType

      pwrEntityParent

      pwrEntityPhysical

      pwrEntityLogical

The pwrEntity[Physical|Logical] objects would point to the appropriate
index in ENTITY-MIB.

However, if the virtual entities are not enumerated in ENTITY-MIB, the
pwrEntityLogical column is probably overkill.  The virtual entities can
be enumerated in pwrEntityTable and linked to the Physical meter using
the pwrEntityParent column, much like how ENTITY-MIB currently manages
references between parent and component entities.  It would just involve
a new pwrEntityType being added for "virtual" or something similar.

So what's the typical use case as these virtual entities relate to
ENTITY-MIB?

Chris

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Christian Groves
Sent: Sunday, October 31, 2010 8:14 PM
To: [email-masked]
Subject: Re: [eman] Entity identification method

Hello Juergen,

I think distinguishing the two cases is the correct way to go.

I agree that reporting energy consumption for logical entities is a
challenge and not straight forward. I guess there are a number of ways
to look at this problem and how address the problem depends on the
ambition level.

As you rightly mention where there is partial use of a physical resource
by a logical entity it is very difficult to have an exact figure.  

However I think the "grain size" of the measurement comes into play. For
example if we have a large gateway implementing several virtual gateways
if we take an overall measurement then we have the problem determining a
figure. If the however the virtual gateways were each assigned a
physical rack and the measurements were taken at a rack level then we
have a more precise estimate of contribution of each virtual gateway to
the overall picture.

For me logical entities ultimately will be realised by physical
hardware. The key is understanding the relationship between these
entities. Thus I think virtualization can in some way be supported by
allowing energy consumption reports for physical entities to indicate
if/what logical entities they relate to. As they say information is
power (pardon the pun :-) ).

Regards, Christian

On 29/10/2010 12:48 PM, Juergen Quittek wrote:

> Hi Christian,

> Thank you for this interesting comment.

> It looks like we have to distinguish two cases: power state monitoring

> and energy consumption monitoring.

> For a logical entity and a virtual machine it may be well possible to 

> report on their power state. However, reporting their energy 

> consumption seems to be a tough challenge at least if be base it on
real measurements.

> It is well feasible to measure energy consumption of physical 

> entities, but if you have a logical entity that consumes only partial 

> resources of a physical entity, then precise measurement of the energy

> consumption will be hardly possible. Estimations may be possible based

> on a model of the logical entity, but I don't know how good such
models can be.

> Thanks,

>      Juergen

> On 29.10.10 03:25  "Christian Groves"<[email-masked]
<mailto:[email-masked]> >  wrote:

>> Hello

>> I've just started to digest the various drafts of the eman wg and I 

>> see the comment below on logical entities. It's not clear to me 

>> how/if network virtualization is intended on being handled in the
eman work.

>> In networks there's now a trend towards network virtualization and 

>> cloud computing with the different "X" as a service models (i.e. 

>> Amazon EC2 "platform as a service" running different server 

>> instances). Even in today's networks we have physical gateways that 

>> are virtualized into different logical gateways. For example: 

>> H.248/Megaco virtual MG (VMG) concept.

>> I think it would be advantageous to be able to assess the energy 

>> consumption of the VMGs or instances (which are logical entities) 

>> independent of the physical gateway so that a customers energy use 

>> may be recording. Will this sort of functionality be allowed?

>> Regards, Christian

>> On 29/10/2010 1:22 AM, Juergen Quittek wrote:

>>> Hi Chris,

>>> I think it may become very difficult if we want to assess energy 

>>> consumption of logical entities. It seems to be reasonable and 

>>> practical to limit energy consumption measurements to physical 

>>> entities. The ENERGY AWARE MIB module

>>> (draft-parello-eman-energy-aware-mib) and the POWER MIB

>>> (draft-quittek-power-mib-02) went this way.

>>> Just using entPhysicalIndex solves some of the problem you
mentioned.

>>> Particularly, you don't need a mapping table to entities if you 

>>> restrict Yourself to entPhysicalIndex.

>>> Thanks,

>>>       Juergen

>>> On 28.10.10 13:34  "Chris Verges"<[email-masked]
<mailto:[email-masked]> >   wrote:

>>>> Hi Jurgen and all,

>>>> The more I've been thinking about these scenarios, the more 

>>>> something doesn't seem quite right.  I'll try to explain my thought

>>>> process behind this and see if my concerns/confusions get captured.

>>>> ENTITY-MIB provides enumerated lists of physical and logical 

>>>> entities on the system.  The indices in both lists are maintained 

>>>> separately, so there is not a single, unique identifier provided.  

>>>> Because of this, the MIBs being discussed by the EMAN WG must only 

>>>> reference to one type (physical or logical) and are limited to that
without additional work.

>>>> At some point in my mind, I'm asking myself the question, "What's 

>>>> the point of linking to ENTITY-MIB?"

>>>> More on that question in later scenarios below ...

>>>> One possibility would be to create a separate mapping table that 

>>>> gives a guaranteed unique index based on all entities, regardless 

>>>> of type.  The physical vs. logical extensions would then become 

>>>> sparse tables that augment this.  For example:

>>>>     ENTITY-MIB : entityGeneral.entGeneralTable.entGeneralEntry

>>>>     +-----------------+-----------------+-----+

>>>>     | entGeneralIndex | entGeneralDescr | ... |

>>>>     +-----------------+-----------------+-----+

>>>>     |        1        | PDU Chassis     |     |

>>>>     +-----------------+-----------------+-----+

>>>>     |        2        | Meter #1        |     |

>>>>     +-----------------+-----------------+-----+

>>>>     |        3        | Meter #2        |     |

>>>>     +-----------------+-----------------+-----+

>>>>     |        4        | Remote Meter #1 |     |

>>>>     +-----------------+-----------------+-----+

>>>>     ENTITY-MIB : entityPhysical.entPhysicalTable.entPhysicalEntry

>>>>     +-----------------+-----------------+-----+

>>>>     | entGeneralIndex | entVendorType   | ... |

>>>>     +-----------------+-----------------+-----+

>>>>     |        1        | enterprises.foo |     |

>>>>     +-----------------+-----------------+-----+

>>>>     |        2        | enterprises.foo |     |

>>>>     +-----------------+-----------------+-----+

>>>>     |        3        | enterprises.foo |     |

>>>>     +-----------------+-----------------+-----+

>>>>     ENTITY-MIB : entityLogical.entLogicalTable.entLogicalEntry

>>>>     +-----------------+----------------------+-----+

>>>>     | entGeneralIndex | entLogicalCommunity  | ... |

>>>>     +-----------------+----------------------+-----+

>>>>     |        4        | remote-power-group-1 |     |

>>>>     +-----------------+----------------------+-----+

>>>> It seems like this 'entGeneralIndex' concept is what's missing from

>>>> the current ENTITY-MIB, and is causing problems when thinking about

>>>> local and remote power data being reported by the same agent.

>>>> Another possibility would be to choose one -- entLogicalIndex, most

>>>> likely -- and require that anyone who wants to implement the EMAN 

>>>> WG MIBs to always create an entLogicalEntry for the power data.  In

>>>> this context, the entLogicalTable fills the role of entGeneralTable

>>>> as described above.  For all entPhysicalTable entries, there would 

>>>> need to be a corresponding entry in entLogicalTable with a 1:1 

>>>> mapping between them.  For all remote entries, there would be a 

>>>> unique entry in entLogicalTable.  Of course, this would add some 

>>>> additional overhead on the part of the implementer, but isn't too
overburdening.

>>>> A third possibility is that ENTITY-MIB just doesn't do what we need

>>>> in its present form and we need to think about enumerating power 

>>>> sensors separately from other components/sensors on the system.  It

>>>> seems like entPhysicalTable gives a good starting template for a 

>>>> new 'POWER-ENTITY-MIB', but can be extended to include support for 

>>>> remote agents.  The difference would be a clean break from the 

>>>> legacy ENTITY-MIB structure.

>>>> And with that, I look forward to others' feedback and ideas on 

>>>> solving this problem!

>>>> Thanks,

>>>> Chris

>>>> -----Original Message-----

>>>> From: [email-masked] [mailto:[email-masked]] On 

>>>> Behalf Of Chris Verges

>>>> Sent: Tuesday, October 26, 2010 9:26 PM

>>>> To: Juergen Quittek; Benoit Claise; eman mailing list

>>>> Subject: Re: [eman] Entity identification method (was: EMAN 

>>>> chartered itemsversus drafts)

>>>> Hi Juergen and all,

>>>> Agreed that a common ID method makes sense, and ENTITY-MIB seems to

>>>> be a good choice.  In case (c) from your list, wouldn't the remote 

>>>> agent condition be handled by ENTITY-MIB supporting a logical 

>>>> entity that is the remote agent iself?  Since the logical entity is

>>>> in the ENTITY-MIB table, it would be given a unique ID mixed 

>>>> amongst the other logical and physical entities.

>>>> At that point, should all of the MIBs being discussed in the EMAN 

>>>> WG be sparse table augmentations of ENTITY-MIB?

>>>> Thanks,

>>>> Chris

>>>> -----Original Message-----

>>>> From: [email-masked] [mailto:[email-masked]] On 

>>>> Behalf Of Juergen Quittek

>>>> Sent: Tuesday, October 26, 2010 6:01 AM

>>>> To: Benoit Claise; eman mailing list

>>>> Subject: [eman] Entity identification method (was: EMAN chartered 

>>>> items versus drafts)

>>>> Hi Benoit and all,

>>>> I agree that we should find a common identification method for 

>>>> entities used by all MIB modules. This does not apply to the POWER 

>>>> MIB only, but to all MIB modules in the eman WG.

>>>> The modules will use an entity ID as index for their tables.

>>>> Which entity is identified by the index can be resolved by other 

>>>> MIB modules, such as the POWER AWARE MIB module

>>>> (draft-parello-eman-energy-aware-mib) or the ENTITY MIB module (RFC

>>>> 4133).

>>>> I see three basic scenarios for the entity identification:

>>>>     a) a device just reports on its own power state and energy

>>>>        consumption and it reports on its own as a single unit.

>>>>        Then we have a single index only stating "it's me".

>>>>        Such a device usually does not need further identification

>>>>        of itself, because typically there are sufficient other

>>>>        MIB modules for this purpose running in the same SNMP
engine,

>>>>        that provide information about the device's IP address,

>>>>        manufacturer, operating system, etc. In such a case a

>>>>        'trivial' index, such as '0' should be used in order to

>>>>        keep it simple in this most simple case.

>>>>     b) a device reports on its own but not just as a single unit

>>>>        but it reports power states and energy consumption for its

>>>>        individual components, for example it may report separately

>>>>        on contained hard drives, line cards, back planes,

>>>>        processor boards, etc.

>>>>        In such a case, identification of these components as

>>>>        individual entities would be required. The ENTITY MIB

>>>>        module was designed for this purpose and would be a good

>>>>        choice here. Also the POWER AWARE MIB module would be

>>>>        useful in this case.

>>>>     c) a device reports on energy consumption of other, remote

>>>>        devices. Then remote devices (and potentially also their

>>>>        contained components need to be identified. For

>>>>        identifying remote components there is the POWER AWARE MIB

>>>>        module that has been designed for this purpose. As far as

>>>>        I understand, the ENTITY MIB module is not applicable to

>>>>        remote devices.

>>>> In summary,

>>>>     - if you need to report on remote entities (case c)),

>>>>       you need the POWER AWARE MIB module,

>>>>     - if you report only on entities locally contained

>>>>       in the reporting device (case b)), you can use

>>>>       the POWER AWARE MIB or the ENTITY MIB

>>>>     - if you report just on your own as a single device

>>>>       (case a)), identification is trivial

>>>> Hence, my recommendation (stated for POWER-STATE MIB and ENERGY MIB

>>>> in

>>>> draft-quittek-power-mib-02) would be:

>>>> If there is an implementation of the POWER AWARE MIB module 

>>>> instantiated in the local SNMP engine, then you SHOULD (or MUST?) 

>>>> use it for indexing (pmIndex).

>>>> If this is not the case but there is an ENTITY MIB instance 

>>>> available, then you SHOULD use this one (entPhysicalIndex).

>>>> If neither of this MIB modules is available you should use index 0 

>>>> only and be limited to report on the local device as a single
entity only.

>>>> That's just my view. Certainly, there are more ways of entity 

>>>> identification. I look forward to discussing them.

>>>> Thanks,

>>>>       Juergen

>>>> On 26.10.10 11:44  "Benoit Claise"<[email-masked]
<mailto:[email-masked]> >   wrote:

>>>>> Hi Juergen,

>>>>> Thanks for the clarification.

>>>>> Something key is to agree on the Power Monitor index for all MIB 

>>>>> modules, which IMHO should be part of the Energy-aware Networks 

>>>>> and Devices MIB module, but reuse in the other MIB modules.

>>>>> Regards, Benoit.

>>>>>> Hi Benoit,

>>>>>> Thanks for checking all drafts.

>>>>>> I don't think that draft-quittek-power-mib-02 makes significant 

>>>>>> contributions to the Energy-aware Networks and Devices MIB.

>>>>>> It just covers the Power and Energy Monitoring MIB and the 

>>>>>> Battery

>>>> MIB.

>>>>>> Thanks,

>>>>>>        Juergen

>>>>>> On 26.10.10 08:22  "Benoit Claise"<[email-masked]
<mailto:[email-masked]> >    wrote:

>>>>>>> Dear all,

>>>>>>> I went through the exercise of mapping the existing six 

>>>>>>> chartered items with the existing draft content.

>>>>>>> _Energy Management Requirements_ 

>>>>>>> http://tools.ietf.org/html/draft-quittek-power-monitoring-requir
<http://tools.ietf.org/html/draft-quittek-power-monitoring-requir> 

>>>>>>> emen

>>>>>>> ts-02 

>>>>>>> https://datatracker.ietf.org/doc/draft-norwin-energy-consider/
<https://datatracker.ietf.org/doc/draft-norwin-energy-consider/> 

>>>>>>> _

>>>>>>> Energy Management Framework_

>>>>>>> http://tools.ietf.org/html/draft-claise-power-management-arch-02
<http://tools.ietf.org/html/draft-claise-power-management-arch-02> 

>>>>>>> _Energy-aware Networks and Devices MIB_

>>>>>>> http://tools.ietf.org/html/draft-parello-eman-energy-aware-mib-0
<http://tools.ietf.org/html/draft-parello-eman-energy-aware-mib-0> 

>>>>>>> 0

>>>>>>> http://tools.ietf.org/html/draft-quittek-power-mib-02
<http://tools.ietf.org/html/draft-quittek-power-mib-02> 

>>>>>>> _Power and Energy Monitoring MIB_

>>>>>>> http://tools.ietf.org/html/draft-claise-energy-monitoring-mib-06
<http://tools.ietf.org/html/draft-claise-energy-monitoring-mib-06> 

>>>>>>> http://tools.ietf.org/html/draft-quittek-power-mib-02
<http://tools.ietf.org/html/draft-quittek-power-mib-02> 

>>>>>>> _Battery MIB_

>>>>>>> http://tools.ietf.org/html/draft-quittek-power-mib-02
<http://tools.ietf.org/html/draft-quittek-power-mib-02> 

>>>>>>> _Energy Management Applicability_ 

>>>>>>> http://datatracker.ietf.org/doc/draft-tychon-eman-applicability-
<http://datatracker.ietf.org/doc/draft-tychon-eman-applicability-> 

>>>>>>> stat

>>>>>>> ement/

>>>>>>> Please let me know if I made any mistakes or if I missed any
draft?

>>>>>>> Regards, Benoit.

>>>>>>> _______________________________________________

>>>>>>> eman mailing list

>>>>>>> [email-masked] <mailto:[email-masked]> 

>>>>>>> https://www.ietf.org/mailman/listinfo/eman
<https://www.ietf.org/mailman/listinfo/eman> 

>>>> _______________________________________________

>>>> eman mailing list

>>>> [email-masked] <mailto:[email-masked]> 

>>>> https://www.ietf.org/mailman/listinfo/eman
<https://www.ietf.org/mailman/listinfo/eman> 

>>> _______________________________________________

>>> eman mailing list

>>> [email-masked] <mailto:[email-masked]> 

>>> https://www.ietf.org/mailman/listinfo/eman
<https://www.ietf.org/mailman/listinfo/eman> 

>> _______________________________________________

>> eman mailing list

>> [email-masked] <mailto:[email-masked]> 

>> https://www.ietf.org/mailman/listinfo/eman
<https://www.ietf.org/mailman/listinfo/eman> 

> _______________________________________________

> eman mailing list

> [email-masked] <mailto:[email-masked]> 

> https://www.ietf.org/mailman/listinfo/eman
<https://www.ietf.org/mailman/listinfo/eman> 

_______________________________________________

eman mailing list

[email-masked] <mailto:[email-masked]> 

https://www.ietf.org/mailman/listinfo/eman
<https://www.ietf.org/mailman/listinfo/eman>

---

From: "Mouli Chandramouli (moulchan)" <[email-masked]>
Date: Tue, 2 Nov 2010 01:37:59 -0500

Hi Christian, 

Presently, the power measurement technology and instrumentation (power
sensors) exist for physical entities. 
While the MIBs in circulation attempt to provide power consumption by
physical components, cost of measurement sensors should be taken into
account. 

Clearly, the power cost of software features or virtual machines
(logical entities) should be determined, the technology/methodology is
work in progress as given in the research paper. 

Power measurement of virtual machines is still at research stage.

http://www.sigmetrics.org/sigmetrics2010/greenmetrics/greenMetrics10.Kri
shnan.pdf

On another point, it has been widely reported in several bench marking
studies that most of the power consumption is due to physical entities.
Clearly, this notion should be extended to logical entities as you have
pointed out. 

Thanks
Mouli

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Christian Groves
Sent: Monday, November 01, 2010 8:44 AM
To: [email-masked]
Subject: Re: [eman] Entity identification method

Hello Juergen,

Regards, Christian

> It looks like we have to distinguish two cases: power state monitoring
and
> energy consumption monitoring.
>
report
> on their power state. However, reporting their energy consumption
seems to
> be a tough challenge at least if be base it on real measurements.
>
entities, but
> if you have a logical entity that consumes only partial resources of a
> physical entity, then precise measurement of the energy consumption
will be
> hardly possible. Estimations may be possible based on a model of the
logical
> entity, but I don't know how good such models can be.
>
> Thanks,
>
>      Juergen
>
>
> On 29.10.10 03:25  "Christian Groves"<[email-masked]>
>> I've just started to digest the various drafts of the eman wg and I
see
>> the comment below on logical entities. It's not clear to me how/if
>> network virtualization is intended on being handled in the eman work.
>>
cloud
>> computing with the different "X" as a service models (i.e. Amazon EC2
>> "platform as a service" running different server instances). Even in
>> today's networks we have physical gateways that are virtualized into
>> different logical gateways. For example: H.248/Megaco virtual MG
(VMG)
>> concept.
may
>> be recording. Will this sort of functionality be allowed?
>>
>> Regards, Christian
>>
consumption
>>> of logical entities. It seems to be reasonable and practical to
limit energy
>>> consumption measurements to physical entities. The ENERGY AWARE MIB
module
mentioned.
>>> Particularly, you don't need a mapping table to entities if you
restrict
>>> Yourself to entPhysicalIndex.
>>>
>>> Thanks,
>>>
>>>       Juergen
>>>
>>> On 28.10.10 13:34  "Chris Verges"<[email-masked]>
>>>> The more I've been thinking about these scenarios, the more
something
>>>> doesn't seem quite right.  I'll try to explain my thought process
behind
>>>> this and see if my concerns/confusions get captured.
>>>>
entities on
>>>> the system.  The indices in both lists are maintained separately,
so
>>>> there is not a single, unique identifier provided.  Because of
this, the
>>>> MIBs being discussed by the EMAN WG must only reference to one type
>>>> (physical or logical) and are limited to that without additional
work.
>>>> At some point in my mind, I'm asking myself the question, "What's
the
>>>> point of linking to ENTITY-MIB?"
>>>>
gives a
>>>> guaranteed unique index based on all entities, regardless of type.
The
>>>> physical vs. logical extensions would then become sparse tables
that
>>>> augment this.  For example:
>>>>
the
>>>> current ENTITY-MIB, and is causing problems when thinking about
local
>>>> and remote power data being reported by the same agent.
>>>>
WG
>>>> MIBs to always create an entLogicalEntry for the power data.  In
this
>>>> context, the entLogicalTable fills the role of entGeneralTable as
>>>> described above.  For all entPhysicalTable entries, there would
need to
>>>> be a corresponding entry in entLogicalTable with a 1:1 mapping
between
>>>> them.  For all remote entries, there would be a unique entry in
>>>> entLogicalTable.  Of course, this would add some additional
overhead on
>>>> the part of the implementer, but isn't too overburdening.
>>>>
in
>>>> its present form and we need to think about enumerating power
sensors
>>>> separately from other components/sensors on the system.  It seems
like
>>>> entPhysicalTable gives a good starting template for a new
>>>> 'POWER-ENTITY-MIB', but can be extended to include support for
remote
>>>> agents.  The difference would be a clean break from the legacy
>>>> ENTITY-MIB structure.
>>>>
solving
>>>> this problem!
Behalf Of
>>>> Chris Verges
chartered
>>>> itemsversus drafts)
be a
>>>> good choice.  In case (c) from your list, wouldn't the remote agent
>>>> condition be handled by ENTITY-MIB supporting a logical entity that
is
>>>> the remote agent iself?  Since the logical entity is in the
ENTITY-MIB
>>>> table, it would be given a unique ID mixed amongst the other
logical and
>>>> physical entities.
WG be
>>>> sparse table augmentations of ENTITY-MIB?
>>>>
>>>> Thanks,
>>>> Chris
>>>>
Behalf Of
>>>> Juergen Quittek
items
>>>> versus drafts)
entities
>>>> used by all MIB modules. This does not apply to the POWER MIB only,
but
>>>> to all MIB modules in the eman WG.
MIB
>>>> modules, such as the POWER AWARE MIB module
engine,
in
>>>> If there is an implementation of the POWER AWARE MIB module
instantiated
>>>> in the local SNMP engine, then you SHOULD (or MUST?) use it for
indexing
>>>> (pmIndex).
available,
>>>> then you SHOULD use this one (entPhysicalIndex).
only
>>>> and be limited to report on the local device as a single entity
only.
>>>> On 26.10.10 11:44  "Benoit Claise"<[email-masked]>   wrote:
>>>>
>>>>> Hi Juergen,
>>>>>
and
>>>>> Devices MIB module, but reuse in the other MIB modules.
>>>>>
>>>>> Regards, Benoit.
>>>>>> Hi Benoit,
>>>>>>
Battery
>>>> MIB.
>>>>>> Thanks,
>>>>>>        Juergen
>>>>>>
>>>>>>
>>>>>> On 26.10.10 08:22  "Benoit Claise"<[email-masked]>    wrote:
>>>>>>
>>>>>>> Dear all,
>>>>>>>
chartered
>>>>>>> items with the existing draft content.
>>>>>>>
http://tools.ietf.org/html/draft-quittek-power-monitoring-requiremen
>>>>>>> _Energy-aware Networks and Devices MIB_
http://tools.ietf.org/html/draft-parello-eman-energy-aware-mib-00
>>>>>>> _Energy Management Applicability_
http://datatracker.ietf.org/doc/draft-tychon-eman-applicability-stat
draft?
>>>>>>> [email-masked]
>>>> [email-masked]
>>> [email-masked]
>> [email-masked]
> [email-masked]
[email-masked]

---

From: Christian Groves <[email-masked]>
Date: Wed, 03 Nov 2010 13:13:52 +1100

Hello Chris,

I don't have any concrete suggestions for implementing virtual entity 
energy usage support. I'm still at the stage of trying to see what the 
requirements and framework is for the work. There seems to be a focus on 
the MIBs themselves, when without good requirements its hard to see what 
needs they are meeting. The work of the eman WG could address IP type 
servers only or the MIBs could also be used in other telecommunication 
equipment i.e. those used in outside plant. I'm encouraged to see 
Bruce's email on topics for discussion as think its in important to get 
consensus/understanding on these before the MIBs are developed.

Please see my comment below marked [CNG].

Regards, Christian

On 1/11/2010 11:39 PM, Chris Verges wrote:
> like the amount of energy used by a virtual machine running on a 
> shared server? Certainly an interesting problem ... gives a lot of 
> room for innovation for different vendors to find solutions.
>
[CNG] I agree its an interesting problem. In some instances it may be 
possible to allocate the actual energy however in many cases it would be 
very difficult/impossible. In these more difficult cases it may be 
possible to simply report the identities of the virtual/logical entities 
that are included in the physical measurement.
> entities. One question is, “where do you want those entities enumerated?”
>
> enumerated in ENTITY-MIB in the “logical” table?
>
> references between parent and component entities. It would just 
> involve a new pwrEntityType being added for “virtual” or something 
> similar.
>
> So what’s the typical use case as these virtual entities relate to 
> ENTITY-MIB?
>
[CNG] I'm not sure the typical case. Either of the approaches above 
could be sufficient but I think it depends on what people agree on with 
respect to virtual entities and how the data collected on them would be 
used.
> From: [email-masked] [mailto:[email-masked]] On Behalf 
> Of Christian Groves
> As you rightly mention where there is partial use of a physical 
> resource by a logical entity it is very difficult to have an exact 
> figure.
>
> However I think the "grain size" of the measurement comes into play. 
> For example if we have a large gateway implementing several virtual 
> gateways if we take an overall measurement then we have the problem 
> determining a figure. If the however the virtual gateways were each 
> assigned a physical rack and the measurements were taken at a rack 
> level then we have a more precise estimate of contribution of each 
> virtual gateway to the overall picture.
>
> <mailto:[email-masked]>> wrote:
>
> >
>
> >> Hello
>
> >>
>
> >>> Just using entPhysicalIndex solves some of the problem you mentioned.
>
> <mailto:[email-masked]>> wrote:
>
> >>>
>
> >>>> Hi Jurgen and all,
>
> >>>>
>
> >>>> reference to one type (physical or logical) and are limited to 
> that without additional work.
>
> >>>> MIB modules for this purpose running in the same SNMP engine,
>
> <mailto:[email-masked]>> wrote:
>
> >>>>
>
> >>>>> Hi Juergen,
>
> >>>>>
>
> <mailto:[email-masked]>> wrote:
>
> >>>>>>
>
> >>>>>>> Dear all,
>
> >>>>>>>
>

---

From: Christian Groves <[email-masked]>
Date: Wed, 03 Nov 2010 13:21:36 +1100

Hello Mouli,

Please see my comments below.

Regards, Christian

> physical components, cost of measurement sensors should be taken into
[CNG] I certainly agree cost of measurement is an issue. As the IETF are 
producing a new MIB future flexibility/extensibility (and backward 
compatibility) should also be considered. It may be possible to add 
support for logical/virtual entities in a way that does not burden 
vendors who don't want to implement this feature. Considering this issue 
may save costs in the future.
> http://www.sigmetrics.org/sigmetrics2010/greenmetrics/greenMetrics10.Kri
[CNG] Thanks for the link.

---

From: "Chris Verges" <[email-masked]>
Date: Tue, 2 Nov 2010 20:30:23 -0700

Hi Christian et al.,

How about the following two use cases to augment the existing
power-allocation-per-virtual-machine scenario?

	1. "Grouped" outlets on a smart PDU

	2. Tenant submetering on a smart meter

In the first scenario, several smart PDUs allow the user to create a
user-defined grouping that consists of several outlets.  Physical
outlets 1 and 7, for example, could be grouped together as logical
entity "server01".  This mapping could be created using the existing
ENTITY-MIB architecture.  The power data for both the physical and
logical entities are useful to facilities and IT managers to understand
power flow and usage.

In the second scenario, a smart meter provides submetering for a
commercial rental building.  Such buildings often occur turnover in
tenants.  A submeter that meters each circuit in the building, for
example, could maintain that data in both granular and aggregate form.
When a tenant moves in, the facilities manager can create a new logical
entity for that tenant and map the physical circuit entries to the
tenant's logical entity.  This enables both real-time power usage and
management on an individual circuit basis (useful for troubleshooting
facilities problems) as well as on a customized aggregate basis (useful
for the bean counters who have to allocate costs).

Here's an interesting question as far as implementation is concerned ...
why not implement pmPhysicalMap and pmLogicalMap tables (like
entLPMappingTable) to map the pmIndex entity to the appropriate physical
or logical entity (1:1 only?)

Chris

-----Original Message-----
From: Christian Groves [mailto:[email-masked]] 
Sent: Tuesday, November 02, 2010 7:14 PM
To: Chris Verges
Cc: [email-masked]
Subject: Re: [eman] Entity identification method

Hello Chris,

Regards, Christian

enumerated?"

> involve a new pwrEntityType being added for "virtual" or something 
> similar.
>

mentioned.

---

From: Benoit Claise <[email-masked]>
Date: Mon, 08 Nov 2010 15:20:13 +0800

Hi Chris,

Question: could the business context be used to achieve what you want to?
See 
http://tools.ietf.org/html/draft-claise-power-management-arch-02#section-5.4

Regards, Benoit
>>>>>>>>> [email-masked]<mailto:[email-masked]>
>>>>>> [email-masked]<mailto:[email-masked]>
>>>>> [email-masked]<mailto:[email-masked]>
>>>> [email-masked]<mailto:[email-masked]>
>>> [email-masked]<mailto:[email-masked]>
>> [email-masked]<mailto:[email-masked]>
>>

---

From: "Chris Verges" <[email-masked]>
Date: Mon, 8 Nov 2010 07:10:54 -0800

Hi Benoit,

In Section 5.4, the "keywords" and/or "role" fields are perhaps the two
that can *best* achieve the business context being described.  However,
neither of these take into account the data reporting aspect.  For
example:

A server's redundant power supplies are connected to two different
outlets on a smart power strip.  Each outlet is enumerated in the EMAN
MIBs as separate entities, and the "role" is set to "server01",
indicating that these outlets are linking in business context through
the server's hostname.  (Use any other metric you want, the result is
still the same.)  I know how much power each power supply is then
drawing since the MIB splits the data out by outlet.  However, I do not
know the aggregate power draw of the server itself, since there is no
"server01" data entry.  There can't be, as there is no corresponding
pmIndex for the server entity.  What's useful is to know all three
datapoints: the power on supply A, the power on supply B, and the power
on the server as a whole.  Yes, it should be a fairly straight-forward
vector math calculation, but more complex scenarios could be crafted
using three phase, automatic transfer switches, solar panels, etc.

The point is that while the "role" and "keywords" fields can be used to
establish business context for a particular entity in the MIB, I'm wary
over whether they can be used to aggregate data or manage aggregated
control over multiple entities being logically grouped together.  At
least, given the current sparse table extension methodology behind the
proposed set of MIBs.

Perhaps mapping to ENTITY-MIB isn't required at this point.  What if
power entities (POWER-ENTITY-MIB?) are separate from ENTITY-MIB
entirely?  These two tables could later be mapped together through a
separate set of tables to ENTITY-MIB, which would establish the ability
for an M:N mapping between POWER-ENTITY-MIB and the physical and logical
tables of ENTITY-MIB separately.  That would allow us to link the
entities IF IT MAKES SENSE and in a way that makes sense.  But it's not
part of the core definition of a power entity, in a more limited 1:1
fashion.

Thanks,
Chris

-----Original Message-----
From: Benoit Claise [mailto:[email-masked]] 
Sent: Sunday, November 07, 2010 11:20 PM
To: Chris Verges
Cc: Christian Groves; [email-masked]
Subject: Re: [eman] Entity identification method

Hi Chris,

Question: could the business context be used to achieve what you want
to?
See
http://tools.ietf.org/html/draft-claise-power-management-arch-02#section
-5.4

Regards, Benoit
> logical entities are useful to facilities and IT managers to 
> understand power flow and usage.
>
> When a tenant moves in, the facilities manager can create a new 
> logical entity for that tenant and map the physical circuit entries to

> the tenant's logical entity.  This enables both real-time power usage 
> and management on an individual circuit basis (useful for 
> troubleshooting facilities problems) as well as on a customized 
> aggregate basis (useful for the bean counters who have to allocate
costs).
> entLPMappingTable) to map the pmIndex entity to the appropriate 
> physical or logical entity (1:1 only?)
>
> Chris
>
>

> requirements and framework is for the work. There seems to be a focus 
> on the MIBs themselves, when without good requirements its hard to see

> what needs they are meeting. The work of the eman WG could address IP 
> type servers only or the MIBs could also be used in other 
> telecommunication equipment i.e. those used in outside plant. I'm 
> encouraged to see Bruce's email on topics for discussion as think its 
> in important to get consensus/understanding on these before the MIBs
are developed.
> possible to allocate the actual energy however in many cases it would 
> be very difficult/impossible. In these more difficult cases it may be 
> possible to simply report the identities of the virtual/logical 
> entities that are included in the physical measurement.

>> The pwrEntity[Physical|Logical] objects would point to the 
>> appropriate index in ENTITY-MIB.
>>
>> However, if the virtual entities are not enumerated in ENTITY-MIB, 
>> the pwrEntityLogical column is probably overkill. The virtual 
>> entities can be enumerated in pwrEntityTable and linked to the 
>> Physical meter using the pwrEntityParent column, much like how 
>> ENTITY-MIB currently manages references between parent and component 
>> entities. It would just involve a new pwrEntityType being added for 
>> "virtual" or something similar.
>>
> could be sufficient but I think it depends on what people agree on 
> with respect to virtual entities and how the data collected on them 
> would be used.
>> Chris
>>
>> challenge and not straight forward. I guess there are a number of 
>> ways to look at this problem and how address the problem depends on 
>> the ambition level.
>>

>>> It looks like we have to distinguish two cases: power state 
>>> monitoring and energy consumption monitoring.
>>> For a logical entity and a virtual machine it may be well possible 
>>> to report on their power state. However, reporting their energy 
>>> entities, but if you have a logical entity that consumes only 
>>> partial resources of a physical entity, then precise measurement of 
>>> the energy consumption will be hardly possible. Estimations may be 
>>> possible based on a model of the logical entity, but I don't know 
>>> how good such
>> models can be.
>>
>>> Thanks,
>>> Juergen

---

From: john parello <[email-masked]>
Date: Tue, 09 Nov 2010 00:40:47 -0800

Hi Chris,

That is a very good point wrt the entity mib. Our draft calls for 
linking to the entity mib if present. You seem to be sayig that even if 
it is present you may do not want to link to it.

One question for the server power: would you want to report the 
aggragate or the remainder?

Example if you have 8 outlets and the server itself.  Wouldn't there be 
8 readings for the attached devices and then one reading for the server 
which is the draw of the PDU itself to function - albeit small.

For PoE switches that's our intent. We report the draw on each interface 
then the draw for the switch itself to operate.

Jp

---

From: "Chris Verges" <[email-masked]>
Date: Tue, 9 Nov 2010 06:40:00 -0800

Hi John,

Here's an example of what I'd ideally report, indented to show parent/child relationships between the entities:

POWER-ENTITY-MIB:

·         Cyber Switching ePower "Smart PDU"

o    Three Phase Input #1

§  Single Phase Bank #1

·         Single Phase Outlet #1

·         Single Phase Outlet #2

·         ...

§  Single Phase Bank #2

·         Single Phase Outlet #9

·         Single Phase Outlet #10

·         ...

§  ...

o    ...

·         Server01

Note that "Server01" is a separate entity hierarchy from the ePower PDU itself.  In this example, let's say that Server01 points to Outlets #1 and #9, meaning the redundant power supplies on server01 are being powered by those physical outlets.  Server01 is, in effect, a "logical" entity from the perspective of the MIB that is representing a "physical" entity in the real world.

To simplify management, I'd report actual power metered on the ePower PDU hierarchy (the PDU itself, the inputs, the banks, and the outlets) AND aggregated power metered on the Server01 entity.  So if Outlet 1 and Outlet 9 report 2.5 A each, I'd aggregate that to 5.0 A for the Server01 entity.  (I'm using Amperage as an example only; a solution should not be limited to only that data type.)

There could even be a "logical-to-physical" mapping table like is in ENTITY-MIB.  The main difference, however, is that there wouldn't need to be a second table to separate logical and physical entities from each other.  That breakdown is just inane, I think.  Add a column that is the pmEntityType (or whatever name is appropriate) that distinguishes whether this entity is physical(1) or logical(2) or ....

But yes, your initial summary is my main point.  With regards to linking to ENTITY-MIB, we should only link to it IF IT MAKES SENSE FOR OUR PRODUCT.  (Emphasis added, not meaning to "yell.")  By splitting this linkage out to tables, we can enhance the flexibility of the framework.

Thanks,

Chris

-----Original Message-----
From: john parello [mailto:[email-masked]] 
Sent: Tuesday, November 09, 2010 12:41 AM
To: Chris Verges
Cc: [email-masked]
Subject: Re: [eman] Entity identification method

Hi Chris,

That is a very good point wrt the entity mib. Our draft calls for linking to the entity mib if present. You seem to be sayig that even if it is present you may do not want to link to it.

One question for the server power: would you want to report the aggragate or the remainder?

8 readings for the attached devices and then one reading for the server which is the draw of the PDU itself to function - albeit small.

For PoE switches that's our intent. We report the draw on each interface then the draw for the switch itself to operate.

Jp

> In Section 5.4, the "keywords" and/or "role" fields are perhaps the 

> two that can *best* achieve the business context being described.  

> However, neither of these take into account the data reporting aspect.  

> For

> drawing since the MIB splits the data out by outlet.  However, I do 

> not know the aggregate power draw of the server itself, since there is 

> no "server01" data entry.  There can't be, as there is no 

> corresponding pmIndex for the server entity.  What's useful is to know 

> all three

> datapoints: the power on supply A, the power on supply B, and the 

> power on the server as a whole.  Yes, it should be a fairly 

> straight-forward vector math calculation, but more complex scenarios 

> could be crafted using three phase, automatic transfer switches, solar panels, etc.

> The point is that while the "role" and "keywords" fields can be used 

> to establish business context for a particular entity in the MIB, I'm 

> wary over whether they can be used to aggregate data or manage 

> aggregated control over multiple entities being logically grouped 

> together.  At least, given the current sparse table extension 

> methodology behind the proposed set of MIBs.

> separate set of tables to ENTITY-MIB, which would establish the 

> ability for an M:N mapping between POWER-ENTITY-MIB and the physical 

> and logical tables of ENTITY-MIB separately.  That would allow us to 

> link the entities IF IT MAKES SENSE and in a way that makes sense.  

> But it's not part of the core definition of a power entity, in a more 

> limited 1:1 fashion.

> http://tools.ietf.org/html/draft-claise-power-management-arch-02#secti <http://tools.ietf.org/html/draft-claise-power-management-arch-02#secti> 

> on

>> logical entity for that tenant and map the physical circuit entries 

>> to

>> Here's an interesting question as far as implementation is concerned

> ...

>> energy usage support. I'm still at the stage of trying to see what 

>> on the MIBs themselves, when without good requirements its hard to 

>>> flexible power-related MIB that can either map to physical or 

>>> entities. Thus I think virtualization can in some way be supported 

>>> by

>>> allowing energy consumption reports for physical entities to 

>>> indicate

>>> <mailto:[email-masked] <mailto:[email-masked]> >>   wrote:

>>>>> I've just started to digest the various drafts of the eman wg and 

>>>>> I see the comment below on logical entities. It's not clear to me 

>>>>> instances). Even in today's networks we have physical gateways 

>>>>> that are virtualized into different logical gateways. For example:

>>> <mailto:[email-masked] <mailto:[email-masked]> >>   wrote:

>>>>>>> something doesn't seem quite right. I'll try to explain my 

>>>>>>> thought process behind this and see if my concerns/confusions 

>>>>>>> get

>> captured.

>>>>>>> Because of this, the MIBs being discussed by the EMAN WG must 

>>>>>>> only reference to one type (physical or logical) and are limited 

>>>>>>> to

>>>>>>> At some point in my mind, I'm asking myself the question, 

>>>>>>> "What's the point of linking to ENTITY-MIB?"

>>>>>>> gives a guaranteed unique index based on all entities, 

>>>>>>> regardless of type. The physical vs. logical extensions would 

>>>>>>> then become sparse tables that augment this. For example:

>>>>>>> It seems like this 'entGeneralIndex' concept is what's missing 

>>>>>>> from the current ENTITY-MIB, and is causing problems when 

>>>>>>> thinking about local and remote power data being reported by the 

>>>>>>> same agent.

>>>>>>> Another possibility would be to choose one -- entLogicalIndex, 

>>>>>>> most likely -- and require that anyone who wants to implement 

>>>>>>> the EMAN WG MIBs to always create an entLogicalEntry for the 

>>>>>>> power data.

>>>>>>> In

>>>>>>> this context, the entLogicalTable fills the role of 

>>>>>>> entGeneralTable as described above. For all entPhysicalTable 

>>>>>>> entries, there would need to be a corresponding entry in 

>>>>>>> entLogicalTable with a 1:1 mapping between them. For all remote 

>>>>>>> entries, there would be a unique entry in entLogicalTable. Of 

>>>>>>> course, this would add some additional overhead on the part of 

>>>>>>> the implementer, but isn't too

>>>>>>> A third possibility is that ENTITY-MIB just doesn't do what we 

>>>>>>> need in its present form and we need to think about enumerating 

>>>>>>> power sensors separately from other components/sensors on the 

>>>>>>> system.

>>>>>>> It

>>>>>>> new 'POWER-ENTITY-MIB', but can be extended to include support 

>>>>>>> for remote agents. The difference would be a clean break from 

>>>>>>> the legacy ENTITY-MIB structure.

>>>>>>> chartered itemsversus drafts) Hi Juergen and all, Agreed that a 

>>>>>>> common ID method makes sense, and ENTITY-MIB seems to be a good 

>>>>>>> choice. In case (c) from your list, wouldn't the remote agent 

>>>>>>> condition be handled by ENTITY-MIB supporting a logical entity 

>>>>>>> that is the remote agent iself? Since the logical entity is in 

>>>>>>> the ENTITY-MIB table, it would be given a unique ID mixed 

>>>>>>> At that point, should all of the MIBs being discussed in the 

>>>>>>> EMAN WG be sparse table augmentations of ENTITY-MIB?

>>>>>>> Subject: [eman] Entity identification method (was: EMAN 

>>>>>>> chartered items versus drafts) Hi Benoit and all, I agree that 

>>>>>>> we should find a common identification method for entities used 

>>>>>>> by all MIB modules. This does not apply to the POWER MIB only, 

>>>>>>> but to all MIB modules in the eman WG.

>>>>>>> (draft-parello-eman-energy-aware-mib) or the ENTITY MIB module 

>>>>>>> (RFC 4133).

>>>>>>> Such a device usually does not need further identification of 

>>>>>>> itself, because typically there are sufficient other MIB modules 

>>>>>>> for this purpose running in the same SNMP engine, that provide 

>>>>>>> information about the device's IP address, manufacturer, 

>>>>>>> operating system, etc. In such a case a 'trivial' index, such as 

>>>>>>> '0' should be used in order to keep it simple in this most 

>>>>>>> simple case.

>>>>>>> b) a device reports on its own but not just as a single unit but 

>>>>>>> it reports power states and energy consumption for its 

>>>>>>> individual components, for example it may report separately on 

>>>>>>> contained hard drives, line cards, back planes, processor 

>>>>>>> boards, etc.

>>>>>>> In such a case, identification of these components as individual 

>>>>>>> entities would be required. The ENTITY MIB module was designed 

>>>>>>> for this purpose and would be a good choice here. Also the POWER 

>>>>>>> AWARE MIB module would be useful in this case.

>>>>>>> contained components need to be identified. For identifying 

>>>>>>> remote components there is the POWER AWARE MIB module that has 

>>>>>>> been designed for this purpose. As far as I understand, the 

>>>>>>> ENTITY MIB module is not applicable to remote devices.

>>>>>>> - if you need to report on remote entities (case c)), you need 

>>>>>>> the POWER AWARE MIB module,

>>>>>>> - if you report only on entities locally contained in the 

>>>>>>> reporting device (case b)), you can use the POWER AWARE MIB or 

>>>>>>> the ENTITY MIB

>>>>>>> - if you report just on your own as a single device (case a)), 

>>>>>>> identification is trivial Hence, my recommendation (stated for 

>>>>>>> POWER-STATE MIB and ENERGY MIB in

>>>>>>> instantiated in the local SNMP engine, then you SHOULD (or 

>>>>>>> MUST?) use it for indexing (pmIndex).

>>>>>>> If neither of this MIB modules is available you should use index

>>> <mailto:[email-masked] <mailto:[email-masked]> >>   wrote:

>>>>>>>> Something key is to agree on the Power Monitor index for all 

>>>>>>>> MIB modules, which IMHO should be part of the Energy-aware 

>>>>>>>> Networks and Devices MIB module, but reuse in the other MIB modules.

>>>>>>>>> I don't think that draft-quittek-power-mib-02 makes 

>>>>>>>>> significant contributions to the Energy-aware Networks and Devices MIB.

>>> <mailto:[email-masked] <mailto:[email-masked]> >>   wrote:

>>>>>>>>>> http://tools.ietf.org/html/draft-quittek-power-monitoring-req <http://tools.ietf.org/html/draft-quittek-power-monitoring-req> 

>>>>>>>>>> u

>>>>>>>>>> ir

>>>>>>>>>> https://datatracker.ietf.org/doc/draft-norwin-energy-consider <https://datatracker.ietf.org/doc/draft-norwin-energy-consider> 

>>>>>>>>>> /

>>>>>>>>>> http://tools.ietf.org/html/draft-claise-power-management-arch <http://tools.ietf.org/html/draft-claise-power-management-arch> 

>>>>>>>>>> -

>>>>>>>>>> 02

>>>>>>>>>> http://tools.ietf.org/html/draft-parello-eman-energy-aware-mi <http://tools.ietf.org/html/draft-parello-eman-energy-aware-mi> 

>>>>>>>>>> b

>>>>>>>>>> -0

>>>>>>>>>> http://tools.ietf.org/html/draft-quittek-power-mib-02 <http://tools.ietf.org/html/draft-quittek-power-mib-02> 

>>>>>>>>>> http://tools.ietf.org/html/draft-claise-energy-monitoring-mib <http://tools.ietf.org/html/draft-claise-energy-monitoring-mib> 

>>>>>>>>>> -

>>>>>>>>>> 06

>>>>>>>>>> http://tools.ietf.org/html/draft-quittek-power-mib-02 <http://tools.ietf.org/html/draft-quittek-power-mib-02> 

>>>>>>>>>> http://tools.ietf.org/html/draft-quittek-power-mib-02 <http://tools.ietf.org/html/draft-quittek-power-mib-02> 

>>>>>>>>>> http://datatracker.ietf.org/doc/draft-tychon-eman-applicabili <http://datatracker.ietf.org/doc/draft-tychon-eman-applicabili> 

>>>>>>>>>> t

>>>>>>>>>> y-

>>>>>>>>>> [email-masked]<mailto:[email-masked] <mailto:[email-masked]%3cmailto:[email-masked]> >

>>>>>>>>>> https://www.ietf.org/mailman/listinfo/eman <https://www.ietf.org/mailman/listinfo/eman> 

>>>>>>> [email-masked]<mailto:[email-masked] <mailto:[email-masked]%3cmailto:[email-masked]> >

>>>>>>> https://www.ietf.org/mailman/listinfo/eman <https://www.ietf.org/mailman/listinfo/eman> 

>>>>>> [email-masked]<mailto:[email-masked] <mailto:[email-masked]%3cmailto:[email-masked]> >

>>>>>> https://www.ietf.org/mailman/listinfo/eman <https://www.ietf.org/mailman/listinfo/eman> 

>>>>> [email-masked]<mailto:[email-masked] <mailto:[email-masked]%3cmailto:[email-masked]> >

>>>>> https://www.ietf.org/mailman/listinfo/eman <https://www.ietf.org/mailman/listinfo/eman> 

>>>> [email-masked]<mailto:[email-masked] <mailto:[email-masked]%3cmailto:[email-masked]> >

>>>> https://www.ietf.org/mailman/listinfo/eman <https://www.ietf.org/mailman/listinfo/eman> 

>>> [email-masked]<mailto:[email-masked] <mailto:[email-masked]%3cmailto:[email-masked]> >

>>> https://www.ietf.org/mailman/listinfo/eman <https://www.ietf.org/mailman/listinfo/eman> 

>> https://www.ietf.org/mailman/listinfo/eman <https://www.ietf.org/mailman/listinfo/eman> 

> https://www.ietf.org/mailman/listinfo/eman <https://www.ietf.org/mailman/listinfo/eman>