Subject: [Last-Call] RtgDir Last Call review: draft-ietf-opsawg-ntf-10

From: Dhruv Dhody <[email-masked]>
Date: Fri, 19 Nov 2021 18:54:51 +0530

Hello,

I have been selected as the Routing Directorate reviewer for this draft.
The Routing Directorate seeks to review all routing or routing-related
drafts as they pass through IETF last call and IESG review, and sometimes
on special request. The purpose of the review is to provide assistance to
the Routing ADs. For more information about the Routing Directorate, please
see http://trac.tools.ietf.org/area/rtg/trac/wiki/RtgDir

Although these comments are primarily for the use of the Routing ADs, it
would be helpful if you could consider them along with any other IETF Last
Call comments that you receive, and strive to resolve them through
discussion or by updating the draft.

Document: draft-ietf-opsawg-ntf-10
Reviewer: Dhruv Dhody
Review Date: 2021-11-19
IETF LC End Date: over
Intended Status: Informational

Summary:
Choose from this list…

   - I have some minor concerns about this document that I think should be
   resolved before publication.

Comments:

   - I find the document to be useful. It is well structured and easy to
   read. Since the aim of the document is to clarify the taxonomy and
   framework, I hope to see more drafts to refer to it when describing network
   telemetry. I have a few suggestions of things that are missing, some
   queries and nits that are easy to resolve, and hopefully improve the
   document further.

Minor Issues:

   - Well almost Major :)
      - Something that I find missing in the document is that the network
      controller could be a valuable source of network telemetry as well.
      Consider a PCE, the controller could be a source of network-wide
data, such
      as the association between network paths, cumulative network metrics,
      global network utilization, etc. The document is currently very
      network-device-specific (as a data source). My suggestion would be to
      handle the centralized controller either as a separate section or part of
      the control plane and management plane telemetry.
      - Something else that I find missing is the multi-domain aspects. You
      could mark it as out of scope or better yet do talk about it how there
      could possibly be a hierarchy and recursive nature in your framework to
      handle multi-domain. Currently, it is mentioned in passing while
describing
      data fusion in section 3.4.
   - Query
      - Section 4.1
         - In figure 2, why MIB is mentioned in the management plane only,
         why not control plane when various control plane protocols have MIBs?
         Similarly, there are forwarding statistics MIB that might work in the
         forwarding plane? Also, add SNMP and ASN.1(?) in the table
corresponding to
         MIB.
         - What is a ‘mirror’? Maybe expand it or put a * and expand it at
         the bottom
         - All external data coming from gRPC only?
      - Others
      - Section 6
         - The Independent management network is mentioned only in passing.
         Shouldn’t there be a much stronger recommendation for this instead?

Nits:

   - From IDNits

  == Outdated reference: draft-ietf-grow-bmp-adj-rib-out has been published
     as RFC 8671

  == Outdated reference: draft-ietf-ippm-multipoint-alt-mark has been
     published as RFC 8889

  == Outdated reference: A later version (-11) exists of
     draft-song-ippm-postcard-based-telemetry-10

   - Section 2
      - Add reference for
         - GPB - https://developers.google.com/protocol-buffers
         - IOAM - draft-ietf-ippm-ioam-data
         - NetFlow - reference is incorrect, it should be RFC 3954
         - SNMP - for the sake of covering all versions, we should mention
         v3 as well - RFC 3414
      - Section 3.3
      - Add reference for
         - Syslog - RFC 5424
         - sFlow - RFC 3176
      - Expand PSAMP - Packet Sampling
   - Section 4.1
      - The list of 6 angles in the text and the 1st column in the table do
      not match.
      - Expand ASICs - Application-Specific Integrated Circuits
   - Section 4.1.1
      - The use of the term “server” can be confusing here. Would you
      consider using “data source”?
   - Section 4.1.2
      - I am unaware of the term “video fluency”. Is it a term of art that
      I am unaware of?
      - Add reference to Y.1731 - “ITU-T, “OAM Functions and Mechanisms for
      Ethernet based networks”, ITU-T Y.1731, 2006.”
   - Section A.1.2
      - gNMI reference is marked as [I-D.openconfig-rtgwg-gnmi-spec]
      whereas, in the main body, it is [gnmi] “gNMI - gRPC Network Management
      Interface”,
      https://github.com/openconfig/reference/tree/master/rpc/gnmi. Any
      reason for different references?
      - same for gRPC!
   - Section A.3.6
      - Expand L2VPN, NVO3, BIER, SFC, DETNET
      - Is there anything about SR and Multicast worth adding to the list?

Thanks!
Dhruv

---

From: Haoyu Song <[email-masked]>
Date: Fri, 19 Nov 2021 20:54:26 +0000

Hi Dhruv,

Thank you very much for the review! Please see inline for our response and proposed modifications. We'll update the document once we hear back from you. Thanks!

Best regards,
Haoyu

From: Dhruv Dhody <[email-masked]>
Sent: Friday, November 19, 2021 5:25 AM
To: [email-masked]; [email-masked]; [email-masked]
Cc: [email-masked]; [email-masked]; opsawg <[email-masked]>
Subject: RtgDir Last Call review: draft-ietf-opsawg-ntf-10

Hello,

I have been selected as the Routing Directorate reviewer for this draft. The Routing Directorate seeks to review all routing or routing-related drafts as they pass through IETF last call and IESG review, and sometimes on special request. The purpose of the review is to provide assistance to the Routing ADs. For more information about the Routing Directorate, please see http://trac.tools.ietf.org/area/rtg/trac/wiki/RtgDir<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftrac.tools.ietf.org%2Farea%2Frtg%2Ftrac%2Fwiki%2FRtgDir&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cb5edb230deb64a7af63808d9ab6016eb%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637729251465898432%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ivEfRbM06rY3Nz6buTpSzp6s5PBIdE6lPKO0lTtzMds%3D&reserved=0>

Although these comments are primarily for the use of the Routing ADs, it would be helpful if you could consider them along with any other IETF Last Call comments that you receive, and strive to resolve them through discussion or by updating the draft.

Summary:
Choose from this list...
* I have some minor concerns about this document that I think should be resolved before publication.

Comments:
* I find the document to be useful. It is well structured and easy to read. Since the aim of the document is to clarify the taxonomy and framework, I hope to see more drafts to refer to it when describing network telemetry. I have a few suggestions of things that are missing, some queries and nits that are easy to resolve, and hopefully improve the document further.

Minor Issues:
* Well almost Major :)
o   Something that I find missing in the document is that the network controller could be a valuable source of network telemetry as well. Consider a PCE, the controller could be a source of network-wide data, such as the association between network paths, cumulative network metrics, global network utilization, etc. The document is currently very network-device-specific (as a data source). My suggestion would be to handle the centralized controller either as a separate section or part of the control plane and management plane telemetry.
[HS] We hold a view on a telemetry system that it collects data from different network planes, in which a centralized controller, if exists,  is the host for the telemetry  applications and a consumer for the data. In a sense, a controller acquires the original data from various network planes and it might further process the data and even generate new data based on telemetry data for applications. We summarize this process as "network operation applications" in the framework. Is this an acceptable view?
o   Something else that I find missing is the multi-domain aspects. You could mark it as out of scope or better yet do talk about it how there could possibly be a hierarchy and recursive nature in your framework to handle multi-domain. Currently, it is mentioned in passing while describing data fusion in section 3.4.
[HS] Multi-domain has many particular issues we think we should leave out of scope, although the general framework would still fit. We will make it clear in the new revision.
* Query
o   Section 4.1
?  In figure 2, why MIB is mentioned in the management plane only, why not control plane when various control plane protocols have MIBs? Similarly, there are forwarding statistics MIB that might work in the forwarding plane? Also, add SNMP and ASN.1(?) in the table corresponding to MIB.
[HS] In the text we "note that the  selected techniques [in the figure] just reflect the de facto state of the art and are by no means exhaustive." Here we only mean to provide some representatives in each category.
?  What is a 'mirror'? Maybe expand it or put a * and expand it at the bottom
[HS] We'll expand it as suggested.
?  All external data coming from gRPC only?
[HS] No necessarily. Note that this is just a representative. We don't mean to provide an exhaustive list.
* Others
o   Section 6
?  The Independent management network is mentioned only in passing. Shouldn't there be a much stronger recommendation for this instead?
[HS] In this section we just list the security considerations and the recommended solution is out of scope of this document. If you have some mature practices to recommend, we'd like to add a few sentences on this.

Nits:
* From IDNits

     as RFC 8671

     published as RFC 8889

     draft-song-ippm-postcard-based-telemetry-10
* Section 2
o   Add reference for
?  GPB - https://developers.google.com/protocol-buffers<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopers.google.com%2Fprotocol-buffers&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cb5edb230deb64a7af63808d9ab6016eb%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637729251465908382%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mQZlJJs4UrLeLkEui0aoeOibD%2B%2BnPybDp2ZkHkbMgZA%3D&reserved=0>
?  IOAM - draft-ietf-ippm-ioam-data
?  NetFlow - reference is incorrect, it should be RFC 3954
?  SNMP - for the sake of covering all versions, we should mention v3 as well - RFC 3414
* Section 3.3
o   Add reference for
?  Syslog - RFC 5424
?  sFlow - RFC 3176
o   Expand PSAMP - Packet Sampling
* Section 4.1
o   The list of 6 angles in the text and the 1st column in the table do not match.
o   Expand ASICs - Application-Specific Integrated Circuits
* Section 4.1.1
o   The use of the term "server" can be confusing here. Would you consider using "data source"?
* Section 4.1.2
o   I am unaware of the term "video fluency". Is it a term of art that I am unaware of?
o   Add reference to Y.1731 - "ITU-T, "OAM Functions and Mechanisms for Ethernet based networks", ITU-T Y.1731, 2006."
* Section A.1.2
o   gNMI reference is marked as [I-D.openconfig-rtgwg-gnmi-spec] whereas, in the main body, it is [gnmi] "gNMI - gRPC Network Management Interface", https://github.com/openconfig/reference/tree/master/rpc/gnmi<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenconfig%2Freference%2Ftree%2Fmaster%2Frpc%2Fgnmi&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cb5edb230deb64a7af63808d9ab6016eb%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637729251465908382%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ELnuFuo1S7ecfZ4juK%2BdRDUoYaZWpnbb1O35IosmQRo%3D&reserved=0>. Any reason for different references?
o   same for gRPC!
* Section A.3.6
o   Expand L2VPN, NVO3, BIER, SFC, DETNET
o   Is there anything about SR and Multicast worth adding to the list?

[HS] Thanks for catching the nits. All these will be fixed in the new revision.

Thanks!
Dhruv

---

From: Dhruv Dhody <[email-masked]>
Date: Mon, 22 Nov 2021 09:52:09 +0530

Hi Haoyu,

Thanks for taking my comments into consideration. Focusing on the open
points...

o   Something that I find missing in the document is that the network
> Consider a PCE, the controller could be a source of network-wide data, such
> [HS] We hold a view on a telemetry system that it collects data from
> different network planes, in which a centralized controller, if exists,  is
> the host for the telemetry  applications and a consumer for the data. In a
> sense, a controller acquires the original data from various network planes
> and it might further process the data and even generate new data based on
> telemetry data for applications. We summarize this process as “network
> operation applications” in the framework. Is this an acceptable view?
>

Dhruv: In most cases that could be true. I do find it limiting to assume
all such applications are hosted at a single central controller. Looking at
figure 3 in RFC 8309 - https://www.rfc-editor.org/rfc/rfc8309.html#section-4,
it is possible that the bottom controller does the telemetry handling but
the analysis and application happen at the orchestrator level.

Anyways, maybe you could add some version of the below text -

Note that in some cases the controller itself may be the source of
> telemetry data that is unique to it or derived from the telemetry data
> collected from the network elements. Some of the principles and taxonomy
> specific to the control plane and management plane telemetry could also be
> applied to the controller when it is required to provide the telemetry data
> to Network Operation Applications hosted outside. The scope of the document
> is focused on the network elements telemetry and further details related to
> controllers are thus out of scope.

> o   Something else that I find missing is the multi-domain aspects. You
> handle multi-domain. Currently, it is mentioned in passing while describing
> [HS] Multi-domain has many particular issues we think we should leave out
> of scope, although the general framework would still fit. We will make it
> clear in the new revision.
>

Dhruv: Looking forward to the text!

> · Query
>
> o   Section 4.1
>
> §  In figure 2, why MIB is mentioned in the management plane only, why
> not control plane when various control plane protocols have MIBs?
> forwarding plane? Also, add SNMP and ASN.1(?) in the table corresponding to
> MIB.
>
> [HS] In the text we “note that the  selected techniques [in the figure]
> just reflect the de facto state of the art and are by no means exhaustive.”
> Here we only mean to provide some representatives in each category.
>

Dhruv: I understand that, it stood out to me because YANG was mentioned in
other planes but not MIB.

Thanks!
Dhruv

---

From: Haoyu Song <[email-masked]>
Date: Mon, 22 Nov 2021 17:39:51 +0000

Thank you very much Dhruv, the text you proposed has been included.

Best regards,
Haoyu

From: Dhruv Dhody <[email-masked]>
Sent: Sunday, November 21, 2021 8:22 PM
To: Haoyu Song <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; opsawg <[email-masked]>
Subject: Re: RtgDir Last Call review: draft-ietf-opsawg-ntf-10

Hi Haoyu,

Thanks for taking my comments into consideration. Focusing on the open points...

Dhruv: In most cases that could be true. I do find it limiting to assume all such applications are hosted at a single central controller. Looking at figure 3 in RFC 8309 - https://www.rfc-editor.org/rfc/rfc8309.html#section-4<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rfc-editor.org%2Frfc%2Frfc8309.html%23section-4&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cf87d7c629ea14c2188ee08d9ad6fbced%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637731517699454988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=phgAdnLv9stOKAz5kQq0Kxq5Ga30Ik5cVd1V9rh5fV8%3D&reserved=0>, it is possible that the bottom controller does the telemetry handling but the analysis and application happen at the orchestrator level.

Note that in some cases the controller itself may be the source of telemetry data that is unique to it or derived from the telemetry data collected from the network elements. Some of the principles and taxonomy specific to the control plane and management plane telemetry could also be applied to the controller when it is required to provide the telemetry data to Network Operation Applications hosted outside. The scope of the document is focused on the network elements telemetry and further details related to controllers are thus out of scope.

* Query
o   Section 4.1
*  In figure 2, why MIB is mentioned in the management plane only, why not control plane when various control plane protocols have MIBs? Similarly, there are forwarding statistics MIB that might work in the forwarding plane? Also, add SNMP and ASN.1(?) in the table corresponding to MIB.
[HS] In the text we "note that the  selected techniques [in the figure] just reflect the de facto state of the art and are by no means exhaustive." Here we only mean to provide some representatives in each category.

Dhruv: I understand that, it stood out to me because YANG was mentioned in other planes but not MIB.

Thanks!
Dhruv