Subject: [OPSAWG] Thoughts on draft-song-opsawg-ntf

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 16 Oct 2018 20:39:36 +0100

Hi authors and working group.

I just had cause to read this document and thought I would share my
comments on the list.

The draft appears as -00, but it is a little more mature than that 
implies because it replaces draft-song-ntf-02.

I think a foundation document on telemetry would be useful for the IETF,
and the OPSAWG may be a good place to discuss and progress that work. 
This document seems like a reasonable starting point for that work 
although there will inevitably need to be some additions and 
modifications. Actually, I found this document pretty complete.

My comments are a combination of substantive issues and nits.

Hope this helps.

Regards,
Adrian

== Discussion points ==

In 1.4 you have
   Since SNMP is poll-based, it incurs
   low data rate and high processing overhead.
I don't think this is quite fair on SNMP. The protocol also includes
Notifications allowing information to be output by a device. Your 
main point about low data rates and high overhead, still stand, but
with a different slant.

---

Although you talk about IPFIX quite a bit later in the document, I 
think that, as part of the scene setting, you might include a paragraph
about it early in 1.4.

---

The text after Figure 1 did not completely convince me of the need for 
interaction between the three "telemetry planes". Perhaps you could
expand the examples and discussion to highlight/clarify this?

---

Clearly you are going to have to write some Security Considerations. 
There is quite a lot to say about what is a layered security model
with plenty of attack vectors, and substantial risks as the telemetry
data can result in operational changes that could harm data delivery and
even bring the network down. Furthermore, examination of telemetry data
can reveal weaknesses in the network.

== Nits ==

The authors will need to reduce the number of front page names to 5.
Move the others to a Contributors section. Might as well do this sooner
rather than later.

---

Abstract

   This document suggests the necessity of an architectural framework
   for network telemetry in order to meet the current and future network
   operation requirements.

Suggest you change this a little because you actually provide the 
framework. How about...

   This document provides an architectural framework for network
   telemetry in order to meet the current and future network operation 
   requirements.

---

You need to write an Introduction section and present it as Section 1.
You don't need a lot of text: start with the Abstract and expand it a
little.

---

The Requirements Language text should move to a new Section 1.1

---

You should expand the abbreviations on first use. I see:
AI
ARCA
ML
OAM
SDN

---

A couple of the terms in 1.3 are confusing.

> gNMI:  gPRC Network Management Interface
Should that be gRPC?

> gRPC:  gRPC Remote Procedure Call
This seems circular.

---

Not sure that RFC 1157 is the best reference for SNMP in section 1.4.
How about 3416?

---

Figure 3
Does gPRC mean gRPC?

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 17 Oct 2018 10:48:37 +0800

Dear Mr. Farrel,

Thank you very muchu for your review and detailed comments. We will update the doc accordingly.
As one of the authors from the operators, I think this is a good start point to contine the telemetry related work in OPSAWG. We belive telemetry will be used in the network to enhance the OPS capability and level. Especially in the SDN era, we need the detail network and service status information, which is suitable to be provided by telemetry, to develop the capability of the SDN archeticture.

Best Regards,
Zhenqiang Li

[email-masked]
 
From: Adrian Farrel
Date: 2018-10-17 03:39
To: [email-masked]
CC: [email-masked]
Subject: Thoughts on draft-song-opsawg-ntf
Hi authors and working group.

 

Hope this helps.
 
Regards,
Adrian
 
== Discussion points ==
 
In 1.4 you have
 
---

---

---

== Nits ==

---
 
Abstract

 
 
---

---

---

---

Should that be gRPC?
 
This seems circular.
 
---

---
 
Figure 3
Does gPRC mean gRPC?

---

From: Tianran Zhou <[email-masked]>
Date: Wed, 17 Oct 2018 03:08:39 +0000

Hi Adrian,

Thank you very much for your careful review and comments.

On this discussion:
"In 1.4 you have
   Since SNMP is poll-based, it incurs
   low data rate and high processing overhead.
I don't think this is quite fair on SNMP. The protocol also includes Notifications allowing information to be output by a device. Your main point about low data rates and high overhead, still stand, but with a different slant."

Do you mean the SNMP trap, which is used for alarm and event?
I agree, it's push based notification with binary encoding. However, 
1. no easy CRUD operations
2. no customizable periodical and on-change export. So have to use poll operation.
3. need special definition and support, not apply for any mib data.

Do you think the above are the SNMP defeat for telemetry use?

Or what your thoughts?

Thanks,
Tianran

> -----Original Message-----
> From: Adrian Farrel [mailto:[email-masked]]
> Sent: Wednesday, October 17, 2018 3:40 AM
> To: [email-masked]
> Cc: [email-masked]
> I just had cause to read this document and thought I would share my comments
> on the list.
> 
> The draft appears as -00, but it is a little more mature than that implies
> because it replaces draft-song-ntf-02.
> 
> I think a foundation document on telemetry would be useful for the IETF, and
> the OPSAWG may be a good place to discuss and progress that work.
> This document seems like a reasonable starting point for that work although
> there will inevitably need to be some additions and modifications. Actually,
> I found this document pretty complete.
> 
> Notifications allowing information to be output by a device. Your main point
> about low data rates and high overhead, still stand, but with a different
> slant.
> 
> ---
> 
> Although you talk about IPFIX quite a bit later in the document, I think that,
> as part of the scene setting, you might include a paragraph about it early
> in 1.4.
> 
> ---
> 
> interaction between the three "telemetry planes". Perhaps you could expand
> the examples and discussion to highlight/clarify this?
> 
> ---
> 
> There is quite a lot to say about what is a layered security model with plenty
> of attack vectors, and substantial risks as the telemetry data can result
> in operational changes that could harm data delivery and even bring the
> network down. Furthermore, examination of telemetry data can reveal
> weaknesses in the network.
> 
> == Nits ==
> 
> Move the others to a Contributors section. Might as well do this sooner rather
> than later.
> 
> ---
> 
> Abstract
> 
> Suggest you change this a little because you actually provide the framework.
> How about...
> 
> You don't need a lot of text: start with the Abstract and expand it a little.
> 
> ---
>

---

From: "Adrian Farrel" <[email-masked]>
Date: Wed, 17 Oct 2018 10:26:34 +0100

Hi Tianran,

Yes, I mean the "Trap" which is now called "Notification."

You're right in your assessment of the drawbacks, and you should add to that
list the drawback of the low data rat and high overhead that come with the
protocol.

I didn't mention the Notification to propose it as a mechanism for telemetry,
but just so that your introductory text has a complete description of the
pre-existing environment.

Best,
Adrian

> From: Tianran Zhou [mailto:[email-masked]]
> Sent: 17 October 2018 04:09
> To: [email-masked]; [email-masked]
> Subject: RE: Thoughts on draft-song-opsawg-ntf
> 
> Hi Adrian,
> 
Notifications
> allowing information to be output by a device. Your main point about low data
> rates and high overhead, still stand, but with a different slant."
> 
rather
little.

---

From: Tianran Zhou <[email-masked]>
Date: Wed, 17 Oct 2018 09:49:50 +0000

Hi Adrian,

I see. Thank you very much for your suggestions.
We will update the document to clarify this.

Thanks,
Tianran

> Sent: Wednesday, October 17, 2018 5:27 PM
> To: Tianran Zhou <[email-masked]>; [email-masked]
> > allowing information to be output by a device. Your main point about
> > low data rates and high overhead, still stand, but with a different slant."
> >
> > 1. no easy CRUD operations 2. no customizable periodical and on-change
> > export. So have to use poll operation.
> > > I think a foundation document on telemetry would be useful for the
> > > IETF, and the OPSAWG may be a good place to discuss and progress that
> work.
> > > think
> that,
> > > as part of the scene setting, you might include a paragraph about it
> > > early in 1.4.
> > >
> > > ---
> > >
> > > The text after Figure 1 did not completely convince me of the need
> > > for interaction between the three "telemetry planes". Perhaps you
> > > could expand the examples and discussion to highlight/clarify this?
> > >
> > > ---
> > >
> > > with
> plenty
> > > of attack vectors, and substantial risks as the telemetry data can
> > > result in operational changes that could harm data delivery and even
> > > bring the network down. Furthermore, examination of telemetry data
> framework.

---

From: "Blumenthal, Uri - 0553 - MITLL" <[email-masked]>
Date: Wed, 17 Oct 2018 12:55:24 +0000

SNMP is a management protocol, not a data transfer one. 

SNMP Trap is like a UDP message - send and forget.

SNMP Notify can require confirmation, so you can make sure it reaches it's destination (or know that it could not).

I rather disagree with the characterization "low data rate and high overhead" - compared to what, and by what criteria?

P.S. IMHO, SNMP Traps are perfect for telemetry.

Regards,
Uri

Sent from my iPhone

> On Oct 17, 2018, at 04:27, Adrian Farrel <[email-masked]> wrote:
> 
> Hi Tianran,
> 
> _______________________________________________
> OPSAWG mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/opsawg

---

From: Haoyu song <[email-masked]>
Date: Wed, 17 Oct 2018 15:37:55 +0000

Hi Uri,

Thanks for the comments. I'd like to clarify there's no black and white separation between conventional OAM and telemetry. Conventional OAM is also indispensable in many cases and some tools is also evolving to bear more characteristics of a typical telemetry system. I'll make the modification and clarification accordingly.

Haoyu

-----Original Message-----
From: Blumenthal, Uri - 0553 - MITLL [mailto:[email-masked]] 
Sent: Wednesday, October 17, 2018 5:55 AM
To: [email-masked]
Cc: Tianran Zhou <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [OPSAWG] Thoughts on draft-song-opsawg-ntf

Regards,
Uri

Sent from my iPhone

---

From: Randy Presuhn <[email-masked]>
Date: Wed, 17 Oct 2018 17:38:01 -0700

Hi -

On 10/16/2018 8:08 PM, Tianran Zhou wrote:
...
> 2. no customizable periodical and on-change export. So have to use poll operation.

Have you tried RFC 2981 or RFC 3877?
If so, what aspects of their design did you find problematic?

> 3. need special definition and support, not apply for any mib data.

I don't understand what this is intended to mean.

Randy

---

From: Tianran Zhou <[email-masked]>
Date: Thu, 18 Oct 2018 01:37:56 +0000

Hi Randy,

Thank you for your pointer.
I do not mean to say the SNMP design is problematic. But I think it's not designed for periodically getting operational data, which is one important case for streaming telemetry.
Compared with current YANG-Push (https://datatracker.ietf.org/doc/draft-ietf-netconf-yang-push) activity, user can get any data described in existing YANG data store (not only the notifications), and can appoint the period when the data originator pushes the event.  

Thanks,
Tianran

> From: OPSAWG [mailto:[email-masked]] On Behalf Of Randy Presuhn
> Sent: Thursday, October 18, 2018 8:38 AM
> To: [email-masked]
> ....

---

From: Randy Presuhn <[email-masked]>
Date: Wed, 17 Oct 2018 21:50:31 -0700

Hi -

On 10/17/2018 6:37 PM, Tianran Zhou wrote:
> I do not mean to say the SNMP design is problematic.
> But I think it's not designed for periodically getting
> operational data, which is one important case for streaming telemetry.

That's one of the possible use cases for RFC 2981 or RFC 3877,
and was considered in their design.  If you think those MIB modules
are inadequate, it would be helpful for you to spell out exactly
why they fail to meet the need.

> Compared with current YANG-Push
> (https://datatracker.ietf.org/doc/draft-ietf-netconf-yang-push) activity,
> user can get any data described in existing YANG data store (not only the
> notifications), and can appoint the period when the data originator pushes
> the event.

I really do recommend you look at RFC 2981.  It provides
exactly that capability.  Again, if you think RFC 2981
does not meet that need, I ask you to spell out exactly
how the design is deficient.

Randy

---

From: Haoyu song <[email-masked]>
Date: Thu, 18 Oct 2018 05:15:36 +0000

Hi Adrian,

Thank you very much for the comments. I'll try to address all your comments in the new revisions. Given the time and bandwidth, I may leave some parts (e.g., security concern) to future revisions. 

Yes, gRPC is a recursive acronym and "g" doesn't mean google although we all know where it comes from ;)

Haoyu
-----Original Message-----
From: Adrian Farrel [mailto:[email-masked]] 
Sent: Tuesday, October 16, 2018 12:40 PM
To: [email-masked]
Cc: [email-masked]
Subject: Thoughts on draft-song-opsawg-ntf

I just had cause to read this document and thought I would share my comments on the list.

The draft appears as -00, but it is a little more mature than that implies because it replaces draft-song-ntf-02.

I think a foundation document on telemetry would be useful for the IETF, and the OPSAWG may be a good place to discuss and progress that work. 
This document seems like a reasonable starting point for that work although there will inevitably need to be some additions and modifications. Actually, I found this document pretty complete.

Hope this helps.

Regards,
Adrian

== Discussion points ==

In 1.4 you have
   Since SNMP is poll-based, it incurs
   low data rate and high processing overhead.
I don't think this is quite fair on SNMP. The protocol also includes Notifications allowing information to be output by a device. Your main point about low data rates and high overhead, still stand, but with a different slant.

---

Although you talk about IPFIX quite a bit later in the document, I think that, as part of the scene setting, you might include a paragraph about it early in 1.4.

---

The text after Figure 1 did not completely convince me of the need for interaction between the three "telemetry planes". Perhaps you could expand the examples and discussion to highlight/clarify this?

---

Clearly you are going to have to write some Security Considerations. 
There is quite a lot to say about what is a layered security model with plenty of attack vectors, and substantial risks as the telemetry data can result in operational changes that could harm data delivery and even bring the network down. Furthermore, examination of telemetry data can reveal weaknesses in the network.

== Nits ==

The authors will need to reduce the number of front page names to 5.
Move the others to a Contributors section. Might as well do this sooner rather than later.

---

Abstract

Suggest you change this a little because you actually provide the framework. How about...

---

---

---

---

Should that be gRPC?

This seems circular.

---

---

Figure 3
Does gPRC mean gRPC?

---

From: Tianran Zhou <[email-masked]>
Date: Thu, 18 Oct 2018 07:58:29 +0000

Hi Randy,

Well, the first paragraph in section 1.4 is neither clear nor necessary.
I would suggest to remove this paragraph. Is that OK for you?

BR,
Tianran

> Sent: Thursday, October 18, 2018 12:51 PM
> To: [email-masked]
> > But I think it's not designed for periodically getting operational
> > data, which is one important case for streaming telemetry.
> 
> That's one of the possible use cases for RFC 2981 or RFC 3877, and was
> considered in their design.  If you think those MIB modules are inadequate,
> it would be helpful for you to spell out exactly why they fail to meet the
> need.
> 
> > (https://datatracker.ietf.org/doc/draft-ietf-netconf-yang-push)
> > activity, user can get any data described in existing YANG data store
> > (not only the notifications), and can appoint the period when the data
> > originator pushes the event.
> 
> I really do recommend you look at RFC 2981.  It provides exactly that
> capability.  Again, if you think RFC 2981 does not meet that need, I ask you
> to spell out exactly how the design is deficient.
> 
> Randy
>

---

From: Randy Presuhn <[email-masked]>
Date: Thu, 18 Oct 2018 12:12:32 -0700

Hi -

> I would suggest to remove this paragraph. Is that OK for you?

The paragraph does seem clear, but is (in my opinion) incorrect.
However, it appears to be an integral part of the rationale for doing
the proposed work.  Removing the paragraph would leave the reader
wondering "why re-invent the wheel?"

Randy

---

From: "Blumenthal, Uri - 0553 - MITLL" <[email-masked]>
Date: Thu, 18 Oct 2018 19:19:50 +0000

I, as a reader, still wonder about re-inventing the wheel - even with the 1st paragraph of 1.4 untouched.

Regards,
Uri

Sent from my iPhone

> On Oct 18, 2018, at 14:12, Randy Presuhn <[email-masked]> wrote:
> 
> Hi -
>

---

From: Haoyu song <[email-masked]>
Date: Thu, 18 Oct 2018 21:11:34 +0000

I don't think we are trying anything "re-inventing the wheel".
First, we didn't invent anything in this draft. Rather, we are providing an overview and summary of existing stuff.
Second, even SNMP notification can do data push, it is still insufficient. Google has deprecated SNMP for a reason. Many other companies hold the similar view.
Here are a few reasons: SNMP is for management plane, but we still need data directly from other planes, that's why we have IOAM and IPFIX. Also, SNMP can't support high bandwidth streaming, that's why we have gPRC.
Sure, SNMP can bear some characteristics of telemetry and it can also continue to evolve to be more like it. But as I said, there's nothing black or white and SNMP is not sufficient and we need many more things and we already have many good and better stuff.

Regards,
Haoyu  

-----Original Message-----
From: OPSAWG [mailto:[email-masked]] On Behalf Of Blumenthal, Uri - 0553 - MITLL
Sent: Thursday, October 18, 2018 12:20 PM
To: Randy Presuhn <[email-masked]>
Cc: [email-masked]
Subject: Re: [OPSAWG] Thoughts on draft-song-opsawg-ntf

Regards,
Uri

Sent from my iPhone

---

From: Michael MacFaden <[email-masked]>
Date: Thu, 18 Oct 2018 21:20:15 +0000

>Company X doesn't' use SNMP.
>It can't do X.
>We need many more things .

Yeah, right. We've heard this before.

https://www.youtube.com/watch?v=wFBXU5MHxtM&feature=youtu.be

Listen to the first 10 minutes to Dr Case.

Mike

________________________________________
From: OPSAWG <[email-masked]> on behalf of Haoyu song <[email-masked]>
Sent: Thursday, October 18, 2018 2:11:34 PM
To: Blumenthal, Uri - 0553 - MITLL; Randy Presuhn
Cc: [email-masked]
Subject: Re: [OPSAWG] Thoughts on draft-song-opsawg-ntf

Regards,
Haoyu

-----Original Message-----
From: OPSAWG [mailto:[email-masked]] On Behalf Of Blumenthal, Uri - 0553 - MITLL
Sent: Thursday, October 18, 2018 12:20 PM
To: Randy Presuhn <[email-masked]>
Cc: [email-masked]
Subject: Re: [OPSAWG] Thoughts on draft-song-opsawg-ntf

Regards,
Uri

Sent from my iPhone

> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fopsawg&amp;data=02%7C01%7Cmrm%40vmware.com%7Ccae277838ebc419fce3908d6353e5601%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C636754939182154540&amp;sdata=gyxXwolkFEBOamwnrpU5DzdjIllgararEyq%2B7LzkzUg%3D&amp;reserved=0

_______________________________________________
OPSAWG mailing list
[email-masked]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fopsawg&amp;data=02%7C01%7Cmrm%40vmware.com%7Ccae277838ebc419fce3908d6353e5601%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C636754939182154540&amp;sdata=gyxXwolkFEBOamwnrpU5DzdjIllgararEyq%2B7LzkzUg%3D&amp;reserved=0

---

From: Joe Clarke <[email-masked]>
Date: Sun, 28 Oct 2018 23:50:43 -0400

> I just had cause to read this document and thought I would share my

Thanks, Adrian.  I have had a chance yo read this new version, and I'll
tack on to your comments.  These are my comments as a contributor.

> modifications. Actually, I found this document pretty complete.

I do agree that we need a baseline for terminology, much like was done
for OAM.  Where I struggle is where this work should be done.  On one
hand, a WG like opsawg does make sense.  However, the existing work
spans multiple WGs and areas.  It seems to me that the authors will have
to do quite a bit of leg work to make sure that their framework aligns
with work that is quickly maturing and being implemented.

In terms of sectional comments, the abstract mentions that telemetry
requires new protocols.  But there are already protocols that provide
"telemetry" (and some like gRPC, YANG push, etc. have emerged
"recently").  From my reading, I do not know if the authors feel more
new protocols are needed.

The abstract also mentions that future directions are discussed for each
section.  I did not see this.  In general, each section describes some
informal requirements and enumerates some challenges with existing work.
 What I was hoping to see is a more in-depth gap analysis with this
existing work with respect to requirements.

I disagree with the point in section 2.4 that a key difference between
telemetry and OAM is human vs. machine operations.  You say that
telemetry is designed to build closed-loop, machine-driven control
systems whereas OAM requires human intervention.  I feel that a holistic
closed loop control system would use telemetry _and_ OAM.  I could
certainly see (as with SNMP traps, IPFIX, or gRPC streams) that a human
might make some post-processing assessment of that data.  That said,
trying to define what this system looks like will be a closing battle
IMHO.  What works for one operator will almost certainly not work for
another.

When the authors list out the existing work in each of the main
telemetry categories, I'm not seeing the gap analysis to the
requirements they list prior.  For example, they describe what iOAM is,
gloss over some challenges, but don't really say where it fits with the
requirements (or what more is required specific to those requirements).
 That is only one example.  I see the same with every other mentioned
example.

I agree with you, Adrian that security needs to be fleshed out.  In
terms of the data that telemetry provides, there should be robust
security requirements around it.

Joe

---

From: Pedro Martinez-Julia <[email-masked]>
Date: Mon, 29 Oct 2018 14:04:00 +0900

Dear Joe,

Thank you very much for your comments. Please find my answer inline.

> with work that is quickly maturing and being implemented.

Regardless of being in opsawg, we are trying to ensure that the network
telemetry framework in our draft is completely consistent with the work
that is being done in other WGs. If you find any specific inconsistency,
please let us know.

> new protocols are needed.

We are first tackling the requirements of a proper telemetry framework,
and it currently requires new protocols to correctly manage telemetry
data. However, when we finish polishing the requirements, we can find
that some can be met by extending other protocols and we will proceed to
update the framework accordingly.

> existing work with respect to requirements.

Thank you for pointing this. As mentioned above, we are now focusing on
polishing requirements, so clarifying it and including a detailed gap
analysis will be our next step. Please be tuned if you are interested.

> IMHO.  What works for one operator will almost certainly not work for

We will rework those statements to avoid this misinterpretation. It is
not our intention to relegate the role of OAM, just to make clear that
it is not what current network management needs. Telemetry is the answer
to it and OAM has its limitations in scope and purpose.

>  That is only one example.  I see the same with every other mentioned

As I mentioned above, we will continue working on this to be sure the
gaps and challenges are clear.

> security requirements around it.

Of course, nowadays, both security and privacy must be totally traversal
requirements to any form of management and transmission of data that can
be used to find or reveal some personal information. We will ensure both
constraints are covered in the network telemetry framework.

> Joe

Again, thanks for your comments.

Regards,
Pedro

-- 
Pedro Martinez-Julia
Network Science and Convergence Device Technology Laboratory
Network System Research Institute
National Institute of Information and Communications Technology (NICT)
4-2-1, Nukui-Kitamachi, Koganei, Tokyo 184-8795, Japan
Email: [email-masked]
---------------------------------------------------------
*** Entia non sunt multiplicanda praeter necessitatem ***

---

From: Haoyu song <[email-masked]>
Date: Mon, 29 Oct 2018 19:41:06 +0000

Hi Joe,

Here I also try to response an earlier question you raised in anther email: 
“Others commented that a detailed comparison and evaluation is required in order to determine what the right approach/recommendation should be.”
I think the nature of the doc is more like a survey than an in-depth analysis and comparison of techniques.

When we say that "new protocol" is needed,  we are actually saying those existing protocols other than the conventional OAM protocols are "new", such as gPRC，YANG PUSH. Maybe we need to rephrase this point. Of course, we don't exclude the possibility that, when new gaps are identified, newer protocols are indeed needed. For example, in control plane and data plane, we have already identified some of the new needs. 

We also don't mean to deprecate the existing OAM protocols and techniques. I agree with you that it will coexist with telemetry even in future automated network operations. We'll make the expression more accurate in the future revisions. 

We admit that there are still work need to be done about this doc and that's exactly why we need the collaboration from the community. What we are trying to do is to make the work useful and align its purpose and scope with OPSAWG.   

Thank you very much!

Best,
Haoyu 

-----Original Message---
From: Joe Clarke [mailto:[email-masked]] 
Sent: Sunday, October 28, 2018 8:51 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: Re: [OPSAWG] Thoughts on draft-song-opsawg-ntf

> I just had cause to read this document and thought I would share my 

Thanks, Adrian.  I have had a chance yo read this new version, and I'll tack on to your comments.  These are my comments as a contributor.

> IETF, and the OPSAWG may be a good place to discuss and progress that work.

I do agree that we need a baseline for terminology, much like was done for OAM.  Where I struggle is where this work should be done.  On one hand, a WG like opsawg does make sense.  However, the existing work spans multiple WGs and areas.  It seems to me that the authors will have to do quite a bit of leg work to make sure that their framework aligns with work that is quickly maturing and being implemented.

In terms of sectional comments, the abstract mentions that telemetry requires new protocols.  But there are already protocols that provide "telemetry" (and some like gRPC, YANG push, etc. have emerged "recently").  From my reading, I do not know if the authors feel more new protocols are needed.

The abstract also mentions that future directions are discussed for each section.  I did not see this.  In general, each section describes some informal requirements and enumerates some challenges with existing work.
 What I was hoping to see is a more in-depth gap analysis with this existing work with respect to requirements.

I disagree with the point in section 2.4 that a key difference between telemetry and OAM is human vs. machine operations.  You say that telemetry is designed to build closed-loop, machine-driven control systems whereas OAM requires human intervention.  I feel that a holistic closed loop control system would use telemetry _and_ OAM.  I could certainly see (as with SNMP traps, IPFIX, or gRPC streams) that a human might make some post-processing assessment of that data.  That said, trying to define what this system looks like will be a closing battle IMHO.  What works for one operator will almost certainly not work for another.

When the authors list out the existing work in each of the main telemetry categories, I'm not seeing the gap analysis to the requirements they list prior.  For example, they describe what iOAM is, gloss over some challenges, but don't really say where it fits with the requirements (or what more is required specific to those requirements).
 That is only one example.  I see the same with every other mentioned example.

I agree with you, Adrian that security needs to be fleshed out.  In terms of the data that telemetry provides, there should be robust security requirements around it.

Joe