Subject: [OPSAWG] Review of draft-song-opsawg-ntf-03

From: Joe Clarke <[email-masked]>
Date: Mon, 8 Apr 2019 10:31:13 -0400

As I promised at the mic at 104, I have a more comprehensive review of
this draft.

I'll tackle this section-by-section.  These are a combination of nits
and more substantive comments.

===

In the abstract, you have a sentence:

Network telemetry promises better flexibility, scalability, accuracy,
coverage, and performance...

This left me wondering, "than what?"  That is, I think you’re trying to
say that with respect to traditional poll-based network monitoring,
telemetry provides these advantages.  If you're going to say something
is better, it would help to compare that to something else in order to
provide better clarity.

===

Section 1:

s/ideal mean/ideal means/

s/telemetry remain/telemetry remains/

===

Section 2:

The first sentence of your motivation bothers me.  I don’t buy it.  Yes,
some are using machine learning to pull insights out of “big data,” but
the way this reads, AI is commonplace in networking.  I would say this
is still very much in niche areas and research.  One key issue with the
vast amounts of data that some networks produce is sorting through it to
find useful information.  Large operators may have the means to do ML,
but others struggle to make sense of this data.

You use the term "intent-driven autonomous network."  I asked around
with some operators and other vendors, and this is 100% buzz word.
People either don't know what it means or define it using even more
buzzwords.  I know each vendor has their own meaning for this, but these
terms on not widely known.  Your document aims to clarify taxonomy.  I
would clarify this buzzy phrase.

In your third paragraph, you say, "the system bottleneck is shifting
from data consumption to data supply."  I disagree.  Telemetry is
furthering the data consumption problem.  More data doesn't mean more
insight.  More data means more headache.  In addition to getting more
data more easily, we _must_ have a way to translate that data into
useful, actionable information.  This goes to Benoit's talk on telemetry
on Thursday at 104.  How do we up-level the value of telemetry?

===

Section 2.1:

s/An intents/An intent/

===

Section 2.2:

The sentence "These convention techniques are not sufficient to support
the above use cases for the following reasons:" doesn't make sense.
There are some grammatical issues here.  Consider rephrasing.  Maybe,
"These conventional techniques".

===

Section 2.3:

You define IDN as Intent-Driven Network, but I would again say you need
to define this in its entirety so the reader is clear.  You do define,
"an intent" but IDN/IBN is more comprehensive than that.

===

Section 4.1:

s/avaiable/available/

You define different types of telemetry data as "Simple Data" and then
"Custom Data".  Might I suggest you rename Custom Data to Complex Data
as "complex" tends to typically refer to some level of multiple
combination or aggregation.

===

Section 4.2:

Can you talk more about what you mean by social data and why
environmental is defined to be external?  I can have on-board sensors
that relay internal environmental data.

Why is external data not offered over HTTP?  I imagine "social" would be
HTTP, but without more definition, I'm not sure.

===

Section 4.4:

Why isn't gRPC listed in the Simple Data query?

Under Custom Data Query, there is a “YANG FSM” draft as an individual
draft in netmod, but it’s not clear what the future is.  There seems to
be a general thought that event-driven YANG is useful, but if that will
be an FSM per se is yet to be seen.

What is PBT?  I didn't see it in glossary.

===

That's it for now.  I think this will give you a lot to chew on.

Joe

---

From: Haoyu song <[email-masked]>
Date: Tue, 9 Apr 2019 16:09:34 +0000

Hi Joe,

Thank you very much for the review! I'll update the draft according to your comments.

Best regards,
Haoyu

-----Original Message-----
From: OPSAWG [mailto:[email-masked]] On Behalf Of Joe Clarke
Sent: Monday, April 08, 2019 7:31 AM
To: [email-masked]
Subject: [OPSAWG] Review of draft-song-opsawg-ntf-03

As I promised at the mic at 104, I have a more comprehensive review of this draft.

I'll tackle this section-by-section.  These are a combination of nits and more substantive comments.

===

Network telemetry promises better flexibility, scalability, accuracy, coverage, and performance...

This left me wondering, "than what?"  That is, I think you’re trying to say that with respect to traditional poll-based network monitoring, telemetry provides these advantages.  If you're going to say something is better, it would help to compare that to something else in order to provide better clarity.

===

Section 1:

===

Section 2:

The first sentence of your motivation bothers me.  I don’t buy it.  Yes, some are using machine learning to pull insights out of “big data,” but the way this reads, AI is commonplace in networking.  I would say this is still very much in niche areas and research.  One key issue with the vast amounts of data that some networks produce is sorting through it to find useful information.  Large operators may have the means to do ML, but others struggle to make sense of this data.

You use the term "intent-driven autonomous network."  I asked around with some operators and other vendors, and this is 100% buzz word.
People either don't know what it means or define it using even more buzzwords.  I know each vendor has their own meaning for this, but these terms on not widely known.  Your document aims to clarify taxonomy.  I would clarify this buzzy phrase.

In your third paragraph, you say, "the system bottleneck is shifting from data consumption to data supply."  I disagree.  Telemetry is furthering the data consumption problem.  More data doesn't mean more insight.  More data means more headache.  In addition to getting more data more easily, we _must_ have a way to translate that data into useful, actionable information.  This goes to Benoit's talk on telemetry on Thursday at 104.  How do we up-level the value of telemetry?

===

Section 2.1:

s/An intents/An intent/

===

Section 2.2:

The sentence "These convention techniques are not sufficient to support the above use cases for the following reasons:" doesn't make sense.
There are some grammatical issues here.  Consider rephrasing.  Maybe, "These conventional techniques".

===

Section 2.3:

You define IDN as Intent-Driven Network, but I would again say you need to define this in its entirety so the reader is clear.  You do define, "an intent" but IDN/IBN is more comprehensive than that.

===

Section 4.1:

s/avaiable/available/

You define different types of telemetry data as "Simple Data" and then "Custom Data".  Might I suggest you rename Custom Data to Complex Data as "complex" tends to typically refer to some level of multiple combination or aggregation.

===

Section 4.2:

Can you talk more about what you mean by social data and why environmental is defined to be external?  I can have on-board sensors that relay internal environmental data.

Why is external data not offered over HTTP?  I imagine "social" would be HTTP, but without more definition, I'm not sure.

===

Section 4.4:

Under Custom Data Query, there is a “YANG FSM” draft as an individual draft in netmod, but it’s not clear what the future is.  There seems to be a general thought that event-driven YANG is useful, but if that will be an FSM per se is yet to be seen.

===

Joe

_______________________________________________
OPSAWG mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/opsawg

---

From: Haoyu song <[email-masked]>
Date: Tue, 30 Apr 2019 01:25:41 +0000

Hi Joe,

I'm working on revising the draft. 

Below are some thoughts on the issues you raised about Section 2:

1) I'll position AI a more forward looking technique that can take advantage of network telemetry.

2) NMRG is working on  provide a formal definition of "intent" so I think we can use that as a reference to clarify the buzz word.

3) When I say "the system bottleneck is shifting from data consumption to data supply",  what I mean is  that now the data processing capability is greatly improved and many applications are hungry for more data yet the network  lags behind in providing enough high quality data in a timely manner. I agree we must have a way to translate that data into useful, actionable information. I think this is also a part of job of network telemetry. Network telemetry is not just supplying raw data, but supplying useful and actionable data.

Best regards,
Haoyu
 
-----Original Message-----
From: OPSAWG [mailto:[email-masked]] On Behalf Of Joe Clarke
Sent: Monday, April 08, 2019 7:31 AM
To: [email-masked]
Subject: [OPSAWG] Review of draft-song-opsawg-ntf-03

===

This left me wondering, "than what?"  That is, I think you’re trying to say that with respect to traditional poll-based network monitoring, telemetry provides these advantages.  If you're going to say something is better, it would help to compare that to something else in order to provide better clarity.pr

===

Section 1:

===

Section 2:

===

Section 2.1:

s/An intents/An intent/

===

Section 2.2:

===

Section 2.3:

===

Section 4.1:

s/avaiable/available/

===

Section 4.2:

===

Section 4.4:

===

Joe