Subject: [OPSAWG] IETF 106 Discussion about draft-gray-sampled-streaming - room moved

From: "Gray, Andrew A" <[email-masked]>
Date: Wed, 20 Nov 2019 04:11:24 +0000

After a couple bits of feedback, I have moved the session to be in the Padang room at 9:00am to avoid conflict with the morning session.

On 11/20/19, 11:59 AM, "OPSAWG on behalf of Gray, Andrew A" <[email-masked] on behalf of [email-masked]> wrote:

    Following the presentation of https://datatracker.ietf.org/doc/draft-gray-sampled-streaming/ during the session, questions and comments were cut off due to time, and there was the suggestion to make a separate session for ongoing conversation.
    
    I reserved the Butterworth room on Thursday at 10am for this, and am very interested in hearing from people on this draft.
    
    Thanks

    E-MAIL CONFIDENTIALITY NOTICE: 
    The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Wed, 20 Nov 2019 09:23:46 +0000

Andrew,

per the discussion in the WG meeting this morning: It would be good if the draft could compare the proposed approach to what is already available with IPFIX and PSAMP (RFC5476). As part of that you could consider adding an IPFIX example as another “data plane option” to section 4 of the draft.

Using your first example as reference, you could choose IPFIX fields (https://www.iana.org/assignments/ipfix/ipfix.xhtml) which match your definitions.
Example (just took a quick glimpse at possible choices – there are other mappings which could apply as well):

  *   “act” fields (forward, drop, bad TTL, …) -> forwardingStatus (u8)
  *    “incoming port” -> ingressInterface (u32)
  *   “timestamp” -> Export time (in IPFIX header)
  *   “frame length” -> totalLengthIPv4 (u32) (example for v4 – assuming we export the entire packet)
  *   “Internal Data 1” -> paddingOctets

Using your example 1 – one possible IPFIX equivalent could look like:

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+---
|        Version Number         |             Length            |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                     Export Time (seconds)                     |IPFIX
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Message
|                        Sequence Number                        |Header
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                     Observation Domain ID                     |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+--
|    Set ID ( = Template ID)    |             Length            |SetHdr
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+--
|                        ingressInterface                       |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                        totalLengthIPv4                        |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
| fwdingStatus  | paddingOctets |     sectionExportedOctets     | Record1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                 ipHeaderPacketSection (start)                 |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                              ...                              |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                 ipHeaderPacketSection (end)                   |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+--
|                        ingressInterface                       |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                        totalLengthIPv4                        |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
| fwdingStatus  | paddingOctets |     sectionExportedOctets     | Record2
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                 ipHeaderPacketSection (start)                 |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                              ...                              |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   |
|                 ipHeaderPacketSection (end)                   |   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+--

Frank

> -----Original Message-----

> From: OPSAWG <[email-masked]> On Behalf Of Gray, Andrew A

> Sent: Mittwoch, 20. November 2019 12:11

> To: [email-masked]

> Subject: [OPSAWG] IETF 106 Discussion about draft-gray-sampled-streaming -

> room moved

> After a couple bits of feedback, I have moved the session to be in the Padang

> room at 9:00am to avoid conflict with the morning session.

> On 11/20/19, 11:59 AM, "OPSAWG on behalf of Gray, Andrew A" <opsawg-<mailto:[email-masked][email-masked]>

> [email-masked] on behalf of [email-masked]<mailto:[email-masked][email-masked]>> wrote:

>     Following the presentation of https://datatracker.ietf.org/doc/draft-gray-<https://datatracker.ietf.org/doc/draft-gray-sampled-streaming/>

> sampled-streaming/<https://datatracker.ietf.org/doc/draft-gray-sampled-streaming/> during the session, questions and comments were cut off

> due to time, and there was the suggestion to make a separate session for

> ongoing conversation.

>     I reserved the Butterworth room on Thursday at 10am for this, and am very

> interested in hearing from people on this draft.

>     The contents of this e-mail message and any attachments are intended solely

> for the addressee(s) and may contain confidential and/or legally privileged

> information. If you are not the intended recipient of this message or if this

> message has been addressed to you in error, please immediately alert the

> sender by reply e-mail and then delete this message and any attachments. If you

> are not the intended recipient, you are notified that any use, dissemination,

> distribution, copying, or storage of this message or any attachment is strictly

> prohibited.

> The contents of this e-mail message and any attachments are intended solely for

> the addressee(s) and may contain confidential and/or legally privileged

> information. If you are not the intended recipient of this message or if this

> message has been addressed to you in error, please immediately alert the

> sender by reply e-mail and then delete this message and any attachments. If you

> are not the intended recipient, you are notified that any use, dissemination,

> distribution, copying, or storage of this message or any attachment is strictly

> prohibited.

> _______________________________________________

> OPSAWG mailing list

> [email-masked]<mailto:[email-masked]>

> https://www.ietf.org/mailman/listinfo/opsawg

---

From: stefan vallin <[email-masked]>
Date: Wed, 20 Nov 2019 11:37:17 +0100

Hi Benoit!
Thanks for bringing the issue of service assurance to the table. More work is needed on this topic.

Some high-level comments on the drafts

The drafts present a YANG module for performing reasoning across a “generic” service tree.
This has existed in assurance systems for a long time: inventory based systems as well as fault managers had modules for this, like Micromuse Impact, OpenView Service Navigator etc. 
The overall idea is to feed KPIs, events, alarms etc into the tree and reasoning upwards to do service impact analysis and downwards to do root-cause analysis.

I have some concerns based on the above:
1) The draft should be renamed. Claiming a service tree being *the* architecture for intent-based network assurance is maybe too ambitious. There are so many other things needed for service assurance in intent-based networking:
- how to represent service tests and service SLA monitoring as part of the intent?
- how to monitor the data-plane as such
- how to represent closed-loop policies
- and more
So I think the draft should be renamed to the relevant, more limited scope: A "YANG data-model for representing generic service trees”, or something

2) Looking back at practical experience with above  mentioned tools, it was very hard to get the approach to work (or someone else out there is smarter?). 
The dependencies between services/subservices are subtle. Either the dependencies get too coarse-grained so that everything gets red, or too fine-grained to have too few hits. Who are to express this knowledge in a large multi-vendor network? The classical service impact tools had fairly advanced algorithms attached to the graph to try to capture this, not just a dependency link. It more or less is as complex as the knowledge acquisition problem for classical rule-based AI, how many dependencies do we need to express until we can do a good job in the service tree? 

Since it is configuration data in the model, I guess it is assumed that the orchestrator should set up all of this. But in many cases this is "hidden" in other domains, like networking protocols and vendor specific details. Also with networks becoming more dynamic and virtual, it is hard to see how the service structures can be statically maintained as *configuration*.

Another underlying challenge is that most of the network problems are configuration related, and not detected by device instrumentation.
Non-optimal QoS config, firewall rules etc, these are not detected by the firewall or router, no alarms.

Hard for you to comment on this, I know, it is just a reality check.

3) Relationship to other YANG service models. 
In your approach the service tree is a separate tree from the concrete service trees like L3VPN service model.
Have you considered an approach to augment these concrete models with generic assurance state and dependency information instead of maintaining a separate tree? Maintaining parallell trees might result in inconsistencies at the end

4) Relationship to the Alarm YANG RFC8632
There are several opportunities to reuse definitions and concepts from RFC8632
- You could add alarms in your module according to the service tree, see especially Section 3.6.  Root Cause, Impacted Resources
- You could use alarm-types as one kind of symptom (there are many others like active measurements with TWAMP etc)

Hope this helps to flesh out more details in your work
br Stefan

---

From: "Gray, Andrew A" <[email-masked]>
Date: Fri, 22 Nov 2019 01:53:22 +0000

Thanks to those who came to the side session and have otherwise provided feedback.  I took notes on the feedback from that meeting combined with the main presentation and other notes:

- Should incorporate the "why not other stuff" into the draft
- Compare/contrast versus RFC 5476 
- How are the resulting entries in a forwarding ASIC's CAM ordered versus real ACL entries populated by another entity?  The replicator already has to do ACL merging between ACLs, QOS policies, and other sampling - is this actually a problem? 
- How are multiple sampling sessions impacting each other (i.e. if a more specific sampler is installed at 1:10 to destination A, and a common sampler is installed at 1:1000 to destination B, does the traffic from the first also appear in the second?).  Generally consensus is that it shouldn't, and depending on platform can't without a recirculation penalty.  Should this be handled in the Proposals somehow?
- Possibly reference IPFIX fields as defined in https://www.iana.org/assignments/ipfix/ipfix.xhtml (possibly as the resulting mapping table on export?)
- Need to do more to explain how this compares and fits into an IPFIX deployment.
- Clarify the current performance penalty is only forwarding capacity
- What about LAGs?  Service/customer/VLAN interfaces?  LAGs are highly problematic between the ports may be sprayed across different ASICs with different capabilities but if a Replicator wants/can offer the option at a reduced set, it should be allowed.  Clients probably should expect to have to grab the member ports individually, however.
- Research more on the filter definition - feedback was this isn't standardized because it's hard to standardize even one item (ACLs for example) across different platforms.  In addition, sampled streaming has unique filtering requirements (filter against what action is going to be taken).  May be best off just sticking with local definition, but still should research other implementations and make sure things like field names/sizes/definitions match up where we can.
- Proposals can offer different filter sets than requested as is, but this is only mentioned in the YANG model (along with some other tidbits) - need to pull into the body of the draft
- Freeform performance penalty (points?) - finger in the wind numbers (make sure to emphasize that it is an estimate) - include things like other resources used (CAM entries, memory, etc?).  Number would only be relative to the device itself.
- Informational or Standards track?

If I missed something, please let me know, and thanks to all who have provided feedback to date.  I'm planning on working through these issues and releasing a -03 within the next couple weeks.

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Fri, 22 Nov 2019 02:03:09 +0000

> - Freeform performance penalty (points?) - finger in the wind numbers (make sure to emphasize that it is an estimate) - include things like other resources used (CAM entries, memory, etc?).  Number…

I had to leave around this discussion, but I really wasn’t following why this would be better than straight forwarding capacity.  That is, an overall “points” value doesn’t necessarily help much since some may care about CAM space and some may care about forwarding capacity, but they don’t know how the points break down.  Like Frank, I’d like to stick to something simple and more well-defined like forwarding capacity.

> - Informational or Standards track?

It’s informational now, and I have no problem with that given the looseness in the draft.  As it evolves, it may be better to move it to Standards.  I think it should stay where it is for now.

Joe

---

From: "Gray, Andrew A" <[email-masked]>
Date: Fri, 22 Nov 2019 02:09:08 +0000

AG: After pondering this overnight, I think I agree with this.  Coming up with a number, while possible, isn't really meaningful.  Operators generally expect turning additional options on reduce the capacity for other items as it stands.

    
AG: I'll leave it Informational for the moment then, but I agree I think Standards probably should be its eventual home.  I had some feedback to split up the draft between defining the model defining the data exchange, and a separate draft talking about the command and control aspect, which may change this up somewhat.  Thoughts?