Subject: [OPSAWG] FW: New Version Notification for draft-gray-sampled-streaming-00.txt

From: "Gray, Andrew A" <[email-masked]>
Date: Mon, 5 Aug 2019 14:45:44 +0000

Hello opsawg,

I spoke to a number of IETF folks at IETF 105, along with Tianran and Ignas (who I thank for their feedback and suggestions), and it was decided that opsawg would be the best place to at least start the conversations about a new draft I am proposing, as detailed below.

I document several use cases in the draft, but the short version is we are wanting to standardize a method for different vendors to handle ASIC-level replication of packets, to a remote server, including any and all information that ASIC may have about the packet, along with at least part of the payload, with a configurable sampling rate and filtering capabilities.

Some things on my TODO list for it still:

1. I want to take a closer look at Flowspec rule definition, and see if that can be adequately encoded here.  It looks like the YANG model for Flowspec rules was draft-wu-idr-flowspec-yang-cfg but that has since expired.
2. I'm planning on talking at NANOG 77 in Austin with other operators to see if there's additional use cases and how much appetite there is.
3. I'm talking with a couple vendors about implementations - there is interest here about it as well.

I am sending out the -00 version of this draft and very much welcome comments and feedback.  

Thanks
Andrew Gray

On 8/5/19, 8:36 AM, "[email-masked]" <[email-masked]> wrote:

    A new version of I-D, draft-gray-sampled-streaming-00.txt
    has been successfully submitted by Andrew Gray and posted to the
    IETF repository.
    
    Name:		draft-gray-sampled-streaming
    Revision:	00
    Title:		Sampled Traffic Streaming
    Document date:	2019-08-05
    Group:		Individual Submission
    Pages:		17
    URL:            https://www.ietf.org/internet-drafts/draft-gray-sampled-streaming-00.txt
    Status:         https://datatracker.ietf.org/doc/draft-gray-sampled-streaming/
    Htmlized:       https://tools.ietf.org/html/draft-gray-sampled-streaming-00
    Htmlized:       https://datatracker.ietf.org/doc/html/draft-gray-sampled-streaming

    Abstract:
       This document standardizes the means of requesting a sampled capture
       stream from a router, receiving details about the resulting data
       flow, and the structure of the data flow itself.  This is
       specifically tailored to having various hardware ASICs be able to
       perform this operation as quickly as possible, by allowing
       communication of the specific bit formats of headers applied to the
       packet flow, in a way that enhances interoperability between sources
       and sinks.  Historically, NetFlow and its ilk have been used for
       these use cases, however the growth in hardware forward speeds is far
       outpacing the growth in CPU speeds, and the CPU-heavy parts of
       NetFlow is resulting in a reduction of sampling rates that include
       all of the fields provided by NetFlow that require CPU lookups.

    
    Please note that it may take a couple of minutes from the time of submission
    until the htmlized version and diff are available at tools.ietf.org.
    
    The IETF Secretariat

E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Mon, 5 Aug 2019 21:20:16 +0000

> On Aug 5, 2019, at 10:45, Gray, Andrew A <[email-masked]> wrote:
> 
> Hello opsawg,
> 

Hello, Andrew.  I’ve read through this draft, and I have a few comments.  I put it up in GitHub to make it a bit easier.  In addition to your TODO, I know Cisco proposed ERSPAN to the IETF a few years ago, but I do not know what the discussion was around that (https://tools.ietf.org/html/draft-foschiano-erspan-03).  Conceptually, your idea seems very similar.  It is perhaps worth exploring what happened with that work. 

More to the contents of your doc, you have it marked informational, but you mention a few times about standardizing the interactions between components.  You’re not standardizing as much as you’re defining a specific interaction.

In https://github.com/xorrkaz/ietf-docs/blob/master/draft-gray-sampled-streaming-00#L195 you mention NETCONF explicitly.  Why can’t this be done with RESTCONF, gRPC, etc.?

In https://github.com/xorrkaz/ietf-docs/blob/master/draft-gray-sampled-streaming-00#L249, why must a Client re-query?  This seems like telemetry would be ideal to update the Client about configuration changes to the stream.

https://github.com/xorrkaz/ietf-docs/blob/master/draft-gray-sampled-streaming-00#L298, are these bit widths or byte?  Either way, they seem too short for things like an interface-ref list.  If it is bit width, then it wouldn’t even be long enough for a list of actions.

https://github.com/xorrkaz/ietf-docs/blob/master/draft-gray-sampled-streaming-00#L309, if this is 24 bytes, that seems excessive for two 32-bit integers.  If it’s bits, it’s not long enough.

I have other more typographical comments at https://github.com/xorrkaz/ietf-docs/commit/7a849d2b12cd2dff151654f3ef37a87ac870a452#diff-580cadc3568cc3a979bb37410222e08b .

Joe

---

From: "Gray, Andrew A" <[email-masked]>
Date: Mon, 5 Aug 2019 22:29:59 +0000

Joe,

Thanks for your review and comments.  Responses in-line.

    

[AG] ERSPAN doesn't include the potentially useful information the ASIC has about the packets (i.e. what it's going to do with it, what the receive timestamp was).  That draft seems to only standardize a single encapsulation - my desire is to get as much information out of the forwarding ASICs as possible, in whatever format works best for the chipset, but still allow interop with different receiving packets.

[AG] True - it started as informational but has become more standards track - will update.

[AG] Good point - it very well could be.  I suppose that could be reworded to be more as an example.

[AG] The re-query is basically the response to the initial configuration request - in effect the client pushes part of the configuration (the desired filters and such), and the Replicator fills in the rest (what the resulting packet format will be, based on ASIC or location or capability or what-have-you).  Telemetry seems a bit overkill for what was intended to be a single re-pull, but I am open to input?

[AG] Those are bits, and it's an example of the resulting data format - the actual format returned by the device could be whatever bit width is desirable for it.  A few reviewers have thought that packet header and table are requirements - I added "example" language to a few different places to try and fix that, but apparently it's still too easy to assume those listings are the requirements.

[AG] See previous comment.

[AG] I'll roll those in - I'm working from a .xml version and separate .yang file, then regenerating the .txt using xml2rfc (but I can't quite upload the .xml as I use a <xi:include href="[email-masked]" parse="text" /> to have the parser pull in the separate yang file for formatting.  I can upload those to the Git repository and use those for work, if that's easier?
    
    Joe

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Tue, 6 Aug 2019 14:15:34 +0000

> On Aug 5, 2019, at 18:29, Gray, Andrew A <[email-masked]> wrote:
> 
> Joe,
> 

It seems like the ASIC bit blends with what IOAM is doing.  In fact, the “framework” you describe might incorporate work like https://tools.ietf.org/html/draft-spiegel-ippm-ioam-rawexport-02 with the exception that it is using IPFIX for export (which is more overhead than you want).  My overarching point is that there is similar work, and it sounds like you’re making the right connections to understand what is out there.

> [AG] True - it started as informational but has become more standards track - will update.

Well, for a standards track, you would have to determine an encap.  And then it might be worth breaking out the encap from the YANG module.

> [AG] The re-query is basically the response to the initial configuration request - in effect the client pushes part of the configuration (the desired filters and such), and the Replicator fills in…

There has been examples of using YANG push as a way push config and then look to see what is actually applied.  This seems to fit in that paradigm of use case.

> [AG] Those are bits, and it's an example of the resulting data format - the actual format returned by the device could be whatever bit width is desirable for it.  A few reviewers have thought that…

I did understand that this was an example.  My point is that I don’t see how this is a feasible example.  Based on your description of the fields, I couldn’t rationalize how they are used or how they would look.  How could a 8-bit field present a list of interface-refs?

> [AG] I'll roll those in - I'm working from a .xml version and separate .yang file, then regenerating the .txt using xml2rfc (but I can't quite upload the .xml as I use a <xi:include href="[email-ma…

As I said in unicast, I think if you could spin up your own repo, it would be easier to add comments/issues or PRs.

Additionally, as a co-chair, I’d like to add that it is always good to see operators presenting work to the IETF that would make their lives easier.  I appreciate you bringing this discussion to opsawg.

Joe

---

From: "Gray, Andrew A" <[email-masked]>
Date: Tue, 6 Aug 2019 14:40:49 +0000

[AG] Yes - there are a number of drafts that are doing different things, but we have the two requirements of getting at least partial packet payloads and wanting the information the forwarding ASIC has (i.e. being able to catch packets the ASIC would normally drop) that seems at this point to be different enough to warrant a new doc. 

[AG] The encap I have proposed in the document is a simple UDP tunnel.  I pondered adding GRE in it, but I at least don't have a great use case that GRE solves that a basic UDP tunnel doesn't.

    

[AG] The trick here is there's a mapping provided back in the "filled in response" portion that gets re-queried (stream-format/port-mappings).  This (hypothetically) says "a value of 20 in the data stream means interface-ref 72, a value of 33 means interface-ref 73, a value of 97 means interface-ref 74" or whatever works best.  An ASIC could say this is 32 bits or what-have-you instead, but the full mapping is provided by the control plane in the filled in response portion describing the data structure.

[AG] Thanks for that advice - I have done so at https://github.com/AGray-Charter/ietf-work and welcome comments/feedback there as well.

[AG] Thanks - this is a problem we see coming up on us very soon, generally with the next generation of platforms where the forwarding/data plane has so far surpassed the control plane that we will need to rethink how we're doing data analytics.  It's one of the use cases I mention in the draft (and the strong desire to standardize how this works as opposed to everyone doing their own thing), so we're pretty motivated to get traction here.
    
    Joe