Subject: Re: [ippm] WGLC for IOAM Flags and Direct Exports

From: Tal Mizrahi <[email-masked]>
Date: Wed, 1 Sep 2021 17:46:18 +0300

Hi,

I support as a coauthor of both drafts.

Both drafts have been reviewed by a large number of people who have
contributed to the maturity of these drafts.
There are no pending open issues about the current versions of these
drafts, and we believe they are ready to be submitted to the IESG for
publication.

Cheers,
Tal.

On Mon, Aug 30, 2021 at 11:58 PM Tommy Pauly <tpauly=
[email-masked]> wrote:

> Hello IPPM,
>
> This email starts a Working Group Last Call on two related IOAM
> documents, draft-ietf-ippm-ioam-flags
> and draft-ietf-ippm-ioam-direct-export.
>
> In-situ OAM Loopback and Active Flags
> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-flags/
> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-flags-06
>
> In-situ OAM Direct Exporting
> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/
> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-direct-export-06
>
> Please provide feedback by replying to this email with your reviews and if
> you think these documents are ready to progress, by *Wednesday, September
> 15*.
>
> Best,
> Tommy & Ian
>
> _______________________________________________
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm
>

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Fri, 3 Sep 2021 13:55:54 +0000

As another co-author of the drafts, I’d like to echo what Tal said below.
IMHO both drafts are ready.

Cheers, Frank

From: ippm <[email-masked]> On Behalf Of Tal Mizrahi
Sent: Mittwoch, 1. September 2021 16:46
To: Tommy Pauly <tpauly=[email-masked]>; IETF IPPM WG ([email-masked]) <[email-masked]>
Subject: Re: [ippm] WGLC for IOAM Flags and Direct Exports

Hi,

Both drafts have been reviewed by a large number of people who have contributed to the maturity of these drafts.
There are no pending open issues about the current versions of these drafts, and we believe they are ready to be submitted to the IESG for publication.

Cheers,
Tal.

On Mon, Aug 30, 2021 at 11:58 PM Tommy Pauly <tpauly=[email-masked]<mailto:[email-masked]>> wrote:
Hello IPPM,

This email starts a Working Group Last Call on two related IOAM documents, draft-ietf-ippm-ioam-flags and draft-ietf-ippm-ioam-direct-export.

Please provide feedback by replying to this email with your reviews and if you think these documents are ready to progress, by Wednesday, September 15.

Best,
Tommy & Ian

_______________________________________________
ippm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ippm

---

From: Haoyu Song <[email-masked]>
Date: Fri, 3 Sep 2021 17:22:41 +0000

As a coauthor for one of the drafts, I also participated in the discussion on the other one. I believe both are ready to progress and therefore support the WGLC.

Cheers,
Haoyu

From: ippm <[email-masked]> On Behalf Of Frank Brockners (fbrockne)
Sent: Friday, September 3, 2021 6:56 AM
To: Tal Mizrahi <[email-masked]>; Tommy Pauly <tpauly=[email-masked]>; IETF IPPM WG ([email-masked]) <[email-masked]>
Subject: Re: [ippm] WGLC for IOAM Flags and Direct Exports

As another co-author of the drafts, I'd like to echo what Tal said below.
IMHO both drafts are ready.

Cheers, Frank

From: ippm <[email-masked]<mailto:[email-masked]>> On Behalf Of Tal Mizrahi
Sent: Mittwoch, 1. September 2021 16:46
To: Tommy Pauly <tpauly=[email-masked]<mailto:tpauly=[email-masked]>>; IETF IPPM WG ([email-masked]<mailto:[email-masked]>) <[email-masked]<mailto:[email-masked]>>
Subject: Re: [ippm] WGLC for IOAM Flags and Direct Exports

Hi,

Cheers,
Tal.

In-situ OAM Loopback and Active Flags
https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-flags/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-ippm-ioam-flags%2F&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cdc1015bb793645febb2008d96ee29b0e%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637662741813832765%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=D5ZxCuXWeLubuWKosEdBS2nDylJfA8GtMuDOrJx4Zsk%3D&reserved=0>
https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-flags-06<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-ippm-ioam-flags-06&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cdc1015bb793645febb2008d96ee29b0e%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637662741813842721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cDLRv%2B3%2FjD3XaE69X0%2BXNibR3uXgGJPYuLp5OVZKgIs%3D&reserved=0>

In-situ OAM Direct Exporting
https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-ippm-ioam-direct-export%2F&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cdc1015bb793645febb2008d96ee29b0e%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637662741813852679%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=oG44IumCRsF%2FNXqWHMtEnqQAvQGQFs2Ncgczp0ANQs8%3D&reserved=0>
https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-direct-export-06<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-ippm-ioam-direct-export-06&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cdc1015bb793645febb2008d96ee29b0e%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637662741813852679%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vx%2B3SKItwGhwpIXPGrHbTZyD36CVkp43Fs9lpAnNDUc%3D&reserved=0>

Best,
Tommy & Ian

_______________________________________________
ippm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ippm<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fippm&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cdc1015bb793645febb2008d96ee29b0e%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637662741813862630%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ijvoz9E%2BNqwmTezOKMlLXQfDfIxC9NZg8lt4NHWr0B4%3D&reserved=0>

---

From: "Ramesh Sivakolundu (sramesh)" <[email-masked]>
Date: Fri, 3 Sep 2021 17:31:09 +0000

Hi all,

As a co-author of these two drafts, I support the WGLC.  These drafts have be reviewed thoroughly over the last few months.

Thanks
-Ramesh

From: ippm <[email-masked]> on behalf of Tommy Pauly <tpauly=[email-masked]>
Date: Monday, August 30, 2021 at 1:58 PM
To: "IETF IPPM WG ([email-masked])" <[email-masked]>
Subject: [ippm] WGLC for IOAM Flags and Direct Exports

Hello IPPM,

Best,
Tommy & Ian

---

From: "Srihari Raghavan (srihari)" <[email-masked]>
Date: Sun, 5 Sep 2021 15:28:49 +0000

Based on past experience with iOAM and having read the drafts, I believe that they are ready for the next step.

Thanks
Srihari

From: ippm <[email-masked]> on behalf of Tommy Pauly <tpauly=[email-masked]>
Date: Tuesday, 31 August 2021 at 2:28 AM
To: "IETF IPPM WG ([email-masked])" <[email-masked]>
Subject: [ippm] WGLC for IOAM Flags and Direct Exports

Hello IPPM,

Best,
Tommy & Ian

---

From: <[email-masked]>
Date: Mon, 6 Sep 2021 14:58:17 +0800 (CST)

Hi Tommy, Ian, et al.,

I think draft-ietf-ippm-ioam-direct-export provides a very useful IOAM mode, which is more attractive than other IOAM modes in some deployments, so I support this draft to progress.
I have no opinion on draft-ietf-ippm-ioam-flags.

Best Regards,
Xiao Min
------------------原始邮件------------------
发件人：TommyPauly
收件人：IETF IPPM WG ([email-masked]);
日 期 ：2021年08月31日 04:58
主 题 ：[ippm] WGLC for IOAM Flags and Direct Exports
_______________________________________________
ippm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ippm

Hello IPPM,
https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-flags/https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-flags-06

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 7 Sep 2021 15:00:12 -0700

Dear Authors, WG Chairs, and All,
I've read the latest version of the draft and have several comments and
questions to share with you. I would much appreciate the opportunity to
discuss these before I can state my support for progressing this draft
to publication.

   - The Abstract states that:

   In-situ Operations, Administration, and Maintenance (IOAM) records
   operational and telemetry information in packets while they traverse
   a path between two points in the network.

I think that is not a complete description of IOAM options as it excludes
the Direct Export option. It might be easier to improve by
s/records/collects/. Also, what, in your opinion, is the distinction
between operational and telemetry information? I think that operational
information collected from remote nodes is one example of telemetry
information. What do you think?

   - nit s/the the/the/
   - I think that Abstract and Introduction can be improved by explaining
   the purpose of defining the Loopback and Active flags. What each of them is
   intended to achieve and improve?
   - It is stated in the Introduction that:

   This document defines
   two new flags in the Pre-allocated and Incremental Trace options: the
   Loopback and Active flags.
Does that mean that neither of the two flags is applicable in the Direct
Export option? Should draft-ietf-ippm-ioam-direct-export have an explicit
statement regarding the applicability of the Loopback and Active flags? And
if any is applicable, describe the behavior of an IOAM node receiving it.

   - In Section 4 the purpose of using the Loopback flag is explained as:

   Loopback allows an IOAM encapsulating node to trace the path to a
   given destination, and to receive per-hop data about both the forward
   and the return path.
In my previous comments, I've pointed out that the Loopback mode is just a
special case of the Direct Export option. In addition to that, I would
appreciate your explanation of how the IOAM encapsulating node receives
per-hop data about the return path. How does that work, for example, in an
MPLS or SFC NSH environment?

   - Further it Section 4, it is stated that:

   The two IOAM looped back copies are
   terminated by the encapsulating node.
But I don't see any technical explanation that could sustain that
assumption. As I understand the diagram in figure 1, a transit node doesn't
know the identity of the IOAM encapsulating node. Correct? If that is the
case, where does the transit node direct the looped packet?
It seems that the text in the last paragraph attempts to walk around my
questions. But I don't find that paragraph sufficient. It lacks the
normative language to guide a developer to produce resilient
implementation. For example, how to interpret:
   Loopback can be used only if a return path from transit nodes and
   destination nodes towards the source (encapsulating node) exists.
That reads like a recommendation, not a requirement. As a result, an
implementation may transmit an IOAM packet with the Loopback fag set
regardless of whether there's a return path or not. Then, how must a
transit node process such an IOAM packet?

   - A very concerning statement I've found in Section 4.1

   The encapsulating node either generates synthetic packets with an
   IOAM trace option that has the Loopback flag set ...
IOAM has been defined as a hybrid measurement protocol. This statement
implies that it is to be used as an active OAM protocol (all per RFC 7799
definitions). The requirements for active OAM protocols in terms of their
applicability and, most of, security are very different from those set for
a hybrid measurement protocol. I believe that extending IOAM into an active
mode of measurement requires more discussion and the audience from other
IETF WGs, e.g., 6man, SFC, BIER, MPLS, BFD, RTGWG.

   - The next statement explains the use case for the Loopback flag:

   Loopback is used
   either proactively or on-demand, i.e., when a failure is detected.
That is the same use of ICMP and LSP Ping. What is the benefit of using the
Loopback flag instead of readily available ICMP? Does it work as L2
Linktrace? I don't see that as a sufficient benefit considering the
complexity, exclusions, and risks associated and identified in the draft.
It is clear that there are tools like ICMP and LSP Ping that adequately
address scenarios listed in the draft. Don't see any technical reason for
the introduction of another mechanism that does not have any significant
benefit compared to the existing OAM tools. Especially, considering the
limited information that is allowed to be collected using the Loopback IOAM
flag ("every transit node that processes this trace option only adds a
single data field, which is the Hop_Lim and node_id data field"). With the
abundance of information and additional verification achieved using ICMP
and LSP Ping, there's even less rationale in the introduction of this
over-complex and unnatural mechanism.

   - The last paragraph states the requirement:

   IOAM encapsulating nodes MUST NOT push an IOAM encapsulation with the
   Loopback flag onto data packets that already include an IOAM
   encapsulation.
I couldn't find any guidance on how an IOAM node that receives such invalid
combination processes the packet.

   - It is assumed that the identity of the IOAM encapsulating node is
   known to the node that is expected to sent a copy of the received packet
   back. But what if that is not the case? How must that node act?
   - I believe that there's inconsistency between Figure 1 and the
   following text in Section 4.2

   Thus, the
   source address of the original packet is used as the destination
   address in the copied packet.
It seems that the text assumes that the IOAM encapsulating node adds also a
network layer encapsulation. But how would it work, for example, in SFC NSH
data plane? Or MPLS?

   - Section 4.4 states that

   If the Node ID matches the current
   IOAM node, it indicates that this is a looped back packet that was
   initiated by the current IOAM node, and processed accordingly.
But according to the definition of node_id in Section 5.4.2.1
of draft-ietf-ippm-ioam-data, "Node identifier field to uniquely identify a
node within the IOAM-Namespace and associated IOAM-Domain." It appears that
simply comparing Node IDs is not sufficient to verify that the node
received its looped back packet. What is your opinion?

   - If, as stated in Section 5,

   It [Active flag] is not intended as a replacement for existing
   active OAM protocols, which may run in higher layers and make use of
   the Active flag.
Then the question, What is the purpose of the Active flag? Existing active
protocols, Fault Management and Performance Monitoring, are identified by,
for example, use of a well-known destination port number or, in case of
MPLS, Generic Associated Channel type. There's no apparent need of using
IOAM option header to identify the payload as a control message of an
active OAM protocol. On the contrary, that would likely cause more
confusion and challenges to interoperation among different implementations.

   - Three cases analyzed in Section 5 require special consideration, as
   these likely present the motivation for the introduction of the Active flag:
      - the first use case is not really using the Active flag at all and
      thus the text doesn't seem adding any value to the draft;
      - it is not clear what is viewed as the benefit of using the Active
      flag compared with using any of the existing active OAM protocols in the
      second use case. I would appreciate it if you can clarify that for me.
      Since an SFC NSH network is used as an example, is it required that a
      Service Function (SF) or an SF Proxy process the IOAM header with Active
      flag set? Or would a Service Function Forwarder (SFF) avoid sending a
      packet with the IOAM header that has an Active flag set? And can you
      compare that with the procedures defined in the Active OAM for SFC
      draft
      <https://datatracker.ietf.org/doc/draft-ietf-sfc-multi-layer-oam/>?
      - My questions for the third use case described in Section 5 are the
      same as above because I don't see the benefit of using a replicated data
      packet (less benefits if it is truncated) compared to a special
test packet.
   - In my conclusion on the Active flag, I don't see any clear benefit of
   introducing it and using whether in place or in addition to the existing
   active OAM protocols.

I am looking forward to your feedback and the continued discussion.

Regards,
Greg

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 7 Sep 2021 15:00:15 -0700

Dear Authors, WG Chairs, and All,
I've read the latest version of the draft and have several comments and
questions to share with you. None of these are blocking, I support
progressing this draft to publication. Please consider the notes below as
an invitation to a discussion:

   - It seems that the characterization of the Direct Export IOAM option in
   the following is not entirely accurate:

The DEX option is used as a trigger to collect and/or export IOAM data.

I propose re-wording:
The DEX option is used as a trigger to collect and export or only collect
locally the IOAM data.

Using "exporting and/or collecting" reflects the use of the Direct Export
option type technically accurate but the order of actions is backward. What
are your thoughts?

   - I find references to RFC 5475 and 7014 very appropriate and helpful.
   But I don't see these RFCs being referenced in draft-ietf-ippm-ioam-data
   <https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-data/>. I think
   that excessive use of not only the Direct Export IAOM option but the IOAM
   method, in general, may adversely affect a network and the recommendations
   listed in RFC 5475 and 7014 are equally applicable.
   - Similarly, the good and helpful discussion of how to practically
   control the use of the Direct Export option is, in my opinion, equally
   applicable to other IOAM trace options and should be in the Security
   Considerations section of draft-ietf-ippm-ioam-data.
   - The second paragraph in Section 3.1.2 is re-phrasing if not repeating
   what is already expressed in the last paragraph of the previous sub-section
   (though the limit is applied to exporting telemetry information). I think
   that it can be removed without any loss to the document.
   - I think that it will be helpful to provide some recommendations, i.e.,
   SHOULD do this, how an implementation handles the excess of packets to
   export. Drop silently, drop and log notification, or try to shape that flow?
   - I was really puzzled by this requirement in Section 3.1.2

   Exported packets SHOULD NOT be exported over a path or a tunnel that

   is subject to IOAM direct exporting.

I hope you can clarify it for me.

   - This requirement

   Furthermore, IOAM encapsulating
   nodes that can identify a packet as an IOAM exported packet MUST NOT
   push a DEX option into such a packet.
brings up questions:

   - How can a node identify a packet as the IOAM exported packet?
   - What should be done if the IOAM exported packet contains the IOAM
   header with the DEX option?

   - What is special about the decapsulating node that is not DEX-capable?

   A decapsulating node that does not support the DEX option
   MUST remove it, along with any other IOAM options carried in the
   packet if such exist.
Shouldn't that be a general behavior of a decapsulating IOAM node?

   - The format of the first four-octet word in Figure 2 is different from
   the format defined in draft-ietf-ippm-ioam-data:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Namespace-ID           |NodeLen  | Flags | RemainingLen|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               IOAM-Trace-Type                 |  Reserved     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Is this intentional? If it is, how does an implementation know which format
to use?

   - It is stated in the draft that:

The length of the DEX option is at least 8 octets.

But the figure 2 in the draft does not include any field that reflects the
length of the IOAM option. Is that because the figure is out of sync? If
not, how can an implementation verify the length? Also, what is the
behavior if the length is less than 8 octets?

   - RE: PerformanceConsiderations

I agree with you that a DEX-encapsulating node and DEX-transit nodes must
have rate-limiting controls (as noted earlier, the behavior of the transit
node needs more detailed specification). But I think that the document
should also note that exported packets could be sent over the network
out-of-band, relative to the monitored flow. As a result, the impact of
packets carrying telemetry information might be not direct and harmful to
the data flow. I see that as one of the important benefits of the Direct
Export method transporting on-path telemetry information.

Regards,
Greg

---

From: "Rakesh Gandhi (rgandhi)" <[email-masked]>
Date: Thu, 9 Sep 2021 13:59:34 +0000

Hi WG,

I have been following the work as part of MPLS encapsulation for IOAM in draft-gandhi-mpls-ioam.

I support the WG adoption of both of these documents.

Thanks,
Rakesh

Best,
Tommy & Ian

---

From: "Rakesh Gandhi (rgandhi)" <[email-masked]>
Date: Thu, 9 Sep 2021 15:24:10 +0000

Hi WG,

I believe both documents are ready to progress to the next steps.

Thanks,
Rakesh

Best,
Tommy & Ian

---

From: Tommy Pauly <[email-masked]>
Date: Wed, 15 Sep 2021 10:32:21 -0700

Hello IPPM,

Thanks to everyone who provided feedback! Our time for the WGLC has concluded. In general, we see good support for progressing the documents.

Authors, would you please provide some responses/updates to Greg’s comments? If we can come to a resolution on those and update the documents, we can progress them.

Best,
Tommy

> On Aug 30, 2021, at 1:57 PM, Tommy Pauly <tpauly=[email-masked]> wrote:
> 
> Hello IPPM,
> 
> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-flags/ <https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-flags/>
> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-flags-06 <https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-flags-06>
> 
> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/ <https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/>
> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-direct-export-06 <https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-direct-export-06>
>

---

From: Tal Mizrahi <[email-masked]>
Date: Thu, 30 Sep 2021 10:17:43 +0300

Hi Greg,

Many thanks for the thorough review.
Below you can find responses to your comments, and in some cases
proposed text edits.

If there are no further comments we will update the document accordingly.

Thanks,
The authors.

On Wed, Sep 8, 2021 at 1:01 AM Greg Mirsky <[email-masked]> wrote:
> I've read the latest version of the draft and have several comments and questions to share with you. None of these are blocking, I support progressing this draft to publication. Please consider the notes below as an invitation to a discussion:
>
> It seems that the characterization of the Direct Export IOAM option in the following is not entirely accurate:
>
> The DEX option is used as a trigger to collect and export or only collect locally the IOAM data.
>
> Using "exporting and/or collecting" reflects the use of the Direct Export option type technically accurate but the order of actions is backward. What are your thoughts?
>

[TM] Right. The following text edit is proposed in the introduction.

OLD:
   This option is used as
   a trigger for IOAM nodes to locally aggregate and process IOAM data,
   and/or to export it to a receiving entity (or entities).
NEW:
   This option is used as
   a trigger for IOAM nodes to locally aggregate and process IOAM data,
   and/or to export it to a receiving entity (or entities). Throughout
   the document this functionality is referred to as collection and/or
exporting.

> I find references to RFC 5475 and 7014 very appropriate and helpful. But I don't see these RFCs being referenced in draft-ietf-ippm-ioam-data. I think that excessive use of not only the Direct Export IAOM option but the IOAM method, in general, may adversely affect a network and the recommendations listed in RFC 5475 and 7014 are equally applicable.
> Similarly, the good and helpful discussion of how to practically control the use of the Direct Export option is, in my opinion, equally applicable to other IOAM trace options and should be in the Security Considerations section of draft-ietf-ippm-ioam-data.

[TM] The last two comments are about draft-ietf-ippm-ioam-data, which
is currently in IESG review. I will take this feedback with the other
authors to see how we can address this input at this point.

> The second paragraph in Section 3.1.2 is re-phrasing if not repeating what is already expressed in the last paragraph of the previous sub-section (though the limit is applied to exporting telemetry information). I think that it can be removed without any loss to the document.

[TM] The second par of 3.1.2 refers to *exporting*, while the last par
of 3.1.1 refers to *encapsulating*, so these are two complementary
requirements.

> I think that it will be helpful to provide some recommendations, i.e., SHOULD do this, how an implementation handles the excess of packets to export. Drop silently, drop and log notification, or try to shape that flow?

[TM] Assuming an amplification scenario, a large number of packets may
be dropped, so logging each drop may defeat the purpose. A drop
counter may be a good idea here, but this document does not really
define counter requirements in general, so it seems like more of an
implementation-specific aspect, or possibly can be considered as part
of the YANG model.

> I was really puzzled by this requirement in Section 3.1.2
>

[TM] As explained later in that paragraph: "This requirement is
intended to prevent nested exporting and/or exporting loops."
This was thoroughly discussed in the working group, and came out of
the security-related review.

> This requirement
> How can a node identify a packet as the IOAM exported packet?

[TM] The requirement means that if an encapsulating node's ACL can
detect exported packets, then DEX should not be applied.

> What should be done if the IOAM exported packet contains the IOAM header with the DEX option?

[TM] The current document does not require an IOAM transit node to
parse beyond the (external) IOAM encapsulation if there are nested
IOAM encapsulations.

>
> What is special about the decapsulating node that is not DEX-capable?
>

[TM] Right. The following text edit is proposed:

OLD:
   A transit IOAM node that does not support the DEX option SHOULD
   ignore it.  A decapsulating node that does not support the DEX option
   MUST remove it, along with any other IOAM options carried in the
   packet if such exist.
NEW:
   A transit or decapsulating IOAM node that does not support the DEX
option SHOULD
   ignore it.  Note that as per <xref target="I-D.ietf-ippm-ioam-data"/> a
   decapsulating node removes the IOAM encapsulation and all its IOAM options,
   and specifically in the case where one of these options is a
(possibly unknown)
   DEX option.

>
> The format of the first four-octet word in Figure 2 is different from the format defined in draft-ietf-ippm-ioam-data:
>
>     0                   1                   2                   3
> Is this intentional? If it is, how does an implementation know which format to use?

[TM] Each IOAM Option-Type has a different header format. The current
document defines a new Option-Type, which was not defined in
draft-ietf-ippm-ioam-data.
The IOAM-Type field in the lower layer header determines which
Option-Type is used and how its header should be parsed.

>
> It is stated in the draft that:
>
> But the figure 2 in the draft does not include any field that reflects the length of the IOAM option. Is that because the figure is out of sync? If not, how can an implementation verify the length? Also, what is the behavior if the length is less than 8 octets?
>

[TM] Quoting the draft:

   The existence of the optional fields is
   indicated by the corresponding flags in the Extension-Flags field.
   ...
   Thus, the Extension-Flags field explicitly indicates
   the length of the DEX option.

> RE: PerformanceConsiderations
>
> I agree with you that a DEX-encapsulating node and DEX-transit nodes must have rate-limiting controls (as noted earlier, the behavior of the transit node needs more detailed specification). But I think that the document should also note that exported packets could be sent over the network out-of-band, relative to the monitored flow. As a result, the impact of packets carrying telemetry information might be not direct and harmful to the data flow. I see that as one of the important benefits of the Direct Export method transporting on-path telemetry information.
>

[TM] Right. The following new text is proposed:

It should be noted that in some networks DEX data may be exported over
an out-of-band network, in which a large volume of exported traffic
does not compromise user traffic. In this case an operator may choose
to disable the exporting rate limiting.

>
> Regards,
> Greg
>
> On Mon, Aug 30, 2021 at 1:58 PM Tommy Pauly <tpauly=[email-masked]> wrote:
>>
>> Hello IPPM,
>>

---

From: Tal Mizrahi <[email-masked]>
Date: Thu, 30 Sep 2021 10:42:14 +0300

Hi Greg,

Many thanks for the detailed comments.
Please see the responses below, as well as suggested text edits in some cases.

Cheers,
The authors.

On Wed, Sep 8, 2021 at 1:00 AM Greg Mirsky <[email-masked]> wrote:
> I've read the latest version of the draft and have several comments and questions to share with you. I would much appreciate the opportunity to discuss these before I can state my support for progressing this draft to publication.
>
> The Abstract states that:
>
> I think that is not a complete description of IOAM options as it excludes the Direct Export option. It might be easier to improve by s/records/collects/. Also, what, in your opinion, is the distinction between operational and telemetry information? I think that operational information collected from remote nodes is one example of telemetry information. What do you think?

[TM] Agreed. Your proposal makes sense: s/records/collects/
Regarding the difference between operational and telemetry
information, the way I see it "telemetry" refers to measurements that
are being exported to an external/remote entity, whereas operational
information is not measurement-based, such as the Node ID or Interface
ID.

> nit s/the the/the/

[TM] Agreed.

> I think that Abstract and Introduction can be improved by explaining the purpose of defining the Loopback and Active flags. What each of them is intended to achieve and improve?

[TM] The point is well-taken, and the following text will be added to
the introduction. In order to avoid text repetition, I believe it
would be best not to add it to the abstract as well.

OLD:
-
NEW:
The Loopback flag is used to request that each transit device along
the path loops back a copy of the data packet to the sender. The
Active flag indicates that a packet is used for active measurement.

> It is stated in the Introduction that:
>
>    This document defines
> Does that mean that neither of the two flags is applicable in the Direct Export option? Should draft-ietf-ippm-ioam-direct-export have an explicit statement regarding the applicability of the Loopback and Active flags? And if any is applicable, describe the behavior of an IOAM node receiving it.

[TM] Right.
The direct exporting draft explicitly states the behavior of each
field in the DEX option header including the flags, so there seems to
be no need to specify flags that are supported in other options.

>
> In Section 4 the purpose of using the Loopback flag is explained as:
>
> In my previous comments, I've pointed out that the Loopback mode is just a special case of the Direct Export option. In addition to that, I would appreciate your explanation of how the IOAM encapsulating node receives per-hop data about the return path. How does that work, for example, in an MPLS or SFC NSH environment?

[TM] It is slightly similar to direct exporting, but is not a special
case of direct exporting. In direct exporting telemetry/operational
data is not pushed into the data packet, but the data packet with the
DEX option triggers the data collection and optionally also exporting,
but exporting is not mandatory and may be performed later, for example
by the management plane. On the other hand, a trace option with the
loopback flag causes all IOAM transit nodes to push IOAM data onto the
trace option, and also loop a copy of the packet back to the sender.

>
> Further it Section 4, it is stated that:
>
> But I don't see any technical explanation that could sustain that assumption. As I understand the diagram in figure 1, a transit node doesn't know the identity of the IOAM encapsulating node. Correct? If that is the case, where does the transit node direct the looped packet?

[TM] As described in the draft, the loopback flag is not necessarily
applicable to all possible encapsulations. The way the return path is
defined is encapsulation-specific. The draft currently states:

   Loopback can be used only if a return path from transit nodes and
   destination nodes towards the source (encapsulating node) exists.
   Specifically, loopback is only applicable in encapsulations in which
   the identity of the encapsulating node is available in the
   encapsulation header.  If an encapsulating node receives a looped
   back packet that was not originated from the current encapsulating
   node, the packet is dropped.

> It seems that the text in the last paragraph attempts to walk around my questions. But I don't find that paragraph sufficient. It lacks the normative language to guide a developer to produce resilient implementation. For example, how to interpret:
> That reads like a recommendation, not a requirement. As a result, an implementation may transmit an IOAM packet with the Loopback fag set regardless of whether there's a return path or not. Then, how must a transit node process such an IOAM packet?
>

[TM] Right, the last response and quoted text is relevant here as
well. Please also note the following text from the draft:

it is assumed that the
   source address is available in the encapsulation header.  Thus, the
   source address of the original packet is used as the destination
   address in the copied packet.

I propose to add the following sentence:
The loopback flag MUST NOT be set if it is not guaranteed that there
is a return path from each of the IOAM transit and IOAM decapsulating
nodes, or if the encapsulating node's address is not available in the
encapsulation header.

> A very concerning statement I've found in Section 4.1
>
> IOAM has been defined as a hybrid measurement protocol. This statement implies that it is to be used as an active OAM protocol (all per RFC 7799 definitions). The requirements for active OAM protocols in terms of their applicability and, most of, security are very different from those set for a hybrid measurement protocol. I believe that extending IOAM into an active mode of measurement requires more discussion and the audience from other IETF WGs, e.g., 6man, SFC, BIER, MPLS, BFD, RTGWG.

[TM] This draft does not define an active measurement protocol. The
point in the draft is that the loopback flag can be used in active
measurement packets.
The following text edit is proposed:

OLD:
   The encapsulating node either generates synthetic packets with an
   IOAM trace option that has the Loopback flag set, or sets the loopack
   flag in a subset of the in-transit data packets.  Loopback is used
   either proactively or on-demand, i.e., when a failure is detected.
NEW:
   The loopback flag can be set in the IOAM trace option either in a
   subset of the in-transit data packets, or in synthetic packets that
   are generated by the encapsulating node. Loopback can be used
   either proactively or on-demand, i.e., when a failure is detected.

>
> The next statement explains the use case for the Loopback flag:
>
>    Loopback is used
> That is the same use of ICMP and LSP Ping. What is the benefit of using the Loopback flag instead of readily available ICMP? Does it work as L2 Linktrace? I don't see that as a sufficient benefit considering the complexity, exclusions, and risks associated and identified in the draft. It is clear that there are tools like ICMP and LSP Ping that adequately address scenarios listed in the draft. Don't see any technical reason for the introduction of another mechanism that does not have any significant benefit compared to the existing OAM tools. Especially, considering the limited information that is allowed to be collected using the Loopback IOAM flag ("every transit node that processes this trace option only adds a single data field, which is the Hop_Lim and node_id data field"). With the abundance of information and additional verification achieved using ICMP and LSP Ping, there's even less rationale in the introduction of this over-complex and unnatural mechanism.

[TM] Right, as the draft says, the loopback flag is an "...alternative
to Traceroute, that allows the encapsulating node to receive responses
from multiple transit nodes along the path". Indeed, it slightly
resembles ETH-LT (Ethernet Linktrace) in the sense that a single
packet from the encapsulating node allows to quickly trace the entire
path, but as with IOAM in general, using the loopback flag does not
mandate a specific encapsulation.

>
> The last paragraph states the requirement:
>
> I couldn't find any guidance on how an IOAM node that receives such invalid combination processes the packet.
>

[TM] The current draft only specifies the flags. Specifically in the
scenario described above, the encapsulating node does not set the
loopback flag. This is intended to reduce the impact of potential
amplification attacks.

>
> It is assumed that the identity of the IOAM encapsulating node is known to the node that is expected to sent a copy of the received packet back. But what if that is not the case? How must that node act?

[TM] Agreed. The following new text is proposed to be added to Section 4.1:
If the address of the encapsulating node is not available in the
encapsulation header, then the transit/decapsulating node does not
loop back a copy of the original packet.

> I believe that there's inconsistency between Figure 1 and the following text in Section 4.2
>
>    Thus, the
> It seems that the text assumes that the IOAM encapsulating node adds also a network layer encapsulation. But how would it work, for example, in SFC NSH data plane? Or MPLS?

[TM] Right, the paragraph beginning with "Loopback can be used
only..." (discussed in response to one of the previous comments) makes
that restriction.

> Section 4.4 states that
> But according to the definition of node_id in Section 5.4.2.1 of draft-ietf-ippm-ioam-data, "Node identifier field to uniquely identify a node within the IOAM-Namespace and associated IOAM-Domain." It appears that simply comparing Node IDs is not sufficient to verify that the node received its looped back packet. What is your opinion?

[TM] Agreed. The following update is proposed:
OLD:
If the Node ID matches
NEW:
If the Node ID and IOAM-Namespace match

>
> If, as stated in Section 5,
>
> Then the question, What is the purpose of the Active flag? Existing active protocols, Fault Management and Performance Monitoring, are identified by, for example, use of a well-known destination port number or, in case of MPLS, Generic Associated Channel type. There's no apparent need of using IOAM option header to identify the payload as a control message of an active OAM protocol. On the contrary, that would likely cause more confusion and challenges to interoperation among different implementations.
>
> Three cases analyzed in Section 5 require special consideration, as these likely present the motivation for the introduction of the Active flag:
>
> the first use case is not really using the Active flag at all and thus the text doesn't seem adding any value to the draft;
> it is not clear what is viewed as the benefit of using the Active flag compared with using any of the existing active OAM protocols in the second use case. I would appreciate it if you can clarify that for me. Since an SFC NSH network is used as an example, is it required that a Service Function (SF) or an SF Proxy process the IOAM header with Active flag set? Or would a Service Function Forwarder (SFF) avoid sending a packet with the IOAM header that has an Active flag set? And can you compare that with the procedures defined in the Active OAM for SFC draft?
> My questions for the third use case described in Section 5 are the same as above because I don't see the benefit of using a replicated data packet (less benefits if it is truncated) compared to a special test packet.
>
> In my conclusion on the Active flag, I don't see any clear benefit of introducing it and using whether in place or in addition to the existing active OAM protocols.

[TM] General response to all the Active flag comments: this was
extensively discussed on the mailing list and in previous IETF
meetings, and your comments about the previous versions of the
document have contributed to greatly improving Section 5, which
describes the main use cases of the Active flag.