Subject: [ippm] Benjamin Kaduk's Discuss on draft-ietf-ippm-multipoint-alt-mark-07: (with DISCUSS and COMMENT)

From: Benjamin Kaduk via Datatracker <[email-masked]>
Date: Wed, 11 Mar 2020 19:02:26 -0700

Benjamin Kaduk has entered the following ballot position for
draft-ietf-ippm-multipoint-alt-mark-07: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-ippm-multipoint-alt-mark/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Let's discuss whether [I-D.mizrahi-ippm-compact-alternate-marking] needs
to be a normative reference, to describe how the Hash selection method
works for multipoint.  This document alone does not even mention what is
used as input to the hash (though I think I have a good guess based on
the context).  Even if the intent is that RFC 5474 suffices (avoiding
the "dependency on individual document" issue), that is also listed only
as an informative reference.

Also, if the grouping procedure (section 6.1) does in fact require a
distinguished (but arbitrary?) choice of initial endpoint as I suspect
it does, that should be clarified.  (See COMMENT.)

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I echo the other reviewers' comments about clarity as to "multipoint"
vs. "multicast" and defining various terms.

I agree with Barry that there's a lot of editorial work to be done; I've
to note many instances in these comments even when the proper resolution
is unclear to me.

Section 1

   This approach fits very well with the Intelligent Network and
   Software Defined Network (SDN) paradigm where the SDN Orchestrator
   and the SDN Controllers are the brains of the network and can manage
   flow control to the switches and routers and, in the same way, can
   calibrate the performance measurements depending on the necessity.

nit: "necessity" doesn't sound right

   An SDN Controller Application can orchestrate how deep the network
   performance monitoring is setup by applying the Multipoint Alternate
   Marking as described in this document.

nit: "how deep the network" doesn't sound right

Section 3

   In this way the flows to be monitored are selected into the
   monitoring points using packet selection rules, that can also change
   the pattern of the monitored network.

nit: I'm not sure what "this way" refers to.  It also seems like this
may be two independent thoughts needlessly joined together with a comma.

Section 4.1

   [I-D.ietf-ippm-route]).  In general there are different options: the
   monitoring network can be obtained by considering all the possible
   paths for the traffic or also by checking the traffic sometimes and
   update the graph consequently.

"also by checking the traffic sometimes and update the graph
consequently" feels pretty informal and under-specified.

Section 5

   Since all the packets of the considered flow leaving the network have
   previously entered the network, the number of packets counted by all
   the input nodes is always greater or equal than the number of packets
   counted by all the output nodes.

[I think I could imagine some exotic cases where this does not hold, but
none of them really seem topical for this work.]

   It is possible to define the Network Packet Loss of one monitored
   flow for a single period: <<In a packet network, the number of lost
   packets is the number of packets counted by the input nodes minus the
   number of packets counted by the output nodes>>.  This is true for
   every packet flow in each marking period.

As another reviewer implies, this discounts the difference between the
number of packets "still in the network" at the start and end of the
measurement period.

      The flow definition is generalized here, indeed, as described
      before, a multipoint packet flow is considered and the
      identification fields of the 5-tuple can be selected without any
      constraints.

I don't think I understand what this means.  If identification is fully
general, how are we still limited to a 5-tuple?

Section 6
   In a completely monitored network (a network where every network
   interface is monitored), each network device corresponds to a Cluster
   and each physical link corresponds to two Clusters (one for each
   direction).

("what about unidirectional links?")

Section 6.1

I'm not following how the first step of this algorithm produces the
listed groups.  Are the links considered to be bidirectional?  Did we
have to pick some arbitrary starting node (R1), and decree that once a
link is in one group it cannot be in some other group?

   In this way the calculation of packet loss can be made on Cluster
   basis.  Note that CIR(Committed Information Rate) and EIR(Excess
   Information Rate) can also be deduced on Cluster basis.

Do we use CIR and/or EIR anywhere else?  (Are they references to some
other body of work?)

   In this way in a very large network there is no need to configure
   detailed filter criteria to inspect the traffic.  You can check
   multipoint network and only in case of problems you can go deep with
   a step-by-step cluster analysis, but only for the cluster or
   combination of clusters where the problem happens.

nits: there at least one missing article here ("a multipoint network")
and I think there was something else that feels not quite right.

   In summary, once defined a flow, the algorithm to build the Cluster
   Partition considers all the possible links and nodes crossed by the

What does "once defined a flow" mean?

   information.  So, if the flow do not enter or traverse all the nodes,
   the counters has a non-zero value for the involved nodes, while a

nit: singular/plural mismatch "the flow"/"do not", "counters"/"has a
[...] value"

   The algorithm described above is an Iterative clustering algorithm,
   but it is also possible to apply a Recursive clustering algorithm by
   using the node-node adjacency matrix representation.

Is there a reference for this?  (Do I assume it's the
[IEEE-ACM-ToN-MPNPM] at the end of the next paragraph?)

Section 7
   Therefore, when we expand to multipoint-to-multipoint flows, we have
   to consider that all source nodes mark the traffic and this adds more
   complexity.

I'm not sure what chain of reasoning "therefore" is attempting to
indicate.

   But we should now consider an additional contribution.  Since all
   source nodes mark the traffic, the source measurement intervals can
   be of different lengths and with different offsets and this mismatch
   m can be added to d, as shown in figure.

Please define m and d.

   So the misalignment between the marking source routers gives an
   additional constraint and the value of m is added to d (that already
   includes clock error and network delay).

   Thus, three different possible constraints are considered: clock
   error between network devices, network delay between measurement
   points and the misalignment between the marking source routers.

Are these "constraints" or "contributions [to error/uncertainty]"?  RFC
8321 does not seem to use "constraint" in this fashion.

   In the end, the condition that must be satisfied to enable the method
   to function properly is that the available counting interval must be
   > 0, and that means: L - 2m - 2d > 0 for each measurement point on
   the multipoint path.  Therefore, the mismatch between measurement
   intervals must satisfy this condition.

Is it bad to just make L really large?

Section 8

   period and this is the time reference that can be used.  It is
   important to highlight that both delay and delay variation
   measurements make sense in a multipoint path.  The Delay Variation is
   calculated by considering the same packets selected for measuring the
   Delay.

Is the "variation" considered here just the variation across the
separate paths that the selected packets take, or over time, or
something else?

   o  Delay measurements on single packets basis means that you can use
      multipoint path just to easily couple packets between inputs and

nit: singular/plural mismatch "packets"/"basis" (also missing "a"?)
nit: "inpur" singular, I think.

Section 8.1.1

I don't understand what weights are used for the "weighted averages".

Section 8.2.1

   since they would not be representative of the entire flow.  The
   packets can follow different paths with various delays and in general
   it can be very difficult to recognize marked packets in a multipoint-
   to-multipoint path especially in case they are more than one per
   period.

nits: "the case when there is", comma before "and in general".

Section 8.2.2

   [I-D.mizrahi-ippm-compact-alternate-marking] introduces how to use
   the Hash method combined with alternate marking method for point-to-

Is it really appropriate for a WG document to depend on an individual
document to introduce a concept?

   In a multipoint environment the behaviour is similar to point-to
   point flow.  In particular, in the context of multipoint-to-
   multipoint flow, the dynamic hash could be the solution to perform

nits: both of these either need "flows" plural or the article "a".

   The management system receives the samples including the timestamps
   and the hash value from all the MPs, and this happens both for point-
   to-point and for multipoint-to-multipoint flow.  Then the longest

nit: "flows"

Also, this seems to be the first time we abbreviate "measurement
points"; it would be good to provide the abbreviation/expansion together
and consistently use the abbreviation.  (Interestingly
https://www.rfc-editor.org/materials/abbrev.expansion.txt only has
"multiprotocol" and "maintenance point".)

   to-point and for multipoint-to-multipoint flow.  Then the longest
   hash used by MPs is deduced and it is applied to couple timestamps of
   same packets of 2 MPs of a point-to-point path or of input and output
   MPs of a Cluster (or a Super Cluster or the entire network).  But

nit: there appear to be several missing words here.

   In summary, the basic hash is logically similar to the double marking
   method, and in case of point-to-point path double marking and basic

I don't really recall any specific discussion of similarities between
basic hash and double marking, so calling this a "summary" is perhaps a
stretch.

Section 9

   An SDN Controller or a Network Management System (NMS) can calibrate
   Performance Measurements since it is aware of the network topology.
   It can start without examining in depth.  In case of necessity
   (packet loss is measured or the delay is too high), the filtering
   criteria could be immediately specified more in order to perform a
   partition of the network by using Clusters and/or different
   combinations of Clusters.  In this way the problem can be localized

nit: "specified more" seems incomplete; perhaps something about detail?

Section 11

I don't see much in RFC 8321 to note that "traffic observed by
measurement points may contain private information if not protected by
transport-layer security protocols, so measurement infrastructure should
be as equally protected/secured as routing hardware".  That said, it is
hopefully obvious, and not specific to this work, so I don't feel a
particular need to have it mentioned here.

---

From: Giuseppe Fioccola <[email-masked]>
Date: Thu, 12 Mar 2020 14:09:58 +0000

Dear Benjamin,
Thanks for your detailed comment.
Please find below my replies tagged as [GF].

Let me know if you agree with my proposed changes to the draft and I will include them in the next revision.

Best Regards,

Giuseppe

-----Original Message-----
From: Benjamin Kaduk via Datatracker [mailto:[email-masked]] 
Sent: Thursday, March 12, 2020 3:02 AM
To: The IESG <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; Tal Mizrahi <[email-masked]>; [email-masked]
Subject: Benjamin Kaduk's Discuss on draft-ietf-ippm-multipoint-alt-mark-07: (with DISCUSS and COMMENT)

When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.)

Let's discuss whether [I-D.mizrahi-ippm-compact-alternate-marking] needs to be a normative reference, to describe how the Hash selection method works for multipoint.  This document alone does not even mention what is used as input to the hash (though I think I have a good guess based on the context).  Even if the intent is that RFC 5474 suffices (avoiding the "dependency on individual document" issue), that is also listed only as an informative reference.

[GF]: Yes, this is an oversight and I think we can surely move RFC 5474 as normative reference. We left [I-D.mizrahi-ippm-compact-alternate-marking] as informative reference because it introduces the use of the Hash Method in combination with Alternate Marking for point-to-point flow and this is not covered by this or other documents. 

Also, if the grouping procedure (section 6.1) does in fact require a distinguished (but arbitrary?) choice of initial endpoint as I suspect it does, that should be clarified.  (See COMMENT.)

[GF]: Ok, I need to clarify better this point in Section 6.1. The grouping procedure implies that the links are unidirectional and you can start from any link. The procedure works anyway. We just need to classify them and put together the links where there is the same starting node in separated groups.

I agree with Barry that there's a lot of editorial work to be done; I've to note many instances in these comments even when the proper resolution is unclear to me.

Section 1

[GF]: I can replace "necessity" with "desired accuracy"

[GF]: I can replace "how deep" with "how accurate"

Section 3

nit: I'm not sure what "this way" refers to.  It also seems like this may be two independent thoughts needlessly joined together with a comma.

[GF]: Yes I can omit "in this way", maybe "so" is better here

Section 4.1

"also by checking the traffic sometimes and update the graph consequently" feels pretty informal and under-specified.

[GF]: I will modify this sentence. It means that the network graph can be updated in several ways (e.g. daily basis, weekly basis,...). It is up to management system configuration.

Section 5

[I think I could imagine some exotic cases where this does not hold, but none of them really seem topical for this work.]

[GF]: Yes correct, the hypothesis is that what goes in is the same as what goes out

As another reviewer implies, this discounts the difference between the number of packets "still in the network" at the start and end of the measurement period.

[GF]: Consider that, with Alternate Marking and this does not happen since we always use a waiting interval to take the counters in order to make sure that they are stable and not still in the network. This is explained in section 7.

I don't think I understand what this means.  If identification is fully general, how are we still limited to a 5-tuple?

[GF]: You are right, no need to refer to 5-tuple here.

Section 6

[GF]: This is an oversight. We are considering the unidirectional flow here. I can replace "direction" with "device".

Section 6.1

I'm not following how the first step of this algorithm produces the listed groups.  Are the links considered to be bidirectional?  Did we have to pick some arbitrary starting node (R1), and decree that once a link is in one group it cannot be in some other group?

[GF]: The links are unidirectional and we can start from any link not from a node. We can imagine to have a list of all the links as connection between two nodes and we can group them considering the same starting node. I will clarify it better within the draft.

Do we use CIR and/or EIR anywhere else?  (Are they references to some other body of work?)

[GF]: this is an oversight. I think we can omit.

nits: there at least one missing article here ("a multipoint network") and I think there was something else that feels not quite right.

[GF]: Agree. I will review this paragraph.

[GF]: It means that you can apply the Cluster Partition based on the Network Graph that is valid for a specific flow or you can also apply the partition in general for the entire network. It depends on how we define the approach. I can review this.

nit: singular/plural mismatch "the flow"/"do not", "counters"/"has a [...] value"

[GF]: Sure, I will correct.

Is there a reference for this?  (Do I assume it's the [IEEE-ACM-ToN-MPNPM] at the end of the next paragraph?)

[GF]: Correct! I can highlight it also here.

Section 7

I'm not sure what chain of reasoning "therefore" is attempting to indicate.

[GF]: Agree. I can remove/replace.

Please define m and d.

[GF]: Sure. I will add more details here. m is misalignment between the marking source routers, while d, already introduced in RFC 8321, takes into account clock error and network delay between network nodes.

[GF]: They are just little contributions to be considered in order to establish the marking period. We need to ensure an interval available for countins that is > 0. I can change the term.

[GF]: Not bad :) and this is the practical approach, but it was just to have a mathematical formulation.

Section 8

Is the "variation" considered here just the variation across the separate paths that the selected packets take, or over time, or something else?

[GF]: It can be both the variation across separate paths or valid for the multipoint path. It is the same as for the delay.

[GF]: Ok, I will correct.

Section 8.1.1

[GF]: I will explain better. In the calculation of average delay we can consider the number of packets for each end points so we can make a multipoint weighted average delay. We just mentioned it but it is not so useful in practice.

Section 8.2.1

[GF]: Ok I will correct.

Section 8.2.2

Is it really appropriate for a WG document to depend on an individual document to introduce a concept?

[GF]: I think we can revise this sentence and add a reference to RFC 5474 too.

[GF]: Ok, I will correct

nit: "flows"

[GF]: Ok, I will correct

Also, this seems to be the first time we abbreviate "measurement points"; it would be good to provide the abbreviation/expansion together and consistently use the abbreviation.  (Interestingly https://www.rfc-editor.org/materials/abbrev.expansion.txt only has "multiprotocol" and "maintenance point".)

[GF]: Ok

[GF]: Ok, I will revise and correct

I don't really recall any specific discussion of similarities between basic hash and double marking, so calling this a "summary" is perhaps a stretch.

[GF]: I will modify the sentence and add the reference to [I-D.mizrahi-ippm-compact-alternate-marking], where it is mentioned.

Section 9

[GF]: Yes, I will correct

Section 11

I don't see much in RFC 8321 to note that "traffic observed by measurement points may contain private information if not protected by transport-layer security protocols, so measurement infrastructure should be as equally protected/secured as routing hardware".  That said, it is hopefully obvious, and not specific to this work, so I don't feel a particular need to have it mentioned here.

[GF]: Ok, we can consider to remove it.

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 19 Mar 2020 13:32:02 -0700

Hi Giuseppe,

> Let me know if you agree with my proposed changes to the draft and I will include them in the next revision.

Sorry for my slow reply, and thanks for the new version.  In general it
looks good; just a few notes inline...

> https://datatracker.ietf.org/doc/draft-ietf-ippm-multipoint-alt-mark/

Ah, if the links are unidirectional then this works fine, of course.
I think I did consider the possibility of unidirectional links, but the way
the figures use "<>" where a link terminates at a router made me think
they were bidirectional.  The explanation added in the -08 looks good, and
I've changed my ballot position to No Objection, but I do wonder if
changing the figures slightly might help further.

I didn't make that connection the first time I read section 7, though I
think I can see it now.

> [GF]: Not bad :) and this is the practical approach, but it was just to have a mathematical formulation.

I definitely appreciate that -- one of my diplomas is in mathematics :)

> [GF]: It can be both the variation across separate paths or valid for the multipoint path. It is the same as for the delay.

Oh, right, I keep forgetting that "delay variation" is a term of art; sorry
about that.

> [GF]: Ok, we can consider to remove it.

Thanks for all the updates!

-Ben

---

From: Giuseppe Fioccola <[email-masked]>
Date: Fri, 20 Mar 2020 13:33:15 +0000

Dear Benjamin,
Thanks a lot for the feedback and for updating the ballot position!
Sure, we could evaluate to do the little change to the figure of Section 6.1 and highlight the direction.

Best Regards,

Giuseppe

-----Original Message-----
From: Benjamin Kaduk [mailto:[email-masked]] 
Sent: Thursday, March 19, 2020 9:32 PM
To: Giuseppe Fioccola <[email-masked]>
Cc: The IESG <[email-masked]>; [email-masked]; [email-masked]; [email-masked]; Tal Mizrahi <[email-masked]>
Subject: Re: Benjamin Kaduk's Discuss on draft-ietf-ippm-multipoint-alt-mark-07: (with DISCUSS and COMMENT)

Hi Giuseppe,

> Let me know if you agree with my proposed changes to the draft and I will include them in the next revision.

Sorry for my slow reply, and thanks for the new version.  In general it looks good; just a few notes inline...

> Cc: [email-masked]; 
> [email-masked]; [email-masked]; Tal Mizrahi 
> <[email-masked]>; [email-masked]
> Subject: Benjamin Kaduk's Discuss on 
> draft-ietf-ippm-multipoint-alt-mark-07: (with DISCUSS and COMMENT)
> 
> email addresses included in the To and CC lines. (Feel free to cut 
> this introductory paragraph, however.)
> 
> 
> Please refer to 
> https://www.ietf.org/iesg/statement/discuss-criteria.html

Ah, if the links are unidirectional then this works fine, of course.
I think I did consider the possibility of unidirectional links, but the way the figures use "<>" where a link terminates at a router made me think they were bidirectional.  The explanation added in the -08 looks good, and I've changed my ballot position to No Objection, but I do wonder if changing the figures slightly might help further.

> [I think I could imagine some exotic cases where this does not hold, 
> but none of them really seem topical for this work.]
> 

I didn't make that connection the first time I read section 7, though I think I can see it now.

> [GF]: It can be both the variation across separate paths or valid for the multipoint path. It is the same as for the delay.

Oh, right, I keep forgetting that "delay variation" is a term of art; sorry about that.

> points"; it would be good to provide the abbreviation/expansion 
> together and consistently use the abbreviation.  (Interestingly 

-Ben