Subject: Re: [Gen-art] Gen-art LC Review of draft-ietf-alto-performance-metrics-17

From: "Y. Richard Yang" <[email-masked]>
Date: Thu, 21 Oct 2021 17:00:08 -0400

Hi Elwyn,

Very sorry to miss this set of reviews. Please see below.

On Sun, Sep 12, 2021 at 10:19 AM elwynd <[email-masked]> wrote:

> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair.  Please treat these comments just
> like any other last call comments.
>
> For more information, please see the FAQ at
>
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>
> Document: draft-ietf-alto-performance-metrics-17.txt
> Reviewer: Elwyn Davies
> Review Date: 2021/09/09
> IETF LC End Date: 2021/08/02
> IESG Telechat date: (if known) -
>
> Summary: Not ready.  My major concern with the document is the lack of
> precision in various aspects that would be needed to ensure an automated
> system could interpret the requests and responses that are  added to the
> basic ALTO protocol by this document.
>
> Major issues:
> The various examples of 'link' parameters:  It is unclear whether these
> links would be useful in an automated ALTO system.  Would it be expected
> that an ALTO server would read the contents of the link pointed
> to by the URL?  If so what structure would be expected?  This is
> particularly relevant in the 'estimation' cases where without a machine
> interpretable or set of standard mchanisms, the estimation option seems of
> minimal  use.  Do the authors anticipate that estimation methodologies
> might be standardized in the foreseeable future?   Similarly,  machine
> interpretable versions of SLA specifications are not something that sre
> conventionally available.
>
>
This is a core design issue, and thanks for discussing it! A short answer
is that in this design, the parameter/link is an opaque identifier to the
methodology. Specific deployment settings may provide machine-readable
content through the parameter/link field, but it is not standardized (yet)
in this document. One way of thinking about the parameter/link is that it
is a mechanism for an ALTO server to provide (BGP-inspired) "community
attributes" to a specific ALTO client and still be grammar conforming.

For a longer answer, in the early phase, there were substantial efforts in
providing the details of the measurement methodology when defining a
metric. The WG invited participation from IPPM and had an extensive
discussion. The decision was that the basic function of ALTO is to provide
estimation and guidance; the ALTO information will be one of multiple
sources of information to applications; there can be multiple types of
estimation methods for the same metric; some estimation methodology can be
very complex to fully specify. Hence, the way to move forward is to (1)
provide high level information about the type of processes (i.e., the
source, estimation, sla) that the metric is computed; and (2) give a
generic URI based approach so that the server has a protocol mechanism to
convey additional information to the client. By defining parameter as a
generic JSON object and link as an URI, the information pointed to by the
link has the capability of self-typing, and eventually (in suitable
deployment settings) provides structured information that can be
interpretable by machines. As an example of ongoing work, the WG is looking
at both learning based estimation of tput [1] and g2 based [2]. Both can
provide an estimation of tput, and as the field becomes mature, the WG can
come back to specify a format.  Make sense?

[1] "Prophet: Toward Fast, Error-Tolerant Model-based Throughput Prediction
for Reactive Flows in DC Networks" (TNET-2018-00567.R2). Appeared in
INFOCOM 2018; Extended version to appear In ACM/IEEE Transactions in
Networking 2020.
[2] https://www.reservoir.com/publication/g2-network-optimization/

> Minor issues:
> s2.1, defininition of CostContext:  Given the name, I would expect that
> there could be more then one parameter specified.  For convenience and to
> make the information more machne readable, I would have expected the
> parameters to be passed over in a JSON object rather than an unspecified
> JSONvalue. [I observe that RFC 7285 does not define JSONobject.]   This
> particularly applies to the 'link' parameter case where the name and value
> need to be encoded.
>

This is related with the previous discussion.

> s7, ALTO Cost Source Registry:  The specification for this new registry
> is incomplete.  The review mechanism for new assgnments plus the
> definitions of the two fields are needed.   It may also be worth
> considering whether this field really nedes a registry.  Can the authors
> think of any other possibilities that might arise?
>
>
Good comment! This is an issue that AD caught as well. We have an updated
text in -18. Could you please take a look at it? It starts from "This
document requests the creation of the "ALTO Cost Source Registry ... and to
the end of Sec. 7".

> Nits/editorial comments:
>
> General:  An RFC is not an academic paper and the form 'We xxxx' is not
> used.  A depersonalised form such as 'In this document...' needs to b used
> instead.  There are three instances that need fixing
>  (s2, para 2;  s5, para 2; and s8. )
>
> Agree. We are taking a pass and making sure to use depersonalised forms.

> Abstract:  I suggest here a number of minor wording chages to improve the
> abstract:
> OLD:
>
>    Cost metric is a basic concept in Application-Layer Traffic
>    Optimization (ALTO), and different applications may use different
>    cost metrics.  Since the ALTO base protocol (RFC 7285) defines only a
>    single cost metric (i.e., the generic "routingcost" metric), if an
>    application wants to issue a cost map or an endpoint cost request to
>    determine the resource provider that offers better delay performance,
>    the base protocol does not define the cost metric to be used.
>
>    This document addresses the issue by introducing network performance
>    metrics, including network delay, jitter, packet loss rate, hop
>    count, and bandwidth.
>
>    There are multiple sources (e.g., estimation based on measurements or
>    service-level agreement) to derive a performance metric.  This
>    document introduces an additional "cost-context" field to the ALTO
>    "cost-type" field to convey the source of a performance metric.
>
> NEW:
>
>    The cost metric is a basic concept in Application-Layer Traffic
>    Optimization (ALTO), and different applications may use different types of
>    cost metric.  Since the ALTO base protocol (RFC 7285) defines only a
>    single cost metric (namely, the generic "routingcost" metric), if an
>    application wants to issue a cost map or an endpoint cost request in order to
>    identify a resource provider that offers a better delay performance,
>    the base protocol does not define the cost metric to be used.
>
>    This document addresses this issue by extending the specification to provide a variety  network performance
>
>    metrics, including network delay, jitter, packet loss rate, hop
>    count, and bandwidth.
>
>    There are multiple sources (e.g., estimation based on measurements or
>    service-level agreement) to derive a performance metric.  This
>    document introduces an additional "cost-context" field to the ALTO
>    "cost-type" field to convey the source of a performance metric.
>
> END
>
>
Looks good. Will adopt.

> s1, para 1: s/Cost metric/The cost metric/
>
>
Fixed.

> s1, para 5 (first on page 5): s/related with bandwith/related to bandwidth/
>
>
Fixed.

> s1, para 6: A refererence to RFC 7285 Section 9.2 should be given when the
> IRD is introduced.
>
>
Added.

> s1:  Some pieces of terminology are carried over from RFC 7285, notably
> JSONxxxx and PID.  These, together with the various media types defined in
> RFC 7285 and used in examples, should be documented in s1.
>
>
Added references.

> s2.1, next to last para (above Figure 1): s/A potential architecture on
> estimating these metrics/A outline of potential information flows used
> for estimating these metrics/
>
>
Changed to "An illustration of potential information flows used for
estimating these metrics ..."

> Figure 1 title: s/A framework to compute estimation to performance
> metrics/A framework for computing estimations of performance metrics/
>
>
Changed.

> s3.1.3, para 1 and s3.3.3, para 1: A reference to RFC 7285 Section 5.1
> should be given when introducing PIDs.
>
>
Fix.

> s4.3.4, last para: s/estimtation/estimation/
>
>
Fixed.

Thank you so much!

Richard

> Sent from my Galaxy