Subject: [alto] ALTO Extension: Defining a Cost Metrics document?

From: "Y. Richard Yang" <[email-masked]>
Date: Sun, 13 Oct 2013 19:19:36 -0400

Dear all,

I am reading up on the documents that define cost metrics.

The motivation is that the base ALTO protocol (
http://www.ietf.org/id/draft-ietf-alto-protocol-20.txt) has defined only
one Cost Metric: 'routingcost':

- Defined the semantics at Sec. 6.1.1.1 of , and then listed it at Table 3.

- Used "hopcount" in examples of Sec. 9.2.3 and 9.2.4, but the semantics of
not formally defined.

Given the aforementioned state of the base protocol, I see good value in
that the WG produces a WG document that defines a relatively complete set
of Cost Metrics.

I particular, I read the following:

- http://tools.ietf.org/html/draft-lee-alto-app-net-info-exchange-02
  (Sec. 3.4 introduced three metrics: hopcount, latency, pktcost, and cost)

- http://tools.ietf.org/html/draft-wu-alto-json-te-01
  Defined a set of metrics: in Sec. 4. This work, as stated in the
document, is motivated by

- http://www.ietf.org/id/draft-ietf-ospf-te-metric-extensions-04.txt

During the review of ALTO base protocol, we are suggested to document
performance metrics (cost metrics) per the guideline of

- RFC 6390 Guidelines for Considering New Performance Metric Development.
A. Clark, B. Claise. October 2011. (Format: TXT=49930 bytes) (Also BCP0170)
(Status: BEST CURRENT PRACTICE)

Here a first question, I have, is whether the authors will produce a
"simple" document, at the upcoming IETF, whose only purpose is to:

  define a set of cost metrics, including the nameing, the semantics, ...
following the guideline per RFC 6390, that can benefit the base protocol.

I feel that such a document is focused, and has good value by itself.

The implications of the introducing multiple cost metrics can be explored
in another document, which I will send in another email shortly.

Thanks.

Richard

---

From: Qin Wu <[email-masked]>
Date: Mon, 14 Oct 2013 06:30:35 +0000

From: [email-masked] [mailto:[email-masked]] On Behalf Of Y. Richard Yang
Sent: Monday, October 14, 2013 7:20 AM
To: IETF ALTO
Cc: [email-masked]; Qin Wu
Subject: [alto] ALTO Extension: Defining a Cost Metrics document?

Dear all,

The motivation is that the base ALTO protocol (http://www.ietf.org/id/draft-ietf-alto-protocol-20.txt) has defined only one Cost Metric: 'routingcost':

- Used "hopcount" in examples of Sec. 9.2.3 and 9.2.4, but the semantics of not formally defined.

Given the aforementioned state of the base protocol, I see good value in that the WG produces a WG document that defines a relatively complete set of Cost Metrics.

- http://tools.ietf.org/html/draft-wu-alto-json-te-01
  Defined a set of metrics: in Sec. 4. This work, as stated in the document, is motivated by

During the review of ALTO base protocol, we are suggested to document performance metrics (cost metrics) per the guideline of
- RFC 6390 Guidelines for Considering New Performance Metric Development. A. Clark, B. Claise. October 2011. (Format: TXT=49930 bytes) (Also BCP0170) (Status: BEST CURRENT PRACTICE)

Here a first question, I have, is whether the authors will produce a "simple" document, at the upcoming IETF, whose only purpose is to:

  define a set of cost metrics, including the nameing, the semantics, ... following the guideline per RFC 6390, that can benefit the base protocol.

[Qin] This is exactly what I we are doing in draft-wu-alto-json-te. We are checking if we can give a complete list of cost metrics that are built based on

draft-ietf-idr-ls-distribution-03,RFC5305, draft-wu-idr-te-pm-bgp<http://tools.ietf.org/id/draft-wu-idr-te-pm-bgp-02.txt>,draft-ietf-ospf-te-metric-extensions-04, draft-ietf-isis-te-metric-extensions-01.

We will further generalize them to firstly have some base metrics that can applied either to the whole path or any link in the path and then have

Derived metrics that are link specific.

The update (v-02) will come in a few days.

The implications of the introducing multiple cost metrics can be explored in another document, which I will send in another email shortly.

Thanks.

Richard

---

From: "RANDRIAMASY, SABINE (SABINE)" <[email-masked]>
Date: Thu, 17 Oct 2013 18:17:02 +0000

Hello,

This is definitely a good idea and it will be interesting to merge the metric definitions of the current drafts. I started the exercise, using the RFC 6390 definition template and found that there are other definition attributes that highlight the attractiveness of the ALTO Service that may be used at several layers of the network, with several levels of control and by several parties.

There has been a number of metrics proposed besides 'routingcost' and 'hopcount'.  One aspect that should be documented is the usage of such metrics in different environments. For instance, some drafts assume a controlled environment where metric values are closer or equal to real values where as other drafts assume a less or no access control and propose abstracted metric values to reflect preferences w.r.t. these metrics.

So it could be useful to include attributes in the metric definition reflecting their degree (may be yes/no) of abstraction, or whether their values are explicitly provided or are just used in a hidden way to filter or tie-break other metric values.

Besides, most currently documented ALTO metrics relate to the transport topology where as the protocol architecture and initial extension discussions mentioned metrics relating to capabilities, of endpoints such as storage or CPU capacity. So I could be useful to document the potential providers of the different metrics and the potential users.

What is your opinion?

Thanks
Sabine

De : [email-masked] [mailto:[email-masked]] De la part de Y. Richard Yang
Envoyé : lundi 14 octobre 2013 01:20
À : IETF ALTO
Cc : [email-masked]; RANDRIAMASY, SABINE (SABINE); [email-masked]; Greg Bernstein; Qin Wu; Young Lee
Objet : ALTO Extension: Defining a Cost Metrics document?

Dear all,

Thanks.

Richard

---

From: "Y. Richard Yang" <[email-masked]>
Date: Thu, 17 Oct 2013 17:05:56 -0400

Dear Sabine, Qin,

On Thu, Oct 17, 2013 at 2:17 PM, RANDRIAMASY, SABINE (SABINE) <
[email-masked]> wrote:

>  Hello,****
>
> ** **
>
> This is definitely a good idea and it will be interesting to merge the
> metric definitions of the current drafts.
>

Thanks a lot for the work on systematically defining a set of metrics.
Please see below. I agree that it is a good idea to merge the definitions.

> I started the exercise, using the RFC 6390 definition template
>

This is a good starting point indeed. We should also define "routingcost"
in this metrics document.

> and found that there are other definition attributes that highlight the
> attractiveness of the ALTO Service that may be used at several layers of
> the network, with several levels of control and by several parties.
>

This multi-level/layer concept is quite interesting. Can you please
elaborate a bit?

> **
>
> There has been a number of metrics proposed besides ‘routingcost’ and
> ‘hopcount’.  One aspect that should be documented is the usage of such
> metrics in different environments. For instance, some drafts assume a
> controlled environment where metric values are closer or equal to real
> values where as other drafts assume a less or no access control and propose
> abstracted metric values to reflect preferences w.r.t. these metrics. ****
>
> ** **
>
> So it could be useful to include attributes in the metric definition
> reflecting their degree (may be yes/no) of abstraction,
>

Let me try to understand. The hopcount metric might say 5 hops, and then a
"hopcount-detail" will give the exact 5 hops; an AScount metric might say 5
ASes, and ASPath then lists the 5 AS's, and then routePath lists the
spwcifi routes, for example? Or this is too detailed?

> or whether their values are explicitly provided or are just used in a
> hidden way to filter or tie-break other metric values.
>

Filtering-only metrics can be tricky, but of course interesting.

> **
>
> Besides, most currently documented ALTO metrics relate to the transport
> topology where as the protocol architecture and initial extension
> discussions mentioned metrics relating to capabilities, of endpoints such
> as storage or CPU capacity. So I could be useful to document the potential
> providers of the different metrics and the potential users.
>

I am not sure we may want to combine transport and storage/CPU metrics in a
single documents. My first reaction is that separate documents may be a
more modular design, and I can think more.

Thanks!

Richard

>  ****
>
> ** **
>
> What is your opinion?****
>
> ** **
>
> Thanks****
>
> Sabine****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> *De :* [email-masked] [mailto:[email-masked]] *De la part de
> * Y. Richard Yang
> *Envoyé :* lundi 14 octobre 2013 01:20
> *À :* IETF ALTO
> *Cc :* [email-masked]; RANDRIAMASY, SABINE (SABINE);
> [email-masked]; Greg Bernstein; Qin Wu; Young Lee
> *Objet :* ALTO Extension: Defining a Cost Metrics document?****
>
> ** **
>
> Dear all,****
>
> ** **
>
> I am reading up on the documents that define cost metrics. ****
>
> ** **
>
> one Cost Metric: 'routingcost':****
>
> ** **
>
> ****
> ** **
> - Used "hopcount" in examples of Sec. 9.2.3 and 9.2.4, but the semantics
> of not formally defined.****
>
> ** **
>
> of Cost Metrics.****
> ** **
> I particular, I read the following:****
>
> ** **
>
> - http://tools.ietf.org/html/draft-lee-alto-app-net-info-exchange-02****
>
> ****
> ** **
> - http://tools.ietf.org/html/draft-wu-alto-json-te-01****
>
> document, is motivated by ****
>
> ** **
>
> - http://www.ietf.org/id/draft-ietf-ospf-te-metric-extensions-04.txt****
>
> ** **
>
> performance metrics (cost metrics) per the guideline of ****
>
> (Status: BEST CURRENT PRACTICE)****
>
> ** **
>
> "simple" document, at the upcoming IETF, whose only purpose is to:****
>
> ** **
>
> following the guideline per RFC 6390, that can benefit the base protocol.*
> ***
>
> ** **
>
> I feel that such a document is focused, and has good value by itself.****
>
> ** **
>
> in another document, which I will send in another email shortly.****
>
> ** **
>
> Thanks.****
>
> ** **
>
> Richard****
>
>   ****
>

---

From: "RANDRIAMASY, SABINE (SABINE)" <[email-masked]>
Date: Fri, 18 Oct 2013 18:52:41 +0000

Hi all,

De : [email-masked] [mailto:[email-masked]] De la part de Y. Richard Yang
Envoyé : jeudi 17 octobre 2013 23:06
À : RANDRIAMASY, SABINE (SABINE)
Cc : IETF ALTO; [email-masked]; [email-masked]; Greg Bernstein; Qin Wu; Young Lee
Objet : Re: ALTO Extension: Defining a Cost Metrics document?

Dear Sabine, Qin,

On Thu, Oct 17, 2013 at 2:17 PM, RANDRIAMASY, SABINE (SABINE) <[email-masked]<mailto:[email-masked]>> wrote:
Hello,

This is definitely a good idea and it will be interesting to merge the metric definitions of the current drafts.

Thanks a lot for the work on systematically defining a set of metrics. Please see below. I agree that it is a good idea to merge the definitions.

This is a good starting point indeed. We should also define "routingcost" in this metrics document.

and found that there are other definition attributes that highlight the attractiveness of the ALTO Service that may be used at several layers of the network, with several levels of control and by several parties.

This multi-level/layer concept is quite interesting. Can you please elaborate a bit?
[     ] I was referring  to the large bandwidth use case where  the ALTO Service exposes a topology and paths abstractions with a granularity below the e2e level.

So it could be useful to include attributes in the metric definition reflecting their degree (may be yes/no) of abstraction,

Let me try to understand. The hopcount metric might say 5 hops, and then a "hopcount-detail" will give the exact 5 hops; an AScount metric might say 5 ASes, and ASPath then lists the 5 AS's, and then routePath lists the spwcifi routes, for example? Or this is too detailed?
[     ] Yes, That is a possibility and up to the server to define the level of detail in terms of units that may be for instance IP hops, AS hops, some other unit or no unit that is the value of hopcount reflects the preference of the Server operator, with the optimum being MIN and the composition operator is SUM.
In such a case, should'nt we introduce different metric names, such as "AShopcount", "IPhopcount", "hopcount" ? What is the opinion in the ALTO WG?

I was also thinking on the "available path bandwidth" example:  In a controlled environment an ALTO Server may be ready to expose  the real value or a statistic, possibly say over which time period, so the metric values would look like N Gbs. In that case, the metric description may indicate that the ALTO value reflects or estimates reality.  In a less controlled environment or for whatever reason, the ALTO Server just wants to express a preference w.r.t.  available bandwidth and express this as a "bandwidth score", for example between 0 and N=20 or 100, indicating that the higher the value, the more available bw there is. A score will keep the real value confidential but reflects at least proportionality. In that case, the description should indicate that the value is an abstraction. Note that in both cases the description would indicate that the optimum is MAX and the composition operator is MIN.

or whether their values are explicitly provided or are just used in a hidden way to filter or tie-break other metric values.

Filtering-only metrics can be tricky, but of course interesting.
[     ] Agree. We may leave this for the moment.

I am not sure we may want to combine transport and storage/CPU metrics in a single documents. My first reaction is that separate documents may be a more modular design, and I can think more.
[     ] These metrics refer to the CDN and Virtualized applications in Data Centers use cases mentioned in draft tools.ietf.org/html/draft-marocco-alto-next-00#page-5 submitted for IETF83 in Paris to prepare the i2aex meeting. Such metrics can be supported using the base protocol and the Basic ALTO  architecture figure 1 mentions third parties and content providers, besides routing protocols etc. as possible entities to provision ALTO information.
But  I agree, we may first focus on metrics reporting on e2e network paths and may start defining them in a separate document.

Thanks!
Richard

What is your opinion?

Thanks
Sabine

De : [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] De la part de Y. Richard Yang
Envoyé : lundi 14 octobre 2013 01:20
À : IETF ALTO
Cc : [email-masked]<mailto:[email-masked]>; RANDRIAMASY, SABINE (SABINE); [email-masked]<mailto:[email-masked]>; Greg Bernstein; Qin Wu; Young Lee
Objet : ALTO Extension: Defining a Cost Metrics document?

Dear all,

Thanks.

Richard

---

From: "Y. Richard Yang" <[email-masked]>
Date: Sun, 20 Oct 2013 21:25:27 -0400

Hi all,

Just a thought, some networks already provide public performance metrics,
and ALTO should be able to provide such info in a standard way and cover
these metrics, if we agree:

-AT&T
 - http://ipnetwork.bgtmo.ip.att.net/pws/averages.html
 - http://ipnetwork.bgtmo.ip.att.net/pws/network_delay.html
   shows latency, loss, jitter, reliability, modem success rate

 - In particular, the link provides a methodology page (
http://ipnetwork.bgtmo.ip.att.net/pws/glossary.html), which points to a
major challenge in defining the metrics: metrics have parameters (e.g., the
AT&T link specifies 15-min interval for latency), and I assume that ALTO
cannot work with a single interval, but then how do we handle parameters?

- CenturyLink (formerly Qwest):
  - https://kai04.centurylink.com/PtapRpts/Public/BackboneReport.aspx
    shows jitter, latency, pkt delivery rate, availability

...

And we could think that ALTO could be extended to be used as a standard way
to check on the service outage of an endpoint (
http://customer.comcast.com/help-and-support/cable-tv/outages-in-your-area/),
which may imply performance metrics as well...

Thanks.

Richard

On Mon, Oct 14, 2013 at 2:30 AM, Qin Wu <[email-masked]> wrote:

>   *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Y. Richard Yang
> *Sent:* Monday, October 14, 2013 7:20 AM
> *To:* IETF ALTO
> *Cc:* [email-masked]; Qin Wu
> *Subject:* [alto] ALTO Extension: Defining a Cost Metrics document?****
>
> ** **
>
> Dear all,****
>
> ** **
>
> [Qin] This is exactly what I we are doing in draft-wu-alto-json-te. We
> are checking if we can give a complete list of cost metrics that are built
> based on****
>
> draft-ietf-idr-ls-distribution-03,RFC5305, draft-wu-idr-te-pm-bgp <http://tools.ietf.org/id/draft-wu-idr-te-pm-bgp-02.txt>,draft-ietf-ospf-te-metric-extensions-04, draft-ietf-isis-te-metric-extensions-01.****
>
> We will further generalize them to firstly have some base metrics that can applied either to the whole path or any link in the path and then have****
>
> Derived metrics that are link specific.****
>
> ** **
>
> The update (v-02) will come in a few days. ****
>
> ** **
>
> ** **
>

---

From: Qin Wu <[email-masked]>
Date: Sat, 26 Oct 2013 03:13:52 +0000

Hi,Richard:
Thank for bring these examples.
I see these examples from two perspectives:

1.       What cost metrics we should define?

2.       When, where and how to calculate these metrics (i.e., methodology ).
I think we can not define these cost metrics from scratch, we can not define all the metrics we think useful. Instead, we should define them mostly based on standardized work,
AT&T have a good practice for measuring these metrics. But I believe AT&T also doing a good job in standardizing these metrics and associated methodology,e.g.,
in IETF IPPM WG.
Therefore I think we should try to reuse the existing methodology associated metrics defined somewhere.

Also I believe we are not intending to design ALTO as the whole measurement system or measurement platform.
What ALTO server should do is to gather/aggregate/abstract/filter useful metrics in a standard way and  provide them
to the alto client or use them as filtering to choose appropriate endpoint to connect.

Regards!
-Qin
From: [email-masked] [mailto:[email-masked]] On Behalf Of Y. Richard Yang
Sent: Monday, October 21, 2013 9:25 AM
To: Qin Wu
Cc: IETF ALTO; [email-masked]
Subject: Re: [alto] ALTO Extension: Defining a Cost Metrics document?

Hi all,

Just a thought, some networks already provide public performance metrics, and ALTO should be able to provide such info in a standard way and cover these metrics, if we agree:

-AT&T

 - In particular, the link provides a methodology page (http://ipnetwork.bgtmo.ip.att.net/pws/glossary.html), which points to a major challenge in defining the metrics: metrics have parameters (e.g., the AT&T link specifies 15-min interval for latency), and I assume that ALTO cannot work with a single interval, but then how do we handle parameters?

...

And we could think that ALTO could be extended to be used as a standard way to check on the service outage of an endpoint (http://customer.comcast.com/help-and-support/cable-tv/outages-in-your-area/), which may imply performance metrics as well...

Thanks.

Richard

On Mon, Oct 14, 2013 at 2:30 AM, Qin Wu <[email-masked]<mailto:[email-masked]>> wrote:
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Y. Richard Yang
Sent: Monday, October 14, 2013 7:20 AM
To: IETF ALTO
Cc: [email-masked]<mailto:[email-masked]>; Qin Wu
Subject: [alto] ALTO Extension: Defining a Cost Metrics document?

Dear all,

Thanks.

Richard

---

From: "Y. Richard Yang" <[email-masked]>
Date: Sun, 27 Oct 2013 16:46:30 -0400

Hi Qin,

On Oct 25, 2013 11:14 PM, "Qin Wu" <[email-masked]> wrote:
> I think we can not define these cost metrics from scratch, we can not
define all the metrics we think useful. Instead, we should define them
mostly based on standardized work,
>
> AT&T have a good practice for measuring these metrics. But I believe AT&T
also doing a good job in standardizing these metrics and associated
methodology,e.g.,
> Therefore I think we should try to reuse the existing methodology
associated metrics defined somewhere.

It may not be necessary to limit ALTO to metrics only defined by other WGs.
In other words, limiting ALTO to the representation layer. But as a first
step, this can be fine. I am not very familiar with IPPM. Here is a
question. How does IPPM handle parameterized metrics, e.g., 15-min vs 5-min
measurement interval of latency? I see two problems:

- How does a network expose the parameters available (e.g., a network says
I can support 5 or 15 min intervals)? ALTO IRD allows exposures of such
capabilities.

-How does a client specify the parameter chosen or desired?

Does IPPM have the mechanisms that ALTO can use?
>
>
>
> Also I believe we are not intending to design ALTO as the whole
measurement system or measurement platform.
>
> What ALTO server should do is to gather/aggregate/abstract/filter useful
metrics in a standard way and  provide them
> to the alto client or use them as filtering to choose appropriate
endpoint to connect.

Agree. The measurement infrastructure might collect data at 1 min interval,
and ALTO exposes only hourly data to certain clients? It is policy
controlled.

Richard
> And we could think that ALTO could be extended to be used as a standard
way to check on the service outage of an endpoint (
http://customer.comcast.com/help-and-support/cable-tv/outages-in-your-area/),
which may imply performance metrics as well...
> - Used "hopcount" in examples of Sec. 9.2.3 and 9.2.4, but the semantics
of not formally defined.
based on
>
> draft-ietf-idr-ls-distribution-03,RFC5305,
draft-wu-idr-te-pm-bgp,draft-ietf-ospf-te-metric-extensions-04,
draft-ietf-isis-te-metric-extensions-01.
>
> We will further generalize them to firstly have some base metrics that
can applied either to the whole path or any link in the path and then have

---

From: Qin Wu <[email-masked]>
Date: Tue, 29 Oct 2013 09:47:39 +0000

Hi, Richard:

From: [email-masked] [mailto:[email-masked]] On Behalf Of Y. Richard Yang
Sent: Monday, October 28, 2013 4:47 AM
To: Qin Wu
Cc: [email-masked]; IETF ALTO
Subject: RE: [alto] ALTO Extension: Defining a Cost Metrics document?

Hi Qin,

> Therefore I think we should try to reuse the existing methodology associated metrics defined somewhere.

It may not be necessary to limit ALTO to metrics only defined by other WGs. In other words, limiting ALTO to the representation layer. But as a first step, this can be fine. I am not very familiar with IPPM. Here is a question. How does IPPM handle parameterized metrics, e.g., 15-min vs 5-min measurement interval of latency? I see two problems:

- How does a network expose the parameters available (e.g., a network says I can support 5 or 15 min intervals)? ALTO IRD allows exposures of such capabilities.

[Qin]: IPPM defines methodology for each metric. In the methodology, it include parameters like source address, dst address, time to measure this metric.

However IPPM don't give default value for interval. IPPM didn't define how to report those metrics.  Routing protocol can be extended to report those metrics. However Routing protocol didn't report measurement interval directly in the routing message. Instead, such value of interval can be pre-configured.

[Qin]: Is a client referred to "ALTO server" that gathers metrics from Routing protocol, or ALTO client that request information from ALTO server?

If it is the former, I think ALTO server has its own local policy to decide how to gather those metrics and in which frequency to gather those metrics.

If it is the latter, I think ALTO client only care about which endpoint he can connect and which path he can traverse. He does not need to care about

Whether this metric is latest measured or measured one hour ago. He can just assume the metric he get is the latest update.

[Qin]: IPPM does define Delay Metric, Packet loss metric, Jitter Metric, bandwidth metrics, see RFC5136, RFC2681, RFC2679,RFC3393,RFC2680.

We can consider to use them as a reference when we follow RFC6390 performance template.

> to the alto client or use them as filtering to choose appropriate endpoint to connect.

Agree. The measurement infrastructure might collect data at 1 min interval, and ALTO exposes only hourly data to certain clients? It is policy controlled.

Richard
>
>
>
> Regards!
>
> -Qin
>
> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Y. Richard Yang
> Cc: IETF ALTO; [email-masked]<mailto:[email-masked]>
> draft-ietf-idr-ls-distribution-03,RFC5305, draft-wu-idr-te-pm-bgp,draft-ietf-ospf-te-metric-extensions-04, draft-ietf-isis-te-metric-extensions-01.
>

---

From: "Y. Richard Yang" <[email-masked]>
Date: Tue, 29 Oct 2013 18:08:25 -0400

> >
> > AT&T have a good practice for measuring these metrics. But I believe
AT&T also doing a good job in standardizing these metrics and associated
methodology,e.g.,
> It may not be necessary to limit ALTO to metrics only defined by other
WGs. In other words, limiting ALTO to the representation layer. But as a
first step, this can be fine. I am not very familiar with IPPM. Here is a
question. How does IPPM handle parameterized metrics, e.g., 15-min vs 5-min
measurement interval of latency? I see two problems:
>
> - How does a network expose the parameters available (e.g., a network
says I can support 5 or 15 min intervals)? ALTO IRD allows exposures of
such capabilities.
>
> [Qin]: IPPM defines methodology for each metric. In the methodology, it
include parameters like source address, dst address, time to measure this
metric.
>
> However IPPM don’t give default value for interval. IPPM didn’t define
how to report those metrics.  Routing protocol can be extended to report
those metrics. However Routing protocol didn’t report measurement interval
directly in the routing message. Instead, such value of interval can be
pre-configured.

So we have a problem here? Hence, we need to include more fields in
CostType: CostMetric, CostMode, CostParams? Then IRD needs to expose the
parameters?

> [Qin]: Is a client referred to “ALTO server” that gathers metrics from
Routing protocol, or ALTO client that request information from ALTO server?
>
> If it is the former, I think ALTO server has its own local policy to
decide how to gather those metrics and in which frequency to gather those
metrics.

I meant the client. But thank you for pointing out that the ALTO Server can
be the client of the Routing system. So we go back to manual configuration?

> If it is the latter, I think ALTO client only care about which endpoint
he can connect and which path he can traverse. He does not need to care
about
>
> Whether this metric is latest measured or measured one hour ago. He can
just assume the metric he get is the latest update.

Not sure about this. The latest assumption assumes real-time operation.
Also, the measurement interval can have large impact on interpreting the
results. For example, jitter is essentially variance and can go through
substantial smoothing (law of large numbers) if the interval is long.
Hence, the client can benefit knowing it.
> [Qin]: IPPM does define Delay Metric, Packet loss metric, Jitter Metric,
bandwidth metrics, see RFC5136, RFC2681, RFC2679,RFC3393,RFC2680.
> We can consider to use them as a reference when we follow RFC6390
performance template.

Good to know.

Thanks!

Richard
> >
> > What ALTO server should do is to gather/aggregate/abstract/filter
useful metrics in a standard way and  provide them
> Agree. The measurement infrastructure might collect data at 1 min
interval, and ALTO exposes only hourly data to certain clients? It is
policy controlled.
> > Just a thought, some networks already provide public performance
metrics, and ALTO should be able to provide such info in a standard way and
http://www.ietf.org/id/draft-ietf-alto-protocol-20.txt) has defined only
one Cost Metric: 'routingcost':
> > Given the aforementioned state of the base protocol, I see good value
in that the WG produces a WG document that defines a relatively complete
set of Cost Metrics.
> >
> > - RFC 6390 Guidelines for Considering New Performance Metric
Development. A. Clark, B. Claise. October 2011. (Format: TXT=49930 bytes)
(Also BCP0170) (Status: BEST CURRENT PRACTICE)
"simple" document, at the upcoming IETF, whose only purpose is to:
> >
> >
> >
> >   define a set of cost metrics, including the nameing, the semantics,
... following the guideline per RFC 6390, that can benefit the base
protocol.
> > The implications of the introducing multiple cost metrics can be
explored in another document, which I will send in another email shortly.