Subject: [Last-Call] Opsdir last call review of draft-ietf-alto-cost-calendar-16

From: Jürgen Schönwälder via Datatracker <[email-masked]>
Date: Tue, 11 Feb 2020 01:49:31 -0800

Reviewer: Jürgen Schönwälder
Review result: Has Issues

I am not an ALTO expert so keep this in mind in case some of my
comments have little meaning for ALTO experts.

- If multiple clients retrieve information about costs in the future
  and take independent scheduling decisions based on that information,
  it might turn out that a "cheap" time turns into "expensive" time,
  i.e., the result may not be an improvement of QoE but rather the
  opposite. How is this dealt with? I see this issue touched on in
  some places and the suggestion seems to be the advice to implement
  SSE. But should ALTO servers make promises long into the future if
  clients do not implement SSE?

  I am concerned that announcements of "cheap" times can easily turn
  into instabilities if many clients start to opt for the announced
  "cheap" times and I think this deserves to be discussed explicitly
  in the Operational Considerations section.

- p6: "time zone (in UTC)" ?? do you mean "time zone offset relative
  to UTC"? After reading, I think this is misleading. Perhaps you
  wanted to say "start time (including time zone)"

- p7: broken sentence

   The extensions in this document and encode requests and responses
   using JSON [RFC8259].

- p7: editorial nit

  OLD

  this document extends: the IRD, the ALTO

  NEW

  this document extends the IRD and the ALTO

- I am a bit concerned about changing the type of json fields, i.e.,
  from a number to an array of numbers. The ALTO specifications may
  allow this but is there a certain risk of implementations not being
  smart in handling this correctly? Were alternatives considered that
  do not change the type of existing json fields? Given the explicit
  text why this is OK from an ALTO perspective, it might be that the
  WG did iterate on this (sorry if I raise a closed issue again).

- p9: can a time-interval-size be negative? can it be zero?

- p9: what does 'at least equal to 1' mean? Do you mean 'a positive
  integer number of values'?

- p14: I am confused about the time zone aspect. The time zone pops up
  in section 3.2 but it is a bit unclear what is meant there (see
  above). On page 14, there is a statement about a 'reference time
  zone' (what is this?)  and then there is text about HTTP header
  fields, and I am lost how that format relates to the rest of the
  document. Was the idea to say that calendar-start-time uses the HTTP
  header time format? Then say this where calendar-start-time is
  defined. And why do you use the HTTP format and not RFC 3339 format?
  Perhaps this data format for date and time is popular in ALTO and
  hence it makes sense, I guess I do not know enough about it. Anyway,
  the time zone seems to be part of the calendar-start-time - if so
  this was not clear while reading top-down. And if there is free
  choice for the data and time format, I would find RFC 3339 probably
  a good robust alternative.

- p17: Why is this SHOULD needed? Because you want to ensure that
  there is always a value that can be used 'right now'? What about
  values that are stale, i.e., calendar-start-time is way in the past?

      [...] The value provided for the "calendar-
      start-time" attribute SHOULD NOT be later than the request date.

- p17: I am confused by the description of "repeated". Is the intent
  to say that the calendar value array simply repeats N times? Am I
  right in assuming that if "repeated" is not present, it defaults to
  1?  Or does "repeated" always have to be present? (Similarly for
  number-of-intervals, can it be absent and then it defaults to 1?

- In which sense is draft-ietf-alto-incr-update-sse normative given
  that the current text says 'can be used' but does not mandate its
  use? In which sense is RFC 8259 normative? Is it required to make
  ALTO clients and servers switch to RFC 8259? The Operational
  Considerations section says "RECOMMENDED" to switch. But how does an
  implementation work that does both RFC7285 (w/o UTF-8) and RFC8259
  (with UTF-8)? I think a clearer message that ALTO clients and
  servers implementing this extension must comply to RFC 8259 is
  desirable.

- Since you write "SHOULD at least IEEE 754 double-precision floating
  point", does this not	make the IEEE 754 reference a normative
  reference? Similarly, if you import the data format from HTTP/1.1,
  does this not make the HTTP/1.1 RFC a normative reference?

---

From: "Y. Richard Yang" <[email-masked]>
Date: Tue, 18 Feb 2020 18:39:04 -0500

Dear Jürgen,

Thank you so much for the careful reviews. The lead author of the document
is Sabine and the response below is formulated by her and then discussed
among the authors. Since she is off this week, so let me send the response
on her behalf to make progress.

Her response for an item will start with ---- [[SR]] and end with ----

Please see below.

>   in the Operational Considerations section.
Good comment! We will do some reorganization to more explicitly,
systematically discuss this issue. In particular, we plan to make explicit
the following considerations in the Operations Considerations section
(Section 8):

PROPOSE TEXT:
It is important that both the operator of the network and the operator of
the applications consider both the feedback aspect and the prediction-based
(uncertainty) aspect of using the cost calendar. Consider the cost calendar
as a traffic-aware map service (e.g., Google map). Using the service
without considering its own effect, a large fleet can turn a not-congested
road into a congested one; a large number of individual cars each choosing
a road with light traffic ("cheap link") can also result in congestion or
result in a less optimal global outcome (e.g., the Braess's Paradox
[citation]); an accident may happen and hence cause unexpected delays.

Although individual network operators and application operators can choose
their own approaches to address the aforementioned issues, this document
recommends the following considerations. First, a typical approach to
reducing instability and handling uncertainty is to ensure timely update of
information. Hence, the "ALTO Incremental Updates Using Server-Sent Events
(SSE)" Service [draft-ietf-alto-incr-update-sse] SHOULD be used to update
the calendar faster if supported by both the server and the client."
Second, Although there are theoretical analysis [RoughgardenThesis] and
Internet based evaluations [SIGCOMM2013SelfishRouting] showing that
uncoordinated behaviors may not result in substantial sub-optimal
results, when a network operator updates the cost calendar, and when an
application reacts to the update, they should still consider the potential
feedback effects.

> ----- [[SR]]
As the time zone in the provided start time is always UTC+0, would the
following update be OK?:

- remove text item "time zone (in UTC)"
- update the "calendar-start-date" description in the next paragraph as
follows:

OLD
"calendar-start-date": specifying when the Calendar starts, that is to
which date the first value of the Cost Calendar is applicable.

NEW
"calendar-start-date": specifying when the Calendar starts, that is to
which date the first value of the Cost Calendar is
applicable. The reference time zone for the provided time values is UTC.
-----

> ----- [[SR]]
yes, thanks: "and" should be removed. New text will be:
"The extensions in this document encode requests and responses using JSON
[RFC8259]."
-----

>   this document extends the IRD and the ALTO

----- [[SR]] thanks, and we will update.

>   WG did iterate on this (sorry if I raise a closed issue again).
----- [[SR]]
Indeed the ALTO WG did iterate on this. In the base protocol RFC7285
section 11.2.3.6, an ALTO cost is specified as a generic JSONValue, to
allow extensions. Implementations of 7285 SHOULD assume it is a JSONNumber,
and the implementor of this extension should consider the issue. We will
add a sentence to alert the implementor.
-----

>  ---- [[SR]]
it can neither be negative nor equal to zero. Would the following addition
be OK?
OLD
A "time-interval-size" value contains a JSONNumber.
NEW
A "time-interval-size" value contains a positive integer JSONNumber (i..e,
greater than zero integer).

>   integer number of values'?
----- [[SR]] yes, Would the following update be OK?
OLD
the integer number of values of the Cost Calendar array, at least equal to
1.
NEW
the positive integer number of values of the Cost Calendar array, at least
equal to 1.

>   a good robust alternative.
----- [[SR]]
Section 3.2 is part of an overview and was meant to define "what" the items
describe rather than how they are encoded. So the format specification as
an HTTP header field format was left to normative section 5 in page 14.
How about we add a sentence in section 3.2, in addition to the update
proposed above, as follows:
OLD
"calendar-start-date": specifying when the Calendar starts, that is to
which date the first value of the Cost Calendar is applicable.
NEW
"calendar-start-date": specifying when the Calendar starts, that is to
which date the first value of the Cost Calendar is applicable. The
reference time zone for the provided time values is UTC.

This document specifies the use of the HTTP header time field format
specified in [RFC7231].

Indeed, this data format for date and time is popular in ALTO.
-----

>       start-time" attribute SHOULD NOT be later than the request date.
----- [[SR]] Yes, this is the idea. If the IRD indicates it provides a
Calendar for a given cost type of a given resource, it should provide a
value that the Client can use when it requests one. The Client can figure
out when to request the next Calendar, if needed, by multiplying the
Calendar attributes "time-interval-size" and "number-of-intervals".

We plan to add this comment on page 18, before the comments on attribute
"repeated". How does this sound?

-----

>   1?

----- [[SR]]
yes, if member "repeated" is not present, it means that the calendar value
array is only valid for 1 time.
Would the re-phrasing below for paragraph 1 be clearer?

OLD
[...] The number N includes the provided iteration

NEW
[...] The number N includes the iteration provided in the returned response.
----

> Or does "repeated" always have to be present? (Similarly for

----- [[SR]] the member "number-of-intervals" is not optional and is
defined in section 4.1 as an in integer at least equal to 1
-----

>   use?

---- [[SR]]
this draft is cited in section 3 that is actually non-normative, in which
it was thus decided to avoid normative terms.
How about we start section 3 with the following text, like in RFC8189 which
is also an ALTO extension?
  "The following is a non-normative overview of the multi-cost ALTO
extensions defined in this document. It assumes the reader is familiar with
the ALTO protocol [RFC7285]."
-----

> In which sense is RFC 8259 normative? Is it required to make
----- [[SR]]
RFC7285 reference RFC7159 and encodes JSON text in UTF-8, UTF-16,UTF-32,
UTF-8 is default encoding format according to section 8.1 of RFC7159.
Since RFC7159 has been replaced by RFC8259 after the publication of
RFC7285, and since RFC7285 sets mandatory encoding to UTF-8, we will
change RECOMMENDED to MUST, as you suggested. How does this sound?

-----

>   reference?

----- [[SR]] The text follows RFC7285, section 11.3.2.3,  in which IEEE 754
is an informative reference and cited with a SHOULD.
----

> Similarly, if you import the data format from HTTP/1.1,
----- [[SR]]
Our plan is to make HTTP header fields format [RFC7231] a normative
reference, but a question for you: since RFC7285 lists RFC7231 as an
informative reference, if we were to follow RFC7285, 7231 would be
informative as well, but we agree that listing 7231 as a normative
reference makes more sense. OK?

-----

Thank you so much!

Sabine, Qin, Richard

---

From: Schönwälder, Jürgen <[email-masked]>
Date: Wed, 19 Feb 2020 09:37:48 +0000

Dear authors,

I will comment inline...

> application reacts to the update, they should still consider the potential

Thanks. I was looking for something like this. 
 
> applicable. The reference time zone for the provided time values is UTC.

Is this not 'calendar-start-time' later on? Or are calendar-start-date
and calendar-start-time two different things?

If times are always in UTC, then why not say: 

      The calendar-start-(date|time) is always expressed in UTC.

This way the reader does not have to think about what a reference time
zone might be.
 
> "The extensions in this document encode requests and responses using JSON

OK 
 
> ----- [[SR]] thanks, and we will update.

OK 
 
> add a sentence to alert the implementor.

OK
 
> greater than zero integer).

This is a change from double precision floating point number to an
integer.  Was that the intention of this change?
 
> the positive integer number of values of the Cost Calendar array, at least

OK 
 
> Indeed, this data format for date and time is popular in ALTO.
 
I am fine if section 3.2 is silent about the timezone as long as the
definition in section 5.1.2 is clear about the timezone. (And under
the assumption that calendar-start-time is the same as
calendar-start-date).
 
> "repeated". How does this sound?

OK 
 
> [...] The number N includes the iteration provided in the returned response.

OK 
 
> defined in section 4.1 as an in integer at least equal to 1

OK 
 
> the ALTO protocol [RFC7285]."

I do not think that all references in a normative section
automatically need to be normative. The question is typically "Can I
implement the specification without using a certain reference?" Given
that SSE is not mandatory to implement, it seemed to me that the
reference to SSE is non-normative.

> change RECOMMENDED to MUST, as you suggested. How does this sound?

This seems more consistent (but ultimately ALTO implementors must
decide whether requiring UTF-8 only is an issue for them).
 
> is an informative reference and cited with a SHOULD.

On second thought, I wonder whether you really want to mandate how a
value is 'stored'. Perhaps you wanted to require 'SHOULD use at least
support IEEE 754 double-precision floating point [IEEE.754.2008]
precision'.

Note that several lines up you suggested to change to integer, but I
am not sure you wanted that - please double check that you get this
right. Anyway, since you refer to properties of a concrete formats
defined in another document, it seems to me the reference is actually
normative.

<soap>
In general, I find the double precision requirement a bit irritating
given the large range of numbers doubles can represent (roughly from
2.2x10^-308 to 1.8x10^308). And yet, many practically useful numbers
such as 0.1 can't be represented exactly as floating point numbers,
i.e., there is always rounding involved, making comparisons rather
complicated.

Often people want to have doubles in the hope that this maximizes the
chances of not getting hit by rounding issues but the sad truth is that
floating point numbers remain dangerous.
</soap>

> reference makes more sense. OK?

I think your definition of calendar-start-(date|time) normatively
imports the Date/Time Format defined in section 7.1.1.1 of RFC 7231
and hence I think your document has a normative reference to RFC 7231.
It does not matter for your document what RFC 7285 has a normative /
informative references.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: "Y. Richard Yang" <[email-masked]>
Date: Fri, 21 Feb 2020 15:54:30 -0500

Dear Jürgen,

Sorry for the late reply, as our lead author is off without a stable
connection.

Please see below.

> Thanks. I was looking for something like this.
Great. We will add the text.

> and calendar-start-time two different things?
Good point. I believe that they are the same. Let us unify.

> This way the reader does not have to think about what a reference time
Sounds good. Simpler is always better.

> This is my typo edit and you caught it. Really careful reader!! It cannot
be integer because the unit is second and we want to have values lower than
second such as ms (0.001) values.

> reference to SSE is non-normative.
OK. Good guidance.

> OK.

> floating point numbers remain dangerous.
Totally agree. It is interesting that you use the 0.1 second example. I
assume that you have the classical Patriot 0.1 second rounding bug in mind (
https://www.viva64.com/en/b/0445/), which is quite close to our setting,
where 0.1 is their interval size, although I do not foresee that ALTO cost
calendar will be used in such a setting :-)

I believe that the initial design used only integers, with a unit (s, ms,
ns, ...) to avoid the floating-point issue (instead of 0.1 sec, it is 100
ms), and then the feedback and WG decision is to use generic floating
points to have a more uniform, single unit (second) design. One proposal,
which I am not sure others like, is to use a smaller unit, say ms, and then
most likely we will deal with only integers (in the range of [-(2**53)+1,
(2**53)-1]) and can still handle smaller units should the setting arise.
The downside is that there might be more zeros if the interval is longer
(60 sec -> 60000 ms). What do you think?

Another possibility is to keep the current design but add some text to
discuss the precision issue. A Proposal is the following:

"time-interval-size":

      *  is the duration of an ALTO Calendar time interval in seconds.
         A "time-interval-size" value contains a JSONNumber.  ALTO
         Servers SHOULD use at least IEEE 754 double-precision floating
         point [IEEE.754.2008] to store this value.  Example values are:
         300 , 7200, meaning that each Calendar value applies on a time
         interval that lasts 5 minutes and 2 hours, respectively.

=>

"time-interval-size":

      *  is the duration of an ALTO Calendar time interval in seconds.
         A "time-interval-size" value contains a JSONNumber.  Example
values
        are: 300 , 7200, meaning that each Calendar value applies on a time
         interval that lasts 5 minutes and 2 hours, respectively. Since the
        interval size can be smaller than 1 second (e.g., 100 ms), the
value
        specified can also be a floating point (e.g., 0.1). Both ALTO
clients and
        servers should be aware of potential issues caused by the precision
         issues caused by using floating point numbers. For example,
         the server may have an internal, integer representation in ms, and
store
        100 as an int (0.1 sec) in the local storage; the sever sends "0.1"
as
        the interval size to the client; the client may use a float/double
to store
        the JSON number, which may not represent 0.1 precisely. To improve
        interoperability, the server and the client SHOULD use IEEE 754
        double-precision floating point [IEEE.754.2008] to store this
value, in unit
       of seconds.

I am a bit ambivalent of the last sentence though.

Any comment?

> It does not matter for your document what RFC 7285 has a normative /
OK. W

Thanks a lot!

Richard

---

From: Schönwälder, Jürgen <[email-masked]>
Date: Mon, 24 Feb 2020 09:54:51 +0000

I will remove all closed issues and comment further inline.

> I am a bit ambivalent of the last sentence though.

This is something the WG has to discuss and decide. If the WG can find
agreement that there is a smallest sensible unit (e.g., milliseconds
or microseconds), then I personally would find it reasonable to
implement this specification by storing values in an integral type of
this smallest sensible unit internally (if the WG does not want to use
integral numbers of that type). Hence, for me personally, a statement
how implementations SHOULD store values goes too far as this is an
implementation concern.

/js

---

From: "Y. Richard Yang" <[email-masked]>
Date: Mon, 24 Feb 2020 12:22:01 -0500

Thank you so much, Jürgen!

The WG decision is to use a single unit and hence to allow scaling below
the chosen unit, we have to deal with float. As much as we thought 1 second
is typically the smallest, from my understanding, some use cases from a
major app developer is already below this. We will revise the sentences to
be reasonable and check with you to finalize say by Wednesday the latest.

Thanks again,
Richard

On Mon, Feb 24, 2020 at 4:55 AM Schönwälder, Jürgen <
[email-masked]> wrote:

> > > > ----- [[SR]] The text follows RFC7285, section 11.3.2.3,  in which
> IEEE
> > > 754
-- 
Richard