Subject: [CDNi] incorporating a few requirements from cdni-publisher-use-cases into cdni-requirements

From: Francois Le Faucheur <[email-masked]>
Date: Tue, 24 May 2011 15:48:34 +0200

Hello Kevin,

I believe we had agreed in Prague that a couple of requirements that were implied in cdni-publisher-use-cases should make their way into cdni-requirements.
Could you recap on those so we can consider/discuss as we rev up cdni-requirements?

Thanks

Francois

PS: I believe the IESG has not yet made a final determination with respect to creation of a CDNI WG. But I suggest we resume the work/progress on this list nonetheless.

---

From: Kevin J Ma <[email-masked]>
Date: Tue, 24 May 2011 20:01:21 -0400

Francois,

  We had discussed updating three of the requirements:

  - R36 discusses limiting the number of redirects for a content request.
    We interpret this as a hop count?  For certain types of live content
    (e.g., HLS) it might be advantageous to include a time-based limit.
    Can we modify the text to include an optional temporal limit?  e.g.:

    "R36  The CDNI Request-Routing protocol SHOULD support an optional
          mechanism enforcing a limit on the number of successive CDN
          redirections or the total elapsed time for a given request."

  - R38 discusses pushing upstream metadata to the downstream CDN.  It
    was not clear if this was inclusive of all hierarchical metadata,
    i.e., can/should the upstream send aggregate default metadata, which
    may be useful in future requests for related content?  Can we modify
    the text to clarify the conveyance of inherited metadata?  e.g.:

    "R38  The CDNI Request-Routing protocol MAY also allow the Upstream
          CDN to convey information pointing to CDNI metadata associated
          with the requested content or an object set to which the
          requested content belongs."

  - R58 lists potential content distribution control policies.  There are
    sometimes contractual obligations which limit which CDNs may be used.
    Can we add a bullet for whitelisting/blacklisting CDNs?  e.g.:

      "*   delegation whitelist/blacklist (i.e.  Information defining
           which Downstream CDNs the content may be delivered through)"

thanx!

--  Kevin J. Ma

> -----Original Message-----
> From: Francois Le Faucheur [mailto:[email-masked]]
> Sent: Tuesday, May 24, 2011 9:49 AM
> To: Kevin J Ma
> Cc: Le Faucheur Francois; [email-masked]
> Subject: incorporating a few requirements from cdni-publisher-use-cases
> into cdni-requirements
> 
> Hello Kevin,
> 
> I believe we had agreed in Prague that a couple of requirements that were
> implied in cdni-publisher-use-cases should make their way into cdni-
> requirements.
> PS: I believe the IESG has not yet made a final determination with respect
> to creation of a CDNI WG. But I suggest we resume the work/progress on
> this list nonetheless.

---

From: Songhaibin <[email-masked]>
Date: Wed, 25 May 2011 06:22:39 +0000

> For certain types of live content

It can be done if there is time synchronization.

BR,
-Haibin

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Kevin J Ma
> Sent: Wednesday, May 25, 2011 8:01 AM
> To: Francois Le Faucheur
> Cc: [email-masked]
> Subject: Re: [CDNi] incorporating a few requirements from
> cdni-publisher-use-cases into cdni-requirements
> 
> Francois,
> 
> _______________________________________________
> CDNi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cdni

---

From: Eric Burger <[email-masked]>
Date: Wed, 25 May 2011 13:20:54 -0400

Any time you have a SHOULD, you really need to enumerate when it is not appropriate to do the behaviour.  Otherwise you are really promulgating a MAY.

In this case, I cannot think of the situation where not to do it, particularly since the mechanism is optional.  Therefore, I would suggest either:
   "R36  The CDNI Request-Routing protocol MUST support an optional
         mechanism enforcing a limit on the number of successive CDN
         redirections or the total elapsed time for a given request."

or
   "R36  The CDNI Request-Routing protocol MAY support a
         mechanism enforcing a limit on the number of successive CDN
         redirections or the total elapsed time for a given request."

---

From: Bhumip Khasnabish <[email-masked]>
Date: Thu, 26 May 2011 10:36:21 -0400

Question/concern: It may not be feasible to share all of the available "
CDNI metadata associated with the requested content " al the time .. may
need a qualifier as to when to share and how much to share with whom (may be
a decision based on congestion, policy, traffic-type, basic/prime peering,
etc. etc.)
Thanks.

Bhumip

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 26 May 2011 17:38:06 +0200

Hi Kevin,

Breaking this discussion into several threads to facilitate resolution:

On 25 May 2011, at 02:01, Kevin J Ma wrote:

>    We interpret this as a hop count?

Right. ie effectively limit the number of CDNs that are involved in cascaded redirections.
Expected to be simple, yet achieves good things (redirection loop prevention, rough bounding of overall redirection time, limit number of players that will share the cost/revenue of delivery,..). 

Note that this hop count limit applies to the scenario of "recursive redirection" (not "iterative redirection").

>    Can we modify the text to include an optional temporal limit?  e.g.:

So, the trade-off here is less obvious to me. It may not be too hard to include < timestamp +  TTL> knobs in the CDNI request routing request, but:
	* to enforce it, you'd need all CDNs in the path to use fairly accurately synchronized timing (as Haibin pointed out)
	* the CDN-hop limit effectively already provides a very rough form of control on redirection time (albeit less accurate)
	* a delay TTL could always be autonomously enforced by the most-upstream CDN (i.e. when it gets the final CDNI Request Routing response that has travelled through all CDNs) without any specific mechanisms inside the CDNI request routing API itself. 
	* one potential value of adding a knob inside the CDNI request routing API is that a downstream CDN could try become smart in order to minimize occurrences of requests being dropped dead (ie exceeding the TTL). To that end the CDN request routing systems on the redirection path would have to try be smart and make judgment calls along the lines of "oh, I see that this request has already lived quite a long life and is getting close to its TTL, so should I try redirect at the risk of exceeding the TTL and having it dropped dead, or should I overwrite my normal redirection policy and not reroute further and serve myself?" . This may not be a trivial decision, since request routing times of the downstream CDN may greatly vary and be hard to predict. So this will end up having a positive value in some case, and a negative value in other cases.

Are you more after:
	 (i) a mechanism that simply discards a response that would be too old (but could be implemented autonomously by the upmost CDN) or 
	(ii) a mechanisms that allows the CDNI system to avoid taking too long to redirect (and "violates" its normal redirection policy when needed)?

If the latter, are you more after:
	(i) avoiding the transient situations (eg one of the request routing system got temporarily overloaded and is atypically slow) or 
	(ii) avoiding inappropriate CDN routing policies that result in very large redirection times (eg CDN1 in Europe redirects to a CDN in Asia, that redirects to a CDN in America, that redirects to a CDN in Europe,..) ?

Is this something you see as really needed in Initial Scope? or in a subsequent phase?

Thanks

Francois

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 26 May 2011 18:06:51 +0200

Hi Eric,

On 25 May 2011, at 19:20, Eric Burger wrote:

> Any time you have a SHOULD, you really need to enumerate when it is not appropriate to do the behaviour.

We plan to follow that convention for the actual protocol specs. 
ie those that define what an implementation MUST/SHOULD/MAY do.

But we followed a slightly different convention for the "Requirements" document because it does not define the behavior required of an implementation, I'd say it probably defines the behavior required of the WG. 
Considering requirements that are "Within Initial CDNI Scope": 
The MUST is intended to mean something like: "the WG will specify a CDNI protocol that does X. It is already clear that this is compatible with the overall WG priorities, or it is clear that shipping a solution without this does not make sense and would justify re-adjusting the WG priorities, or both." 
The SHOULD is intended to mean something like: "the WG will define a CDNI protocol that does X unless the WG realizes that this would compromise the overall WG priorities (for example it would involve complexities that would result in delaying the milestones)."
The MAY is intended to mean something like: "the WG will define a CDNI protocol that does X if it has done the more important items and this would not compromise the overall WG priorities."

This question has come up before on the list and I had the impression the group was fairly comfortable with that sort of convention .

If it is acceptable to all, we could include these terminology definitions into the Requirements document.
Another option, would be to revisit all the SHOULDs/MAYs and, for each of them, try make the final call as to whether we actually want to support them or not (but that will involve detailed discussions into the protocol themselves and making judgment calls on complexities_vs_timing).
So my preference is to stick to our current convention (ie define how hard the WG needs to try address each requirement) and then have the final calls actually made during the protocol definition phase.

Makes sense?

Francois

>  Otherwise you are really promulgating a MAY.
>

---

From: Kevin J Ma <[email-masked]>
Date: Thu, 26 May 2011 12:34:50 -0400

Hi Francois,

  The possible lack of time synchronization brings up a separate question
  about enforcement of signed URL expiration and more generally content
  expiration, though, the latter is admittedly on a much less granualar
  time scale, but we can leave that digression for the moment.

  To your later points, I was thinking about (ii) and (ii), i.e., not
  redirecting to a downstream CDN that would probably violate the R36
  constraint.  It could probably be done in other ways, but, if in a
  recursive redirection case, the upstream CDNs have latency estimates
  for the downstream CDNs, the aggregate estimates might be sufficient
  for providing a more accurate TTL than just hop count, without needing
  precise time synchronization.  Granted thats probably a more detailed
  implementation consideration than we want to make at this time, it is
  just to say that time synchronization should not be the limiting factor.

  As for implementing it at the most-upstream CDN, that is certainly an
  option as well, but using an aggregate latency estimate might still
  require the accumulation of data during the request routing process.
  It may or may not affect the final protocol, but I guess the question
  is: is a time-based TTL a valid requirement?

  I don't think it would be unreasonable for the initial scope, but I
  don't necessarily think it is mandatory.

thanx.

--  Kevin J. Ma

> Sent: Thursday, May 26, 2011 11:38 AM
> To: Kevin J Ma
> Cc: Francois Le Faucheur; [email-masked]
> Subject: Delay TTL in CDNI Request Routing (was Re: incorporating a few
> requirements from cdni-publisher-use-cases into cdni-requirements)
> 
> Hi Kevin,
> 
> Expected to be simple, yet achieves good things (redirection loop
> prevention, rough bounding of overall redirection time, limit number of
> players that will share the cost/revenue of delivery,..).
> 
> Note that this hop count limit applies to the scenario of "recursive
> redirection" (not "iterative redirection").
> 
> So, the trade-off here is less obvious to me. It may not be too hard to
> include < timestamp +  TTL> knobs in the CDNI request routing request,
> but:
> 	* to enforce it, you'd need all CDNs in the path to use fairly
> accurately synchronized timing (as Haibin pointed out)
> 	* the CDN-hop limit effectively already provides a very rough form
> of control on redirection time (albeit less accurate)
> 	* a delay TTL could always be autonomously enforced by the most-
> upstream CDN (i.e. when it gets the final CDNI Request Routing response
> that has travelled through all CDNs) without any specific mechanisms
> inside the CDNI request routing API itself.
> 	* one potential value of adding a knob inside the CDNI request
> routing API is that a downstream CDN could try become smart in order to
> minimize occurrences of requests being dropped dead (ie exceeding the
> TTL). To that end the CDN request routing systems on the redirection path
> would have to try be smart and make judgment calls along the lines of "oh,
> I see that this request has already lived quite a long life and is getting
> close to its TTL, so should I try redirect at the risk of exceeding the
> TTL and having it dropped dead, or should I overwrite my normal
> redirection policy and not reroute further and serve myself?" . This may
> not be a trivial decision, since request routing times of the downstream
> CDN may greatly vary and be hard to predict. So this will end up having a
> positive value in some case, and a negative value in other cases.
> 
> Are you more after:
> 	 (i) a mechanism that simply discards a response that would be too
> old (but could be implemented autonomously by the upmost CDN) or
> 	(ii) a mechanisms that allows the CDNI system to avoid taking too
> long to redirect (and "violates" its normal redirection policy when
> needed)?
> 
> 	(i) avoiding the transient situations (eg one of the request routing
> system got temporarily overloaded and is atypically slow) or
> 	(ii) avoiding inappropriate CDN routing policies that result in very
> large redirection times (eg CDN1 in Europe redirects to a CDN in Asia,
> that redirects to a CDN in America, that redirects to a CDN in Europe,..)
> ?
>

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 26 May 2011 18:52:47 +0200

> If it is acceptable to all, we could include these terminology definitions into the Requirements document.

and, perhaps, to avoid confusion with the RFC2119  terminology, we could drop the ALL CAPS notation ie use must/should/may. 

>>>         redirections or the total elapsed time for a given request."

Francois Le Faucheur
Distinguished Engineer
Corporate Development
[email-masked]
Phone: +33 49 723 2619
Mobile: +33 6 19 98 50 90

Cisco Systems France
Greenside
400 Ave de Roumanille
06410 Sophia Antipolis
France
Cisco.com

 Think before you print.

This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.

Cisco Systems France, Société à responsabiité limitée, Rue Camille Desmoulins – Imm Atlantis Zac Forum Seine Ilot 7 92130 Issy les Moulineaux, Au capital de 91.470 €, 349 166 561 RCS Nanterre, Directeur de la publication: Jean-Luc Michel Givone.

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 26 May 2011 19:33:37 +0200

On 26 May 2011, at 18:34, Kevin J Ma wrote:

>  time scale,

Right. I don't see a practical issue here. Just like it is not an issue with existing HTTP cache expiration mechanisms.

> but we can leave that digression for the moment.
> 
>  constraint.

I see.
But I think having a knob in the CDNI Request Routing API would only provide value if you want to support situations where:
	(i) redirecting some requests to a CDN would result in a redirection time that exceeds the max redirection time for these requests, AND
	(ii) redirecting some other requests to a CDN would NOT result in a redirection time that exceeds the max redirection time for these requests.
Because if you are in a situation where all requests have the same max redirection time, then you may as well update the whole redirection policy (to no longer redirect to the dCDN) and be done with it. 

>  It could probably be done in other ways, but, if in a

So my proposal is to add a standalone requirement for the delay-based TTL (that is separate from the CDN-hop-count TTL), have it as a MAY in Initial scope, and have it reflect the objective of affecting the request routing decision (as opposed to just dropping the request if it takes too long to redirect).

Is that alright?

Francois

---

From: Kevin J Ma <[email-masked]>
Date: Fri, 27 May 2011 00:45:57 -0400

Francois,

  comments inline:

> Sent: Thursday, May 26, 2011 1:34 PM
> To: Kevin J Ma
> Subject: Re: Delay TTL in CDNI Request Routing (was Re: incorporating a
> few requirements from cdni-publisher-use-cases into cdni-requirements)
> 
> 
> Right. I don't see a practical issue here. Just like it is not an issue
> with existing HTTP cache expiration mechanisms.
> 
> 
> But I think having a knob in the CDNI Request Routing API would only
> provide value if you want to support situations where:
> 	(i) redirecting some requests to a CDN would result in a redirection
> time that exceeds the max redirection time for these requests, AND
> 	(ii) redirecting some other requests to a CDN would NOT result in a
> redirection time that exceeds the max redirection time for these requests.
> Because if you are in a situation where all requests have the same max
> redirection time, then you may as well update the whole redirection policy
> (to no longer redirect to the dCDN) and be done with it.

This is probably true for a given CDN, if we ignore the transient latency case
(though maybe the transient latency case is valid).  However, there may be many
uCDNs which service User Agent requests, each with its own redirection priorities
based on proximity, business rules, etc.  Note, I am still not convinced that a
single authoritative CDN receiving all initial requests makes sense.  There may
be a need for regional aCDNs to optimize proximity, or subscriber home aCDNs to
meeting billing or other business requirements.  Coordinating redirection policies
between multiple aCDNs may be difficult.  Having an abstraction for a latency
metric on which redirection decisions may be made seems useful.

> So my proposal is to add a standalone requirement for the delay-based TTL
> (that is separate from the CDN-hop-count TTL), have it as a MAY in Initial
> scope, and have it reflect the objective of affecting the request routing
> decision (as opposed to just dropping the request if it takes too long to
> redirect).
> 
> Is that alright?

That sounds fine to me.

thanx!

--  Kevin J. Ma

---

From: "David Harrington" <[email-masked]>
Date: Fri, 27 May 2011 11:47:52 -0400

Hi,
 
This is a status update from your Responsible AD.
 
1) The IESG has discussed where a CDNI WG would be best located. It
was decided that, if approved, CDNI WG belonged in the TSV area and I
would remain the Responsible area director.
 
2) I have requested the IESG discuss the proposed WG charter in the
IESG telechat 6/9.
 
3) I have asked the BOF chairs to request a BOF session at ietf81
Quebec. This ensures the CDNI community will have a session at ietf81.
 
4) If the IESG approves the WG, the BOF session will simply become a
WG session.

David Harrington
Director, IETF Transport Area
[email-masked] (preferred for ietf)
[email-masked]
+1 603 828 1401 (cell)

---

From: Bhumip Khasnabish <[email-masked]>
Date: Fri, 27 May 2011 14:35:30 -0400

Hi David,

Accordinig to Fig.1 of
http://tools.ietf.org/html/draft-jenkins-cdni-problem-statement-01 (shown
below)  acquisition looks like out of scope of CDNI problem area. Would that
remain the case going forward? Thanks.

Bhumip

   -

-------
     /        \
     |   CSP  |
     \        /
      --------
          *
          *
          *                        /\
          *                       /  \
      ---------------------      |CDNI|       ---------------------
     /    Upstream CDN     \     |    |      /   Downstream CDN    \
     |     +-------------+ |   Control API   | +-------------+     |
     |     |CDN Control  |<======|====|=======>| CDN Control |     |
     |     +------*-*-*--+ |     |    |      | +-*-*-*-------+     |
     |            * * *    |     |    |      |   * * *             |
     |     +------*------+ |   Logging API   | +-----*-------+     |
     | ****| Logging     |<======|====|=======>|  Logging    |**** |
     | *   --------------+ |     |    |      | +-------------+   * |
     | *            * *    |     |    |      |   * *             * |
     | *   +--------*----+ | Req-Routing API | +---*---------+   * |
     | * **|Req-Routing  |<======|====|=======>| Req-Routing |** * |
     | * * +-------------+ |     |    |      | +-------------+ * * |
     | * *            *    |     |    |      |   *             * * |
     | * * +----------*--+ |CDNI Metadata API| +-*-----------+ * * |
     | * * |Distribution |<======|====|=======>| Distribution| * * |
     | * * |             | |      \  /       | |             | * * |
     | * * |             | |       \/        | |             | * * |
     | * ****+---------+ | |                 | | +---------+**** * |
     | ******|Surrogate|*************************|Surrogate|****** |
     |     | +---------+ | |   Acquisition   | | +-----*---+ |     |
     |     +-------------+ |                 | +-------*-----+     |
     \                     /                 \         *           /
      ---------------------                   ---------*-----------
                                                       *
                                                       * Delivery
                                                       *
                                                    +------+
                                                    | User |
                                                    | Agent|
                                                    +------+

On Fri, May 27, 2011 at 11:47 AM, David Harrington <[email-masked]>wrote:

>  1) The IESG has discussed where a CDNI WG would be best located. It was
> decided that, if approved, CDNI WG belonged in the TSV area and I would
> remain the Responsible area director.
>
> 2) I have requested the IESG discuss the proposed WG charter in the IESG
> telechat 6/9.
>
> 3) I have asked the BOF chairs to request a BOF session at ietf81 Quebec.
> This ensures the CDNI community will have a session at ietf81.
>
> 4) If the IESG approves the WG, the BOF session will simply become a WG
> session.
>
>
> David Harrington

---

From: Francois Le Faucheur <[email-masked]>
Date: Mon, 30 May 2011 08:42:43 +0200

Hi Bhumip,

On 27 May 2011, at 20:35, Bhumip Khasnabish wrote:

> Hi David,
>  
> Accordinig to Fig.1 of http://tools.ietf.org/html/draft-jenkins-cdni-problem-statement-01 (shown below)  acquisition looks like out of scope of CDNI problem area.

Just to be clear, the problem statement argues that there already exist protocols that can be used "as is" (without any extensions) for acquisition across CDNs, so there is no need for this group to work on this aspect of the problem (at least initially). However, the problem statement also indicates that potential exchange of information across CDNs in order to bootstrap/facilitate/allow such acquisition to happen, is in scope: for example the CDNI interfaces could convey, to the downstream CDN, information about where the content can be acquired.

> Would that remain the case going forward? Thanks.

This remained the proposal to the IESG.
Dave can let us know if the IESG wants to change that for some reason.

Cheers

Francois

> On Fri, May 27, 2011 at 11:47 AM, David Harrington <[email-masked]> wrote:
> Hi,
>  
> 1) The IESG has discussed where a CDNI WG would be best located. It was decided that, if approved, CDNI WG belonged in the TSV area and I would remain the Responsible area director.
>  
> 2) I have requested the IESG discuss the proposed WG charter in the IESG telechat 6/9.
>  
> 3) I have asked the BOF chairs to request a BOF session at ietf81 Quebec. This ensures the CDNI community will have a session at ietf81.
>  
> 4) If the IESG approves the WG, the BOF session will simply become a WG session.
>  
>  
> David Harrington