Subject: Re: [CDNi] CDNI and byte-range requests

From: Ben Niven-Jenkins <[email-masked]>
Date: Mon, 14 Oct 2013 04:09:37 +0100

Ray,

On 30 Jul 2013, at 12:27, Brandenburg, R. (Ray) van wrote:

> Hi all,
> 
> As I'm not sure whether this should be part of the Redirection Interface document or the Framework document, let me just post this as a general question to the list:
> 
> To my knowledge, the current CDNI documents do not explicitly address the case of HTTP Byte-range requests. Although in most cases byte-range requests will function just as any other requests, I think there are some cases where byte-range requests warrant further attention. An example is the Redirection interface. If a uCDN asks a dCDN to deliver a particular user request (containing a byte range), and the dCDN answers positively, can the uCDN assume that the dCDN accepts any future requests for the same file with a different byte range? And if that is the case, is this default behavior or do we need an explicit flag in the inter-CDN redirection request to indicate the uCDN/dCDN's preference? 
> 
> To be clear: I don't think we need any new functionality to handle byte-range requests, but a few sentences in either the Framework or the RI document might clear up any future uncertainty. 

I don't see how this is any different to a sequence of non byte-range requests.

It will likely depend on the behaviour of the client (and possibly whether the server closes the connection after each request if that drives a different client behaviour) as to where the client ends up going (uCDN or dCDN) when it makes a subsequent request. 

Trying to say anything here may take us down the rat hole of trying to describe client behaviour which we don't control.

Ben

> 
> Best regards,
> 
> Ray
> This e-mail and its contents are subject to the DISCLAIMER at http://www.tno.nl/emaildisclaimer
> 
> _______________________________________________
> CDNi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cdni

---

From: "Brandenburg, R. (Ray) van" <[email-masked]>
Date: Tue, 15 Oct 2013 09:23:34 +0000

Hi Ben,

> -----Original Message-----
> From: Ben Niven-Jenkins [mailto:[email-masked]]
> Sent: maandag 14 oktober 2013 5:10
> To: Brandenburg, R. (Ray) van
> Cc: [email-masked]
> Subject: Re: [CDNi] CDNI and byte-range requests
> 
> Ray,
> 
> > As I'm not sure whether this should be part of the Redirection Interface
> document or the Framework document, let me just post this as a general
> question to the list:
> >
> > To my knowledge, the current CDNI documents do not explicitly address
> the case of HTTP Byte-range requests. Although in most cases byte-range
> requests will function just as any other requests, I think there are some cases
> where byte-range requests warrant further attention. An example is the
> Redirection interface. If a uCDN asks a dCDN to deliver a particular user
> request (containing a byte range), and the dCDN answers positively, can the
> uCDN assume that the dCDN accepts any future requests for the same file
> with a different byte range? And if that is the case, is this default behavior or
> do we need an explicit flag in the inter-CDN redirection request to indicate
> the uCDN/dCDN's preference?
> >
> > To be clear: I don't think we need any new functionality to handle byte-
> range requests, but a few sentences in either the Framework or the RI
> document might clear up any future uncertainty.
> 
> It will likely depend on the behaviour of the client (and possibly whether the
> server closes the connection after each request if that drives a different
> client behaviour) as to where the client ends up going (uCDN or dCDN) when
> it makes a subsequent request.
> 
> Trying to say anything here may take us down the rat hole of trying to
> describe client behaviour which we don't control.
> 

Let me give a practical example: Let's say the uCDN is hosting a 2gb TS file. At some point it receives an incoming request for a particular byte range within that file, let's say the first 10mb. For some reason the uCDN decides that it wants to redirect the request to a dCDN. The dCDN receives the request. At that point it has to decide what to do: does it download the full 2gb file from the uCDN, or does it download just the first 10mb? Of course it's fully up to internal logic in the dCDN to decide what it should do here, however it does have an impact on the uCDN as well (potentially unnecessary traffic reducing the effectiveness of the inter-CDN redirect). My question is, do we need to say anything about this case to help both the dCDN and uCDN in making this decision? 

Ray

---

From: Scott Leibrand <[email-masked]>
Date: Tue, 15 Oct 2013 02:30:55 -0700

This feature (partial object caching) is absolutely a requirement before we
could think about offloading on-demand video traffic to a dCDN.  If the
dCDN downloads the entire file, that completely ruins the uCDN's efforts to
do partial caching, and the resulting increase in cache use makes things
worse than not using a dCDN at all.

-Scott

On Tue, Oct 15, 2013 at 2:23 AM, Brandenburg, R. (Ray) van <
[email-masked]> wrote:

> Let me give a practical example: Let's say the uCDN is hosting a 2gb TS
> file. At some point it receives an incoming request for a particular byte
> range within that file, let's say the first 10mb. For some reason the uCDN
> decides that it wants to redirect the request to a dCDN. The dCDN receives
> the request. At that point it has to decide what to do: does it download
> the full 2gb file from the uCDN, or does it download just the first 10mb?
> Of course it's fully up to internal logic in the dCDN to decide what it
> should do here, however it does have an impact on the uCDN as well
> (potentially unnecessary traffic reducing the effectiveness of the
> inter-CDN redirect). My question is, do we need to say anything about this
> case to help both the dCDN and uCDN in making this decision?
>
> Ray
>
> > Ben
> >
> > >
> > > Best regards,
> > >
> > > Ray

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Tue, 15 Oct 2013 11:17:11 +0100

Ray,

I'm not sure it warrants special mention, for example why is it any different to a client doing a GET for the complete resource and terminating the connection early. 

There are many situations that may lead to some form of caching inefficiency depending on the circumstances, client behaviour and the feature set of each CDN. 

I don't think it is feasible to document them all and I don't see byte range requests as being a particularly special case worthy of specific mention. 

Ben

> On 15 Oct 2013, at 10:23, "Brandenburg, R. (Ray) van" <[email-masked]> wrote:
> 
> Hi Ben,
>

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Tue, 15 Oct 2013 11:21:16 +0100

Scott,

What two CDNs may agree is a minimum feature set and what we need to specify in the RFCs are separate things IMO. 

Partial caching may be considered a required feature for dCDNs but I don't see it as within scope for the Framework or Redirection Interface documents to mandate that as it is not related to either IMO. 

Ben

> On 15 Oct 2013, at 10:30, Scott Leibrand <[email-masked]> wrote:
> 
> This feature (partial object caching) is absolutely a requirement before we could think about offloading on-demand video traffic to a dCDN.  If the dCDN downloads the entire file, that completely ruins the uCDN's efforts to do partial caching, and the resulting increase in cache use makes things worse than not using a dCDN at all.
> 
> -Scott
> 
> 
>> On Tue, Oct 15, 2013 at 2:23 AM, Brandenburg, R. (Ray) van <[email-masked]> wrote:
>> Hi Ben,
>>

---

From: Scott Leibrand <[email-masked]>
Date: Tue, 15 Oct 2013 04:15:25 -0700

On Tue, Oct 15, 2013 at 3:17 AM, Ben Niven-Jenkins
<[email-masked]>wrote:

> Ray,
>
> I'm not sure it warrants special mention,

I'm not sure it does, but...

> for example why is it any different to a client doing a GET for the
> complete resource and terminating the connection early.
>

Because clients only request content they actually want.  In particular, if
part of a video is unpopular, clients will tend not to request it.  But a
dCDN that requests the full file will ask for it anyway, wrecking the
upstream's ability to cache just the popular bits.

>
> There are many situations that may lead to some form of caching
> inefficiency depending on the circumstances, client behaviour and the
> feature set of each CDN.
>
> I don't think it is feasible to document them all and I don't see byte
> range requests as being a particularly special case worthy of specific
> mention.
>

I don't disagree there.  In my experience most CDN equipment vendors
support some form of partial caching, and there is no particular reason
they have to do it the same way, as long as some sort of sensible behavior
is configurable.

-Scott

---

From: "Francois Le Faucheur (flefauch)" <[email-masked]>
Date: Tue, 15 Oct 2013 12:40:57 +0000

Scott, Jan, and all,

On 15 Oct 2013, at 11:30, Scott Leibrand <[email-masked]<mailto:[email-masked]>> wrote:

Then perhaps it would make sense to include something like "Partial Object Caching" in the set of capabilities that can be advertised in the CDNI Footprint and Capabilities Advertisement interface (FCI). This would allow the uCDN to select a dCDN that supports partial object caching when required (or possibly elect to serve itself).

Jan,
Would you mind including this as a candidate in your list so we rememebr to make a concious decision about it?

Thanks

Francois

-Scott

On Tue, Oct 15, 2013 at 2:23 AM, Brandenburg, R. (Ray) van <[email-masked]<mailto:[email-masked]>> wrote:
Hi Ben,

> From: Ben Niven-Jenkins [mailto:[email-masked]<mailto:[email-masked]>]
> Cc: [email-masked]<mailto:[email-masked]>

Ray

> > [email-masked]<mailto:[email-masked]>

_______________________________________________
CDNi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/cdni

_______________________________________________
CDNi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/cdni