Subject: [icnrg] Reflection on the QoE with ICN

From: Damien Saucez <[email-masked]>
Date: Mon, 20 Oct 2014 23:44:23 +0200

Dear all,

The very nature of ICN may impact QoS and hence QoE.  We believe this is worth
to be studied and is by itself a challenge.  Do you believe that the following
would merit to be added to the research challenge draft? What is your opinion
about the subject in general?

With traditional networking, all the chunks composing an object are commonly
delivered from the same entity meaning that the delivery of all these chunks
are impacted the same way by the network conditions.  On the contrary, with ICN
name data objects may be delivered by virtually any network element to the
requestor.  Therefore, each single chunk may be subject to completely different
network behaviour.  This behaviour may have a direct impact on objective QoS
metrics (e.g., delay or bandwidth) and therefore may impact user experience
that is usually measured with subjective QoE metrics (e.g., MOS).  Moreover,
nowadays the traffic at a given time has little if no impact on the traffic at
subsequent time and the exact content consumed by a user has little or no
impact on the QoE of other users.  With ICN, this is no longer true as caching
leverages popularity of contents, and raises potentially much higher
correlation between a user behaviour to other users QoE than exists today.  As
a result, for the same network condition the QoE can be very different
depending whether or not a name data object has already been retrieved in the
past.

It is therefore essential to understand how much the shift toward ICN can
impact QoS and consequently QoE.  The understanding of the impact of ICN on QoS
will potentially lead to the conception of new models linking QoS and QoE,
taking into account the new form of correlation of traffic.  For instance,
variability is usually considered harmful in current networks because it is
often the result of queuing phenomenas.  However, in ICN this variability might
exist in perfectly provisioned networks and just be an artefact of caching.

Thank you in advance for your comments, 

Damien Saucez

---

From: Cedric Westphal <[email-masked]>
Date: Mon, 20 Oct 2014 22:14:38 +0000

Hi Damien,

This is a very interesting topic. But as in the issue of fairness, you would have to ask: against what are you comparing the QoE? Let me add to this in-line...

-----Original Message-----
From: icnrg [mailto:[email-masked]] On Behalf Of Damien Saucez
Sent: Monday, October 20, 2014 2:44 PM
To: icnrg
Cc: Chadi Barakat; Rautio Teemu
Subject: [icnrg] Reflection on the QoE with ICN

Dear all,

The very nature of ICN may impact QoS and hence QoE.  We believe this is worth to be studied and is by itself a challenge.  Do you believe that the following would merit to be added to the research challenge draft? What is your opinion about the subject in general?

With traditional networking, all the chunks composing an object are commonly delivered from the same entity meaning that the delivery of all these chunks are impacted the same way by the network conditions.  On the contrary, with ICN name data objects may be delivered by virtually any network element to the requestor.  Therefore, each single chunk may be subject to completely different network behaviour.  This behaviour may have a direct impact on objective QoS metrics (e.g., delay or bandwidth) and therefore may impact user experience that is usually measured with subjective QoE metrics (e.g., MOS). 

>> CWe: if the content is retrieved locally in an ICN vs from the server in a "traditional" network, then while there may be more variation on the delay or bandwidth, it should be bounded above by the delay or bandwidth of the traditional network. In the worst case, the ICN will retrieve the chunk from the server; it may get it earlier, therefore reducing bandwidth usage and reducing retrieval delay. There may be more variation, but the QoS (as measured by delay or bandwidth) should be uniformly better.

 Moreover, nowadays the traffic at a given time has little if no impact on the traffic at subsequent time and the exact content consumed by a user has little or no impact on the QoE of other users.  With ICN, this is no longer true as caching leverages popularity of contents, and raises potentially much higher correlation between a user behaviour to other users QoE than exists today.  As a result, for the same network condition the QoE can be very different depending whether or not a name data object has already been retrieved in the past.

>> CWe: again, if the content has been retrieved in the past can only help you. So if there is an impact on QoS, it can only be positive, right? 

It is therefore essential to understand how much the shift toward ICN can impact QoS and consequently QoE.  The understanding of the impact of ICN on QoS will potentially lead to the conception of new models linking QoS and QoE, taking into account the new form of correlation of traffic.  For instance, variability is usually considered harmful in current networks because it is often the result of queuing phenomenas.  However, in ICN this variability might exist in perfectly provisioned networks and just be an artefact of caching.

>> CWe: this is an interesting question, as a shift toward ICN should improve QoS. The variability you mention should be about multiple requests for the same file, where the QoE improves for subsequent requests. If different chunks come from different servers, the variability is harmful only if the application can't handle it; we've seen this for video, say in "On the Interaction of Adaptive Video Streaming with Content-Centric Networking" Grandl et al, PacketVideo 2013, or the ICN/video draft. I think it would be interesting to add some considerations on the research challenge draft, and if you have some video specific considerations for QoE/MOS, then maybe into the video draft.

>> Best, 

>> C.

Damien Saucez
_______________________________________________
icnrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/icnrg

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Mon, 20 Oct 2014 22:25:39 +0000

More inline - but everytime I hear QoS hence QoE I wince... QoS is for your operator - QoE is you.
Marie-Jose Montpetit, Ph.D.
[email-masked]
@SocialTVMIT

On Oct 20, 2014, at 6:14 PM, Cedric Westphal <[email-masked]> wrote:

> Moreover, nowadays the traffic at a given time has little if no impact on the traffic at subsequent time

what do you mean? Netflix has 40% of the traffic at night - so I can tell you that a stream from Netflix will be there for a long while.
> and the exact content consumed by a user has little or no impact on the QoE of other users.

Again disagree. If I get into a buffering event and I can't watch anymore the QoE of my friends will be impacted - I cannot communicated with them what I am seeing vs what they do - huge QoE impact.
>  With ICN, this is no longer true as caching leverages popularity of contents, and raises potentially much higher correlation between a user behaviour to other users QoE than exists today.  As a result, for the same network condition the QoE can be very different depending whether or not a name data object has already been retrieved in the past.
I think you need to define QoE
> It is therefore essential to understand how much the shift toward ICN can impact QoS and consequently QoE.  
Orthogonal...

> The understanding of the impact of ICN on QoS will potentially lead to the conception of new models linking QoS and QoE, taking into account the new form of correlation of traffic.  For instance, variability is usually considered harmful in current networks because it is often the result of queuing phenomenas.  However, in ICN this variability might exist in perfectly provisioned networks and just be an artefact of caching.
I think you need to define what you  think ICN is.
> Thank you in advance for your comments, 
You have mine,

/mjm

---

From: Damien Saucez <[email-masked]>
Date: Tue, 21 Oct 2014 01:06:27 +0200

Hello,

On 21 Oct 2014, at 00:14, Cedric Westphal <[email-masked]> wrote:

> Hi Damien,
> 
> This is a very interesting topic. But as in the issue of fairness, you would have to ask: against what are you comparing the QoE? Let me add to this in-line…
> 
you are right that it has to be compared to something.  The QoE is a subjective
measure taken at the user and the question we could ask is the following:

“For a given infrastructure, would the QoE as perceived by a user be better, equal,
or worth with ICN than without ICN?”

comments inline.

>>> CWe: if the content is retrieved locally in an ICN vs from the server in a "traditional" network, then while there may be more variation on the delay or bandwidth, it should be bounded above by t…

On the other hand, you might have fluctuations (=jitter) induced by the fact
that some chunks may be retrieved from different places.  Maybe there is a
lower average delay or a higher average bandwidth, but it might come at the
cost of a higher jitter.  In some situations, users are more sensitive to
variations than pure performances.

>>> CWe: again, if the content has been retrieved in the past can only help you. So if there is an impact on QoS, it can only be positive, right? 

yes an no.

Yes in the sense that today (unless proxies are used), you never benefit from
the traffic of others while with the possibility to cache, you can improve your
performance based on what other people did before you.

No if you think in terms of variability. Imagine that today to get a service you
need X. With caches, in the worst case you would have X (assuming using
a cache is always faster than not using it, which may not be true) so not
worst than today, and in the general case you will have it in Y <= X. The 
problem is that sometime you have in X, sometime faster. How would this
variability be perceived by the user?

Also, caching contents can potentially negatively impact performance of
non-cached contents, so when you are cached you somehow win, but
what about the instants you are not cached? How much will you suffer
from the presence of other elements that, them, are cached?

>>> CWe: this is an interesting question, as a shift toward ICN should improve QoS. The variability you mention should be about multiple requests for the same file, where the QoE improves for subsequ…

This is definitely something that has to be studied in details, and having the
choice of paths (and origins) may not always be beneficial. This is not only
limited to video, it can also be applied to congestion control where you could
see spurious timeout events just because the system is not able to provision
its timers properly.

Damien Saucez

---

From: Dirk Kutscher <[email-masked]>
Date: Mon, 20 Oct 2014 23:15:48 +0000

Hi Damien and Cedric,

Thanks for bringing this up -- the general topic is certainly relevant to our work.

Ideally, we'd be able to define QoS, QoE in ICN and explain the challenges and the opportunities for ICN.

The challenges draft has a Transport Services section -- should this be extended?

Best regards,
Dirk

> From: icnrg [mailto:[email-masked]] On Behalf Of Cedric Westphal
> Sent: Dienstag, 21. Oktober 2014 00:15
> To: Damien Saucez; icnrg
> Subject: Re: [icnrg] Reflection on the QoE with ICN
> 
> Hi Damien,
> 
> This is a very interesting topic. But as in the issue of fairness, you would have
> to ask: against what are you comparing the QoE? Let me add to this in-line...
> 
> The very nature of ICN may impact QoS and hence QoE.  We believe this is
> worth to be studied and is by itself a challenge.  Do you believe that the
> following would merit to be added to the research challenge draft? What is
> your opinion about the subject in general?
> 
> With traditional networking, all the chunks composing an object are
> commonly delivered from the same entity meaning that the delivery of all
> these chunks are impacted the same way by the network conditions.  On the
> contrary, with ICN name data objects may be delivered by virtually any
> network element to the requestor.  Therefore, each single chunk may be
> subject to completely different network behaviour.  This behaviour may
> have a direct impact on objective QoS metrics (e.g., delay or bandwidth) and
> therefore may impact user experience that is usually measured with
> subjective QoE metrics (e.g., MOS).
> 
> >> CWe: if the content is retrieved locally in an ICN vs from the server in a
> "traditional" network, then while there may be more variation on the delay
> or bandwidth, it should be bounded above by the delay or bandwidth of the
> traditional network. In the worst case, the ICN will retrieve the chunk from
> the server; it may get it earlier, therefore reducing bandwidth usage and
> reducing retrieval delay. There may be more variation, but the QoS (as
> measured by delay or bandwidth) should be uniformly better.
> 
>  Moreover, nowadays the traffic at a given time has little if no impact on the
> traffic at subsequent time and the exact content consumed by a user has
> little or no impact on the QoE of other users.  With ICN, this is no longer true
> as caching leverages popularity of contents, and raises potentially much
> higher correlation between a user behaviour to other users QoE than exists
> today.  As a result, for the same network condition the QoE can be very
> different depending whether or not a name data object has already been
> retrieved in the past.
> 
> >> CWe: again, if the content has been retrieved in the past can only help
> you. So if there is an impact on QoS, it can only be positive, right?
> 
> impact QoS and consequently QoE.  The understanding of the impact of ICN
> on QoS will potentially lead to the conception of new models linking QoS and
> QoE, taking into account the new form of correlation of traffic.  For instance,
> often the result of queuing phenomenas.  However, in ICN this variability
> might exist in perfectly provisioned networks and just be an artefact of
> caching.
> 
> >> CWe: this is an interesting question, as a shift toward ICN should improve
> QoS. The variability you mention should be about multiple requests for the
> same file, where the QoE improves for subsequent requests. If different
> chunks come from different servers, the variability is harmful only if the
> application can't handle it; we've seen this for video, say in "On the
> Interaction of Adaptive Video Streaming with Content-Centric Networking"
> Grandl et al, PacketVideo 2013, or the ICN/video draft. I think it would be
> interesting to add some considerations on the research challenge draft, and
> if you have some video specific considerations for QoE/MOS, then maybe
> into the video draft.
> 
> >> Best,
> 
> >> C.
>

---

From: Cedric Westphal <[email-masked]>
Date: Mon, 20 Oct 2014 23:19:34 +0000

Inline...

-----Original Message-----
From: Damien Saucez [mailto:[email-masked]] 
Sent: Monday, October 20, 2014 4:06 PM
To: Cedric Westphal
Cc: icnrg; Chadi Barakat; Rautio Teemu
Subject: Re: [icnrg] Reflection on the QoE with ICN

Hello,

On 21 Oct 2014, at 00:14, Cedric Westphal <[email-masked]> wrote:

> Hi Damien,
> 
> This is a very interesting topic. But as in the issue of fairness, you 
> would have to ask: against what are you comparing the QoE? Let me add 
> to this in-line...
> 
you are right that it has to be compared to something.  The QoE is a subjective measure taken at the user and the question we could ask is the following:

"For a given infrastructure, would the QoE as perceived by a user be better, equal, or worth with ICN than without ICN?"

comments inline.

>>> CWe: if the content is retrieved locally in an ICN vs from the server in a "traditional" network, then while there may be more variation on the delay or bandwidth, it should be bounded above by t…

On the other hand, you might have fluctuations (=jitter) induced by the fact that some chunks may be retrieved from different places.  Maybe there is a lower average delay or a higher average bandwidth, but it might come at the cost of a higher jitter.  In some situations, users are more sensitive to variations than pure performances.

yes an no.

Yes in the sense that today (unless proxies are used), you never benefit from the traffic of others while with the possibility to cache, you can improve your performance based on what other people did before you.

No if you think in terms of variability. Imagine that today to get a service you need X. With caches, in the worst case you would have X (assuming using a cache is always faster than not using it, which may not be true) so not worst than today, and in the general case you will have it in Y <= X. The problem is that sometime you have in X, sometime faster. How would this variability be perceived by the user?

CWe: well, you can always recreate the variability of "all non-cached content" at (or to fool) the application layer. 

Also, caching contents can potentially negatively impact performance of non-cached contents, so when you are cached you somehow win, but what about the instants you are not cached? How much will you suffer from the presence of other elements that, them, are cached?

CWe: this I don't get. Again, if you have two non-cached content, or one cached, one non-cached, the latter will be better. You will not suffer from stuff being cached. It's like saying taking the bus will affect negatively someone driving her car. You are removing traffic out of the way, which improves performance both for cached and non-cached.

>>> CWe: this is an interesting question, as a shift toward ICN should improve QoS. The variability you mention should be about multiple requests for the same file, where the QoE improves for subsequ…

This is definitely something that has to be studied in details, and having the choice of paths (and origins) may not always be beneficial. This is not only limited to video, it can also be applied to congestion control where you could see spurious timeout events just because the system is not able to provision its timers properly.

Damien Saucez

---

From: Damien Saucez <[email-masked]>
Date: Tue, 21 Oct 2014 01:21:25 +0200

Hello,

> More inline - but everytime I hear QoS hence QoE I wince... QoS is for your operator - QoE is you.

maybe the terminology is not adequate. By QoS I mean objective information and QoE is subjective information.

comments inline.

> what do you mean? Netflix has 40% of the traffic at night - so I can tell you that a stream from Netflix will be there for a long while.

from that point of view you are right, but it has to be understood as the fact
that, in general (still not always true of course), if the traffic at time t
was about content A or B, it does not change anything for another user at time
t+1.

> Again disagree. If I get into a buffering event and I can't watch anymore the QoE of my friends will be impacted - I cannot communicated with them what I am seeing vs what they do - huge QoE impact.

yes, but if there is a download of A or of B changes nothing if A and B incur
the same bandwidth/buffer/delay/loss/... events.

Normally what impact your performance are the performances of the other flows,
not their content.

If you start to have caching or dynamic routing, it is not only the performance
of the other flows that impact you but also what they are about.

> I think you need to define QoE

I will not be original and consider the usual definition that the QoE is a subjective
measure of a service as perceived by the user.

> I think you need to define what you  think ICN is.

I like the definition given in the research challenge document:

"
Information-Centric Networking is an approach to evolve the Internet
infrastructure to directly support information distribution by introducing
uniquely named data as a core Internet principle.  Data becomes independent
from location, application, storage, and means of transportation, enabling
in-network caching and replication.
"

>> Thank you in advance for your comments, 

Thank you,

Damien Saucez

---

From: Damien Saucez <[email-masked]>
Date: Tue, 21 Oct 2014 09:23:34 +0200

> CWe: well, you can always recreate the variability of "all non-cached content" at (or to fool) the application layer. 

yes, and that’s worth mentioning that maybe such mechanism would be necessary.

> CWe: this I don't get. Again, if you have two non-cached content, or one cached, one non-cached, the latter will be better. You will not suffer from stuff being cached. It's like saying taking the…

Indeed when you are cached you go faster than when you are not cached (assuming
the cache is closer than the sever, which might not always be true, depending
on how you implement routing).

Nevertheless, saying that caching is purely beneficial is incorrect.  We have
proven (see [1]) that when some contents are cached, it may slow down
non-cached contents if you don’t use an appropriate congestion control
mechanism.  The intuition behind is that if you get closer (i.e., cached) you
get a better throughput than if no cache is used at all, but the capacity of the
link shared between the cached content and the non-cached content is not
changed, meaning that the non-cached content has less bandwidth for it and
is hence slowed down.

[1] When AIMD meets ICN: a bandwidth sharing perspective, D. Saucez, I. Cianci, L. A. Grieco, C. Barakat, to appear in Proc. IFIP Networking 2014, June 2014.

Damien Saucez

---

From: Damien Saucez <[email-masked]>
Date: Tue, 21 Oct 2014 09:30:09 +0200

Hello,

What about extending last paragraph of Sec. 4.6?

"   NDOs can represent content used in different types of applications
   with different QoS requirements, for example, interactive real-time
   applications, media streaming, file download.  Each of these
   applications imposes different QoS requirements on different elements
   in an information-centric network, e.g., regarding cache placement,
   request-to-cache/source routing.  Achieving the necessary quality-of-
   service levels in a shared network is an active ICN research topic.”

Damien Saucez

---

From: Daniel Corujo <[email-masked]>
Date: Tue, 21 Oct 2014 10:39:35 +0100

Dear all,

I quite agree with this addition. In fact, this also raises the impact that the service-side requirements (in addition to user-side requirements as well) can place over ICN-based networking operation mechanisms. 

With your permission, I would like to propose an addition/change to the last phrase suggested by Daminen:

“ (…) Achieving the necessary quality-of-service levels in a shared network is an active ICN research topic, which considers aspects and interaction opportunities between the network, the application services and the user terminals for the optimisation of content reception.”

Is that ok for all?

Best regards,  

Daniel Corujo, Ph.D., Research Fellow
Universidade de Aveiro, Portugal	
http://re.vu/dcorujo
Advanced Telecommunications and Networks Group - http://atnog.av.it.pt
Follow us - @ATNoG_ITAv

> On 21 Oct 2014, at 08:30 , Damien Saucez <[email-masked]> wrote:
> 
> Hello,
>

---

From: Spiros Spirou <[email-masked]>
Date: Tue, 21 Oct 2014 09:56:10 +0300

Hi Dirk, all

> Ideally, we'd be able to define QoS, QoE in ICN and explain the challenges and the opportunities for ICN.

Section 2.4 'Choosing Relevant Metrics' of 
draft-irtf-icnrg-evaluation-methodology is an early attempt to do that. 
At some point we even discussed going to IPPM with an ICN outlook, but 
this fizzled out.

Thanks,

Spiros

---

From: Damien Saucez <[email-masked]>
Date: Tue, 21 Oct 2014 20:52:49 +0200

Hello,

> I quite agree with this addition. In fact, this also raises the impact that the service-side requirements (in addition to user-side requirements as well) can place over ICN-based networking operati…

good point to generalise the discussion to the service-side.
> “ (…) Achieving the necessary quality-of-service levels in a shared network is an active ICN research topic, which considers aspects and interaction opportunities between the network, the applicati…

would you include users also in this or do you stick to quality-of-service, so
independent of user’s perception?

Damien Saucez

---

From: Daniel Corujo <[email-masked]>
Date: Wed, 22 Oct 2014 00:12:36 +0100

Hi Damien, 

You place a very good question about the QoS independence of user’s perception. Well, I never really thought about the concept of having users with better QoS having access to “better names” for reaching content in a “better way”… but I bet that ICN would allow such a thing :)
Anyone would fancy to address this?

Best regards,

> On 21 Oct 2014, at 19:52 , Damien Saucez <[email-masked]> wrote:
> 
> Hello,
>

---

From: <[email-masked]>
Date: Tue, 21 Oct 2014 23:22:41 +0000

At PARC, we have often used an example like “/creditcard/platnum/mybill” vs “/creditcard/silver/mybill” as ways a company can segment their traffic.  For example, the globally routable prefix /creditcard would get the user to the closest any cast instance of the credit card company.  The data-center routable part “/platnum” vs “/silver” would then route the request to different clusters with different performance.  Finally, the application name “/mybill” identifies the action to take (there would be other data in the name or interest payload).

Marc

---

From: Daniel Corujo <[email-masked]>
Date: Wed, 22 Oct 2014 00:32:13 +0100

Thanks, Mark. This means that in the draft we can definitely consider user perceived QoS.
Also, is there any document where we could get more information on this item, or scenarios exploring it?

Thank you and best regards,

> On 22 Oct 2014, at 24:22 , <[email-masked]> <[email-masked]> wrote:
>

---

From: "Christian Timmerer (ITEC)" <[email-masked]>
Date: Tue, 21 Oct 2014 10:28:38 +0200

Regarding QoE you may consult http://www.qualinet.eu/ and their white paper on definitions of QoE which can be downloaded there. This network is very much experienced in assessing QoE and ICN-based services haven't been discussed so far. However, this network is very open and everybody could join or you could setup a liaison with them...

Best regards,
 -Christian

:--
:- Priv.-Doz. Dr. Christian Timmerer
:- Assoc.-Prof., Multimedia Communication, Alpen-Adria-Universität Klagenfurt
:- research.timmerer.com | selab.itec.aau.at | dash.itec.aau.at | bitmovin.com
------------------------------------------------------------------------------------------------------

> On 20 Oct 2014, at 23:44, Damien Saucez <[email-masked]> wrote:
> 
> Dear all,
>