Subject: Re: [rmcat] [iccrg] draft-johansson-rmcat-scream-cc

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 25 Jun 2014 12:46:14 +0000

Hi

Thanks for the interest in this work.

Crosspost to RMCAT as well, hope that is Ok
Answers/comments inline

/Ingemar

From: Luca De Cicco [mailto:[email-masked]]
Sent: den 25 juni 2014 13:23
To: Ingemar Johansson S
Cc: Dave Taht; [email-masked]
Subject: Re: [iccrg] draft-johansson-rmcat-scream-cc

Dear Ingemar,

I have some questions.

1) In the architecture figure (Figure 2) you show the case of two videos produced by the
same source and going to the same destination (both videos are encoded and fed to the same UDP socket),
am I right? In the case SCReAM is used in a WebRTC context (in a browser), what use case
are you imagining?
[IJ] This was just to highlight how two sources are handled, I am fully aware that it is probably not the most common use case, on the other hand I don’t know how WebRTC will be used in the future. If there are two destinations then one will need to congestion control contexts (one for each UDP socket). The bandwidth distribution may then be based on rate shaping but another, possibly more useful, option is to target the congestion window functions directly, for instance if the congetsion window for one flow is knocked down just a little then the other flow will grab the free capacity automatically. Another option is to adjust the gain factors for the CWND calculations.

2) How did you implement GCC algorithm? Do you have a paper where I can see the dynamics of GCC (for instance
the rate computed by the receiver side controller, the estimated one way queuing delay variation "m")?
[IJ] The GCC algorithm was implemented into our simulation platform by Zahed, it is based in the availabe code and the drafts. We have logging possibilities in our simulator but I don’t know of any paper around this.

3) You show average bitrate in the tables. There's an odd situation where GCC gets a 50% packet loss ratio in the case of
16 users/cell but with an average rate of 61 kbps. Can you elaborate on this?
[IJ] Could be an effect of a much too high load in combination with a high mobility, could be an effect of large amount of packet losses due to network queue overflow. I will BTW update the evaluation results for GCC soon, it seems like it was tuned a but too sensitive, with the new setting one get higher bitrates, but unfortunately also the tail latency will increase more quickly with increased load.

Cheers,
Luca

On Wed, Jun 18, 2014 at 11:37 AM, Ingemar Johansson S <[email-masked]<mailto:[email-masked]>> wrote:
Hi

Besides the example code there is no ready to compile code available (at least nothing that I know of). But on the other hand the solution has similarities to TFWC (TCP Friendly Window-based Congestion control) that was devised by Soo-Hyun Choi (http://tfwc.hackerslab.eu/ ), perhaps that can be used as a basis ?

/Ingemar

> -----Original Message-----
> From: Dave Taht [mailto:[email-masked]<mailto:[email-masked]>]
> Sent: den 17 juni 2014 20:00
> To: Ingemar Johansson S
> Cc: [email-masked]<mailto:[email-masked]>
> Subject: Re: [iccrg] draft-johansson-rmcat-scream-cc
>
> My usual request: Is there running code somewhere?
>
> On Tue, Jun 17, 2014 at 10:35 AM, Ingemar Johansson S
> <[email-masked]<mailto:[email-masked]>> wrote:
> > Hi
> >
> >
> >
> > This draft can be of interest for those of you interested in
> > congestion control for realtime media.
> >
> > http://www.ietf.org/internet-drafts/draft-johansson-rmcat-scream-cc-00
> > .txt
> >
> >
> >
> > Abstract : “This memo describes a rate adaptation framework for
> > conversational
> >
> >    video services.  The solution conforms to the packet conservation
> >
> >    principle and uses a hybrid loss and delay based congestion control
> >
> >    algorithm.  The framework is evaluated over both simulated
> > bottleneck
> >
> >    scenarios as well as in a LTE (Long Term Evolution) system
> > simulator
> >
> >    and is shown to achieve both low latency and high video throughput
> > in
> >
> >    these scenarios”
> >
> >
> >
> > /Ingemar
> >
> >
> >
> >
> >
> > =================================
> >
> > Ingemar Johansson  M.Sc.
> >
> > Senior Researcher
> >
> >
> >
> > Ericsson AB
> >
> > Wireless Access Networks
> >
> > Labratoriegränd 11
> >
> > 971 28, Luleå, Sweden
> >
> > Phone +46-1071 43042<tel:%2B46-1071%2043042>
> >
> > SMS/MMS +46-73 078 3289<tel:%2B46-73%20078%203289>
> >
> > [email-masked]<mailto:[email-masked]>
> >
> > www.ericsson.com<http://www.ericsson.com>
> >
> >
> >
> > “The difference between stupidity and
> >
> > genius is that genius has its limits."
> > Albert Einstein
> > =================================
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > iccrg mailing list
> > [email-masked]<mailto:[email-masked]>
> > https://www.irtf.org/mailman/listinfo/iccrg
> >
>
>
>
> --
> Dave Täht
>
> NSFW:
> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indec
> ent.article
_______________________________________________
iccrg mailing list
[email-masked]<mailto:[email-masked]>
https://www.irtf.org/mailman/listinfo/iccrg

---

From: Luca De Cicco <[email-masked]>
Date: Wed, 25 Jun 2014 17:32:07 +0200

Dear Zahed,

can you share some information about the GCC simulation code?

thanks,
Luca

On Wed, Jun 25, 2014 at 2:46 PM, Ingemar Johansson S <
[email-masked]> wrote:

> *From:* Luca De Cicco [mailto:[email-masked]]
> *Sent:* den 25 juni 2014 13:23
> *To:* Ingemar Johansson S
> *Cc:* Dave Taht; [email-masked]
> *Subject:* Re: [iccrg] draft-johansson-rmcat-scream-cc
>
>
>
> Dear Ingemar,
>
>
>
> I have some questions.
>
>
>
> [IJ] This was just to highlight how two sources are handled, I am fully
> aware that it is probably not the most common use case, on the other hand I
> don’t know how WebRTC will be used in the future. If there are two
> destinations then one will need to congestion control contexts (one for
> each UDP socket). The bandwidth distribution may then be based on rate
> shaping but another, possibly more useful, option is to target the
> congestion window functions directly, for instance if the congetsion window
> for one flow is knocked down just a little then the other flow will grab
> the free capacity automatically. Another option is to adjust the gain
> factors for the CWND calculations.
>
>
>
> [IJ] The GCC algorithm was implemented into our simulation platform by
> Zahed, it is based in the availabe code and the drafts. We have logging
> possibilities in our simulator but I don’t know of any paper around this.
>
>
>
> 3) You show average bitrate in the tables. There's an odd situation where
> GCC gets a 50% packet loss ratio in the case of
>
> [IJ] Could be an effect of a much too high load in combination with a high
> mobility, could be an effect of large amount of packet losses due to
> network queue overflow. I will BTW update the evaluation results for GCC
> soon, it seems like it was tuned a but too sensitive, with the new setting
> one get higher bitrates, but unfortunately also the tail latency will
> increase more quickly with increased load.
>
>
>
> Cheers,
>
> Luca
>
>
>
> On Wed, Jun 18, 2014 at 11:37 AM, Ingemar Johansson S <
> [email-masked]> wrote:
>
> Hi
>
> Besides the example code there is no ready to compile code available (at
> least nothing that I know of). But on the other hand the solution has
> similarities to TFWC (TCP Friendly Window-based Congestion control) that
> was devised by Soo-Hyun Choi (http://tfwc.hackerslab.eu/ ), perhaps that
> can be used as a basis ?
>
> /Ingemar
>
>
> > From: Dave Taht [mailto:[email-masked]]
> > Cc: [email-masked]
> > <[email-masked]> wrote:
> > > Hi
> > >
> > >
> > >
> > > Phone +46-1071 43042
> > > SMS/MMS +46-73 078 3289
> > > [email-masked]
> > >
> > > www.ericsson.com
> > >
> > >
> > >
> > > [email-masked]
> [email-masked]

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Thu, 26 Jun 2014 14:10:49 +0200

Hi Luca,

The simulation code is implemented in our proprietary simulator (I 
should thank my MS thesis workers who contributed a lot). Thus cant 
really share the code but what I can share is some simulation results.

Note that in our implementation we followed latest update for varying 
threshold from the GCC draft

If you want GCC related logs particularly for LTE simulations I cant not 
provide that.We simulate lots of users and dont log all the GCC related 
variables.

However, I have attached simulation logs from two test cases-
1. a staircase bandwidth test , that I found interesting in one of the 
papers.
2. and test case 5.1 from 
http://tools.ietf.org/html/draft-sarker-rmcat-eval-test-01.

I hope those plots will give you an idea about the implementation we 
have and how it is behaving.

I would suggest you run tests from 
http://tools.ietf.org/html/draft-sarker-rmcat-eval-test-01 as well and 
then we can compare the results.

BR

Zahed

On 2014-06-25 17:32, Luca De Cicco wrote:
> On Wed, Jun 25, 2014 at 2:46 PM, Ingemar Johansson S
> <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Hi____
>
>     __ __
>
>     Thanks for the interest in this work. ____
>
>     __ __
>
>     Crosspost to RMCAT as well, hope that is Ok____
>
>     Answers/comments inline____
>
>     __ __
>
>     /Ingemar____
>
>     __ __
>
>     *From:*Luca De Cicco [mailto:[email-masked]
>     <mailto:[email-masked]>]
>     *Cc:* Dave Taht; [email-masked] <mailto:[email-masked]>
>     *Subject:* Re: [iccrg] draft-johansson-rmcat-scream-cc____
>
>     __ __
>
>     Dear Ingemar,____
>
>     __ __
>
>     I have some questions. ____
>
>     __ __
>
>     1) In the architecture figure (Figure 2) you show the case of two
>     videos produced by the ____
>
>     same source and going to the same destination (both videos are
>     encoded and fed to the same UDP socket),____
>
>     am I right? In the case SCReAM is used in a WebRTC context (in a
>     browser), what use case ____
>
>     are you imagining?____
>
>     [IJ] This was just to highlight how two sources are handled, I am
>     fully aware that it is probably not the most common use case, on the
>     other hand I don’t know how WebRTC will be used in the future. If
>     there are two destinations then one will need to congestion control
>     contexts (one for each UDP socket). The bandwidth distribution may
>     then be based on rate shaping but another, possibly more useful,
>     option is to target the congestion window functions directly, for
>     instance if the congetsion window for one flow is knocked down just
>     a little then the other flow will grab the free capacity
>     automatically. Another option is to adjust the gain factors for the
>     CWND calculations.____
>
>     __ __
>
>     2) How did you implement GCC algorithm? Do you have a paper where I
>     can see the dynamics of GCC (for instance____
>
>     the rate computed by the receiver side controller, the estimated one
>     way queuing delay variation "m")?____
>
>     [IJ] The GCC algorithm was implemented into our simulation platform
>     by Zahed, it is based in the availabe code and the drafts. We have
>     logging possibilities in our simulator but I don’t know of any paper
>     around this.____
>
>     __ __
>
>     3) You show average bitrate in the tables. There's an odd situation
>     where GCC gets a 50% packet loss ratio in the case of____
>
>     16 users/cell but with an average rate of 61 kbps. Can you elaborate
>     on this?____
>
>     [IJ] Could be an effect of a much too high load in combination with
>     a high mobility, could be an effect of large amount of packet losses
>     due to network queue overflow. I will BTW update the evaluation
>     results for GCC soon, it seems like it was tuned a but too
>     sensitive, with the new setting one get higher bitrates, but
>     unfortunately also the tail latency will increase more quickly with
>     increased load. ____
>
>     __ __
>
>     Cheers,____
>
>     Luca____
>
>     __ __
>
>     On Wed, Jun 18, 2014 at 11:37 AM, Ingemar Johansson S
>     <[email-masked]
>     <mailto:[email-masked]>> wrote:____
>
>     Hi
>
>     Besides the example code there is no ready to compile code available
>     (at least nothing that I know of). But on the other hand the
>     solution has similarities to TFWC (TCP Friendly Window-based
>     Congestion control) that was devised by Soo-Hyun Choi
>     (http://tfwc.hackerslab.eu/ ), perhaps that can be used as a basis ?
>
>     /Ingemar____
>
>
>      > Cc: [email-masked] <mailto:[email-masked]>
>      > <[email-masked]
>     <mailto:[email-masked]>> wrote:
>      > > Hi
>      > >
>      > >
>      > >
>      > > Phone +46-1071 43042 <tel:%2B46-1071%2043042>
>      > >
>      > > SMS/MMS +46-73 078 3289 <tel:%2B46-73%20078%203289>
>      > >
>     <mailto:[email-masked]>
>      > >
>      > > www.ericsson.com <http://www.ericsson.com>
>      > >
>      > >
>      > >
>      > > [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>
>     https://www.irtf.org/mailman/listinfo/iccrg____
>
>     __ __
>
>

-- 

Zahed

==================================================
ANM ZAHEDUZZAMAN SARKER

Ericsson AB
Services, Media and Network Features
Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43
Fax +46 920 996 21
SMS/MMS +46 76 115 37 43
[email-masked]
www.ericsson.com

==================================================

---

From: Varun Singh <[email-masked]>
Date: Thu, 26 Jun 2014 16:11:33 +0300

Hi Zahed,

Does your GCC simulation code implement any FEC or Retx? Since the gcc
draft does not imply use of any repair mechanism even though the chrome
browser implements both.
 On 26 Jun 2014 14:11, "Zaheduzzaman Sarker" <
[email-masked]> wrote:

> Hi Luca,
>
> The simulation code is implemented in our proprietary simulator (I should
> thank my MS thesis workers who contributed a lot). Thus cant really share
> the code but what I can share is some simulation results.
>
> 2. and test case 5.1 from http://tools.ietf.org/html/
> draft-sarker-rmcat-eval-test-01.
>
> I hope those plots will give you an idea about the implementation we have
> and how it is behaving.
>
> I would suggest you run tests from http://tools.ietf.org/html/
> draft-sarker-rmcat-eval-test-01 as well and then we can compare the
> results.
>
> BR
>
> Zahed
>
>

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Thu, 26 Jun 2014 15:25:05 +0200

Hi Varun,

No, it does not.

BR

Zahed

On 2014-06-26 15:11, Varun Singh wrote:
> On 26 Jun 2014 14:11, "Zaheduzzaman Sarker"
> <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Hi Luca,
>
>     Note that in our implementation we followed latest update for
>     varying threshold from the GCC draft
>
>     If you want GCC related logs particularly for LTE simulations I cant
>     not provide that.We simulate lots of users and dont log all the GCC
>     related variables.
>
>     1. a staircase bandwidth test , that I found interesting in one of
>     the papers.
>     http://tools.ietf.org/html/__draft-sarker-rmcat-eval-test-__01
>     <http://tools.ietf.org/html/draft-sarker-rmcat-eval-test-01>.
>
>     http://tools.ietf.org/html/__draft-sarker-rmcat-eval-test-__01
>     <http://tools.ietf.org/html/draft-sarker-rmcat-eval-test-01> as well
>     and then we can compare the results.
>
>     BR
>
>     Zahed
>
>
>         <[email-masked]
>         <mailto:[email-masked]
>         <mailto:[email-masked]>>> wrote:
>
>              Hi____
>
>              __ __
>
>         <mailto:[email-masked]>
>              <mailto:[email-masked] <mailto:[email-masked]>>]
>         <mailto:[email-masked] <mailto:[email-masked]>>
>              *Subject:* Re: [iccrg] draft-johansson-rmcat-scream-__cc____
>
>              __ __
>
>              Dear Ingemar,____
>
>              __ __
>
>              <[email-masked]
>              <mailto:[email-masked]
>         <mailto:[email-masked]>>> wrote:____
>
>              Hi
>
>               > From: Dave Taht [mailto:[email-masked]
>         <mailto:[email-masked]>
>              <mailto:[email-masked] <mailto:[email-masked]>>]
>         <mailto:[email-masked] <mailto:[email-masked]>>
>               > Subject: Re: [iccrg] draft-johansson-rmcat-scream-__cc
>               >
>               > <[email-masked]
>              <mailto:[email-masked]
>         <mailto:[email-masked]>>> wrote:
>               > > Hi
>               > >
>               > >
>               > >
>         http://www.ietf.org/internet-__drafts/draft-johansson-rmcat-__scream-cc-00
>         <http://www.ietf.org/internet-drafts/draft-johansson-rmcat-scream-cc-00>
>               > > .txt
>               > >
>               > >
>               > >
>               > >    principle and uses a hybrid loss and delay based
>         congestion
>              control
>               > >
>               > > ==============================__===
>               > >
>               > > Ingemar Johansson  M.Sc.
>               > >
>               > > Senior Researcher
>               > >
>               > >
>               > >
>               > > Ericsson AB
>               > >
>               > > Wireless Access Networks
>               > >
>               > > Labratoriegränd 11
>               > >
>               > > 971 28, Luleå, Sweden
>               > >
>         <tel:%2B46-1071%2043042>
>         <tel:%2B46-73%20078%203289>
>               > >
>               > > [email-masked]
>              <mailto:[email-masked]
>         <mailto:[email-masked]>>
>               > >
>         <http://www.ericsson.com>
>               > >
>               > >
>               > >
>               > > ==============================__===
>               > >
>               > >
>               > >
>               > >
>               > >
>               > >
>               > > _________________________________________________
>               > > iccrg mailing list
>         <mailto:[email-masked] <mailto:[email-masked]>>
>               > > https://www.irtf.org/mailman/__listinfo/iccrg
>         <https://www.irtf.org/mailman/listinfo/iccrg>
>               > >
>               >
>               >
>               >
>               > --
>               > Dave Täht
>               >
>               > NSFW:
>               >
>         https://w2.eff.org/Censorship/__Internet_censorship_bills/__russell_0296_indec
>         <https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indec>
>               > ent.article
>              _________________________________________________
>              iccrg mailing list
>         [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>         <mailto:[email-masked]>>
>         https://www.irtf.org/mailman/__listinfo/iccrg____
>         <https://www.irtf.org/mailman/listinfo/iccrg____>
>
>              __ __
>
>
>
>     --
>
>     Zahed
>
>     ==============================__====================
>     ANM ZAHEDUZZAMAN SARKER
>
>
>     Ericsson AB
>     Phone +46 10 717 37 43 <tel:%2B46%2010%20717%2037%2043>
>     Fax +46 920 996 21 <tel:%2B46%20920%20996%2021>
>     SMS/MMS +46 76 115 37 43 <tel:%2B46%2076%20115%2037%2043>
>     [email-masked]
>     <mailto:[email-masked]>
>     ==============================__====================
>

-- 

Zahed

Ericsson AB

---

From: Luca De Cicco <[email-masked]>
Date: Thu, 26 Jun 2014 15:38:33 +0200

Dear Varun,

the way RTX or FEC are computed shouldn't impact GCC sending rate dynamics,
but only
the user's QoE.

Cheers,
Luca

On Thu, Jun 26, 2014 at 3:11 PM, Varun Singh <[email-masked]> wrote:

>>>     <mailto:[email-masked]>]

---

From: Dave Taht <[email-masked]>
Date: Thu, 26 Jun 2014 10:06:29 -0700

> the code but what I can share is some simulation results.

I have decided that I will no longer consider experimental results
that cannot easily be reproduced independently, and will ignore future
articles, internet drafts, and proposals that do not come with
sufficient source code to reproduce independently.

Nearly every published experiment that actually supplied sufficient
information, raw data and code to reproduce it - that I've had the
chance to examine closely and/or reproduce in more varied
circumstances - has been flawed, and I have no reason whatsoever to
believe that code I cannot see and run is not flawed in some respect,
either.

If I apply this criterion as a stage-1 filter, my workload drops enormously.

Both chrome and firefox have mostly working webrtc implementations
now, and whatever future algorithms proposed should be able to be
easily hacked into those and tried in real life.

open source ns2, ns3, and mininet simulators exist as well as a basis
for future work.

My policy is obviously not the ietf's nor the position of the chairs of this wg,
but it is in the spirit of "rough consensus and running code".

> http://tools.ietf.org/html/draft-sarker-rmcat-eval-test-01 as well and then
> we can compare the results.
>
> BR
>
> Zahed
>
>
>

-- 
Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 27 Jun 2014 06:42:30 +0000

Hi Dave

I am  not sure if you refer to the Google Congestion Control algorithm (GCC) or the self-clocked alternative (SCReAM). 
GCC is implemented in running code (I guess?) not sure though if the latest addition in (draft-alvestrand-rmcat-congestion-02) are implemented, I leave to others to comment on that.

As regards to SCReAM : No, there is no (not yet) WebRTC implementation available, I hope there will be one at least based on the same philosophy available. It depends partly on the level of attention that draft gets. The example code is in any case quite complete and it should be possible for those interested to implement the framework in NS2, NS3 or mininet (and also implement it as running code in the browsers too!). If you examine the code you will also notice that it resembles TCP quite a lot,  the equations to compute the congestion window are quite simple so I don't expect things to blow up in a real implementation. 
There are some details missing around loss event detection and congestion window validation and I plan to add it in a later version after the summer (it is already part of our proprietary code). So the bottom line is that with a little luck you should be able to run WebRTC with SCReAM as congestion control in the future but we are not there yet.

Regards
/Ingemar

> Sent: den 26 juni 2014 19:06
> To: Zaheduzzaman Sarker
> Cc: Luca De Cicco; Ingemar Johansson S; [email-masked]; rmcat WG
> ([email-masked])
> I have decided that I will no longer consider experimental results that cannot
> easily be reproduced independently, and will ignore future articles, internet
> drafts, and proposals that do not come with sufficient source code to
> reproduce independently.
> 
> information, raw data and code to reproduce it - that I've had the chance to
> examine closely and/or reproduce in more varied circumstances - has been
> flawed, and I have no reason whatsoever to believe that code I cannot see
> and run is not flawed in some respect, either.
> 
> Both chrome and firefox have mostly working webrtc implementations now,
> and whatever future algorithms proposed should be able to be easily hacked
> into those and tried in real life.
> 
> open source ns2, ns3, and mininet simulators exist as well as a basis for
> future work.
> 
> > However, I have attached simulation logs from two test cases- 1. a
> > staircase bandwidth test , that I found interesting in one of the
> > papers.