Subject: Re: [iccrg] Presentation on Sprout at today's ICCRG

From: Zaheduzzaman Sarker <[email-masked]>
Date: Tue, 3 Dec 2013 12:32:04 +0000

HI Keith,

We had to port sproutbt2 into our LTE simulator and we are confident the port is OK. You can see that also in the results that the file transfer simulations were showing some results.

Yes, we think there are differences when you use tracefile based simulation/emulation and the simulator we use. The fact that the LTE link is shared and left over bandwidth is used by other users makes the difference. When you measure the network capacity using some kind of trace that gives you certain throughput which is based on certain data you send. However, this misses the interaction in a shared link when somebody else is also trying to do that same. In our simulator we can simulate this behavior hence the view of the link capacity will be more dynamic.

BR

Zahed

===============================================================================
ANM ZAHEDUZZAMAN SARKER

Multimedia Technologies

Ericsson
Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43
Mobile +46 76 115 37 43
Office +46 76 115 37 43
Fax +46 920 996 21
[email-masked]
www.ericsson.com<http://www.ericsson.com>
===============================================================================

From: [email-masked] [mailto:[email-masked]] On Behalf Of Keith Winstein
Sent: den 16 november 2013 01:20
To: Zaheduzzaman Sarker
Cc: [email-masked]; Hari Balakrishnan; Anirudh Sivaraman; [email-masked]
Subject: Re: Presentation on Sprout at today's ICCRG

Thanks Zahed. Very interesting -- we'll be eager to see and test the code when it is available.

Have you run the unmodified sproutbt2 in your LTE simulator? Separate from the difficulties created by these periodic sources, I am curious if there is any substantive difference between our results (in an emulated LTE network, based on a trace of Verizon and AT&T's LTE networks) and yours (in the LTE simulator).

Best regards,
Keith

On Fri, Nov 15, 2013 at 6:06 AM, Zaheduzzaman Sarker <[email-masked]<mailto:[email-masked]>> wrote:
Hi Keith,

Sorry for late reply.

Nikolas is finalizing his thesis now. He has implemented some ideas to improve Sprout which he calls "E-Sprout". Soon he will publish his thesis and you will get to know more about it. He has implemented similar ideas like 1) and 2) you mentioned. According to the results it was not good enough to improve the Sprout by just doing 1) or 2).

please see inline below for some answers to your questions.

BR

Zahed

On 2013-11-05 19:23, Keith Winstein wrote:
Hello Zahed,

Thanks for sending! I'm afraid I wasn't able to find a previous email
from you.

This is very interesting -- I would love to see the implementation of
how your team grafted the periodic data source to the Sprout sender. I
assume you modified "sproutbt2" (Sprout "bulk transport") but wanted to
double-check. Is your code available for us to take a look?
Basically the tunnel version of Sprout is implemented in our propitiatory LTE Simulator (this was available in the Sprout website). If you want to about how we model the periodic data source then you can have a look at my presentation in RMCAT WG http://www.ietf.org/proceedings/88/slides/slides-88-rmcat-9.pdf.

I agree with you that having a sender that only produces data
periodically is considerably tougher than "bulk transport," or at least
that it introduces more tradeoffs into the design. Sprout has a few
tools to deal with this kind of issue --

(1) The "time_to_next" header field, which tells the receiver when it
should expect the next datagram. The receiver uses this to decide
whether it should count a gap in packets received against the capacity
of the network or just as an indication that none have been sent. Did
your team set this to 1/30 second for the last datagram in a flight, or
how did you adjust it?
yes it was set to the framerate.

It may take tuning, especially if there is some
latency in coming up with the next flight (next coded frame).

(2) Buffering 1-2 frames at the sender. This is the easiest way to make
sure there is always data available to send! Of course there is a
tradeoff between latency and throughput, which is sort of the basic
message of the Sprout work. But you probably do not literally want your
coded video frame sizes to be bouncing around as fast as Sprout's
network forecasts, so it would be better to have 1-2 frames of
sender-side buffering (at 60 fps) and adjust the video coder a bit more
smoothly.
yes, buffering is also done as the sender but the frames were not hold back unnecessarily.

(3) Sampling from the video camera more often (generally 60 fps) and
only plucking frames off the camera and coding them when there is
(enough) capacity available in Sprout's forecast to fire off a
good-quality frame. The slower the camera generates frames (e.g. 30 Hz
in your case), the harder it is to fill the link. Especially if you
don't buffer anything at the sender.
This might work however I don't know how this fits into the Sprout framework. Here you are talking about a direct relation between camera, encoder and congestion control algorithm (in case of Sprout the transport protocol) which might be needed for a transport congestion control algorithm to work. In RMCAT there have been discussions around this please join those discussion if possible.

Thanks again for your interest -- we'd be eager to try your code in our
emulation environment and see if we can get Sprout to do better with a
"periodic" source.

Best regards,
Keith

On Tue, Nov 5, 2013 at 12:53 PM, Zaheduzzaman Sarker
<[email-masked]<mailto:[email-masked]>
<mailto:[email-masked]<mailto:[email-masked]>>> wrote:

    Hi Keith,____

    __ __

    My apologies if I have not send you the slides already. I thought I
    send something to you (the mail address from the web site) on last
    Sunday and asking your comment on the slides.____

    __ __

    This work is done by one of our MS thesis student. I found it
    interesting and he was also wondering if you guys have also seen
    this and if there is any fix for it.____

    Please bear in mind that the presentation is not to show Sprout is
    not working rather showing what could go wrong if the periodicity
    and data limitation is not taken into consideration.____

    __ __

    Anyway, Here the slides. I would appreciate any comments from you.____

    __ __

    BR____

    __ __

    Zahed____

    __ __

    *===============================================================================____*

    *ANM ZAHEDUZZAMAN SARKER *
    Senior Researcher
    Multimedia Technologies____

    *Ericsson*

    Laboratoriegränd 11
    97128 Luleå, Sweden
    Phone +46 10 717 37 43<tel:%2B46%2010%20717%2037%2043> <tel:%2B46%2010%20717%2037%2043>
    Mobile +46 76 115 37 43<tel:%2B46%2076%20115%2037%2043> <tel:%2B46%2076%20115%2037%2043>
    Office +46 76 115 37 43<tel:%2B46%2076%20115%2037%2043> <tel:%2B46%2076%20115%2037%2043>
    Fax +46 920 996 21<tel:%2B46%20920%20996%2021> <tel:%2B46%20920%20996%2021>
    [email-masked]<mailto:[email-masked]>
    <mailto:[email-masked]<mailto:[email-masked]>>
    www.ericsson.com<http://www.ericsson.com> <http://www.ericsson.com>____

    ===============================================================================____

    __ __

    *From:[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>
    [mailto:[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>] *On Behalf
    Of *Keith Winstein
    *Sent:* den 5 november 2013 09:44
    *To:* Zaheduzzaman Sarker
    *Cc:* [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>; Hari Balakrishnan;
    Anirudh Sivaraman
    *Subject:* Presentation on Sprout at today's ICCRG____

    __ __

    Hello,____

    __ __

    Just saw you are presenting an evaluation of Sprout at today's ICCRG
    at the IETF meeting. That's great! We appreciate your interest.____

    __ __

    Unfortunately I'm not attending this IETF -- can you please send me
    a copy of your slides and any conclusions you have found? I'm eager
    to learn more.____

    __ __

    Best regards, and thank you,____

    Keith____

    __ __

--

Zahed

==================================================
ANM ZAHEDUZZAMAN SARKER

Ericsson AB
Multimedia Technologies

Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43<tel:%2B46%2010%20717%2037%2043>
Fax +46 920 996 21<tel:%2B46%20920%20996%2021>
SMS/MMS +46 76 115 37 43<tel:%2B46%2076%20115%2037%2043>
[email-masked]<mailto:[email-masked]>
www.ericsson.com<http://www.ericsson.com>

==================================================

---

From: Keith Winstein <[email-masked]>
Date: Fri, 6 Dec 2013 15:47:41 -0500

Zahed,

Thanks for this. I'd be very interested to see any results from sproutbt2
on your LTE simulator -- are you able to share?

I did not understand your reference to how our emulation strategy "gives
you certain throughput which is based on certain data you send," and I'd be
eager to learn anything that bears on the realism of this style of
emulation. Could you expand a bit more on this?

We have measured the behavior of Verizon and AT&T LTE base stations in the
middle of the night in response to increasing demand (we boot up 6-12
phones on the same cell ID, where there is unlikely to be much other
demand, and start pounding away). From our measurements, the behavior of
the base station appears simply to divide up its resource blocks equally
among clients who have pending data at a given time.

That is, in practice the eNodeB appears to have a separate queue for each
client that has pending data and divides the downlink among them. My UE
might feel the affect of the _presence_ or _absence_ of another flow to
another UE, but my UE does not have to care how many bits per second a
server is trying to send to that UE at a given time. That's if our
inferences about isolation between UEs are correct.

If this is true, the only thing that matters to a particular client is the
number of resource blocks allocated to *that* client and the number of bits
that can be sent in each RB -- which is essentially what our trace
captures. So I'd of course be interested to learn about material feedback
effects that we're not capturing.

Best regards,
Keith

On Tue, Dec 3, 2013 at 7:32 AM, Zaheduzzaman Sarker <
[email-masked]> wrote:

>  HI Keith,
>
>
>
> We had to port sproutbt2 into our LTE simulator and we are confident the
> port is OK. You can see that also in the results that the file transfer
> simulations were showing some results.
>
>
>
> Yes, we think there are differences when you use tracefile based
> simulation/emulation and the simulator we use. The fact that the LTE link
> is shared and left over bandwidth is used by other users makes the
> difference. When you measure the network capacity using some kind of trace
> that gives you certain throughput which is based on certain data you send.
> However, this misses the interaction in a shared link when somebody else is
> also trying to do that same. In our simulator we can simulate this behavior
> hence the view of the link capacity will be more dynamic.
>
>
>
> BR
>
>
>
> Zahed
>
>
>
>
> *===============================================================================*
>
> www.ericsson.com
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *Keith
> Winstein
> *Sent:* den 16 november 2013 01:20
> *Cc:* [email-masked]; Hari Balakrishnan; Anirudh Sivaraman;
> [email-masked]
> *Subject:* Re: Presentation on Sprout at today's ICCRG
>
>
>
> Have you run the unmodified sproutbt2 in your LTE simulator? Separate from
> the difficulties created by these periodic sources, I am curious if there
> is any substantive difference between our results (in an emulated LTE
> network, based on a trace of Verizon and AT&T's LTE networks) and yours (in
> the LTE simulator).
>
>
>
> Best regards,
>
> Keith
>
>
>
> On Fri, Nov 15, 2013 at 6:06 AM, Zaheduzzaman Sarker <
> [email-masked]> wrote:
>
> Hi Keith,
>
> Sorry for late reply.
>
> Nikolas is finalizing his thesis now. He has implemented some ideas to
> improve Sprout which he calls "E-Sprout". Soon he will publish his thesis
> and you will get to know more about it. He has implemented similar ideas
> like 1) and 2) you mentioned. According to the results it was not good
> enough to improve the Sprout by just doing 1) or 2).
>
> Basically the tunnel version of Sprout is implemented in our propitiatory
> LTE Simulator (this was available in the Sprout website). If you want to
> about how we model the periodic data source then you can have a look at my
> presentation in RMCAT WG
> http://www.ietf.org/proceedings/88/slides/slides-88-rmcat-9.pdf.
>
>
>
>
>  This might work however I don't know how this fits into the Sprout
> framework. Here you are talking about a direct relation between camera,
> encoder and congestion control algorithm (in case of Sprout the transport
> protocol) which might be needed for a transport congestion control
> algorithm to work. In RMCAT there have been discussions around this please
> join those discussion if possible.
>
>
> <[email-masked]
>
> <mailto:[email-masked]>> wrote:
>
>     Hi Keith,____
>
>     __ __
>
>
>
>     Phone +46 10 717 37 43 <tel:%2B46%2010%20717%2037%2043<%2B46%2010%20717%2037%2043>
> >
>     Mobile +46 76 115 37 43 <tel:%2B46%2076%20115%2037%2043<%2B46%2076%20115%2037%2043>
> >
>     Office +46 76 115 37 43 <tel:%2B46%2076%20115%2037%2043<%2B46%2076%20115%2037%2043>
> >
>     Fax +46 920 996 21 <tel:%2B46%20920%20996%2021<%2B46%20920%20996%2021>
> >
>     <mailto:[email-masked]>
>     www.ericsson.com <http://www.ericsson.com>____
>
>
>     *From:[email-masked] <mailto:[email-masked]>
>     [mailto:[email-masked] <mailto:[email-masked]>] *On Behalf
>     Of *Keith Winstein
>     *Cc:* [email-masked] <mailto:[email-masked]>; Hari Balakrishnan;
>     Anirudh Sivaraman
> SMS/MMS +46 76 115 37 43
> www.ericsson.com
> _______________________________________________
> iccrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/iccrg
>
>

---

From: "Robinson, Dave C (Dave)" <[email-masked]>
Date: Mon, 9 Dec 2013 10:31:36 +0000

Keith, Zahed,

The most common scheduler for Best Effort data traffic on QCI is "fair shares". That is, the eNodeB allocates an equal number of PRBs to the current flows with traffic to send. So in your example of 6 or 12 phones all transmitting, they should get an equal number of PRBs. However, this does not mean they will get the same bandwidth. This depends on the Signal to Interference and Noise Ratio (SINR). I.e the radio conditions will affect now many radio resources are needed to send one bit of information. Signal strength, reflections, etc. all play a part. In your late night tests, Interference from nearby cells will be minimal. During the daytime, however, this is not the case. And LTE schedulers go to lengths with allocating PRBs to minimise interference between two phones in adjacent cells.

Hope this helps you as you plan your LTE simulator.

Regards
    Dave

Sent from my iPad

On 6 Dec 2013, at 20:47, Keith Winstein <[email-masked]<mailto:[email-masked]>> wrote:

Zahed,

Thanks for this. I'd be very interested to see any results from sproutbt2 on your LTE simulator -- are you able to share?

I did not understand your reference to how our emulation strategy "gives you certain throughput which is based on certain data you send," and I'd be eager to learn anything that bears on the realism of this style of emulation. Could you expand a bit more on this?

We have measured the behavior of Verizon and AT&T LTE base stations in the middle of the night in response to increasing demand (we boot up 6-12 phones on the same cell ID, where there is unlikely to be much other demand, and start pounding away). From our measurements, the behavior of the base station appears simply to divide up its resource blocks equally among clients who have pending data at a given time.

That is, in practice the eNodeB appears to have a separate queue for each client that has pending data and divides the downlink among them. My UE might feel the affect of the _presence_ or _absence_ of another flow to another UE, but my UE does not have to care how many bits per second a server is trying to send to that UE at a given time. That's if our inferences about isolation between UEs are correct.

If this is true, the only thing that matters to a particular client is the number of resource blocks allocated to *that* client and the number of bits that can be sent in each RB -- which is essentially what our trace captures. So I'd of course be interested to learn about material feedback effects that we're not capturing.

Best regards,
Keith

On Tue, Dec 3, 2013 at 7:32 AM, Zaheduzzaman Sarker <[email-masked]<mailto:[email-masked]>> wrote:
HI Keith,

BR

Zahed

Multimedia Technologies

Ericsson

Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43<tel:%2B46%2010%20717%2037%2043>
Mobile +46 76 115 37 43<tel:%2B46%2076%20115%2037%2043>
Office +46 76 115 37 43<tel:%2B46%2076%20115%2037%2043>
Fax +46 920 996 21<tel:%2B46%20920%20996%2021>
[email-masked]<mailto:[email-masked]>
www.ericsson.com<http://www.ericsson.com>
===============================================================================

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Keith Winstein
Sent: den 16 november 2013 01:20
To: Zaheduzzaman Sarker
Cc: [email-masked]<mailto:[email-masked]>; Hari Balakrishnan; Anirudh Sivaraman; [email-masked]<mailto:[email-masked]>
Subject: Re: Presentation on Sprout at today's ICCRG

Best regards,
Keith

Sorry for late reply.

BR

Zahed

Best regards,
Keith

    Hi Keith,____

    __ __

    __ __

    __ __

    __ __

    BR____

    __ __

    Zahed____

    __ __

    *Ericsson*

    Laboratoriegränd 11
    97128 Luleå, Sweden

    __ __

    __ __

    Hello,____

    __ __

    __ __

    __ __

    Keith____

    __ __

--

Zahed

Ericsson AB
Multimedia Technologies

Laboratoriegränd 11
97128 Luleå, Sweden

_______________________________________________
iccrg mailing list
[email-masked]<mailto:[email-masked]>
https://www.irtf.org/mailman/listinfo/iccrg

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Thu, 12 Dec 2013 00:14:03 +0000

Hi Keith,

Please see inline below. Prefixed with [Zaheduzzaman Sarker]

I did not understand your reference to how our emulation strategy "gives you certain throughput which is based on certain data you send," and I'd be eager to learn anything that bears on the realism of this style of emulation. Could you expand a bit more on this?
[Zaheduzzaman Sarker] As you might already know that in LTE, base station allocates numbers of resource blocks to the user to send ( and receive) data. The allocation of such resource blocks also depends on current radio conditions. Now, If the size of the packets are big then it might require higher number of resource blocks to send or receive and if the packets are small it might require lower numbers of resource blocks. As you can see the same is true for how frequently the user have data to send.  Hence, depending on the characteristics of data (size big or small, frequency periodic or continuous) you will measure different link capacity irrespective of how queues are maintained in the base stations. This is what I wanted to mean with that particular line. Your data generation characteristics matters in what you capture in the tracefiles.
Another important aspect is that a LTE scheduler typically takes the historical behavior (e.g. historical rate) of the user into account when deciding scheduling weights. Since the historical rate depends on how much data that previously has been available for transmission you get a direct dependence between the momentary "throughput" and the data you feed into the system. Unless the eNB always is "saturated" with data you will get a cross-dependency between "data in" and the available bandwidth.

We have measured the behavior of Verizon and AT&T LTE base stations in the middle of the night in response to increasing demand (we boot up 6-12 phones on the same cell ID, where there is unlikely to be much other demand, and start pounding away). From our measurements, the behavior of the base station appears simply to divide up its resource blocks equally among clients who have pending data at a given time.
[Zaheduzzaman Sarker]  does this mean at a particular time all your phones measure same link capacity irrespective where they are and their movement?
Note that the bandwidth the user will measure will not only depend on the activities in the particular cell it is in but all depend on what is happening in other cells. At night as the other cells are 'unlikely to have much other demand', you don't have realistic interference and thus the traces are most likely to have smaller variation in bandwidth than you would measure at day time. At night you also get low traffic and low signaling those usually have higher priority. Again this will make the tracefilse at night not to match that of the day.

By the way how do you measure the resource blocks? Do you use special test mobiles?

That is, in practice the eNodeB appears to have a separate queue for each client that has pending data and divides the downlink among them. My UE might feel the affect of the _presence_ or _absence_ of another flow to another UE, but my UE does not have to care how many bits per second a server is trying to send to that UE at a given time. That's if our inferences about isolation between UEs are correct.
[Zaheduzzaman Sarker] The schedulers does not do UE "isolation" meaning static allocation of resources as that will be wasted if the UE don't have enough data to send. In reality the resource are allocated on TTI basis based on if there is data in the buffers, the historical rate etc. If number of UEs manages to keep the base station buffers saturated all the time it will result in a long term division of resources that depends only on radio quality perceived by each UE. Hence a mobile user should see different bandwidth when standing close to base station and at the cell edge.
For the same reason, in case of your measurement setup which is to saturate all the per UE base station buffers, the end  result (long term) will not depend on how many bits per second each UE receives from the server, but the assumption that you always have plenty of data in all the buffers. This is not the case with realistic traffic.

If this is true, the only thing that matters to a particular client is the number of resource blocks allocated to *that* client and the number of bits that can be sent in each RB -- which is essentially what our trace captures. So I'd of course be interested to learn about material feedback effects that we're not capturing.
[Zaheduzzaman Sarker] I think the way you are producing trace files and your inference on LTE queuing makes sense. However, there are problems when you try to use those traces in rate adaptation study. As a response to congestion the user will lower the bitrate resulting in releasing of resources. Unless there exists some sort of priority there is no guarantee  that other users will not see that an opportunity to use those resources. In a trace file you capture the current view of the link capacity of a particular user and the user reacts to fits it's current sending rate to that capacity. This adaption solves the problem of that user in the trace file based emulation as the link capacity remains static at that point. In reality, the link capacity may change depending on demand from users and other factor I discussed above. If you don't have some scheduling/resource management in your emulation you will not be able to capture what happens when user changes it sending rate i.e. the distribution of resource blocks when the users adapt. This is I think is missing in your emulation .

I hope this helps.

BR

Zahed

Ericsson
Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43
Mobile +46 76 115 37 43
Office +46 76 115 37 43
Fax +46 920 996 21

BR

Zahed

Multimedia Technologies

Ericsson

Laboratoriegränd 11
97128 Luleå, Sweden

Best regards,
Keith

Sorry for late reply.

BR

Zahed

Best regards,
Keith

    Hi Keith,____

    __ __

    __ __

    __ __

    __ __

    BR____

    __ __

    Zahed____

    __ __

    *Ericsson*

    Laboratoriegränd 11
    97128 Luleå, Sweden

    __ __

    __ __

    Hello,____

    __ __

    __ __

    __ __

    Keith____

    __ __

--

Zahed

Ericsson AB
Multimedia Technologies

Laboratoriegränd 11
97128 Luleå, Sweden

---

From: Zaheduzzaman Sarker <[email-masked]>
Date: Thu, 12 Dec 2013 00:16:53 +0000

Hi Dave,

Thanks and agreed.

BR

Zahed

Ericsson
Laboratoriegränd 11
97128 Luleå, Sweden
Phone +46 10 717 37 43
Mobile +46 76 115 37 43
Office +46 76 115 37 43
Fax +46 920 996 21

From: Robinson, Dave C (Dave) [mailto:[email-masked]]
Sent: den 9 december 2013 11:32
To: Keith Winstein
Cc: Zaheduzzaman Sarker; Hari Balakrishnan; [email-masked]; [email-masked]; Anirudh Sivaraman
Subject: Re: [iccrg] Presentation on Sprout at today's ICCRG

Keith, Zahed,

Regards
    Dave

Sent from my iPad

Best regards,
Keith

BR

Zahed

Multimedia Technologies

Ericsson

Laboratoriegränd 11
97128 Luleå, Sweden

Best regards,
Keith

Sorry for late reply.

BR

Zahed

Best regards,
Keith

    Hi Keith,____

    __ __

    __ __

    __ __

    __ __

    BR____

    __ __

    Zahed____

    __ __

    *Ericsson*

    Laboratoriegränd 11
    97128 Luleå, Sweden

    __ __

    __ __

    Hello,____

    __ __

    __ __

    __ __

    Keith____

    __ __

--

Zahed

Ericsson AB
Multimedia Technologies

Laboratoriegränd 11
97128 Luleå, Sweden