Subject: [Banana] Charter proposal

From: Mingui Zhang <[email-masked]>
Date: Tue, 13 Sep 2016 07:48:30 +0000

Folks,

In order to set up a WG, we need a charter proposal. Based on the bar-BoF meetings and emails we've ever had on BANANA, I put together the following Charter of BANANA for discussion. Please read it and comment on it.

<CHARTER BEGINS>
Charter of BANANA

Bandwidth aggregation capabilities for Internet access services can significantly improve end user's Quality of Experience. Such capabilities are especially attractive in environments where multi-interfaced boxes become commonplace and can technically connect to various access networks, both wired and wireless.

The purpose of the BANANA (Bandwidth Aggregation for interNet Access) WG is to develop a set of protocols that enable the access to the Internet through a bond of heterogeneous links which are all connected to an aggregation device. The access bandwidth should be apparently increased in comparison with the case when those links are separately used. The peer to peer latency between the access device and aggregation device should be under an acceptable level. The aggregation device in question should support concurrent connections from tens of thousands of access devices. 

For the considered BANANA use cases, access devices are possessed by end users therefore very few concurrent flows will be produced by each access device. Traditional per-flow load balancing solutions do not help much. For this reason, the WG will only conduct per-packet load balancing methods. At the egress, packet disorder is likely to happen due to the per-packet load splitting therefore packet reordering is required to be supported. 

A control plane will be developed to support the exchange of IP addresses (both IPv4 and IPv6) between the peering access and aggregation devices. The access and aggregation devices rely on this control plane to automatically setup/destroy the connections between them. The control plane will also support the measurement and report of measured parameters, e.g., bypassing traffic volume.

The WG will select, extend, and/or develop one or more data plane encapsulation format(s).

The WG will develop or adopt a mechanism of dynamically distributing traffic load onto the multiple connections. This traffic distribution mechanism will take the measurement of the connections as its input. The parameters to be measured may include the congestion level, the latency as well as the state of the connection. When a connection is failed or exhibiting bad quality, attached access or aggregation devices should refrain from distributing traffic onto this connection or even deactivate the whole bonding mode in order to exclude this connection. For the possible route oscillation issue in the measurement time, the WG will select, extend and/or develop counter-measurement(s). In pursuit of the traffic distribution mechanism, existing dynamic routing methods, which have been proved to be effective, would be investigated and employed by the WG. 

The WG will work on network-based solutions while host-based solutions are out of the question.
<CHARTER ENDS>

Cheers,
Mingui

---

From: Michael Menth <[email-masked]>
Date: Tue, 13 Sep 2016 10:26:50 +0200

Hi Mingui,

thanks for your efforts in setting up an agenda.

Am 13.09.2016 um 09:48 schrieb Mingui Zhang:
> For the considered BANANA use cases, access devices are possessed by end users therefore very few concurrent flows will be produced by each access device. Traditional per-flow load balancing solutions do not help much. For this reason, the WG will only conduct per-packet load balancing methods. 
I think under some conditions packet-based solutions may be inefficient
so that flow-based could be valuable and should not be ruled out from
the beginning.

At the egress, packet disorder is likely to happen due to the per-packet
load splitting therefore packet reordering is required to be supported.
> The WG will develop or adopt a mechanism of dynamically distributing traffic load onto the multiple connections. This traffic distribution mechanism will take the measurement of the connections as…
This text seems to have some idea in mind that I am not aware of. I
think it is too detailed for an initial charter and should be rephrased.

> The WG will work on network-based solutions while host-based solutions are out of the question.

Regards,

Michael

> 
> Cheers,
> Mingui 
> 
> _______________________________________________
> Banana mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/banana
> 

-- 
Prof. Dr. habil. Michael Menth
University of Tuebingen
Faculty of Science
Department of Computer Science
Chair of Communication Networks
Sand 13, 72076 Tuebingen, Germany
phone: (+49)-7071/29-70505
fax: (+49)-7071/29-5220
mailto:[email-masked]
http://kn.inf.uni-tuebingen.de

---

From: <[email-masked]>
Date: Tue, 13 Sep 2016 08:33:39 +0000

> -----Message d'origine-----
> De : Banana [mailto:[email-masked]] De la part de Michael Menth
> Envoyé : mardi 13 septembre 2016 10:27
> À : [email-masked]
> Objet : Re: [Banana] Charter proposal
> 
> Hi Mingui,
> 
> > In order to set up a WG, we need a charter proposal. Based on the bar-BoF
> meetings and emails we've ever had on BANANA, I put together the following
> Charter of BANANA for discussion. Please read it and comment on it.
> >
> > <CHARTER BEGINS>
> > Charter of BANANA
> >
> > Bandwidth aggregation capabilities for Internet access services can significantly
> improve end user's Quality of Experience. Such capabilities are especially
> attractive in environments where multi-interfaced boxes become commonplace and
> can technically connect to various access networks, both wired and wireless.
> >
> > The purpose of the BANANA (Bandwidth Aggregation for interNet Access) WG is
> to develop a set of protocols that enable the access to the Internet through a bond
> of heterogeneous links which are all connected to an aggregation device. The
> access bandwidth should be apparently increased in comparison with the case
> when those links are separately used. The peer to peer latency between the access
> device and aggregation device should be under an acceptable level. The
> aggregation device in question should support concurrent connections from tens
> of thousands of access devices.
> >
> > For the considered BANANA use cases, access devices are possessed by end
> users therefore very few concurrent flows will be produced by each access device.
> Traditional per-flow load balancing solutions do not help much. For this reason, the
> WG will only conduct per-packet load balancing methods.
> I think under some conditions packet-based solutions may be inefficient so that
> flow-based could be valuable and should not be ruled out from the beginning.
> 
> At the egress, packet disorder is likely to happen due to the per-packet load
> splitting therefore packet reordering is required to be supported.
> >
> > A control plane will be developed to support the exchange of IP addresses (both
> IPv4 and IPv6) between the peering access and aggregation devices. The access
> and aggregation devices rely on this control plane to automatically setup/destroy
> the connections between them. The control plane will also support the
> measurement and report of measured parameters, e.g., bypassing traffic volume.
> >
> > The WG will develop or adopt a mechanism of dynamically distributing traffic
> load onto the multiple connections. This traffic distribution mechanism will take the
> measurement of the connections as its input. The parameters to be measured may
> include the congestion level, the latency as well as the state of the connection.
> When a connection is failed or exhibiting bad quality, attached access or
> aggregation devices should refrain from distributing traffic onto this connection or
> even deactivate the whole bonding mode in order to exclude this connection. For
> the possible route oscillation issue in the measurement time, the WG will select,
> extend and/or develop counter-measurement(s). In pursuit of the traffic distribution
> mechanism, existing dynamic routing methods, which have been proved to be
> effective, would be investigated and employed by the WG.

> This text seems to have some idea in mind that I am not aware of. I think it is too
> detailed for an initial charter and should be rephrased.
> 

I agree. The way to distribute traffic is implementation specific and should be out the scope. This part should be removed.

Regards,
pierrick

> > The WG will work on network-based solutions while host-based solutions are out

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Mingui Zhang <[email-masked]>
Date: Wed, 14 Sep 2016 07:07:24 +0000

Hi Michael,

Thanks for your comments! Please see the response inline.

> -----Original Message-----
> From: Banana [mailto:[email-masked]] On Behalf Of Michael Menth
> Sent: Tuesday, September 13, 2016 4:27 PM
> To: [email-masked]
> Subject: Re: [Banana] Charter proposal
> 
> Hi Mingui,
> 
> attractive in environments where multi-interfaced boxes become commonplace
> and can technically connect to various access networks, both wired and wireless.
> >
> > The purpose of the BANANA (Bandwidth Aggregation for interNet Access) WG
> is to develop a set of protocols that enable the access to the Internet through a
> bond of heterogeneous links which are all connected to an aggregation device.
> The access bandwidth should be apparently increased in comparison with the
> case when those links are separately used. The peer to peer latency between
> the access device and aggregation device should be under an acceptable level.
> The aggregation device in question should support concurrent connections from
> tens of thousands of access devices.
> >
> users therefore very few concurrent flows will be produced by each access
> device. Traditional per-flow load balancing solutions do not help much. For this
> reason, the WG will only conduct per-packet load balancing methods.

[Mingui] The problem spaces of flow-based and packet-based are quite different. For example, flow based methods would not create the packet reorder issue. The major use cases require per-packet solutions therefore I am afraid flow-based will not be strongly driven. In addition, flow-based seems already well supported by existing implementations. Is there much standardization work that need to be done?

> > A control plane will be developed to support the exchange of IP addresses
> (both IPv4 and IPv6) between the peering access and aggregation devices. The
> access and aggregation devices rely on this control plane to automatically
> setup/destroy the connections between them. The control plane will also
> support the measurement and report of measured parameters, e.g., bypassing
> traffic volume.
> >
> load onto the multiple connections. This traffic distribution mechanism will take
> the measurement of the connections as its input. The parameters to be
> measured may include the congestion level, the latency as well as the state of
> the connection. When a connection is failed or exhibiting bad quality, attached
> access or aggregation devices should refrain from distributing traffic onto this
> connection or even deactivate the whole bonding mode in order to exclude this
> connection. For the possible route oscillation issue in the measurement time, the
> WG will select, extend and/or develop counter-measurement(s). In pursuit of
> the traffic distribution mechanism, existing dynamic routing methods, which have
> been proved to be effective, would be investigated and employed by the WG.

[Mingui] Yes, it really looks too detailed for the current start-up stage. Let's think how to make it simple.

Regards,
Mingui

---

From: Mingui Zhang <[email-masked]>
Date: Wed, 14 Sep 2016 07:07:43 +0000

Hi Pierrick, 

> the scope. This part should be removed.

Yes, the way to distribute traffic (e.g., round-robin, coloring method) is a local implementation issue. However, I believe we should retain the part of measurement. I will rephrase this paragraph.

Thanks,
Mingui

> From: Banana [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: Tuesday, September 13, 2016 4:34 PM
> To: Michael Menth; [email-masked]
> > De : Banana [mailto:[email-masked]] De la part de Michael
> > Menth Envoyé : mardi 13 septembre 2016 10:27 À : [email-masked] Objet
> > : Re: [Banana] Charter proposal
> >
> > Hi Mingui,
> >
> > meetings and emails we've ever had on BANANA, I put together the
> > following Charter of BANANA for discussion. Please read it and comment on it.
> > >
> > > <CHARTER BEGINS>
> > > Charter of BANANA
> > >
> > improve end user's Quality of Experience. Such capabilities are
> > especially attractive in environments where multi-interfaced boxes
> > become commonplace and can technically connect to various access networks,
> both wired and wireless.
> > >
> > to develop a set of protocols that enable the access to the Internet
> > through a bond of heterogeneous links which are all connected to an
> > aggregation device. The access bandwidth should be apparently
> > increased in comparison with the case when those links are separately
> > used. The peer to peer latency between the access device and
> > aggregation device should be under an acceptable level. The
> > aggregation device in question should support concurrent connections from
> device.
> > Traditional per-flow load balancing solutions do not help much. For
> > this reason, the WG will only conduct per-packet load balancing methods.
> > I think under some conditions packet-based solutions may be
> > inefficient so that flow-based could be valuable and should not be ruled out
> from the beginning.
> >
> > At the egress, packet disorder is likely to happen due to the
> > per-packet load splitting therefore packet reordering is required to be
> supported.
> > >
> > IPv4 and IPv6) between the peering access and aggregation devices. The
> > access and aggregation devices rely on this control plane to
> > automatically setup/destroy the connections between them. The control
> > plane will also support the measurement and report of measured parameters,
> e.g., bypassing traffic volume.
> > >
> > load onto the multiple connections. This traffic distribution
> > mechanism will take the measurement of the connections as its input.
> > The parameters to be measured may include the congestion level, the latency
> as well as the state of the connection.
> > When a connection is failed or exhibiting bad quality, attached access
> > or aggregation devices should refrain from distributing traffic onto
> > this connection or even deactivate the whole bonding mode in order to
> > exclude this connection. For the possible route oscillation issue in
> > the measurement time, the WG will select, extend and/or develop
> > counter-measurement(s). In pursuit of the traffic distribution
> > > The WG will work on network-based solutions while host-based
> > > solutions are out
> > of the question.
> > > <CHARTER ENDS>
> >
> > Regards,
> >
> > Michael
> >
> > >
> > > Cheers,
> > > Mingui
> > >
> _____________________________________________________________
> ____________________________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou
> copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le
> signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration, Orange decline toute
> responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or privileged
> information that may be protected by law; they should not be distributed, used
> or copied without authorisation.

---

From: Michael Menth <[email-masked]>
Date: Wed, 14 Sep 2016 09:43:38 +0200

Hi Mingui,

> [Mingui] The problem spaces of flow-based and packet-based are quite different. For example, flow based methods would not create the packet reorder issue. The major use cases require per-packet sol…
I'm not sure. I just believe there are situation where packet-based load
balancing may not very effective so that a fallback on flow-based load
balancing may be helpful. Maybe such functionality can be integrated in
a common framework. We haven't looked into details.

Regards,

Michael

---

From: Margaret Cullen <[email-masked]>
Date: Tue, 20 Sep 2016 20:42:52 -0400

> On Sep 14, 2016, at 3:43 AM, Michael Menth <[email-masked]> wrote:
> 
> Hi Mingui,
> 

I agree with both of you…  :-)

Mingui, I think it would be useful to separate the “problem space” from the “solution space”.  While there are distinct trade-offs between packet-based and flow-based approaches, it is possible that there are parts of the problem space that could be served by either approach. 

Also, as both of you have pointed out, it is possible that a packet-based tunnel and a flow-based solution could be unified at other layers.

Personally, I would prefer to see this work put in a single group or area, so that we can come up with a more unified set of solutions, however I realize that there are some people who don’t agree with that.

I’m thinking that we should propose a BOF and a WG to work on the problem space (including an IETF problem statement combining the problems of various groups, and a considerations document), and work with the ADs to decide whether the MP-TCP solution work and the tunneling solution should be done in a single group, or separate groups.

Margaret

---

From: Mingui Zhang <[email-masked]>
Date: Thu, 22 Sep 2016 07:30:32 +0000

Hi Margaret,

It’s a good point to “separate the “problem space” from the “solution space”. Okay. I will revise the charter not to rule out per-flow solutions.

Thanks,
Mingui
From: Margaret Cullen [mailto:[email-masked]]
Sent: Wednesday, September 21, 2016 8:43 AM
To: Michael Menth
Cc: Mingui Zhang; [email-masked]
Subject: Re: [Banana] Charter proposal

On Sep 14, 2016, at 3:43 AM, Michael Menth <[email-masked]<mailto:[email-masked]>> wrote:

Hi Mingui,

Margaret