Subject: Re: [Banana] BOF Description and Agenda

From: <[email-masked]>
Date: Tue, 6 Dec 2016 15:37:39 +0000

Is it possible to send the minutes, even if they’re draft, please? Or a summary of next steps after the BoF. I wanted to catch up with what happened, as unfortunately I didn’t make it to Seoul
Thanks
phil

From: Banana [mailto:[email-masked]] On Behalf Of Margaret Cullen
Sent: 02 November 2016 10:59
To: [email-masked]
Cc: [email-masked]; Brian Trammell <[email-masked]>; Mirja Kühlewind <[email-masked]>; Suresh Krishnan <[email-masked]>
Subject: Re: [Banana] BOF Description and Agenda

Oh, it might have helped to include the description… :-)

- Name: BANdwidth Aggregation for interNet Access (BANANA)

- Description:
This BOF will discuss how we can take advantage of multiple access links, provided by one or more access providers, in cases where end nodes and applications may not be multi-access-aware.  Use of multiple access links could provide bandwidth aggregation when multiple links are available (i.e. improved performance), and session continuation when a link becomes unavailable (i.e. increased reliability).

Solutions to this problem are intended to apply to home and small-office networks, so solutions must support load-sharing of a small number of flows (in some cases only one) over multiple paths, recombining the traffic in ways that do not cause problems for the network (i.e. congestion, multi-level retransmission, etc.) or for upper-layer protocols (i.e. packet reordering within flows).  Solutions in this area must allow for some traffic to bypass bandwidth aggregation (e.g. for flows that are mandated to travel a specific path, or for flows that are already using a more end-to-end bandwidth aggregation solution).  Solutions should provide minimal (if any) performance degradation when multiple links are not available, or when one or more of the links is not performing sufficiently to support increased performance to the end nodes.

This is a Non-WG-forming BOF.  The purposes of the BOF are to discuss the problem space, and determine if there is sufficient interest and energy in the IETF to pursue any work in this area.  If the interest and energy exist, further effort may be exerted to scope and charter an IETF WG.

There has been an informal group, also called BANANA, looking at a subset of this space:  the case where multiple links are provided by a single Internet access provider/operator.  Multiple solutions that could apply to this subset have been discussed within that group, including:  a MIP-Based Solution (draft-ietf-dmm-mag-multihoming), an MPTCP Proxy-based solution (merged draft forthcoming), a GRE Tunnel Bonding Solution (draft-zhang-gre-tunnel-bonding), and a 3GPP-specific solution (draft-muley-bonding-solution-hybrid-access).  The MIP-based solution was developed in the DMM WG and is proposed for publication as a Standards Track RFC.  The GRE Tunnel Bonding solution, proposed for publication as an Independent Stream RFC, has been deployed by Deutsche Telecom in a large-scale operational network.  The MPTCP solution, though not yet fully documented, presents a promising, minimal-overhead solution, especially for situations where it is only necessary to share TCP traffic between multiple links.  There are significant differences between these proposals, resulting in different technical trade-offs, many of which have been explored in the BANANA Considerations Draft (draft-mrc-banana-considerations).  Some of the solutions that have been under discussion may apply (with or without additional signaling mechanisms) to the broader problem space where the access links may not be provided by a single operator.  This ongoing work will be discussed during the BOF, as it applies to the more general problem.

There has also been ongoing work in multiple other standards organizations to describe related problems.   The Broadband Forum has produced a “Hybrid Access” problem statement for DSL/LTE bandwidth aggregation specifically for the case where the DSL and LTE links are provided by a single operator (TR-348).  Documents from the other groups are not yet publicly available.

- Agenda:
0) Administrivia (5 mins)
1) BOF Scope and Problem Description (Chairs, 10 min)
2) BANANA Solution Space & Experiences
   - GRE Tunnel Bonding
                - Solution Overview (Mingui Zhang, 5 min)
                - Deployment Experience (Nicolai Leymann, 5 min)
   - MPTCP Proxy (??, 10 min)
   - Others?
3) Related Research
   - Multipath Bonding at Layer 3 (Brian Trammel, 10 min)
4) Related Work in Other SDOs
   - Broadband Forum TR-348 (Dave Allan, 5 mins)
   - Others?
5) Open Discussion (30 mins)
6) Questions & Next Steps (Chairs & ADs, 10 mins)

- Status: Non-WG Forming
- Responsible AD: Suresh Krishnan <[email-masked]<mailto:[email-masked]>>
- BoF proponents: Mingui Zhang <[email-masked]<mailto:[email-masked]>>, Nicolai Leymann <[email-masked]<mailto:[email-masked]>>, Margaret Cullen <[email-masked]<mailto:[email-masked]>>
- BoF chairs: Brian Trammel <[email-masked]<mailto:[email-masked]>>, Margaret Cullen <[email-masked]<mailto:[email-masked]>>
- Number of people expected to attend: 100
- Length of session (1, 1.5, 2, or 2.5 hours): 1.5 hours
- Conflicts to avoid (whole Areas and/or WGs): Entire INT Area, Entire TSV area, TSVAREA, INTAREA WG, MPTCP, DMM, TRILL, HOMENET, BABEL, CAPPORT
- Mailing List: https://www.ietf.org/mailman/listinfo/banana
- Draft charter:  None

- Relevant drafts/documents:
https://datatracker.ietf.org/doc/draft-zhang-banana-problem-statement
https://datatracker.ietf.org/doc/draft-mrc-banana-considerations
https://datatracker.ietf.org/doc/draft-zhang-gre-tunnel-bonding
https://datatracker.ietf.org/doc/draft-boucadair-mptcp-plain-mode
https://datatracker.ietf.org/doc/draft-peirens-mptcp-transparent
https://datatracker.ietf.org/doc/draft-ietf-dmm-mag-multihoming-02
https://www.ietf.org/staging/draft-muley-bonding-solution-hybrid-access-00.txt
https://irtf.org/anrw/2016/anrw16-final21.pdf
https://www.broadband-forum.org/technical/download/TR-348.pdf

On Nov 2, 2016, at 6:57 AM, Margaret Cullen <[email-masked]<mailto:[email-masked]>> wrote:

Hi All,

Here is the current BOF description and agenda (which will be posted in the next few minutes) for our upcoming BANANA BOF.

As you can see, there is room in the agenda for someone to do a short presentation on the state of the MPTCP solution(s) that are applicable to this problem space, however a presenter has not been assigned.  We know that Olivier will not be in Seoul.  Will any of the advocates of an MPTCP solution (Med, perhaps?) be in Seoul and willing to present a quick overview?

Also, please let us know if you have any comments on the description or would like to add anything to the agenda.

Margaret

---

From: Brian Trammell <[email-masked]>
Date: Tue, 6 Dec 2016 16:40:48 +0100

hi Phil,

These are in my queue for completion last Friday (as you can see, I'm running a bit behind), along with kicking off the "next steps" discussion that we identified needs to happen on the list.

(In the meantime, those who came up to the microphone in Seoul who stated interest in solving a problem in this space: please be prepared draft a paragraph or two on what you think the important parts of the problem to solve are, to send to the list :) )

Cheers,

Brian

> On 06 Dec 2016, at 16:37, <[email-masked]> <[email-masked]> wrote:
> 
> - Responsible AD: Suresh Krishnan <[email-masked]>
> - BoF proponents: Mingui Zhang <[email-masked]>, Nicolai Leymann <[email-masked]>, Margaret Cullen <[email-masked]>
> - BoF chairs: Brian Trammel <[email-masked]>, Margaret Cullen <[email-masked]>
> On Nov 2, 2016, at 6:57 AM, Margaret Cullen <[email-masked]> wrote:
> 
> Hi All,
> 
> _______________________________________________
> Banana mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/banana

---

From: Brian Trammell <[email-masked]>
Date: Wed, 7 Dec 2016 17:56:54 +0100

Hello, Banana!

I've posted draft minutes at https://datatracker.ietf.org/doc/minutes-97-banana/ -- please send corrections/clarifications to the chairs (I suppose [email-masked] works for this?)

Another mail framing the problem statement discussion to follow shortly (for some value of shortly), but there's no need to wait for the chairs to kick off the discussion. :)

Cheers,

Brian

> On 06 Dec 2016, at 16:40, Brian Trammell <[email-masked]> wrote:
> 
> hi Phil,
>

---

From: Mingui Zhang <[email-masked]>
Date: Thu, 8 Dec 2016 03:05:18 +0000

Hi,

From our discussions, I observe the measurement feedback loop is a very good thing to be standardized. The information about latency, loss rate, connection state, etc. exposed by the feedback could be used (e.g., as the input into the load distribution function at the ingress). Flow control and congestion control can fit into this loop. It has been well motivated since users/providers are able to see higher throughput & reliability out of the bandwidth aggregation with this measurement-feedback loop. I would be happy to contribute to it.

Thanks,
Mingui

> -----Original Message-----
> From: Banana [mailto:[email-masked]] On Behalf Of Brian Trammell
> Sent: Thursday, December 08, 2016 12:57 AM
> To: [email-masked]
> I've posted draft minutes at
> https://datatracker.ietf.org/doc/minutes-97-banana/ -- please send
> corrections/clarifications to the chairs (I suppose [email-masked]
> works for this?)
> 
> Another mail framing the problem statement discussion to follow shortly (for
> some value of shortly), but there's no need to wait for the chairs to kick off the
> discussion. :)
> 
> Cheers,
> 
> Brian
> 
> > These are in my queue for completion last Friday (as you can see, I'm running a
> bit behind), along with kicking off the "next steps" discussion that we identified
> needs to happen on the list.
> >
> > (In the meantime, those who came up to the microphone in Seoul who
> > stated interest in solving a problem in this space: please be prepared
> > draft a paragraph or two on what you think the important parts of the
> > problem to solve are, to send to the list :) )
> >
> > Cheers,
> >
> > Brian
> >
> >> Is it possible to send the minutes, even if they’re draft, please? Or
> >> a summary of next steps after the BoF. I wanted to catch up with what
> >> happened, as unfortunately I didn’t make it to Seoul Thanks phil
> >>
> >> Cc: [email-masked]; Brian Trammell <[email-masked]>;
> >> Mirja Kühlewind <[email-masked]>; Suresh Krishnan
> >> <[email-masked]>
> >> This BOF will discuss how we can take advantage of multiple access links,
> provided by one or more access providers, in cases where end nodes and
> applications may not be multi-access-aware.  Use of multiple access links could
> provide bandwidth aggregation when multiple links are available (i.e. improved
> performance), and session continuation when a link becomes unavailable (i.e.
> increased reliability).
> >>
> >> Solutions to this problem are intended to apply to home and small-office
> networks, so solutions must support load-sharing of a small number of flows (in
> some cases only one) over multiple paths, recombining the traffic in ways that do
> not cause problems for the network (i.e. congestion, multi-level retransmission,
> etc.) or for upper-layer protocols (i.e. packet reordering within flows).  Solutions
> in this area must allow for some traffic to bypass bandwidth aggregation (e.g. for
> flows that are mandated to travel a specific path, or for flows that are already
> using a more end-to-end bandwidth aggregation solution).  Solutions should
> provide minimal (if any) performance degradation when multiple links are not
> available, or when one or more of the links is not performing sufficiently to
> support increased performance to the end nodes.
> >>
> >> This is a Non-WG-forming BOF.  The purposes of the BOF are to discuss the
> problem space, and determine if there is sufficient interest and energy in the
> IETF to pursue any work in this area.  If the interest and energy exist, further
> effort may be exerted to scope and charter an IETF WG.
> >>
> >> There has been an informal group, also called BANANA, looking at a subset of
> this space:  the case where multiple links are provided by a single Internet
> access provider/operator.  Multiple solutions that could apply to this subset
> have been discussed within that group, including:  a MIP-Based Solution
> (draft-ietf-dmm-mag-multihoming), an MPTCP Proxy-based solution (merged
> draft forthcoming), a GRE Tunnel Bonding Solution
> (draft-zhang-gre-tunnel-bonding), and a 3GPP-specific solution
> (draft-muley-bonding-solution-hybrid-access).  The MIP-based solution was
> developed in the DMM WG and is proposed for publication as a Standards Track
> RFC.  The GRE Tunnel Bonding solution, proposed for publication as an
> Independent Stream RFC, has been deployed by Deutsche Telecom in a
> large-scale operational network.  The MPTCP solution, though not yet fully
> documented, presents a promising, minimal-overhead solution, especially for
> situations where it is only necessary to share TCP traffic between multiple links.
> There are significant differences between these proposals, resulting in different
> technical trade-offs, many of which have been explored in the BANANA
> Considerations Draft (draft-mrc-banana-considerations).  Some of the solutions
> that have been under discussion may apply (with or without additional signaling
> mechanisms) to the broader problem space where the access links may not be
> provided by a single operator.  This ongoing work will be discussed during the
> BOF, as it applies to the more general problem.
> >>
> >> There has also been ongoing work in multiple other standards organizations to
> describe related problems.   The Broadband Forum has produced a “Hybrid
> Access” problem statement for DSL/LTE bandwidth aggregation specifically for
> the case where the DSL and LTE links are provided by a single operator (TR-348).
> Documents from the other groups are not yet publicly available.
> >>
> >> - Agenda:
> >> - BoF proponents: Mingui Zhang <[email-masked]>, Nicolai
> >> Leymann <[email-masked]>, Margaret Cullen
> >> <[email-masked]>
> >> - Conflicts to avoid (whole Areas and/or WGs): Entire INT Area,
> >> Entire TSV area, TSVAREA, INTAREA WG, MPTCP, DMM, TRILL, HOMENET,
> >> BABEL, CAPPORT
> >> https://www.ietf.org/staging/draft-muley-bonding-solution-hybrid-acce
> >> ss-00.txt https://irtf.org/anrw/2016/anrw16-final21.pdf
> >> Here is the current BOF description and agenda (which will be posted in the
> next few minutes) for our upcoming BANANA BOF.
> >>
> >> As you can see, there is room in the agenda for someone to do a short
> presentation on the state of the MPTCP solution(s) that are applicable to this
> problem space, however a presenter has not been assigned.  We know that
> Olivier will not be in Seoul.  Will any of the advocates of an MPTCP solution
> (Med, perhaps?) be in Seoul and willing to present a quick overview?
> >>
> >> Also, please let us know if you have any comments on the description or
> would like to add anything to the agenda.
> >>
> >> Margaret
> >>
> >>
> >>
> >>

---

From: Brian Trammell <[email-masked]>
Date: Mon, 12 Dec 2016 13:57:37 +0100

hi Mingui,

A clarification question and (I think) agreement, below...

> On 08 Dec 2016, at 04:05, Mingui Zhang <[email-masked]> wrote:
> 
> Hi,
> 
> From our discussions, I observe the measurement feedback loop is a very good thing to be standardized. The information about latency, loss rate, connection state, etc. exposed by the feedback could be used (e.g., as the input into the load distribution function at the ingress).

The measurement feedback loop you envision here is sent from the bonding tunnel egress back to the bonding tunnel ingress, correct?

> Flow control and congestion control can fit into this loop. It has been well motivated since users/providers are able to see higher throughput & reliability out of the bandwidth aggregation with this measurement-feedback loop. I would be happy to contribute to it.

This seems reasonable, though I suspect there is some OAM work in the routing area (with which I have only passing familiarity) that could be (re-)used here.

Cheers,

Brian

---

From: Mingui Zhang <[email-masked]>
Date: Tue, 13 Dec 2016 07:55:34 +0000

Hi Brian,

> The measurement feedback loop you envision here is sent from the bonding
> tunnel egress back to the bonding tunnel ingress, correct?

Yes, one-way measurements are on the table. But we should not rule out the round-trip measurements which are frequently used in practice. 
I imagine the loop is more about how the measurements impact the load distribution which in turn impacts the measurements result. 

> This seems reasonable, though I suspect there is some OAM work in the routing
> area (with which I have only passing familiarity) that could be (re-)used here.

I agree. There is a parade of OAM work (Y1731, OWAMP& TWAMP defined by IPPM WG, etc) that can be reused to execute measurements. However, Service Providers are more concerned about making use of the measurement result to do traffic re-distribution. RFC 7424 can be looked as a related work. There, utilizations of component links are monitored and rebalance is performed when the imbalance of the utilizations exceed a certain threshold. Note that banana requires packet-based load balancing and cares more about throughput so that RFC 7424 is not applicable here. 

Thanks,
Mingui

> From: Brian Trammell [mailto:[email-masked]]
> Sent: Monday, December 12, 2016 8:58 PM
> To: Mingui Zhang
> Cc: [email-masked]
> > From our discussions, I observe the measurement feedback loop is a very good
> thing to be standardized. The information about latency, loss rate, connection
> state, etc. exposed by the feedback could be used (e.g., as the input into the
> load distribution function at the ingress).
> 
> The measurement feedback loop you envision here is sent from the bonding
> tunnel egress back to the bonding tunnel ingress, correct?
> 
> > Flow control and congestion control can fit into this loop. It has been well
> motivated since users/providers are able to see higher throughput & reliability
> out of the bandwidth aggregation with this measurement-feedback loop. I would
> be happy to contribute to it.
> 
> This seems reasonable, though I suspect there is some OAM work in the routing
> area (with which I have only passing familiarity) that could be (re-)used here.
> 
> Cheers,
> 
> Brian
> 
> > Thanks,
> > Mingui
> >
> >> corrections/clarifications to the chairs (I suppose
> >> [email-masked] works for this?)
> >>
> >> Another mail framing the problem statement discussion to follow
> >> shortly (for some value of shortly), but there's no need to wait for
> >> the chairs to kick off the discussion. :)
> >>
> >> Cheers,
> >>
> >> Brian
> >>
> >> bit behind), along with kicking off the "next steps" discussion that
> >> we identified needs to happen on the list.
> >>>
> >>> stated interest in solving a problem in this space: please be
> >>> prepared draft a paragraph or two on what you think the important
> >>> parts of the problem to solve are, to send to the list :) )
> >>>
> >>> Cheers,
> >>>
> >>> Brian
> >>>
> >>>> On 06 Dec 2016, at 16:37, <[email-masked]>
> >>>> <[email-masked]>
> >> wrote:
> >>>>
> >>>> Is it possible to send the minutes, even if they’re draft, please?
> >>>> Or a summary of next steps after the BoF. I wanted to catch up with
> >>>> what happened, as unfortunately I didn’t make it to Seoul Thanks
> >>>> phil
> >>>>
> >>>> Cc: [email-masked]; Brian Trammell
> >>>> <[email-masked]>; Mirja Kühlewind
> >>>> <[email-masked]>; Suresh Krishnan
> >> provided by one or more access providers, in cases where end nodes
> >> and applications may not be multi-access-aware.  Use of multiple
> >> access links could provide bandwidth aggregation when multiple links
> >> are available (i.e. improved performance), and session continuation when a
> link becomes unavailable (i.e.
> >> increased reliability).
> >>>>
> >> networks, so solutions must support load-sharing of a small number of
> >> flows (in some cases only one) over multiple paths, recombining the
> >> traffic in ways that do not cause problems for the network (i.e.
> >> congestion, multi-level retransmission,
> >> etc.) or for upper-layer protocols (i.e. packet reordering within
> >> flows).  Solutions in this area must allow for some traffic to bypass
> >> bandwidth aggregation (e.g. for flows that are mandated to travel a
> >> specific path, or for flows that are already using a more end-to-end
> >> bandwidth aggregation solution).  Solutions should provide minimal
> >> (if any) performance degradation when multiple links are not
> >> problem space, and determine if there is sufficient interest and
> >> energy in the IETF to pursue any work in this area.  If the interest
> >> and energy exist, further effort may be exerted to scope and charter an IETF
> WG.
> >>>>
> >> this space:  the case where multiple links are provided by a single
> >> Internet access provider/operator.  Multiple solutions that could
> >> apply to this subset have been discussed within that group,
> >> including:  a MIP-Based Solution (draft-ietf-dmm-mag-multihoming), an
> >> MPTCP Proxy-based solution (merged draft forthcoming), a GRE Tunnel
> >> Bonding Solution (draft-zhang-gre-tunnel-bonding), and a
> >> 3GPP-specific solution (draft-muley-bonding-solution-hybrid-access).
> >> The MIP-based solution was developed in the DMM WG and is proposed
> >> for publication as a Standards Track RFC.  The GRE Tunnel Bonding
> >> solution, proposed for publication as an Independent Stream RFC, has
> >> been deployed by Deutsche Telecom in a large-scale operational
> >> network.  The MPTCP solution, though not yet fully documented,
> >> presents a promising, minimal-overhead solution, especially for situations
> where it is only necessary to share TCP traffic between multiple links.
> >> There are significant differences between these proposals, resulting
> >> in different technical trade-offs, many of which have been explored
> >> in the BANANA Considerations Draft (draft-mrc-banana-considerations).
> >> Some of the solutions that have been under discussion may apply (with
> >> or without additional signaling
> >> mechanisms) to the broader problem space where the access links may
> >> not be provided by a single operator.  This ongoing work will be
> >> discussed during the BOF, as it applies to the more general problem.
> >>>>
> >> Access” problem statement for DSL/LTE bandwidth aggregation
> >> specifically for the case where the DSL and LTE links are provided by a single
> operator (TR-348).
> >>>> https://datatracker.ietf.org/doc/draft-zhang-banana-problem-stateme
> >>>> nt https://datatracker.ietf.org/doc/draft-mrc-banana-considerations
> >>>> https://www.ietf.org/staging/draft-muley-bonding-solution-hybrid-ac
> >>>> ce ss-00.txt https://irtf.org/anrw/2016/anrw16-final21.pdf
> >> presentation on the state of the MPTCP solution(s) that are
> >> applicable to this problem space, however a presenter has not been
> >> assigned.  We know that Olivier will not be in Seoul.  Will any of
> >> the advocates of an MPTCP solution (Med, perhaps?) be in Seoul and willing
> to present a quick overview?
> >>>>