Subject: Re: [Banana] BANANA BOF in Prague?

From: "Zhangmingui (Martin)" <[email-masked]>
Date: Tue, 4 Jul 2017 07:31:03 +0000

Hi Mirja,

Thanks for the formulation of the problem. Compared to the non-aggregation case, the load balancer does become an extra tool that can be used by the ingress to react to the downstream congestion status. I think the problem and thoughts we are having here deserve a memo document. 

Thanks,
Mingui

________________________________________
From: Mirja Kühlewind [[email-masked]]
Sent: Saturday, July 01, 2017 0:08
To: Zhangmingui (Martin)
Cc: Brian Trammell (IETF); Margaret Cullen; [email-masked]; Dave Dolson
Subject: Re: [Banana] BANANA BOF in Prague?

Actually, you can potentially use ECN for the tunnel only, as long as you have a way to feedback the congestion information to the tunnel ingress and the ingress node has a way to react to the feedback appropriately, e.g. moving traffic from the congestion path to another path. If you use this for the tunnel, you probably even don’t want the endpoint to react as well because you just moved to a bigger link and then the endpoint reducing it’s rate doesn’t make any sense anymore. However, I case all available link are congestion and you can’t react at the ingress anymore, you need to provide feedback to the sending endpoint. In this case the tunnel ingress could mark the inner IP packets if ECN-enabled or drop some packets. So it’s not super easy but probably worth looking into.

Mirja

> Am 30.06.2017 um 10:58 schrieb Zhangmingui (Martin) <[email-masked]>:
>
> Right. Only if all routers carrying the tunnel support ECN. This is a prerequisite.
>
> One issue that might be also interesting: who is the "sender"? We know ECN is to indicate the status of the path so that the TCP sender, which is actually the host, can adjust its sending rate accordingly. But, the sender mentioned here is one of the two BANANA boxes. The load balancer of the BANANA box can determine proportion of the traffic transmitted on each tunnel rather that the overall traffic sending rate. The delivery and operations with the ECN bits are protocol issues while the load balancing strategy based on the ECN bits is local and seems more of an implementation issue.
>
> Thanks,
> Mingui
>
>> -----Original Message-----
>> From: Brian Trammell (IETF) [mailto:[email-masked]]
>> Sent: Thursday, June 29, 2017 7:06 PM
>> To: Zhangmingui (Martin)
>> Cc: Dave Dolson; Margaret Cullen; [email-masked]
>> Subject: Re: [Banana] BANANA BOF in Prague?
>>
>> There's also an issue of the network carrying the tunnel having impaired
>> support for the ECN codepoints -- this is often because of networks configured
>> to use the entire ToS byte for forwarding decisions, rather than the 6 dscp bits.
>> So even if both BANANA endpoints support ECN, they might need to do
>> cooperative network capability sensing to deal with these networks. Though
>> this isn't negotiation per se, it uses equivalent mechanisms.
>>
>> Cheers,
>>
>> Brian (as individual)
>>
>>
>>> On 29 Jun 2017, at 04:51, Zhangmingui (Martin) <[email-masked]>
>> wrote:
>>>
>>> Hi Dave,
>>>
>>> For bonding tunnels to be specified by the BANANA group we may well make
>> this feature be mandatory. So I think you propose a fair point.
>>>
>>> That said, a BANANA box might communicate with a legacy box. For this
>> scenario, negotiation seems still necessary.
>>>
>>> Thanks,
>>> Mingui
>>>
>>>> -----Original Message-----
>>>> From: Dave Dolson [mailto:[email-masked]]
>>>> Sent: Wednesday, June 28, 2017 7:42 PM
>>>> To: Zhangmingui (Martin); Margaret Cullen; [email-masked]
>>>> Subject: Re: [Banana] BANANA BOF in Prague?
>>>>
>>>> Mingui,
>>>> Negotiation? Is it not reasonable to require support at both BANANA
>>>> tunnel end-points?
>>>>
>>>> David Dolson
>>>> Sandvine
>>>> Original Message
>>>> From: Zhangmingui (Martin)
>>>> Sent: Wednesday, June 28, 2017 6:01 AM
>>>> To: Dave Dolson; Margaret Cullen; [email-masked]
>>>> Subject: RE: [Banana] BANANA BOF in Prague?
>>>>
>>>>
>>>> Hi Dave,
>>>>
>>>> The ECN capability is a good thing for bonding tunnels. The ECN
>>>> capability for a specific tunnel has been specified by 3.3.2 of a
>>>> recently updated draft-ietf-tsvwg-rfc6040update-shim-03. Basically,
>>>> the ECN bits will be carried within the outer IP header that is
>>>> immediately visible to the forwarding element [RFC6040]. BANANA boxes
>>>> may support this capability as well if each tunnel supports it.
>>>>
>>>> There is one additional thing can be done by this group.
>>>>
>>>>  "GRE tunnels do not support dynamic configuration based on capability
>>>>  negotiation, so the ECN capability has to be manually configured,
>>>>  which is specified in Section 4.3 of RFC 6040."
>>>>
>>>> With the signaling protocol, this capability can also be
>>>> automatically negotiated by the peering BANANA boxes other than manually
>> configured.
>>>>
>>>> Thanks,
>>>> Mingui
>>>>
>>>>> -----Original Message-----
>>>>> From: Banana [mailto:[email-masked]] On Behalf Of Dave
>>>>> Dolson
>>>>> Sent: Tuesday, May 23, 2017 10:45 PM
>>>>> To: Margaret Cullen; [email-masked]
>>>>> Subject: Re: [Banana] BANANA BOF in Prague?
>>>>>
>>>>> Margaret,
>>>>> Thanks for submitting the request.
>>>>> I think the charter needs to include scope for the algorithm
>>>>> deciding what proportion of traffic the sender must put on each link.
>>>>>
>>>>> In my mind, the sender must model the congestion window for each
>>>>> link, and therefore know the loss (or ECN) experienced on each link.
>>>>> There may be other approaches. But I don't think this can be left to
>>>>> the implementers.
>>>>>
>>>>> -Dave
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Banana [mailto:[email-masked]] On Behalf Of Margaret
>>>>> Cullen
>>>>> Sent: Monday, May 22, 2017 8:11 PM
>>>>> To: [email-masked]
>>>>> Subject: [Banana] BANANA BOF in Prague?
>>>>>
>>>>> Just FYI —
>>>>>
>>>>> I have submitted a request for a WG Forming BANANA BOF in Prague
>>>>> with the information included below.  This information has also been
>>>>> added to the BOF Wiki.
>>>>>
>>>>> Please let me know if you notice any errors or omissions, or if you
>>>>> have any questions or suggestions.
>>>>>
>>>>> Thanks,
>>>>> Margaret
>>>>>
>>>>> BOF Name:  Bandwidth Aggregation for Network Access (BANANA)
>>>>>
>>>>> Description:
>>>>>
>>>>> Bandwith Aggregation consists of splitting local traffic across
>>>>> multiple Internet links on a per-packet basis, including the ability
>>>>> to split a single flow across multiple links when necessary.
>>>>>
>>>>> It is the goal of this WG to produce a Bandwidth Aggregation
>>>>> solution that will provide the following benefits:
>>>>>
>>>>> - Higher Per-Flow Bandwidth: Many Internet links available to homes
>>>>> and small offices (DSL, Cable, LTE, Satellite, etc.) have relatively
>>>>> low bandwidth.  Users may wish to run applications (such as
>>>>> streaming video, or content up/downloads) that require (or could
>>>>> benefit from) more bandwidth for a single traffic flow than is
>>>>> available on any of the local links.  A Bandwidth Aggregation
>>>>> solution could supply the needed bandwidth by splitting a single
>>>>> traffic flow across multiple Internet links.
>>>>>
>>>>> - Reduced Cost: Traffic sharing on a per-packet basis allows the
>>>>> full  bandwidth of the lowest-cost link to be used first, only using
>>>>> a  higher-cost link when the lowest-cost link is full.
>>>>>
>>>>> - Increased Reliability: When one Internet link goes down, ongoing
>>>>> application flows can be moved to another link, preventing service
>>>>> disruption.
>>>>>
>>>>> Agenda
>>>>> - Agenda bash, scribe, minute taker [5min]
>>>>> - Review of proposed charter text (see below) [10 mins]
>>>>> - Charter discussion [45 mins]
>>>>> - Questions: [30 mins]
>>>>>   - Is the charter text clear and understandable?
>>>>>   - Should the IETF do this work?
>>>>>   - Are you willing to contribute (write, review, email, etc.)
>>>>>
>>>>> Status: WG Forming
>>>>> Responsible AD: Suresh Krishnan
>>>>> BoF proponents: Margaret Cullen / Brian Trammel / Mingui Zhang BoF
>> chairs:
>>>>> Margaret Cullen / Brian Trammel Number of people expected to attend:
>>>>> 100 Length of session (1, 1.5, 2, or 2.5 hours): 1.5 hours Conflicts
>>>>> to avoid (whole Areas and/or WGs): Internet area, Homenet, TRILL,
>>>>> MPTCP, QUIC
>>>>>
>>>>> -     Mailing List: ​https://www.ietf.org/mailman/listinfo/banana
>>>>> -     Draft charter: see below
>>>>> -     Relevant drafts: [[BR]]
>>>>> https://datatracker.ietf.org/doc/draft-leymann-banana-signalling
>>>>> https://datatracker.ietf.org/doc/draft-leymann-banana-data-encap
>>>>> https://datatracker.ietf.org/doc/draft-leymann-banana-integration
>>>>> https://datatracker.ietf.org/doc/draft-leymann-banana-discard-load-r
>>>>> eb
>>>>> alance
>>>>> https://datatracker.ietf.org/doc/draft-leymann-banana-discard-timer
>>>>> https://tools.ietf.org/html/draft-kanugovi-intarea-mams-protocol-04
>>>>> https://tools.ietf.org/html/draft-zhu-intarea-mams-control-protocol-
>>>>> 01
>>>>>
>>>>>
>>>>> Proposed Charter Text and Milestones:
>>>>>
>>>>> The BANdwidth Aggregation for Network Access (BANANA) Working Group
>>>>> is chartered to develop solution(s) to support dynamic path
>>>>> selection on a per-packet basis in networks that have more than one
>>>>> point of attachment to the Internet.
>>>>>
>>>>> Bandwith Aggregation consists of splitting local traffic across
>>>>> multiple Internet links on a per-packet basis, including the ability
>>>>> to split a single flow across multiple links when necessary.
>>>>>
>>>>> It is the goal of this WG to produce a Bandwidth Aggregation
>>>>> solution that will provide the following benefits:
>>>>>
>>>>> - Higher Per-Flow Bandwidth: Many Internet links available to homes
>>>>> and small offices (DSL, Cable, LTE, Satellite, etc.) have relatively
>>>>> low bandwidth.  Users may wish to run applications (such as
>>>>> streaming video, or content up/downloads) that require (or could
>>>>> benefit from) more bandwidth for a single traffic flow than is
>>>>> available on any of the local links.  A Bandwidth Aggregation
>>>>> solution could supply the needed bandwidth by splitting a single
>>>>> traffic flow across multiple Internet links.
>>>>>
>>>>> - Reduced Cost: Traffic sharing on a per-packet basis allows the
>>>>> full  bandwidth of the lowest-cost link to be used first, only using
>>>>> a  higher-cost link when the lowest-cost link is full.
>>>>>
>>>>> - Increased Reliability: When one Internet link goes down, ongoing
>>>>> application flows can be moved to another link, preventing service
>>>>> disruption.
>>>>>
>>>>> Proposed BANANA solutions use different approaches (e.g. tunnels,
>>>>> proxies,
>>>>> etc.) to split and recombine traffic, but at an abstract level, they
>>>>> involve a local (hardware or software) component on the multi-access
>>>>> network, a remote component within the Internet, and mechanisms for
>>>>> those components to find each other, exchange signalling
>>>>> information, and
>>>> direct traffic to each other.
>>>>> We refer to these functional components as the Local and Remote
>>>>> "BANANA Boxes", and we refer to the method they use to direct
>>>>> traffic to each other as a "BANANA Encapsulation".
>>>>>
>>>>> The Bandwidth Aggregation solutions developed in this group will
>>>>> work whether the attached links are provided by a single Internet
>>>>> Service Provider or multiple Providers.
>>>>>
>>>>> The BANANA WG will have the following work items:
>>>>>
>>>>> - Determine how Local and Remote BANANA Boxes find each other.
>>>>>
>>>>> - Specify a signalling protocol that can be used to send
>>>>> configuration  and control information between BANANA boxes, including:
>>>>>   -  IP Prefixes of local links
>>>>>   -  Information about link properties & status
>>>>>   -  Information needed by the encapsulations
>>>>>
>>>>> - Select (and extend, if necessary) an existing tunneling
>>>>> encapsulation for sending traffic between BANANA Boxes.
>>>>>
>>>>> - Work with other IETF WGs defining BANANA encapsulations  (if any)
>>>>> to ensure that the discovery mechanism and signalling  protocol will
>>>>> meet their needs.
>>>>>
>>>>> BANANA Boxes will determine if a specific flow is eligible for
>>>>> Bandwith Aggregation. If a flow is not eligible, it will not be
>>>>> split across multiple attached links.
>>>>>
>>>>> For this initial charter, we will focus on how Local BANANA Boxes
>>>>> communicate with Remote BANANA Boxes.  We will not address the topic
>>>>> of cooperation between multiple Local BANANA Boxes.
>>>>>
>>>>> MILESTONES
>>>>> Apr 2018 Adopt WG draft for discovery/configuration mechanism Apr
>>>>> 2018 Adopt WG draft for signalling proocol Apr 2018 Adopt WG draft
>>>>> for tunnel encapsulation Feb 2019 WGLC on discovery/configuration
>>>>> mechanism Feb 2019 WGLC on signalling protocol Feb 2019 WGLC on
>>>>> tunnel encapsulation Aug 2019 Send discovery/configuration mechanism
>>>>> to the IESG Aug 2019 Send signalling protocl to the IESG Aug 2019
>>>>> Send tunnel encapsulation to the IESG
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Banana mailing list
>>>>> [email-masked]
>>>>> https://www.ietf.org/mailman/listinfo/banana
>>>>> _______________________________________________
>>>>> Banana mailing list
>>>>> [email-masked]
>>>>> https://www.ietf.org/mailman/listinfo/banana
>>> _______________________________________________
>>> Banana mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/banana
>
> _______________________________________________
> Banana mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/banana

---

From: Dave Dolson <[email-masked]>
Date: Tue, 4 Jul 2017 10:35:48 +0000

Note my original point was ‎this needs to be in the scope of the charter.

David Dolson
Sandvine
  Original Message
From: Zhangmingui (Martin)
Sent: Tuesday, July 4, 2017 3:31 AM
To: Mirja Kühlewind
Cc: Brian Trammell (IETF); Margaret Cullen; Dave Dolson; [email-masked]
Subject: Re: [Banana] BANANA BOF in Prague?

Hi Mirja,

Thanks,
Mingui

Mirja

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 17 Jul 2017 12:26:24 +0200

Hi Dave,

This is clearly in scope for the current charter, even though it is not explicitly mentioned.

Are you saying that you think the charter _should_ explicitly mention this?  If so, could you send a suggested text change?

Margaret

> On Jul 4, 2017, at 12:35 PM, Dave Dolson <[email-masked]> wrote:
>

---

From: Dave Dolson <[email-masked]>
Date: Mon, 17 Jul 2017 12:13:26 +0000

As a work item, I suggest:
- specify measurements, protocols, and control algorithm ‎for the scheduling of packets into each link based on packet loss and ECN experienced per link.

David Dolson
Sandvine
  Original Message
From: Margaret Cullen
Sent: Monday, July 17, 2017 12:26 PM
To: Dave Dolson
Cc: Zhangmingui (Martin); Mirja Kühlewind; [email-masked]; Brian Trammell (IETF)
Subject: Re: [Banana] BANANA BOF in Prague?

Hi Dave,

Margaret

---

From: Mirja Kühlewind <[email-masked]>
Date: Mon, 17 Jul 2017 14:50:59 +0200

So I think a/the signaling protocol (extensions) are already in scope for the group. However, I’m not sure about the measurements and control algorithms points you mentioned below. For measurements there might be other groups that are a better fit. And regarding control algorithms, you actually don’t have to standardize one single scheduling algorithm for all to use (as that does not needs to defined for interoperability). So there is some room for research and experimentation here, and further those questions might be very similar for other scenarios where there is any kind of multipath scheduling needed. So this is probably rather in scope for panrg which is a proposed research group than for an IETF working.

> Am 17.07.2017 um 14:13 schrieb Dave Dolson <[email-masked]>:
>

---

From: Dave Dolson <[email-masked]>
Date: Mon, 17 Jul 2017 12:58:19 +0000

OK, fair enough.
But I think there will be a protocol aspect to reporting loss and/or ECN Congestion-Experienced per tunnel.

-----Original Message-----
From: Mirja Kühlewind [mailto:[email-masked]] 
Sent: Monday, July 17, 2017 2:51 PM
To: Dave Dolson
Cc: Margaret Cullen; Zhangmingui (Martin); [email-masked]; Brian Trammell (IETF)
Subject: Re: [Banana] BANANA BOF in Prague?

>>>> There's also an issue of the network carrying the tunnel having 
>>>> impaired support for the ECN codepoints -- this is often because of 
>>>> networks configured to use the entire ToS byte for forwarding decisions, rather than the 6 dscp bits.
>>>> cooperative network capability sensing to deal with these networks. 
>>>> Though this isn't negotiation per se, it uses equivalent mechanisms.
>>>> 
>>>> Cheers,
>>>> 
>>>> Brian (as individual)
>>>> 
>>>> 
>>>>>> Negotiation? Is it not reasonable to require support at both 
>>>>>> BANANA tunnel end-points?
>>>>>> 
>>>>>> David Dolson
>>>>>> Sandvine
>>>>>> Original Message
>>>>>> recently updated draft-ietf-tsvwg-rfc6040update-shim-03. 
>>>>>> Basically, the ECN bits will be carried within the outer IP 
>>>>>> header that is immediately visible to the forwarding element 
>>>>>> [RFC6040]. BANANA boxes may support this capability as well if each tunnel supports it.
>>>>>> 
>>>>>> "GRE tunnels do not support dynamic configuration based on 
>>>>>> capability negotiation, so the ECN capability has to be manually 
>>>>>> configured, which is specified in Section 4.3 of RFC 6040."
>>>>>> 
>>>>>>> There may be other approaches. But I don't think this can be 
>>>>>>> left to the implementers.
>>>>>>> 
>>>>>>> -Dave
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> From: Banana [mailto:[email-masked]] On Behalf Of 
>>>>>>> Margaret Cullen
>>>>>>> with the information included below.  This information has also 
>>>>>>> been added to the BOF Wiki.
>>>>>>> 
>>>>>>> Please let me know if you notice any errors or omissions, or if 
>>>>>>> you have any questions or suggestions.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Margaret
>>>>>>> 
>>>>>>> multiple Internet links on a per-packet basis, including the 
>>>>>>> ability to split a single flow across multiple links when necessary.
>>>>>>> 
>>>>>>> - Higher Per-Flow Bandwidth: Many Internet links available to 
>>>>>>> homes and small offices (DSL, Cable, LTE, Satellite, etc.) have 
>>>>>>> relatively low bandwidth.  Users may wish to run applications 
>>>>>>> (such as streaming video, or content up/downloads) that require 
>>>>>>> (or could benefit from) more bandwidth for a single traffic flow 
>>>>>>> than is available on any of the local links.  A Bandwidth 
>>>>>>> Aggregation solution could supply the needed bandwidth by 
>>>>>>> splitting a single traffic flow across multiple Internet links.
>>>>>>> 
>>>>>>> full  bandwidth of the lowest-cost link to be used first, only 
>>>>>>> using a  higher-cost link when the lowest-cost link is full.
>>>>>>> 
>>>>>>> - Increased Reliability: When one Internet link goes down, 
>>>>>>> ongoing application flows can be moved to another link, 
>>>>>>> preventing service disruption.
>>>>>>> 
>>>>>>> Agenda
>>>>>>> 100 Length of session (1, 1.5, 2, or 2.5 hours): 1.5 hours 
>>>>>>> Conflicts to avoid (whole Areas and/or WGs): Internet area, 
>>>>>>> Homenet, TRILL, MPTCP, QUIC
>>>>>>> 
>>>>>>> The BANdwidth Aggregation for Network Access (BANANA) Working 
>>>>>>> Group is chartered to develop solution(s) to support dynamic 
>>>>>>> path selection on a per-packet basis in networks that have more 
>>>>>>> than one point of attachment to the Internet.
>>>>>>> 
>>>>>>> multiple Internet links on a per-packet basis, including the 
>>>>>>> ability to split a single flow across multiple links when necessary.
>>>>>>> 
>>>>>>> - Higher Per-Flow Bandwidth: Many Internet links available to 
>>>>>>> homes and small offices (DSL, Cable, LTE, Satellite, etc.) have 
>>>>>>> relatively low bandwidth.  Users may wish to run applications 
>>>>>>> (such as streaming video, or content up/downloads) that require 
>>>>>>> (or could benefit from) more bandwidth for a single traffic flow 
>>>>>>> than is available on any of the local links.  A Bandwidth 
>>>>>>> Aggregation solution could supply the needed bandwidth by 
>>>>>>> splitting a single traffic flow across multiple Internet links.
>>>>>>> 
>>>>>>> full  bandwidth of the lowest-cost link to be used first, only 
>>>>>>> using a  higher-cost link when the lowest-cost link is full.
>>>>>>> 
>>>>>>> - Increased Reliability: When one Internet link goes down, 
>>>>>>> ongoing application flows can be moved to another link, 
>>>>>>> preventing service disruption.
>>>>>>> 
>>>>>>> Proposed BANANA solutions use different approaches (e.g. 
>>>>>>> tunnels, proxies,
>>>>>>> etc.) to split and recombine traffic, but at an abstract level, 
>>>>>>> they involve a local (hardware or software) component on the 
>>>>>>> multi-access network, a remote component within the Internet, 
>>>>>>> and mechanisms for those components to find each other, exchange 
>>>>>>> signalling information, and
>>>>>>> work whether the attached links are provided by a single 
>>>>>>> Internet Service Provider or multiple Providers.
>>>>>>> 
>>>>>>> - Work with other IETF WGs defining BANANA encapsulations  (if 
>>>>>>> any) to ensure that the discovery mechanism and signalling  
>>>>>>> protocol will meet their needs.
>>>>>>> 
>>>>>>> For this initial charter, we will focus on how Local BANANA 
>>>>>>> Boxes communicate with Remote BANANA Boxes.  We will not address 
>>>>>>> the topic of cooperation between multiple Local BANANA Boxes.
>>>>>>> 
>>>>>>> MILESTONES
>>>>>>> 2018 Adopt WG draft for signalling proocol Apr 2018 Adopt WG 
>>>>>>> draft for tunnel encapsulation Feb 2019 WGLC on 
>>>>>>> discovery/configuration mechanism Feb 2019 WGLC on signalling 
>>>>>>> protocol Feb 2019 WGLC on tunnel encapsulation Aug 2019 Send 
>>>>>>> discovery/configuration mechanism to the IESG Aug 2019 Send 
>>>>>>> signalling protocl to the IESG Aug 2019 Send tunnel 
>>>>>>> encapsulation to the IESG
>>>>>>> 
>>>>>>>

---

From: Mirja Kühlewind <[email-masked]>
Date: Mon, 17 Jul 2017 14:59:43 +0200

Yes, and I believe loss is already considered in the charter and ECN should be considered as well.

Mirja

> Am 17.07.2017 um 14:58 schrieb Dave Dolson <[email-masked]>:
> 
> OK, fair enough.

---

From: Dave Dolson <[email-masked]>
Date: Mon, 17 Jul 2017 13:03:58 +0000

I see signaling protocol " Information about link properties & status"
Can that be " Per-link Information about loss, congestion-experienced, properties & status" ?

-----Original Message-----
From: Mirja Kühlewind [mailto:[email-masked]] 
Sent: Monday, July 17, 2017 3:00 PM
To: Dave Dolson
Cc: Margaret Cullen; Zhangmingui (Martin); [email-masked]; Brian Trammell (IETF)
Subject: Re: [Banana] BANANA BOF in Prague?

Mirja

>>>>> impaired support for the ECN codepoints -- this is often because 
>>>>> of networks configured to use the entire ToS byte for forwarding decisions, rather than the 6 dscp bits.
>>>>> So even if both BANANA endpoints support ECN, they might need to 
>>>>> do cooperative network capability sensing to deal with these networks.
>>>>>>> capability for a specific tunnel has been specified by 3.3.2 of 
>>>>>>> a recently updated draft-ietf-tsvwg-rfc6040update-shim-03.
>>>>>>>> In my mind, the sender must model the congestion window for 
>>>>>>>> each link, and therefore know the loss (or ECN) experienced on each link.
>>>>>>>> I have submitted a request for a WG Forming BANANA BOF in 
>>>>>>>> Prague with the information included below.  This information 
>>>>>>>> has also been added to the BOF Wiki.
>>>>>>>> 
>>>>>>>> (or could benefit from) more bandwidth for a single traffic 
>>>>>>>> flow than is available on any of the local links.  A Bandwidth 
>>>>>>>> - Reduced Cost: Traffic sharing on a per-packet basis allows 
>>>>>>>> the full  bandwidth of the lowest-cost link to be used first, 
>>>>>>>> only using a  higher-cost link when the lowest-cost link is full.
>>>>>>>> 
>>>>>>>> Responsible AD: Suresh Krishnan BoF proponents: Margaret Cullen 
>>>>>>>> / Brian Trammel / Mingui Zhang BoF
>>>>> chairs:
>>>>>>>> https://datatracker.ietf.org/doc/draft-leymann-banana-integrati
>>>>>>>> o
>>>>>>>> n
>>>>>>>> https://datatracker.ietf.org/doc/draft-leymann-banana-discard-l
>>>>>>>> o
>>>>>>>> ad-r
>>>>>>>> eb
>>>>>>>> alance
>>>>>>>> https://datatracker.ietf.org/doc/draft-leymann-banana-discard-t
>>>>>>>> i
>>>>>>>> mer
>>>>>>>> https://tools.ietf.org/html/draft-kanugovi-intarea-mams-protoco
>>>>>>>> l
>>>>>>>> -04
>>>>>>>> https://tools.ietf.org/html/draft-zhu-intarea-mams-control-prot
>>>>>>>> o
>>>>>>>> col-
>>>>>>>> 01
>>>>>>>> 
>>>>>>>> 
>>>>>>>> (or could benefit from) more bandwidth for a single traffic 
>>>>>>>> flow than is available on any of the local links.  A Bandwidth 
>>>>>>>> - Reduced Cost: Traffic sharing on a per-packet basis allows 
>>>>>>>> the full  bandwidth of the lowest-cost link to be used first, 
>>>>>>>> only using a  higher-cost link when the lowest-cost link is full.
>>>>>>>> 
>>>>>>>> and mechanisms for those components to find each other, 
>>>>>>>> exchange signalling information, and
>>>>>>>> The Bandwidth Aggregation solutions developed in this group 
>>>>>>>> will work whether the attached links are provided by a single 
>>>>>>>> Boxes communicate with Remote BANANA Boxes.  We will not 
>>>>>>>> address the topic of cooperation between multiple Local BANANA Boxes.
>>>>>>>> 
>>>>>>>> MILESTONES