Subject: Re: [Banana] Charter

From: [email-masked]
Date: Wed, 13 Sep 2017 09:27:49 -0400

Hi Wim,

Sent from my iPhone

> If I hear GRE as a proposal it is very NAT unfriendly and the solution need to work across multiple providers, so this is an important consideration.

Sorry, I somehow dropped this thread while I was in vacation...

Why do you think that (all) GRE-based proposal(s) would be NAT unfriendly?  

Margaret

---

From: [email-masked]
Date: Wed, 13 Sep 2017 09:52:33 -0400

Hi Alex,

Sent from my iPhone

> On Jul 17, 2017, at 11:06 AM, Alexandre Petrescu <[email-masked]> wrote:
> 
> Mic: for the encapsulation methods.
> 
> IP encapsulation can mean to add another IP header similar to the original, or to add another header like a Routing Header, or to add another non-IP header.

As the charter text now attempts to make clear, when we talk about a "BANANA Encapsulation" we are not necessarily talking about adding an extra header, as the "Encapsulation" could be a tunnel-based mechanism (of which the bonding tunnels proposal is an example) OR a proxy-based solution (of which MPTCP Proxy is an example).

> There are many encapsulation methods at IETF and some other outside of IETF (e.g. GTP at 3GPP).  People also work new encapsulation methods at IETF like "skinny encap" in 6man WG.
> 
> There could be a means between BANANA Local box and BANANA Remote box to negotiate that method of "encapsulation", and maybe default to some value such as "Generic Encapsulaiton"

There is definitely a desire, reflected in the charter, to support more than one type of BANANA Encapsulation.  There has even been discussion about using more than one BANANA Encapsulation in a single deployment (I.e. MPTCP Proxy for TCP traffic and tunnels for non-TCP traffic).  The signaling protocol could include negotiating which Encapsulation is used for which traffic, if there are multiple options available on both ends.

The most recent proposed BANANA charter text indicates that the BANANA WG will define a tunnel-based Encapsulation. There is also ongoing discussion of defining an MPTCP-based Encapsulation in the MPTCP WG.

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Wed, 13 Sep 2017 14:11:48 +0000

On 8/29/17, 6:51 PM, "Banana on behalf of Muley, Praveen (Nokia -
US/Mountain View)" <[email-masked] on behalf of
[email-masked]> wrote:

>Since the discussion on encapsulations  has NOT progressed further. Would
>like to understand also about other candidate solutions  which were
>mentioned in the BOF like MP-TCP or MP-UDP/QUIC ?  If these solutions are
>progressing already in other WG what are we  planning to do here ?

I have the same comment. It is not clear to me either on what exactly this
WG intends to do. In the BOF (WG-forming) meeting, there was no
opportunity given to raise any questions on the problem statement. The
entire focus of the meeting was on some charter text without any
discussions on 1.) problem statement, 2.) currently known art, 3.) efforts
in other WG groups,4.) the gaps.  There was no agreement on IETF 98 BOF
meeting (non-WG forming) either on the problem statement. So, I do no
understand why we are jumping on charter text discussion without clear
agreements on the problem statement.

I really hope IESG will review all the notes/discussions carefully.

Sri

---

From: "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 13 Sep 2017 16:51:16 +0000

How will you sent GRE through NAT. GRE has no port number

On 13/09/2017, 17:27, "[email-masked]" <[email-masked]> wrote:

    Hi Wim,
    
    Sent from my iPhone

    
    
    Margaret

---

From: "Zhangmingui (Martin)" <[email-masked]>
Date: Thu, 14 Sep 2017 03:07:19 +0000

Hi Alex,

If "GRE passthrough NAT" was proved to be really required, there are two options:
1. GRE in UDP while the UDP provides you the port number.
2. The GRE Key field to be used to carry the port number. 
For the second option, there are some existing implementations. But it is not an option if the Key field has already used for other purpose, e.g., security.

However, I would say the popular usage is that the NAT happens just before the GRE tunnel. Why do we have to insert a NAT device in between two GRE tunnel endpoints?

Thanks,
Mingui

________________________________________
From: Banana [[email-masked]] on behalf of Henderickx, Wim (Nokia - BE/Antwerp) [[email-masked]]
Sent: Thursday, September 14, 2017 0:51
To: [email-masked]
Cc: Alexandre Petrescu; [email-masked]
Subject: Re: [Banana] Charter

    Hi Wim,

    Sent from my iPhone

    Margaret

_______________________________________________
Banana mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/banana

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Thu, 14 Sep 2017 07:36:37 +0200

> On 14 Sep 2017, at 05:07, Zhangmingui (Martin) <[email-masked]> wrote:
> 
> Hi Alex,
> 

I think the issue here would be double-NAT; i.e., a BANANA Box located on a home router, so one device performs both NAT for the local network and bandwidth aggregation, sitting on CG-NATted space on at least one of its access networks.

IMO situations like this are a good reason for BANANA to support multiple encapsulations.

Cheers,

Brian

---

From: Margaret Cullen <[email-masked]>
Date: Thu, 14 Sep 2017 17:02:32 -0400

Hi Sri,

> meeting (non-WG forming) either on the problem statement.

Actually, the non-WG forming BOF was held at IETF-97, and it is true that we did not yet have agreement on a clear and well-understood problem statement at that time. We didn’t hold a BOF at IETF-98.  Instead, about 30 of us met for a publicly-scheduled "Bar BOF" and discussed the problem statement and possible charter text. There was also _considerable_ discussion of the problem statement on the mailing list between IETF-97 and IETF-99, and we reached general agreement on our problem statement on the mailing list.

At IETF-99, we reviewed the charter text, which included our agreed problem statement as the first few paragraphs.  To wit:

"The BANdwidth Aggregation for Network Access (BANANA) Working Group is chartered to develop solution(s) to support dynamic path selection on a per-packet basis in networks that have more than one point of attachment to the Internet.

Bandwidth Aggregation consists of splitting local traffic across multiple Internet links on a per-packet basis, including the ability to split a single flow across multiple links when necessary.

It is the goal of this WG to produce a Bandwidth Aggregation solution that will provide the following benefits:

	• Higher Per-Flow Bandwidth: Many Internet links available to homes and small offices (DSL, Cable, LTE, Satellite, VPNs, etc.) have relatively low bandwidth. Users may wish to run applications (such as streaming video, or content up/downloads) that require (or could benefit from) more bandwidth for a single traffic flow than is available on any of the local links. A Bandwidth Aggregation solution could supply the needed bandwidth by splitting a single traffic flow across multiple Internet links.
	• Reduced Cost: Traffic sharing on a per-packet basis allows the full bandwidth of the lowest-cost link to be used first, only using a higher-cost link when the lowest-cost link is full.
	• Increased Reliability: When one Internet link goes down, ongoing application flows can be moved to another link, preventing service disruption.”

We went through the charter text, sentence by sentence, at the BOF and the only change we made to the problem statement portion of the charter was to add VPNs to the list of low bandwidth Internet links.  There was an opportunity to discuss every sentence in this section, but people didn’t have much to say about it, probably because we had already spent months discussing it on the list and in a Bar BOF.

During the question period at the end of the BOF, we asked whether the problem was clear and well-understood.  The large majority of  the people who responded said “yes”.  This was recorded in the minutes as a “decent hum” for yes, and a “low murmur” for no. 

> So, I do not

Based on the agreement on the mailing list and in the room in Prague, I would say that we _do_ have agreement on a clear and understandable problem statement.  What makes you think we don’t?

> I really hope IESG will review all the notes/discussions carefully.

I am sure that Suresh can be trusted to review things carefully before he puts our charter on the IESG agenda.  If there are any particular notes or discussion that you would like him to review, please feel free to draw his attention to them.

Margaret

---

From: "Muley, Praveen (Nokia - US/Mountain View)" <[email-masked]>
Date: Fri, 15 Sep 2017 01:17:00 +0000

In IETF 99 there was no debate/discussion on problem statement which was working group BOF.  It was more of charter edits assuming that everyone agrees with problem statement. In charter if we exclude the single provider offering service ( i.e the CPE and Aggregation point ) then I am NOT sure how valid is the business/use  case for over the top bandwidth aggregation case ?  It will be better if we can find consensus on the exact use case  which can be of larger interest. One thing also we SHOULD consider here in our debate is,  if there are already MP-solutions are progressing in other working groups which are relevant for the problem statement then do we need another working group ?

-Praveen 

-----Original Message-----
From: Margaret Cullen [mailto:[email-masked]] 
Sent: Thursday, September 14, 2017 2:03 PM
To: Sri Gundavelli (sgundave) <[email-masked]>
Cc: Muley, Praveen (Nokia - US/Mountain View) <[email-masked]>; Alexandre Petrescu <[email-masked]>; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; [email-masked]
Subject: Re: [Banana] Charter

Hi Sri,

> I have the same comment. It is not clear to me either on what exactly 
> this WG intends to do. In the BOF (WG-forming) meeting, there was no 
> opportunity given to raise any questions on the problem statement.  
> The entire focus of the meeting was on some charter text without any 
> discussions on 1.) problem statement, 2.) currently known art, 3.) 
> efforts in other WG groups,4.) the gaps.  There was no agreement on 
> IETF 98 BOF meeting (non-WG forming) either on the problem statement.

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Fri, 15 Sep 2017 02:30:16 +0000

Hi Margaret,

Thank you for your response.

Sure; it was non-WG BOF in IETF 97 which closed with no agreements, then
the BAR BOF in IETF 98 which was not supervised by any AD, or covered
under any IETF process and so does not mean any thing. But, we met in IETF
99 for a WG-forming BOF and straight tweaked the charter text. I tend to
think we missed some important steps in between and those are discussions
on 1.) problem statement,  2.) currently known art, 3.) efforts in other
WG groups, 4.) the gaps 5.) more interestingly SDO/Vendor interest.  I
wonder, why we missed those steps. If the argument is that we have
established consensus in the ML, I looked at all the emails from November
2016 and I did not conclude that there was agreement on this work.

> I am sure that Suresh can be trusted to review things carefully before
>he puts our charter on the IESG agenda.  If there are any particular
>notes or discussion that you would like him to review, please feel free
>to draw his attention to them.

Absolutely! I trust each and every AD/IESG member and I am absolutely sure
they will do their job. They know the process better than me. But, that is
not the point of this discussion. I strongly believe there is no agreement
on the PS and so we need to go back to PS discussion and not randomly
start charter text discussion. That is my feedback to the AD and to the
entire IESG. I hope that will be the conclusion of the IESG review as
well. 

Regards
Sri

On 9/14/17, 2:02 PM, "Margaret Cullen" <[email-masked]> wrote:

>>this
>>efforts
>Actually, the non-WG forming BOF was held at IETF-97, and it is true that
>we did not yet have agreement on a clear and well-understood problem
>statement at that time. We didn¹t hold a BOF at IETF-98.  Instead, about
>30 of us met for a publicly-scheduled "Bar BOF" and discussed the problem
>statement and possible charter text. There was also _considerable_
>discussion of the problem statement on the mailing list between IETF-97
>and IETF-99, and we reached general agreement on our problem statement on
>the mailing list.
>
>At IETF-99, we reviewed the charter text, which included our agreed
>problem statement as the first few paragraphs.  To wit:
>
>"The BANdwidth Aggregation for Network Access (BANANA) Working Group is
>chartered to develop solution(s) to support dynamic path selection on a
>per-packet basis in networks that have more than one point of attachment
>to the Internet.
>
>Bandwidth Aggregation consists of splitting local traffic across multiple
>Internet links on a per-packet basis, including the ability to split a
>single flow across multiple links when necessary.
>
>It is the goal of this WG to produce a Bandwidth Aggregation solution
>that will provide the following benefits:
>
>	€ Higher Per-Flow Bandwidth: Many Internet links available to homes and
>small offices (DSL, Cable, LTE, Satellite, VPNs, etc.) have relatively
>low bandwidth. Users may wish to run applications (such as streaming
>video, or content up/downloads) that require (or could benefit from) more
>bandwidth for a single traffic flow than is available on any of the local
>links. A Bandwidth Aggregation solution could supply the needed bandwidth
>by splitting a single traffic flow across multiple Internet links.
>	€ Reduced Cost: Traffic sharing on a per-packet basis allows the full
>bandwidth of the lowest-cost link to be used first, only using a
>higher-cost link when the lowest-cost link is full.
>	€ Increased Reliability: When one Internet link goes down, ongoing
>application flows can be moved to another link, preventing service
>disruption.²
>
>We went through the charter text, sentence by sentence, at the BOF and
>the only change we made to the problem statement portion of the charter
>was to add VPNs to the list of low bandwidth Internet links.  There was
>an opportunity to discuss every sentence in this section, but people
>didn¹t have much to say about it, probably because we had already spent
>months discussing it on the list and in a Bar BOF.
>
>During the question period at the end of the BOF, we asked whether the
>problem was clear and well-understood.  The large majority of  the people
>who responded said ³yes².  This was recorded in the minutes as a ³decent
>hum² for yes, and a ³low murmur² for no.
>
>> So, I do not
>Based on the agreement on the mailing list and in the room in Prague, I
>would say that we _do_ have agreement on a clear and understandable
>problem statement.  What makes you think we don¹t?
>
>I am sure that Suresh can be trusted to review things carefully before he
>puts our charter on the IESG agenda.  If there are any particular notes
>or discussion that you would like him to review, please feel free to draw
>his attention to them.
>
>Margaret
>
>

---

From: "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]>
Date: Fri, 15 Sep 2017 04:39:00 +0000

The issue I have here is the following. We are chartering a new WG to solve a certain problem.
The charter already hints to GRE while we have not understood the requirements and have looked at what the best solution would be to accommodate these requirements. The environment in the charter is multi-operator, which means we will have to deal with NAT in multiple ways as long as we intend to use v6.

So, the issue I have with the charter in general is that we are trying to define a protocols/signalling extensions, while we have not understood the requirements and have done a gap analysis regarding this. The first thing that should happen is look at the requirements and more importantly look at the algorithms we would need to accommodate these requirements. After do gap analysis for the existing protocols and then define the potential extensions. In the last step we should even see if other WG are not already suited to handle this activity.

On 14/09/2017, 07:07, "Zhangmingui (Martin)" <[email-masked]> wrote:

    Hi Alex,

    
    Thanks,
    Mingui

    
    
        Hi Wim,
    
        Sent from my iPhone

    
    
        Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Fri, 15 Sep 2017 11:25:28 -0400

Given the concerns about GRE and NATs, perhaps it would make sense to remove the explicit mention of GRE from the charter and add some wording about traversal of NATs and other middle boxes?  Maybe something along these lines?

OLD:
The Bandwidth Aggregation solutions developed in this group will be designed to work in multi-provider scenarios (i.e. they will not depend on all of the aggregated links being provided by a single Internet access provider).

NEW:  
The Bandwidth Aggregation solutions developed in this group will be designed to work in multi-provider scenarios (i.e. they will not depend on all of the aggregated links being provided by a single Internet access provider, and they will be designed to work across NATs and other middle boxes, as needed).

OLD:
Select (and extend, if necessary) an existing tunneling encapsulation (e.g. GRE)  for sending traffic between BANANA Boxes.

NEW:
Select (and extend, if necessary) an existing tunneling encapsulation for sending traffic between BANANA boxes.

Do people think these changes would be an improvement to the existing charter text?  If there are no objections over the next few days, I will make these changes to the online charter text.

Margaret

> On Sep 15, 2017, at 12:39 AM, Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]> wrote:
>

---

From: David Allan I <[email-masked]>
Date: Mon, 18 Sep 2017 06:06:38 +0000

I have to admit, my original expectation of the outcome of the discussions was what the focus would be on configuration of the end points for a non-carrier-integrated OTT scenario.

What I am reading in the charter is license to explore the whole space, and invent additional solutions even in the presence of suitable work by other WGs....

Dave

-----Original Message-----
From: Banana [mailto:[email-masked]] On Behalf Of Margaret Cullen
Sent: Friday, September 15, 2017 12:03 AM
To: Sri Gundavelli (sgundave) <[email-masked]>
Cc: Alexandre Petrescu <[email-masked]>; Muley, Praveen (Nokia - US/Mountain View) <[email-masked]>; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; [email-masked]
Subject: Re: [Banana] Charter

Hi Sri,

Margaret

---

From: "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]>
Date: Mon, 18 Sep 2017 13:42:17 +0000

Margaret, GRE is one thing, but on top is the deliverables as outlined in the charter.

In a situation like this we should first do requirements and check gaps with existing protocols to ensure we go down the right direction. Given the scope is specified as multi-operator OTT deployment we need to ensure we understand all these implications.

Referring to this:
Milestones
	• Apr 2018 Adopt WG draft for provisioning/configuration mechanism
	• Apr 2018 Adopt WG draft for signaling protocol
	• Apr 2018 Adopt WG draft(s) for tunnel encapsulation(s)
	• Feb 2019 WGLC on provisioning/configuration mechanism
	• Feb 2019 WGLC on signaling protocol
	• Feb 2019 WGLC on tunnel encapsulation(s)
	• Aug 2019 Send provisioning/configuration mechanism to the IESG
	• Aug 2019 Send signalling protocol to the IESG
	• Aug 2019 Send tunnel encapsulation(s) to the IESG

On 15/09/2017, 17:25, "Margaret Cullen" <[email-masked]> wrote:

    OLD:
    
    NEW:  
    
    OLD:
    
    NEW:

    Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 18 Sep 2017 15:18:40 -0400

Hi Wim,

I have heard and understand that you do not feel that we should proceed with this work without a (potentially lengthy) process to document the requirements, do gap analysis, etc.  That opinion was raised at the BOF meeting in Prague, as well, where several people said that they did not support going through that sort of process, and our AD, Suresh, told us that he wouldn’t charter this group to go through that sort of process.   At the end of the BOF, when we asked the questions, a large majority of the people who responded indicated that they thought the problem was clear and well understood, and that the charter represented work we should do in the IETF.  I understand that there are a few of you who feel differently, and you are welcome to express your opinions, but I would say that there was a fairly strong agreement of the people in the room in Prague _not_ to change the charter to include a requirements/gap analysis phase, so I am not planning to do so.

Margaret

> On Sep 18, 2017, at 9:42 AM, Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]> wrote:
>

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Mon, 18 Sep 2017 19:43:50 +0000

Hi Margaret,

It is not just the question of counting hands in the room, I do not think
that¹s the criteria for ³consensus² determination. We all have been in
IETF long enough and we know that does not mean any thing. Sure, there may
be few excited researchers/students who have no skin in the game to
express interest in the work, but Standardization should have a product
path. Are the key network vendors who have skin in the game have expressed
interest for this work? Is there SDO interest from 3GPP, Wireless or
Broadband guys? Clearly, I do not see that and there is no support for
this work.

Also, many folks asked why there is a need for a new signaling protocol.
When I asked this question, some explicit text was added to say something
along the lines, ³WG will not define a new signaling protocol, unless the
existing protocols do not meet the requirement². I remember Mirja
commenting on that and the chairs editing the text (Mirja can comment if I
am wrong here). But, the below charter text conveniently removed that
entire text and now shows a milestone for the signaling work. So, the work
is getting steered not based on consensus, but on a pre-determined path.

Bottomline, this work has no support. There is no vendor, SDO or operator
interest; we need to have additional discussions. This is a classic case
for IESG Appeal.

‹-
€ Specify a signalling protocol that can be used to send control
information between BANANA Boxes, including:
		€ IP Prefixes of access  links
		€ Information about link status and properties (including congestion)
		€ Information needed by BANANA Encapsulations
		€ Determining which flows are/are not eligible for BANANA Encapsulation
	€ Select (and extend, if necessary) an existing tunneling encapsulation
(e.g. GRE)  for sending traffic between BANANA Boxes.

€ Feb 2019 WGLC on signaling protocol

‹-

Sri

Feb 2019 WGLC on signaling protocol

On 9/18/17, 12:18 PM, "Banana on behalf of Margaret Cullen"
<[email-masked] on behalf of [email-masked]> wrote:

>Hi Wim,
>
>I have heard and understand that you do not feel that we should proceed
>with this work without a (potentially lengthy) process to document the
>requirements, do gap analysis, etc.  That opinion was raised at the BOF
>meeting in Prague, as well, where several people said that they did not
>support going through that sort of process, and our AD, Suresh, told us
>that he wouldn¹t charter this group to go through that sort of process.
>At the end of the BOF, when we asked the questions, a large majority of
>the people who responded indicated that they thought the problem was
>clear and well understood, and that the charter represented work we
>should do in the IETF.  I understand that there are a few of you who feel
>differently, and you are welcome to express your opinions, but I would
>say that there was a fairly strong agreement of the people in the room in
>Prague _not_ to change the charter to include a requirements/gap analysis
>phase, so I am not planning to do so.
>
>Margaret
>
>
>> In a situation like this we should first do requirements and check gaps
>>with existing protocols to ensure we go down the right direction. Given
>>the scope is specified as multi-operator OTT deployment we need to
>>ensure we understand all these implications.
>> 
>> Referring to this:
>> Milestones
>> 	€ Apr 2018 Adopt WG draft for provisioning/configuration mechanism
>> 	€ Apr 2018 Adopt WG draft for signaling protocol
>> 	€ Apr 2018 Adopt WG draft(s) for tunnel encapsulation(s)
>> 	€ Feb 2019 WGLC on provisioning/configuration mechanism
>> 	€ Feb 2019 WGLC on signaling protocol
>> 	€ Feb 2019 WGLC on tunnel encapsulation(s)
>> 	€ Aug 2019 Send provisioning/configuration mechanism to the IESG
>> 	€ Aug 2019 Send signalling protocol to the IESG
>> 	€ Aug 2019 Send tunnel encapsulation(s) to the IESG
>> 
>>    Given the concerns about GRE and NATs, perhaps it would make sense
>>to remove the explicit mention of GRE from the charter and add some
>>wording about traversal of NATs and other middle boxes?  Maybe something
>>along these lines?
>> 
>>    OLD:
>>    The Bandwidth Aggregation solutions developed in this group will be
>>designed to work in multi-provider scenarios (i.e. they will not depend
>>on all of the aggregated links being provided by a single Internet
>>access provider).
>> 
>>    NEW:  
>>    The Bandwidth Aggregation solutions developed in this group will be
>>designed to work in multi-provider scenarios (i.e. they will not depend
>>on all of the aggregated links being provided by a single Internet
>>access provider, and they will be designed to work across NATs and other
>>middle boxes, as needed).
>> 
>>    OLD:
>>    Select (and extend, if necessary) an existing tunneling
>>encapsulation (e.g. GRE)  for sending traffic between BANANA Boxes.
>> 
>>    NEW:
>>    Select (and extend, if necessary) an existing tunneling
>>encapsulation for sending traffic between BANANA boxes.
>> 
>>    Do people think these changes would be an improvement to the
>>existing charter text?  If there are no objections over the next few
>>days, I will make these changes to the online charter text.
>> 
>>    Margaret
>> 
>> 
>>> The charter already hints to GRE while we have not understood the
>>>requirements and have looked at what the best solution would be to
>>>accommodate these requirements. The environment in the charter is
>>>multi-operator, which means we will have to deal with NAT in multiple
>>>ways as long as we intend to use v6.
>>> 
>>> So, the issue I have with the charter in general is that we are trying
>>>to define a protocols/signalling extensions, while we have not
>>>understood the requirements and have done a gap analysis regarding
>>>this. The first thing that should happen is look at the requirements
>>>and more importantly look at the algorithms we would need to
>>>accommodate these requirements. After do gap analysis for the existing
>>>protocols and then define the potential extensions. In the last step we
>>>should even see if other WG are not already suited to handle this
>>>activity.
>>> 
>>>   For the second option, there are some existing implementations. But
>>>it is not an option if the Key field has already used for other
>>>purpose, e.g., security.
>>> 
>>>   However, I would say the popular usage is that the NAT happens just
>>>before the GRE tunnel. Why do we have to insert a NAT device in between
>>>two GRE tunnel endpoints?
>>> 
>>>   Thanks,
>>>   Mingui
>>> 
>>>   From: Banana [[email-masked]] on behalf of Henderickx, Wim
>>>(Nokia - BE/Antwerp) [[email-masked]]
>>>> If I hear GRE as a proposal it is very NAT unfriendly and the
>>>>solution need to work across multiple providers, so this is an
>>>>important consideration.
>>>

---

From: "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]>
Date: Mon, 18 Sep 2017 20:04:41 +0000

Margeret, I can only express my opinion and the simple example I gave already showed we don’t have things clear and as such doing the steps w/o proper analysis is going to end in shortcomings later on.

On 18/09/2017, 21:18, "Margaret Cullen" <[email-masked]> wrote:

    Hi Wim,

    Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 18 Sep 2017 16:50:29 -0400

Hi Sri,

> On Sep 18, 2017, at 3:43 PM, Sri Gundavelli (sgundave) <[email-masked]> wrote:

The IETF WG formation process focuses on making sure that there is an actual community of people who want to do a well-understood set of work, that those people have the skills and knowledge to do the work successfully, and that they represent enough segments of the industry to have a wide-enough perspective to do the work in a way that will be widely applicable.  The IETF is set up to attract competent groups of people who want to do interesting work within the IETF’s scope, and offer them an environment and support that will allow them to do a good job.  There have even been some WGs formed by the IESG with no BOF or consensus call in the community at all.  My statements about the number of people who said that they understand the work and want to do it in the IETF, were not meant to say “we have consensus”, but were more meant to say “We have sufficient people who understand this, who want to do it, and who can do a good job”.  If you personally don’t want to work on this, then you don’t have to work on it.

> is getting steered not based on consensus, but on a pre-determined path.

I also remember that discussion.  There was objection to the wording you suggested, and it was reworded.  Then, people pointed out that it didn’t apply only to the signaling protocol.  After the discussion ended, we ended up with the following sentence:  

"The BANANA WG will consider existing IETF protocols, where applicable, as the basis for the work items listed above.”

That sentence appears in the charter, just below the list of the three protocols that are needed as part of a solution.

Even if we use an existing protocol, we would probably need to document how that protocol would be used to enable the BANANA functionality, as there are no IETF protocols that are currently being used for this purpose, as is.
> interest; we need to have additional discussions. This is a classic case

I don’t believe you are correct about this.  We have proposals authored by multiple vendors and operators, we have consistently had people from a dozen or more companies show up at BANANA Bar BOFs and other meetings, including vendors, operators, and researchers.  There are at least three groups that have shown up who are actively working on developing solutions in this space, including at least one group that has actively deployed a solution.  We have researchers who are doing general research in the area, as well as one group of researchers who are doing very interesting research on the performance of potential BANANA solutions, specifically.  

Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 18 Sep 2017 17:06:02 -0400

> 
> The IETF WG formation process focuses on making sure that there is an actual community of people who want to do a well-understood set of work, that those people have the skills and knowledge to do the work successfully, and that they represent enough segments of the industry to have a wide-enough perspective to do the work in a way that will be widely applicable.  The IETF is set up to attract competent groups of people who want to do interesting work within the IETF’s scope, and offer them an environment and support that will allow them to do a good job.  There have even been some WGs formed by the IESG with no BOF or consensus call in the community at all.

Just to clarify, I am not saying that any WG has been formed without a notice to the whole IETF community, just that they are not always preceeded by a BOF or by a consensus call on a topic-specific mailing list, or within an area meeting, etc.

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Mon, 18 Sep 2017 22:30:12 +0000

> The IETF WG formation process focuses on making sure that there is an actual community of people who want to do a well-understood set of work, that those people have the skills and knowledge to do the work successfully, and that they represent enough segments of the industry to have a wide-enough perspective to do the work in a way that will be widely applicable.  The IETF is set up to attract competent groups of people who want to do interesting work within the IETF's scope, and offer them an environment and support that will allow them to do a good job.

Generally, I agree with your above statement. If there is enough interest to do some new work and people ready to write some drafts and people ready to chair the meetings, creation of a WG should be the most logical thing. But, that goes with the assumption that the work is not overlapping with some other work that is happening some where else. That needs to be ensured and hence the need to agree upon the problem statement from both interested and non-interested parties. We need to review this overlap with QUIC, MPTCP, MobIKE and Mobile IPv6 protocols.

The problem with the proposed charter text is the following. Our starting point was about a load-balancing problem. Given two overlay tunnel paths, how do I split the traffic between these two tunnel paths; is it flow-based routing, packet based routing, or weighted packet distribution, what should be the algorithm and considerations. If you had kept at this at this level, a study item on this might be a reasonable proposal, as we could have learnt if there are any gaps in this area. This is strictly a user-plane discussion and no need for signaling plane.  Instead, the charter talks about "Banana Boxes", "Banana Encapsulation", "Banana Signaling"; This is solution description. IETF does not do system architecture, that is in SDO scope; we work on protocols.

We can define a new signaling protocol, just to solve this distribution problem, which can authenticate/authorize the peers, allows path registrations, or prefix allocations, but it will be exactly the same as some existing protocols, but with a new signaling semantics. So, what would be achieve from this effort? It will be another signaling protocol  called "Banana",  which does exactly the same. I wonder, what is the point of this effort?

I suggest we go back to the drawing board and discuss the Problem Statement. I am saying this very objectively, not with the goal to block this work.

Sri

From: Margaret Cullen <[email-masked]<mailto:[email-masked]>>
Date: Monday, September 18, 2017 at 1:50 PM
To: Sri Gundavelli <[email-masked]<mailto:[email-masked]>>
Cc: "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]<mailto:[email-masked]>>, "Zhangmingui (Martin)" <[email-masked]<mailto:[email-masked]>>, Alexandre Petrescu <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Banana] Charter

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 18 Sep 2017 19:11:46 -0400

> The problem with the proposed charter text is the following. Our starting point was about a load-balancing problem.

What starting point?  

These discussions have never been about a generic load balancing problem.  We have always been talking about a home or small office with two or more connections to the Internet, at least some of which are relatively low-bandwidth, and few flows with perhaps only one large or long-lived flow.  I apparently didn’t do a good job of formulating that problem statement at the Informational BOF, because people didn’t understand it.  If you were one of those people, I apologize for being unclear. There was a time, more than a year ago, when we were talking about constraining the problem to a single-operator scenario, but the IESG and IAB thought (correctly, IMO) that it would be more in-scope for the IETF to consider the larger multi-operator problem.  That was decided before the Informational BOF in November of 2016.   The problem was much better formulated during the BANANA mailing list discussion in February, and a much clearer version was included in the charter text reviewed at the BANANA bar BOF in early April.   That version of the charter, first presented in March 2017, is what grew into the charter text presented at the WG-forming BOF in July, and into the text we have now. 

> Given two overlay tunnel paths, how do I split the traffic between these two tunnel paths; is it flow-based routing, packet based routing, or weighted packet distribution, what should be the algorithm and considerations. If you had kept at this at this level, a study item on this might be a reasonable proposal, as we could have learnt if there are any gaps in this area. This is strictly a user-plane discussion and no need for signaling plane.  Instead, the charter talks about "Banana Boxes", "Banana Encapsulation", "Banana Signaling”; This is solution description. IETF does not do system architecture, that is in SDO scope; we work on protocols.

The term BANANA Boxes was first used on the mailing list on February 4th, and it has been used extensively since then in email, in draft charter text, in minutes, etc.  It isn’t a new term, or a new concept.  There is in-band signaling in MPTCP already, and the GRE Tunnel Bonding proposal included a signaling protocol, so signaling has alway been under discussion to some degree.  I think it was Mirja’s idea, at the Bar BOF at IETF-97, to break the provisioning, signaling and transport/encapsulation out separately, so that we might write more than one transport/encapsulation that uses the same signaling protocol, for instance.  This also opened the door to proposals to have one base protocol (perhaps a tunneling protocol) for all traffic, while using other multi-path protocols (such as MPTCP, perhaps MP-QUIC if/when it arrives) to performed bandwidth aggregation in a coordinated fashion.
> 
> We can define a new signaling protocol, just to solve this distribution problem, which can authenticate/authorize the peers, allows path registrations, or prefix allocations, but it will be exactly the same as some existing protocols, but with a new signaling semantics. So, what would be achieve from this effort? It will be another signaling protocol  called “Banana”,  which does exactly the same. I wonder, what is the point of this effort? 

There is no requirement in the Charter that we develop a new protocol and call it BANANA <anything>.  It could be that we write a document for how to do bandwidth aggregation that defines some new messages to run over an existing signaling protocol, for instance.  If you know of a signaling protocol that could do this, I suggest that you point it out to the group, perhaps with a Draft that defines how additional BANANA-specific information could be signaled, if necessary.  If we did that, then the existing protocol would become the “BANANA Signaling Protocol” referred to in the charter.

> I suggest we go back to the drawing board and discuss the Problem Statement. I am saying this very objectively, not with the goal to block this work. 

I hear and understand what you are saying, and I am pretty sure that other people on the list hear and understand what you are saying.  I don’t personally agree that would be the right step to take at this point.  Other people should feel free to express their opinions.

Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 18 Sep 2017 19:15:44 -0400

> On Sep 18, 2017, at 6:30 PM, Sri Gundavelli (sgundave) <[email-masked]> wrote:

This is one of the reasons we have BOFs.  If you believe that there is some overlap with one of those protocols, or that one of those protocols already solves the problem, say that and explain how.  Also, perhaps include some explanation of what that protocol is not already being used to solve this problem, or what sort of changes or extensions would be needed to make it solve the problem more completely.  I am pretty sure that the IESG doesn’t want to charter a group to solve an already-solved problem.  

There are people solving this problem today, in the field, though, and they don’t seem to be using the protocols you have mentioned.  Can you point to someone who is?

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Tue, 19 Sep 2017 00:20:56 +0000

> There are people solving this problem today, in the field, though, and they don't seem to be using the protocols you have mentioned.  Can you point to someone who is?

OK! I can provide that information. We have large scale deployments of the below based on Proxy Mobile IPv6 (100's of thousands), with the ability to perform per-flow load balancing and per-packet and link-weighted packet distribution.  Many DSL operators have deployed this (DSL RG with LTE dongle)  and we have also deployed for many overlay applications.  You can check cisco CCO pages for the product details. For measuring path characteristics, we use IP SLA. Its a solution based on PMIP mobility and SLA for path performance measurement.

Your problem statement is exactly this. The difference is that its Banana Signaling, Banana encapsulation and with Banana anchor/CPE. Its 100% re-invention of existing art, IMO, but built on the argument that we need improved packet distribution scheme. In the process you are building a solution, IMO.

The aspect of CPE registering with the anchor, discovery of the anchor, path specific registrations, prefix negotiation for the ingress network from the overlay anchor, tunnel parameter negotiation, flow policy negotiation have to be part of your protocol. So, this is the big overlap and so this is not about few willing hands to do the work.

*** I know some others realizing the above with MobIKE extensions, and some others realizing the same based on MPTCP Anchor.

Sri

Date: Monday, September 18, 2017 at 4:15 PM

On Sep 18, 2017, at 6:30 PM, Sri Gundavelli (sgundave) <[email-masked]<mailto:[email-masked]>> wrote:
Generally, I agree with your above statement. If there is enough interest to do some new work and people ready to write some drafts and people ready to chair the meetings, creation of a WG should be the most logical thing. But, that goes with the assumption that the work is not overlapping with some other work that is happening some where else. That needs to be ensured and hence the need to agree upon the problem statement from both interested and non-interested parties. We need to review this overlap with QUIC, MPTCP, MobIKE and Mobile IPv6 protocols.

This is one of the reasons we have BOFs.  If you believe that there is some overlap with one of those protocols, or that one of those protocols already solves the problem, say that and explain how.  Also, perhaps include some explanation of what that protocol is not already being used to solve this problem, or what sort of changes or extensions would be needed to make it solve the problem more completely.  I am pretty sure that the IESG doesn't want to charter a group to solve an already-solved problem.

There are people solving this problem today, in the field, though, and they don't seem to be using the protocols you have mentioned.  Can you point to someone who is?

Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 18 Sep 2017 20:50:40 -0400

HI Sri,

> On Sep 18, 2017, at 8:20 PM, Sri Gundavelli (sgundave) <[email-masked]> wrote:
> 
> OK! I can provide that information. We have large scale deployments of the below based on Proxy Mobile IPv6 (100’s of thousands), with the ability to perform per-flow load balancing and per-packet and link-weighted packet distribution.  Many DSL operators have deployed this (DSL RG with LTE dongle)  and we have also deployed for many overlay applications.  You can check cisco CCO pages for the product details. For measuring path characteristics, we use IP SLA. Its a solution based on PMIP mobility and SLA for path performance measurement.  

Cool.

I tried to look up “Cisco CCO”, and that appears to be a type of Cisco Customer ID for logging into a support/documentation system.  Not being a Cisco Customer, I guess I don’t have “CCO pages”.  Could you provide details here, or links that are accessible to someone who isn’t a Cisco customer?  What is the product actually called, so that I can look for other sources of information about it?

Also, what is the expansion of “SLA”? The only expansion I know of is “Service Level Agreement"  (i.e. a legal contract), but I assume you are referring to some sort of protocol?

> Your problem statement is exactly this. The difference is that its Banana Signaling, Banana encapsulation and with Banana anchor/CPE. Its 100% re-invention of existing art, IMO, but built on the ar…

The existence of multiple deployed, proprietary solutions to this problem from multiple vendors only proves, to me anyway, that this is indeed an interesting problem to solve.  

Obviously, there could be advantages to having an open standard in this space for multi-vendor interoperability.   It sounds like the Cisco solution is based on open protocols, but that there might be proprietary extensions included (for the per-packet handling and signaling of performance data, etc.)?  Would you be willing to document your solution in Internet-Drafts to be considered for standardization?  If so, it sounds like this might be a very interesting candidate for BANANA work.

> The aspect of CPE registering with the anchor, discovery of the anchor, path specific registrations, prefix negotiation for the ingress network from the overlay anchor, tunnel parameter negotiation…

Although we don’t use the same terms, I believe the all of these things have also been considered in the Huawei GRE Tunnel Bonding solution (RFC 8127 and related documents), as well.  We have also had presentations on other solutions to this problem at various levels of specification and deployment.

We have more than just a few willing hands…  

Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 18 Sep 2017 23:14:52 -0400

One more question about this solution, Sri — is this an IPv6-only solution?  Or does it do bandwidth aggregation for IPv4 traffic over an IPv6 tunnel, or something similar?

Margaret

> From: Margaret Cullen <[email-masked] <mailto:[email-masked]>>
> To: Sri Gundavelli <[email-masked] <mailto:[email-masked]>>
> Cc: "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked] <mailto:[email-masked]>>, "Zhangmingui (Martin)" <[email-masked] <mailto:[email-masked]>>, Alexandre Petrescu <[email-masked] <mailto:[email-masked]>>, "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>
>> On Sep 18, 2017, at 6:30 PM, Sri Gundavelli (sgundave) <[email-masked] <mailto:[email-masked]>> wrote:

---

From: "Zhangmingui (Martin)" <[email-masked]>
Date: Tue, 19 Sep 2017 03:58:20 +0000

I cannot agree with you more.

It requires resources to execute a full set of drafts on problem statement, use case, requirements, gap analysis, framework, architecture, applicability etc. I don't think any of them are necessary at this stage. I don’t think it is necessary to step back to work on the problem statement either, since the problem, to me, had already been thoroughly discussed and is clear enough after two BOFs, several bar BOFs and dozens of email discussions. Actually, the charter of the current version neatly describes the problem. I can even see people is developing BANANA solutions. 

I wish this WG to be a solution-based rather than a issue-based one. Otherwise, I do not expect it to be a productive WG on working protocols out. 

Thanks,
Mingui

> From: Margaret Cullen [mailto:[email-masked]]
> Sent: Tuesday, September 19, 2017 3:19 AM
> To: Henderickx, Wim (Nokia - BE/Antwerp)
> Cc: Zhangmingui (Martin); Alexandre Petrescu; [email-masked]
> I have heard and understand that you do not feel that we should proceed with
> this work without a (potentially lengthy) process to document the requirements,
> do gap analysis, etc.  That opinion was raised at the BOF meeting in Prague, as
> well, where several people said that they did not support going through that sort
> of process, and our AD, Suresh, told us that he wouldn’t charter this group to go
> through that sort of process.   At the end of the BOF, when we asked the
> questions, a large majority of the people who responded indicated that they
> thought the problem was clear and well understood, and that the charter
> represented work we should do in the IETF.  I understand that there are a few
> of you who feel differently, and you are welcome to express your opinions, but I
> would say that there was a fairly strong agreement of the people in the room in
> Prague _not_ to change the charter to include a requirements/gap analysis phase,
> so I am not planning to do so.
> 
> Margaret
> 
> 
> > In a situation like this we should first do requirements and check gaps with
> existing protocols to ensure we go down the right direction. Given the scope is
> specified as multi-operator OTT deployment we need to ensure we understand
> all these implications.
> >
> > Referring to this:
> > Milestones
> >    Given the concerns about GRE and NATs, perhaps it would make sense to
> remove the explicit mention of GRE from the charter and add some wording
> about traversal of NATs and other middle boxes?  Maybe something along these
> lines?
> >
> >    OLD:
> designed to work in multi-provider scenarios (i.e. they will not depend on all of
> the aggregated links being provided by a single Internet access provider).
> >
> >    NEW:
> designed to work in multi-provider scenarios (i.e. they will not depend on all of
> the aggregated links being provided by a single Internet access provider, and they
> will be designed to work across NATs and other middle boxes, as needed).
> >
> >    OLD:
> >    Select (and extend, if necessary) an existing tunneling encapsulation (e.g.
> GRE)  for sending traffic between BANANA Boxes.
> >
> >    NEW:
> >    Do people think these changes would be an improvement to the existing
> charter text?  If there are no objections over the next few days, I will make
> these changes to the online charter text.
> >
> >    Margaret
> >
> >
> multi-operator, which means we will have to deal with NAT in multiple ways as
> long as we intend to use v6.
> >>
> >> So, the issue I have with the charter in general is that we are trying to define a
> protocols/signalling extensions, while we have not understood the requirements
> and have done a gap analysis regarding this. The first thing that should happen is
> look at the requirements and more importantly look at the algorithms we would
> need to accommodate these requirements. After do gap analysis for the existing
> protocols and then define the potential extensions. In the last step we should
> even see if other WG are not already suited to handle this activity.
> >>
> >>   For the second option, there are some existing implementations. But it is
> not an option if the Key field has already used for other purpose, e.g., security.
> >>
> before the GRE tunnel. Why do we have to insert a NAT device in between two
> GRE tunnel endpoints?
> >>
> >>   Thanks,
> >>   Mingui
> >>
> >>> If I hear GRE as a proposal it is very NAT unfriendly and the solution need to
> work across multiple providers, so this is an important consideration.
> >>

---

From: Suresh Krishnan <[email-masked]>
Date: Tue, 19 Sep 2017 01:13:25 -0400

Hi all,
  I think extreme positions on the suitability of the posted charter (“It is perfect” to “It is worthless”) are not likely going to help moving forward. The idea of putting up the proposed charter for review is to bring out opinions and to modify the charter to make it more widely applicable to a larger community. I am glad that we are having this discussion right now on the list. If you think the charter does not clearly describe a problem to be solved, that is something that certainly needs to be fixed. The best way to fix the charter is by proposing edits (add/modify/delete) and see if the participants in this ML are fine with those changes. Regarding conflicts, once the charter gets sent to me there will be further levels of review to see if the work conflicts with other ongoing IETF work (e.g. in mptcp) during the IESG Internal Review and the external review by the entire IETF community. So I am not very worried about this part. If people feel that writing down a clear problem statement is a way to make progress I am fine with starting with that too as long as the recommendations specified in the IESG statement at

https://www.ietf.org/iesg/statement/support-documents-in-ietf-wgs.html

are understood and followed.

Thanks
Suresh

> On Sep 18, 2017, at 11:58 PM, Zhangmingui (Martin) <[email-masked]> wrote:
>

---

From: <[email-masked]>
Date: Tue, 19 Sep 2017 07:14:57 +0000

Dear all,

> Bottomline, this work has no support. There is no vendor, SDO or operator interest; we need to have additional discussions. This is a classic case for IESG Appeal.

At this point I can tell you that there is an operator interest to work on it. As we have a running solution, we had and have dialogs with several operators in Europe. I would say that there is a need to develop something like BANANA.

The interest from all parties to support BBF was very high to define the "Hybrid Access Broadband Network architecture" TR-348. That should also give us hints that there is a potential interest on the BANANA topic.

Cornelius 

Deutsche Telekom Technik GmbH 
Fixed Mobile Engineering Deutschland 
Cornelius Heidemann
 
www.telekom.de    
Erleben, was verbindet.  
Die gesetzlichlichen Pflichtangaben finden Sie unter: www.telekom.de/pflichtangaben-dttechnik

Hi Margaret,

It is not just the question of counting hands in the room, I do not think that¹s the criteria for ³consensus² determination. We all have been in IETF long enough and we know that does not mean any thing. Sure, there may be few excited researchers/students who have no skin in the game to express interest in the work, but Standardization should have a product path. Are the key network vendors who have skin in the game have expressed interest for this work? Is there SDO interest from 3GPP, Wireless or Broadband guys? Clearly, I do not see that and there is no support for this work.

Also, many folks asked why there is a need for a new signaling protocol.
When I asked this question, some explicit text was added to say something along the lines, ³WG will not define a new signaling protocol, unless the existing protocols do not meet the requirement². I remember Mirja commenting on that and the chairs editing the text (Mirja can comment if I am wrong here). But, the below charter text conveniently removed that entire text and now shows a milestone for the signaling work. So, the work is getting steered not based on consensus, but on a pre-determined path.

Bottomline, this work has no support. There is no vendor, SDO or operator interest; we need to have additional discussions. This is a classic case for IESG Appeal.

‹-
€ Specify a signalling protocol that can be used to send control information between BANANA Boxes, including:
		€ IP Prefixes of access  links
		€ Information about link status and properties (including congestion)
		€ Information needed by BANANA Encapsulations
		€ Determining which flows are/are not eligible for BANANA Encapsulation
	€ Select (and extend, if necessary) an existing tunneling encapsulation (e.g. GRE)  for sending traffic between BANANA Boxes.

‹-

Sri

On 9/18/17, 12:18 PM, "Banana on behalf of Margaret Cullen"
<[email-masked] on behalf of [email-masked]> wrote:

>should do in the IETF.  I understand that there are a few of you who 
>feel differently, and you are welcome to express your opinions, but I 
>would say that there was a fairly strong agreement of the people in the 
>room in Prague _not_ to change the charter to include a 
>requirements/gap analysis phase, so I am not planning to do so.
>
>Margaret
>
>
>> In a situation like this we should first do requirements and check 
>>gaps with existing protocols to ensure we go down the right direction. 
>>Given the scope is specified as multi-operator OTT deployment we need 
>>to ensure we understand all these implications.
>> 
>> Referring to this:
>> Milestones
>>wording about traversal of NATs and other middle boxes?  Maybe 
>>something along these lines?
>> 
>>    OLD:
>>    The Bandwidth Aggregation solutions developed in this group will 
>>be designed to work in multi-provider scenarios (i.e. they will not 
>>depend on all of the aggregated links being provided by a single 
>>Internet access provider).
>> 
>>    NEW:  
>>    The Bandwidth Aggregation solutions developed in this group will 
>>be designed to work in multi-provider scenarios (i.e. they will not 
>>depend on all of the aggregated links being provided by a single 
>>Internet access provider, and they will be designed to work across 
>>NATs and other middle boxes, as needed).
>> 
>>    OLD:
>>> So, the issue I have with the charter in general is that we are 
>>>trying to define a protocols/signalling extensions, while we have not 
>>>accommodate these requirements. After do gap analysis for the 
>>>existing protocols and then define the potential extensions. In the 
>>>last step we should even see if other WG are not already suited to 
>>>handle this activity.
>>> 
>>> On 14/09/2017, 07:07, "Zhangmingui (Martin)" 
>>><[email-masked]>
>>>wrote:
>>> 
>>>   Hi Alex,
>>> 
>>>   For the second option, there are some existing implementations. 
>>>But it is not an option if the Key field has already used for other 
>>>purpose, e.g., security.
>>> 
>>>   However, I would say the popular usage is that the NAT happens 
>>>just before the GRE tunnel. Why do we have to insert a NAT device in 
>>>between two GRE tunnel endpoints?
>>> 
>>>   Thanks,
>>>   Mingui
>>> 
>>>   From: Banana [[email-masked]] on behalf of Henderickx, 
>>>Wim (Nokia - BE/Antwerp) [[email-masked]]
>>>   On 13/09/2017, 17:27, "[email-masked]" 
>>><[email-masked]>
>>>wrote:
>>> 
>>>       Hi Wim,
>>> 
>>>       Sent from my iPhone
>>>

---

From: <[email-masked]>
Date: Tue, 19 Sep 2017 08:30:11 +0000

Hi,

Von: Banana [mailto:[email-masked]] Im Auftrag von Sri Gundavelli (sgundave)
Gesendet: Dienstag, 19. September 2017 00:30
An: Margaret Cullen <[email-masked]>
Cc: Zhangmingui (Martin) <[email-masked]>; Alexandre Petrescu <[email-masked]>; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; [email-masked]
Betreff: Re: [Banana] Charter

> The IETF WG formation process focuses on making sure that there is an actual community
>   of people who want to do a well-understood set of work, that those people have the skills and
>  knowledge to do the work successfully, and that they represent enough segments of the
> industry to have a wide-enough perspective to do the work in a way that will be widely applicable.
> The IETF is set up to attract competent groups of people who want to do interesting work within
> the IETF's scope, and offer them an environment and support that will allow them to do a good job.

> Generally, I agree with your above statement. If there is enough interest to do some new work and people ready to write some
> drafts and people ready to chair the meetings, creation of a WG should be the most logical thing. But, that goes with the
> assumption that the work is not overlapping with some other work that is happening some where else. That needs to be
> ensured and hence the need to agree upon the problem statement from both interested and non-interested parties.
> We need to review this overlap with QUIC, MPTCP, MobIKE and Mobile IPv6 protocols.
My impression from the last IETF meeting was that there is sufficient support for that type of work. I agree that there are ongoing
discussions in other working groups which might address some of the BANANA aspects but I do not see any WG which addresses
the whole picture and intends to come up with a solution. The (proposed) charter also states that existing protocols will be
considered. Nobody want's to reinvent the wheel.

> We can define a new signaling protocol, just to solve this distribution problem, which can authenticate/authorize the peers,
> allows path registrations, or prefix allocations, but it will be exactly the same as some existing protocols, but with a new
> signaling semantics. So, what would be achieve from this effort? It will be another signaling protocol  called "Banana",
> which does exactly the same. I wonder, what is the point of this effort?

I don't think that we have the requirement to create a new signaling protocol called BANANA. The reuse of an existing
protocol in combination with some extensions is also a very valid approach.

Nic

---

From: Margaret Cullen <[email-masked]>
Date: Tue, 19 Sep 2017 08:14:07 -0400

> On Sep 19, 2017, at 1:13 AM, Suresh Krishnan <[email-masked]> wrote:
> 
> Hi all,
>  I think extreme positions on the suitability of the posted charter (“It is perfect” to “It is worthless”) are not likely going to help moving forward. The idea of putting up the proposed charter for review is to bring out opinions and to modify the charter to make it more widely applicable to a larger community. I am glad that we are having this discussion right now on the list.

I agree Suresh.  It’s good to get some feedback on the charter text that we can use to improve it.  It would also be good to get some feedback on proposed improvements, like the proposal I made to add mention of NAT boxes and remove mention of GRE, for instance.  

> If you think the charter does not clearly describe a problem to be solved, that is something that certainly needs to be fixed. The best way to fix the charter is by proposing edits (add/modify/delete) and see if the participants in this ML are fine with those changes. Regarding conflicts, once the charter gets sent to me there will be further levels of review to see if the work conflicts with other ongoing IETF work (e.g. in mptcp) during the IESG Internal Review and the external review by the entire IETF community. So I am not very worried about this part. If people feel that writing down a clear problem statement is a way to make progress I am fine with starting with that too as long as the recommendations specified in the IESG statement at
> 

I think this is an excellent statement, and will help us to avoid drawn out processes that exhaust participants and kill new work before it really gets started.

We actually talked during the BOF about the idea of having informal WG material on use cases, and I don’t think I captured that discussion in the charter.  I will extend that to the problem statement, and I will make a specific proposed wording change in another thread.  Perhaps that will help to resolve the concerns that the problem is not properly understood? 

There has also been an issue raised that the charter is insufficiently clear that the items listed (provisioning, signaling and encapsulation protocols) may not be new protocols, but may instead involve the reuse (and possible extension?) of existing protocols.  I will borrow from Nic Leymann’s wording and make a proposed change in another thread to address that concern.

If people have other concerns with the charter, or suggestions to improve it, it would be great to send them now, so that we can send the best possible version of this charter to the IESG and IETF for consideration.

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Tue, 19 Sep 2017 16:23:08 +0000

Hi Margaret,

PMIPv6 supports IPv4 or IPv6 for transport, so the tunnel can be based on IPv4-GRE, IPv6-GRE, IPv4-UDP, or IPv6-UDP. The overlay traffic (user plane) can be IPv4, IPv6 or IPv4/IPv6. All based on RFC standards.

With regards to my comment on IP SLA, there are various tools for measuring path performance.  These metrics include path-loss, delay, jitter, packet loss, packet sequencing, peer loss, Voice quality MOS ... These are achieved by sending various packet probes. The IP Performance guys in IETF can point of the available RFC publications on these approaches.

Regards
Sri

Date: Monday, September 18, 2017 at 8:14 PM

Margaret

Sri

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Tue, 19 Sep 2017 16:25:12 +0000

Hi Margaret,

> The existence of multiple deployed, proprietary solutions to this problem from multiple vendors only proves, to me anyway, that this is indeed an interesting problem to solve.

The issue is we are mixing protocol work with solution work. We don't do solutions, that is for the expertise of Vendors, and experts in SDO's at Broadband forum and 3GPP  .

I understand folks are interested in "Banana type of solution". But, that is not a problem statement that IETF can charter.   We need to state the requirement in more specific terms.

For example, if the goal is to define a "a better packet load-glancing mechanism given to two tunnel overlay paths", without any binding to a specific protocol, then its about considerations and the logic for packet spraying on two overlay tunnel paths.  We cannot bring operator, prefix allocations, tunnel setup mechanisms into the discussion.

Given the below pointers on existing art, if we can state the problem in more precise terms, then we can agree upon some charter text. In the current form with "solution type description",  this is a non-starter.

Sri

Date: Monday, September 18, 2017 at 5:50 PM

HI Sri,

Cool.

I tried to look up "Cisco CCO", and that appears to be a type of Cisco Customer ID for logging into a support/documentation system.  Not being a Cisco Customer, I guess I don't have "CCO pages".  Could you provide details here, or links that are accessible to someone who isn't a Cisco customer?  What is the product actually called, so that I can look for other sources of information about it?

Also, what is the expansion of "SLA"? The only expansion I know of is "Service Level Agreement"  (i.e. a legal contract), but I assume you are referring to some sort of protocol?

Although we don't use the same terms, I believe the all of these things have also been considered in the Huawei GRE Tunnel Bonding solution (RFC 8127 and related documents), as well.  We have also had presentations on other solutions to this problem at various levels of specification and deployment.

We have more than just a few willing hands...

Margaret

---

From: Tommy Pauly <[email-masked]>
Date: Tue, 19 Sep 2017 16:31:25 -0700

Hi Sri,

I think your point about the distinction between solutions and protocols is important for this discussion, and the heart of what you are getting at. To that end, I think that this work is indeed charterable by the IETF, and the proposed charter gives the group the flexibility to delve into these discussions further.

Can we frame the work of the WG as "updates to protocols to help build better standards-based solutions"? BANANA isn't set up to be a pure-protocol group; it is about adding options or recommending how to use protocols together. But neither is it simply a solution-based group. We shouldn't end up with a lot of MUST statements about how solutions should be deployed. Instead, we should be looking at existing protocols and identify the gaps or options or extensions we need to address in existing protocols to make it simpler to create interoperable non-proprietary solutions.

As an analogy for this work: having come from the VPN side of things, the IETF has defined protocol extensions for protocols like IPSec and IKE to fill in gaps that VPN solutions commonly needed to extend with proprietary solutions. By defining these protocol bits as standards, we can help encourage interoperability. The work of BANANA would help look at the opportunities in protocols that may be used for bandwidth aggregation solutions. This work may be all done in incremental documents, not necessarily large new protocols, but is still useful and applicable work.

Thanks,
Tommy

> On Sep 19, 2017, at 9:25 AM, Sri Gundavelli (sgundave) <[email-masked]> wrote:
> 
> Hi Margaret,
> 
> The issue is we are mixing protocol work with solution work. We don’t do solutions, that is for the expertise of Vendors, and experts in SDO’s at Broadband forum and 3GPP  .
> 
> I understand folks are interested in “Banana type of solution”. But, that is not a problem statement that IETF can charter.   We need to state the requirement in more specific terms.  
> 
> For example, if the goal is to define a “a better packet load-glancing mechanism given to two tunnel overlay paths”, without any binding to a specific protocol, then its about considerations and the logic for packet spraying on two overlay tunnel paths.  We cannot bring operator, prefix allocations, tunnel setup mechanisms into the discussion.
> 
> Given the below pointers on existing art, if we can state the problem in more precise terms, then we can agree upon some charter text. In the current form with "solution type description”,  this is a non-starter.
> 
> 
> Sri
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>> On Sep 18, 2017, at 8:20 PM, Sri Gundavelli (sgundave) <[email-masked] <mailto:[email-masked]>> wrote:
>>

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Wed, 20 Sep 2017 00:20:05 +0000

Hi Tommy,

Vendors have developed and deployed solutions for hybrid-access use-case (broad-band forum terminology). These are all based on existing IETF protocols and some vendor extensions for vendor differentiation (as in any technology). Now, if there is a non-standard extension in a given solution/protocol, one has the option to standardize that knob in that respective protocol group. If the solution is based on IPSec/IKEv2, there is ipsecme protocol group that is out there to standardize that option, there is MPTCP, QUIC, DMM working groups for other approaches. The protocol expertise resides in those groups. So, that option is available today and any one take the needed extensions and fix it in that WG.

The key issue with the Banana charter is lack of a clear problem statement, other than the description of a high-level solution requirement and Banana terminology. There is a not a single statement in the charter which points to a technical issue. I was hoping its was about packet load-balancing on overlay tunnel paths. That is a meaningful charter item with no relation to any specific solution/signaling protocol/encapsulation.

The idea of chartering a solution-type working group is new to me. I do not remember seeing such working group. The closest that comes to my mind is IPv6 group  focussing on deployments aspects to realize IPv6 more quickly, but clearly they were not chartered to do protocol extensions.  There has to be some value in going that route for this point solution.   Assuming we go that route, what would be deliverable of the WG?  Is it only documentation of various solutions and fix interop issues by engaging the respective protocol groups? Will the WG refrain from defining protocol extensions? Can we put that in the charter (no protocol extensions allowed)? Lets also look at the solutions that are out there. Will Banana pick one of these approach and start calling it Banana?  Or, because there are “n” solutions, Banana will standardize a new protocol/solution and call it Banana solution?

1. Multi path Support with PMIPv6

2.  Multi path Support with MobIKE

3. Multi path Support with DSMIPv6

4. Multi path Support with Static Tunnel Peer Provisioning

5. Multi path Support with QUIC/MPTCP Proxy (Client Initiated)

6. Multi path Support with QUIC/MPTCP Proxy (Proxy Initiated)

Regards
Sri

From: <[email-masked]<mailto:[email-masked]>> on behalf of Tommy Pauly <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, September 19, 2017 at 4:31 PM
To: Sri Gundavelli <[email-masked]<mailto:[email-masked]>>
Cc: Margaret Cullen <[email-masked]<mailto:[email-masked]>>, "Zhangmingui (Martin)" <[email-masked]<mailto:[email-masked]>>, Alexandre Petrescu <[email-masked]<mailto:[email-masked]>>, "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Banana] Charter

Hi Sri,

Thanks,
Tommy

On Sep 19, 2017, at 9:25 AM, Sri Gundavelli (sgundave) <[email-masked]<mailto:[email-masked]>> wrote:

Hi Margaret,

Sri

HI Sri,

Cool.

Margaret

_______________________________________________
Banana mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/banana

---

From: Margaret Wasserman <[email-masked]>
Date: Tue, 19 Sep 2017 22:55:10 -0400

Sri,

>Can we put that in the charter (no protocol extensions allowed)? 

How would that make sense for a group that is chartered to define three protocols?  Yes, we might reuse existing protocols, but we also might define some new ones, or we might define extensions to existing protocols to meet our needs.  

While there is no one complete proposal at this point, and it will be up to the WG to choose what approaches to use for each part of the solution, I could see this group defining multiple protocols and extensions, such as:  a DHCP option or RA record to provision the location of a centralized remote BANANA Box, or (more interestingly, IMO) a DNS record to find the remote BANANA box closest to a specific destination address.  A tunnel-based encapsulation (perhaps based on RFC 8157, or your MIP-based proposal if you choose to document it, or some other proposal), a description of how to use an MPTCP proxy (and maybe an MPQUIC Proxy, if that even makes sense?) in conjunction with a default tunneling solution to perform more optimal aggregration for TCP (or QUIC?) traffic in a coordinated fashion with the aggregation of other traffic, a conceptual description of the information that needs to be signaled between BANANA boxes for two-way load balancing and flow control, etc., and either a new protocol or extensions to an existing signaling protocol to send that information. Maybe we would also choose to write some type of "boxes and arrows" architecture document to describe how the pieces fit together.

It wouldn't make sense, in my opinion, to charter the WG in a way that would stop us from doing any of those things, if the WG has consensus to do them.

> Lets also look at the solutions that are out there. Will Banana pick one of these approach and start calling it Banana?  Or, because there are “n” solutions, Banana will standardize a new protocol/solution and call it Banana solution?   

If we don't end up writing a new protocol at all, there might be no protocol _named_ BANANA.   There would probably be BANANA options, records or extensions, though.  

If you believe that PMIPv6-based approach to this problem is the best solution, perhaps you should write a draft (or drafts) describing that solution, so the WG can consider it?  

Margaret
> From: <[email-masked]> on behalf of Tommy Pauly <[email-masked]>
> To: Sri Gundavelli <[email-masked]>
> Cc: Margaret Cullen <[email-masked]>, "Zhangmingui (Martin)" <[email-masked]>, Alexandre Petrescu <[email-masked]>, "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]>, "[email-masked]" <[email-masked]>
>> From: Margaret Cullen <[email-masked]>
>> To: Sri Gundavelli <[email-masked]>
>> Cc: "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]>, "Zhangmingui (Martin)" <[email-masked]>, Alexandre Petrescu <[email-masked]>, "[email-masked]" <[email-masked]>

---

From: Margaret Wasserman <[email-masked]>
Date: Tue, 19 Sep 2017 23:47:56 -0400

Hi Sri,

> 6. Multi path Support with QUIC/MPTCP Proxy (Proxy Initiated)
Given that the standard parts of this protocol (as you describe them) only includes flow-based distribution between the links, how would this solution allow a single flow to use the bandwidth of multiple links?  How do you control (or at least influence how reverse-path traffic is distributed?

I am at least somewhat familiar with MIP and PMIP (although I will grant that you are much more knowledgable about it), and this is the part I don't understand.  Are you performing packet-based distribution that using some sort of changes or extensions to PMIP?  Or is your solution purely flow based?

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Wed, 20 Sep 2017 04:03:30 +0000

Hi Margaret,

There are two parts to this; the approach of #1) registering multiple paths / establishing multiple tunnels and #2) the approach of flow distribution with flow spec negotiation.

In MIP, #1 is not dependent on #2. When there is no negotiated flow spec, the peers can adopt a locally configured traffic distribution scheme, which can be per-packet, or link-weighted packet distribution scheme .. or what ever is supported and configured on those two peers. In our implementation we do support both the modes, flow based on packet based.

Sri

From: Margaret Wasserman <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, September 19, 2017 at 8:47 PM
To: Sri Gundavelli <[email-masked]<mailto:[email-masked]>>
Cc: Tommy Pauly <[email-masked]<mailto:[email-masked]>>, Margaret Cullen <[email-masked]<mailto:[email-masked]>>, "Zhangmingui (Martin)" <[email-masked]<mailto:[email-masked]>>, Alexandre Petrescu <[email-masked]<mailto:[email-masked]>>, "Henderickx, Wim (Nokia - BE/Antwerp)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Banana] Charter

Hi Sri,

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Wed, 20 Sep 2017 04:04:35 +0000

Margaret,

>  If you believe that PMIPv6-based approach to this problem is the best solution, perhaps you should write a draft (or drafts) describing that solution, so the WG can consider it?

There is nothing like a best solution and that discussion is out of scope for IETF, IMO.  I have already talked about “n” different ways vendors are realizing this today. and PMIPv6 based solution is one among the many approaches. We have large scale deployments and so I do not need to write a draft on that approach.

I think what you are looking for is like an open charter for a solution with the right to define what ever protocol ( or protocols) needed for this solution. Without presenting any data as why those existing “n” mechanisms don’t work, having this in the charter makes no sense to me. If the only argument is, "there are people interested in working in this”,  that IMO is a weak argument, as most of those folks are draft authors.

I think we need much stronger consensus and industry wide participation, for obtaining such broad charter, allowing definition of new protocols based on some solution description.

Sri

Date: Tuesday, September 19, 2017 at 7:55 PM

Sri,

Margaret

Regards
Sri

Hi Sri,

Thanks,
Tommy

Hi Margaret,

Sri

HI Sri,

Cool.

Margaret

---

From: <[email-masked]>
Date: Wed, 20 Sep 2017 09:02:56 +0000

Sri,

<< The problem with the proposed charter text is the following. Our starting point was about a load-balancing problem. Given two overlay tunnel paths, how do I split the traffic between these two tunnel paths; is it flow-based routing, packet based routing, or weighted packet distribution, what should be the algorithm and considerations. If you had kept at this at this level, a study item on this might be a reasonable proposal, as we could have learnt if there are any gaps in this area. This is strictly a user-plane discussion and no need for signaling plane.  Instead, the charter talks about "Banana Boxes", "Banana Encapsulation", "Banana Signaling"; This is solution description. IETF does not do system architecture, that is in SDO scope; we work on protocols.>>

I think it's better for the IETF to define the protocol for the signalling between the Banana 'boxes'. The algorithm for splitting traffic between the paths can be left for vendors, I think.

There's an open question in my mind whether the split would be done per packet or per flow, and if that choice has implications on the signalling protocol between the Banana boxes.  I presume that if, as the charter says, it allows per packet split, then it also supports per flow support?

Best wishes,

phil

From: Banana [mailto:[email-masked]] On Behalf Of Sri Gundavelli (sgundave)
Sent: 18 September 2017 23:30
To: Margaret Cullen <[email-masked]>
Cc: Zhangmingui (Martin) <[email-masked]>; Alexandre Petrescu <[email-masked]>; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; [email-masked]
Subject: Re: [Banana] Charter

Sri

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Wed, 20 Sep 2017 13:29:48 +0000

Hi Phil,

The only technical point that I have seen in all the Banana discussions over the last 2+ years was about per-packet load balancing. How do you support per-packet load balancing given two tunnel overlay paths. In the past IETF transport AD always discouraged this mode. This is a generic  problem, can be used with controller initiated static  (SDWAN) tunnels, IPsec based tunnels, PMIP tunnels..etc. We dont need a new signaling protocol for this, you can use any of the existing protocols. Its a flag in DSMIP IFOM 6088/6089 or in IKEv2 RFC. Why do you need a new protocol for this? If the algorithm is left for vendors as the case today, what are we developing here? What is the point of this effort? How is new signaling between "banana" boxes helping? What is it solving?

Regards
Sri

On Sep 20, 2017, at 2:03 AM, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>> wrote:

Sri,

<< The problem with the proposed charter text is the following. Our starting point was about a load-balancing problem. Given two overlay tunnel paths, how do I split the traffic between these two tunnel paths; is it flow-based routing, packet based routing, or weighted packet distribution, what should be the algorithm and considerations. If you had kept at this at this level, a study item on this might be a reasonable proposal, as we could have learnt if there are any gaps in this area. This is strictly a user-plane discussion and no need for signaling plane.  Instead, the charter talks about "Banana Boxes", "Banana Encapsulation", "Banana Signaling”; This is solution description. IETF does not do system architecture, that is in SDO scope; we work on protocols.>>

Best wishes,

phil

From: Banana [mailto:[email-masked]] On Behalf Of Sri Gundavelli (sgundave)
Sent: 18 September 2017 23:30
To: Margaret Cullen <[email-masked]<mailto:[email-masked]>>
Cc: Zhangmingui (Martin) <[email-masked]<mailto:[email-masked]>>; Alexandre Petrescu <[email-masked]<mailto:[email-masked]>>; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Banana] Charter

> The IETF WG formation process focuses on making sure that there is an actual community of people who want to do a well-understood set of work, that those people have the skills and knowledge to do the work successfully, and that they represent enough segments of the industry to have a wide-enough perspective to do the work in a way that will be widely applicable.  The IETF is set up to attract competent groups of people who want to do interesting work within the IETF’s scope, and offer them an environment and support that will allow them to do a good job.

The problem with the proposed charter text is the following. Our starting point was about a load-balancing problem. Given two overlay tunnel paths, how do I split the traffic between these two tunnel paths; is it flow-based routing, packet based routing, or weighted packet distribution, what should be the algorithm and considerations. If you had kept at this at this level, a study item on this might be a reasonable proposal, as we could have learnt if there are any gaps in this area. This is strictly a user-plane discussion and no need for signaling plane.  Instead, the charter talks about "Banana Boxes", "Banana Encapsulation", "Banana Signaling”; This is solution description. IETF does not do system architecture, that is in SDO scope; we work on protocols.

Sri

---

From: David Sinicrope <[email-masked]>
Date: Wed, 20 Sep 2017 21:24:31 +0000

Hi All,

From the initial discussions of this work a concern was raised (with the authors, ADs, and IAB) that BANANA proposed work overlapping with the Hybrid Access work in the Broadband Forum (BBF) (See TR-348, and WT-378) which deals with both fixed and mobile provider (not solely single provider) multiple access. 
 
There was accommodation of the point of view that it could be useful to explore the multi provider, over the top cases, but the pursuit of any work that would compete with or be considered to overlap the BBF Hybrid Access project, which was well underway at the time BANANA started, would potentially jeopardize IETF's relations with the BBF and was therefore discouraged.   
 
Subsequent to the concerns noted above, the BBF has embarked on a project in cooperation with 3GPP to converge fixed access and the 5G core which would include hybrid access scenarios and fixed wireless access.  BBF and 3GPP currently have work underway and are looking to produce documents in the 3GPP Rel 16 timeframe.  
 
Over the course of the BANANA charter creation this positioning (i.e., BANANA = multi provider, OTT, not in conflict with the BBF) seems to have been increasingly diluted, resulting in the existing charter which, as noted below, gives license to deal with the entire space and seemingly casts aside the previous scope positioning and concerns.  In doing so, the charter has BANANA overlapping the work of the BBF creating a potentially adversarial situation, and jeopardizing what is otherwise a cordial and productive working relationship between BBF and the IETF.

I would suggest:
- clearly address the remaining substantive concerns about the clarity of the problem, scope and requirements, etc.
- Explicitly scope the work to deal with only the multi provider, over the top environment.  This clearly shows it is out of scope for BBF.
- liaise the charter and milestones/work plan to the BBF asking for their input - given the subtle differences in the work and plans, even with the non-overlapping scope, it would greatly help relations to ask before completing the charter process.
 
Best Regards,
Dave Sinicrope
BBF/IETF Liaison Manager

On 9/18/17, 2:06 AM, "Banana on behalf of David Allan I" <[email-masked] on behalf of [email-masked]> wrote:

    
    Dave

    -----Original Message-----
    From: Banana [mailto:[email-masked]] On Behalf Of Margaret Cullen
    Sent: Friday, September 15, 2017 12:03 AM
    To: Sri Gundavelli (sgundave) <[email-masked]>
    Cc: Alexandre Petrescu <[email-masked]>; Muley, Praveen (Nokia - US/Mountain View) <[email-masked]>; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; [email-masked]
    Subject: Re: [Banana] Charter

    Hi Sri,

    

    

    

    

    Margaret

---

From: Florin Baboescu <[email-masked]>
Date: Wed, 20 Sep 2017 21:08:19 -0700

Dear all,

Sorry for stepping late into this conversation . As I'm following both this
group and BBF as well as few other groups in 3GPP, I  can only agree with
David's comments below.
The obvious point is that the problem statement is not clear. From this
point of view, I would like to suggest the group to meet and spend time in
coming up with an agreeable problem statement. We should  at least make sure
this does not overlap with the BBF work.
This is why I simply do not see the need of rushing into chartering this
group without a clear understanding of what the group is trying to address.
Thank you.
-Florin Baboescu

-----Original Message-----
From: Banana [mailto:[email-masked]] On Behalf Of David Sinicrope
Sent: Wednesday, September 20, 2017 2:25 PM
To: David Allan I; Margaret Cullen; Sri Gundavelli (sgundave)
Cc: Alexandre Petrescu; Muley, Praveen (Nokia - US/Mountain View);
Henderickx, Wim (Nokia - BE/Antwerp); [email-masked]
Subject: Re: [Banana] Charter

Hi All,

>From the initial discussions of this work a concern was raised (with the
authors, ADs, and IAB) that BANANA proposed work overlapping with the Hybrid
Access work in the Broadband Forum (BBF) (See TR-348, and WT-378) which
deals with both fixed and mobile provider (not solely single provider)
multiple access.

There was accommodation of the point of view that it could be useful to
explore the multi provider, over the top cases, but the pursuit of any work
that would compete with or be considered to overlap the BBF Hybrid Access
project, which was well underway at the time BANANA started, would
potentially jeopardize IETF's relations with the BBF and was therefore
discouraged.

Subsequent to the concerns noted above, the BBF has embarked on a project in
cooperation with 3GPP to converge fixed access and the 5G core which would
include hybrid access scenarios and fixed wireless access.  BBF and 3GPP
currently have work underway and are looking to produce documents in the
3GPP Rel 16 timeframe.

Over the course of the BANANA charter creation this positioning (i.e.,
BANANA = multi provider, OTT, not in conflict with the BBF) seems to have
been increasingly diluted, resulting in the existing charter which, as noted
below, gives license to deal with the entire space and seemingly casts aside
the previous scope positioning and concerns.  In doing so, the charter has
BANANA overlapping the work of the BBF creating a potentially adversarial
situation, and jeopardizing what is otherwise a cordial and productive
working relationship between BBF and the IETF.

I would suggest:
- clearly address the remaining substantive concerns about the clarity of
the problem, scope and requirements, etc.
- Explicitly scope the work to deal with only the multi provider, over the
top environment.  This clearly shows it is out of scope for BBF.
- liaise the charter and milestones/work plan to the BBF asking for their
input - given the subtle differences in the work and plans, even with the
non-overlapping scope, it would greatly help relations to ask before
completing the charter process.

Best Regards,
Dave Sinicrope
BBF/IETF Liaison Manager

On 9/18/17, 2:06 AM, "Banana on behalf of David Allan I"
<[email-masked] on behalf of [email-masked]> wrote:

    I have to admit, my original expectation of the outcome of the
discussions was what the focus would be on configuration of the end points
for a non-carrier-integrated OTT scenario.

    What I am reading in the charter is license to explore the whole space,
and invent additional solutions even in the presence of suitable work by
other WGs....

    Dave

    -----Original Message-----
    From: Banana [mailto:[email-masked]] On Behalf Of Margaret
Cullen
    Sent: Friday, September 15, 2017 12:03 AM
    To: Sri Gundavelli (sgundave) <[email-masked]>
    Cc: Alexandre Petrescu <[email-masked]>; Muley, Praveen
(Nokia - US/Mountain View) <[email-masked]>; Henderickx, Wim
(Nokia - BE/Antwerp) <[email-masked]>; [email-masked]
    Subject: Re: [Banana] Charter

    Hi Sri,

    > IETF 98 BOF meeting (non-WG forming) either on the problem statement.

    Actually, the non-WG forming BOF was held at IETF-97, and it is true
that we did not yet have agreement on a clear and well-understood problem
statement at that time. We didn’t hold a BOF at IETF-98.  Instead, about 30
of us met for a publicly-scheduled "Bar BOF" and discussed the problem
statement and possible charter text. There was also _considerable_
discussion of the problem statement on the mailing list between IETF-97 and
IETF-99, and we reached general agreement on our problem statement on the
mailing list.

    "The BANdwidth Aggregation for Network Access (BANANA) Working Group is
chartered to develop solution(s) to support dynamic path selection on a
per-packet basis in networks that have more than one point of attachment to
the Internet.

    Bandwidth Aggregation consists of splitting local traffic across
multiple Internet links on a per-packet basis, including the ability to
split a single flow across multiple links when necessary.

    	• Higher Per-Flow Bandwidth: Many Internet links available to homes and
small offices (DSL, Cable, LTE, Satellite, VPNs, etc.) have relatively low
bandwidth. Users may wish to run applications (such as streaming video, or
content up/downloads) that require (or could benefit from) more bandwidth
for a single traffic flow than is available on any of the local links. A
Bandwidth Aggregation solution could supply the needed bandwidth by
splitting a single traffic flow across multiple Internet links.
    	• Reduced Cost: Traffic sharing on a per-packet basis allows the full
bandwidth of the lowest-cost link to be used first, only using a higher-cost
link when the lowest-cost link is full.
    	• Increased Reliability: When one Internet link goes down, ongoing
application flows can be moved to another link, preventing service
disruption.”

    We went through the charter text, sentence by sentence, at the BOF and
the only change we made to the problem statement portion of the charter was
to add VPNs to the list of low bandwidth Internet links.  There was an
opportunity to discuss every sentence in this section, but people didn’t
have much to say about it, probably because we had already spent months
discussing it on the list and in a Bar BOF.

    During the question period at the end of the BOF, we asked whether the
problem was clear and well-understood.  The large majority of  the people
who responded said “yes”.  This was recorded in the minutes as a “decent
 hum” for yes, and a “low murmur” for no.

    Based on the agreement on the mailing list and in the room in Prague, I
would say that we _do_ have agreement on a clear and understandable problem
statement.  What makes you think we don’t?

    I am sure that Suresh can be trusted to review things carefully before
he puts our charter on the IESG agenda.  If there are any particular notes
or discussion that you would like him to review, please feel free to draw
his attention to them.

    Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Thu, 21 Sep 2017 11:54:33 -0400

Hi Dave,

It is not our intent to overlap with work in the BBF or the 3GPP.  It is also quite explicitly our intention to develop a multi-provider, “over-the-top” solution.  If that is unclear in the charter, we should make it clearer.

You wrote:
> Over the course of the BANANA charter creation this positioning (i.e., BANANA = multi provider, OTT, not in conflict with the BBF) seems to have been increasingly diluted, resulting in the existing charter which, as noted below, gives license to deal with the entire space and seemingly casts aside the previous scope positioning and concerns.  

I’m not sure how this message has been “diluted", so I will need your help in un-diluting it.  Is there something in particular that was included in previous descriptions of this work that has been (perhaps unintentionally) removed from the current charter?  This is essentially the same charter text we have been discussing for 6+ months, and the same work we have been discussing for multiple years.

To break down your two points:  
(1) The charter is unclear that this is intended to be a multi-provider solution.  The charter says:

"The Bandwidth Aggregation solutions developed in this group will be designed to work in multi-provider scenarios (i.e. they will not depend on all of the aggregated links being provided by a single Internet access provider)."

How would you change that sentence to make it clearer that these solutions will be multi-provider?  Would it help to say something about the BANANA solutions not requiring explicit cooperation between providers?  I think it would be acceptable to the group to add that wording, as we have been talking about solutions that would work with any set of providers, not just with a single provider or a cooperating set of providers.

(2) The charter does not say that this will be an OTT solution.

It is true that the charter does not explicitly say that we will develop an “over-the-top” solution. It is, however, the scope of the IETF to do IP-based, link-layer-independent work (except in unusual cases), and this work is no exception.  We don’t usually use the term OTT in the IETF, but I wouldn’t object to explicitly saying that this work will be an IP-based and link-layer-independent, if it would make things clearer for people who are not familiar with the usual scope of IETF work.  If there is some other property of OTT solutions that you think is important here, please let me know what it is.

Putting these together, along with the NAT change discussed earlier, I think we would end up with the following text:

"The Bandwidth Aggregation solutions developed in this group will work in true multi-provider scenarios (i.e. they will not depend on all of the aggregated links being provided by a single Internet access provider nor by a group of cooperating providers).  Any protocols defined by this group will be IP-based, link-layer-independent solutions, and they will be designed to work across NATs and other middle boxes, as needed."

Would this change address your concerns?  If so, I will post it in another thread for feedback from the list.  If not, could you suggest a specific change that would address your concerns?

> You also wrote:

Prior to the announcement of the 5G work, it had been stated openly that the BBF planned to produce a requirements document for “hybrid access”, and then the BBF would look at protocols from other standards bodies, such as the IETF, to determine how to meet those requirements.  The BANANA effort was blocked (for over a year, with no discussion on the BANANA list or even a statement on the list explaining why) until the BBF’s requirements were published, so that our work would not interfere (in some unspecified way) with the BBF requirements effort.  After the BBF requirements document was published, the BBF withdrew its objection to the BANANA work proceeding in the IETF.  

Now, the BBF is starting a _new_ standardization effort (started _much later_ than the BANANA work), and you would like the IETF to further block or limit the BANANA work to avoid conflicting with this new effort.   It seems to me that the BBF is the one expanding the scope of their work, not the BANANA group.  

Personally, I object strongly to the notion that the IETF should block or reject work on end-user-controlled, multi-provider, IP-based, link-layer-independent solutions because they “conflict" with provider-controlled (single-provider or cooperating-provider) solutions, or with solutions that are only applicable to specific link layer technologies (i.e. BBF/5G) thus only serving a subset of Internet users. 

Dave, what hat were you wearing when you wrote this message?  Ordinarily if the BBF’s liaison to the IETF wrote to a WG or BOF mailing list that I was chairing claiming some sort of conflict with the BBF’s work and using the threatening tone you have assumed here, I would talk to the IETF’s liaison to the BBF and  get their help in resolving the situation.  In this case, though, the IETF’s liaison to the BBF is also you.  So, I guess I have no recourse but to ask you to help us figure out how to position the BANANA work so that the BBF will understand that we are intending to do a true multi-provider, OTT solution, that this work is in scope for the IETF, and that it does not conflict with the BBF’s work.  Do you think you will be willing and able to do that?  

Margaret

---

From: Suresh Krishnan <[email-masked]>
Date: Thu, 21 Sep 2017 23:47:57 -0400

Hi Margaret,

> On Sep 21, 2017, at 11:54 AM, Margaret Cullen <[email-masked]> wrote:
> 
> Hi Dave,
> 

Can you please elaborate on the things you mention in this paragraph? Specifically "The BANANA effort was blocked (for over a year, with no discussion on the BANANA list or even a statement on the list explaining why)”.

> Personally, I object strongly to the notion that the IETF should block or reject work on end-user-controlled, multi-provider, IP-based, link-layer-independent solutions

I think this is a strawman. Is there anybody suggesting this? 

> because they “conflict" with provider-controlled (single-provider or cooperating-provider) solutions, or with solutions that are only applicable to specific link layer technologies (i.e. BBF/5G) thus only serving a subset of Internet users. 

In my view the conflict is about whether the the "provider-controlled (single-provider or cooperating-provider)" part is in scope of the charter or not. I don’t think we can have a meaningful conversation unless we state this in the charter one way or another.

Thanks
Suresh

---

From: "Zhangmingui (Martin)" <[email-masked]>
Date: Fri, 22 Sep 2017 09:05:38 +0000

Hi Suresh,

> In my view the conflict is about whether the the "provider-controlled
> (single-provider or cooperating-provider)" part is in scope of the charter or not. I
> don’t think we can have a meaningful conversation unless we state this in the
> charter one way or another.

Surely we do not have conflict for the cooperating-provider scenario. But I want to argue there is no conflict even for the single-provider part.

TR-348 is "Hybrid Access Broadband Network Architecture". I knew contributors from vendors and operators worked closely to develop a good set of requirements. 75 of them. They are the core asset of TR-348. Since the TR is public, people can easily download it and check. (https://www.broadband-forum.org/technical/download/TR-348.pdf ) The title of WT-378 is "Nodal Requirements for Hybrid Access Broadband Networks". It's not published yet but we can be sure it will not develop specific protocols either. Working Texts and Technical Reports in BBF often include references to RFCs and working group documents. WT-378 is not an exception. We should understand this manner as the encouragement of protocol development in IETF rather than discouragement. BBF is not the sole organization that relies on some standardized protocols developed by IETF. There are many others. I believe this is a good way of cooperation rather than competition.

If we regarded BANANA effort as competition, then we have many more. For example, Section 5.4.3 of TR-348 points to the two-ended-proxy effort in the MPTCP WG. We also consider it as a competition? We all know it doesn't make any sense. 

We are fear of undermining the good relationship between IETF and BBF so we don't step into the area that might be covered by BBF. Is it an overkill caused by such fear to scope the development of protocols for a single provider scenario out of BANANA WG? If we face the fear, we would find BBF is actually asking for help from IETF. How can we just sit by and watch?

In my view, the documents developed by BBF can be regarded as external support documents of BANANA. Since BANANA WG is to be chartered not to formally publish support documents, I think this kind of conflicts can be nicely avoided. In this sense, the option to develop formal problem statement and requirements documents is more likely to cause conflicts.

Thanks,
Mingui

> From: Banana [mailto:[email-masked]] On Behalf Of Suresh Krishnan
> Sent: Friday, September 22, 2017 11:48 AM
> To: Margaret Cullen
> Cc: David Allan I; Muley, Praveen (Nokia - US/Mountain View); Sri Gundavelli
> (sgundave); Alexandre Petrescu; David Sinicrope; Henderickx, Wim (Nokia -
> BE/Antwerp); [email-masked]
> > It is not our intent to overlap with work in the BBF or the 3GPP.  It is also quite
> explicitly our intention to develop a multi-provider, “over-the-top” solution.  If
> that is unclear in the charter, we should make it clearer.
> >
> > You wrote:
> BANANA = multi provider, OTT, not in conflict with the BBF) seems to have been
> increasingly diluted, resulting in the existing charter which, as noted below, gives
> license to deal with the entire space and seemingly casts aside the previous
> scope positioning and concerns.
> >
> > I’m not sure how this message has been “diluted", so I will need your help in
> un-diluting it.  Is there something in particular that was included in previous
> descriptions of this work that has been (perhaps unintentionally) removed from
> the current charter?  This is essentially the same charter text we have been
> discussing for 6+ months, and the same work we have been discussing for
> multiple years.
> >
> > "The Bandwidth Aggregation solutions developed in this group will be designed
> to work in multi-provider scenarios (i.e. they will not depend on all of the
> aggregated links being provided by a single Internet access provider)."
> >
> > How would you change that sentence to make it clearer that these solutions
> will be multi-provider?  Would it help to say something about the BANANA
> solutions not requiring explicit cooperation between providers?  I think it would
> be acceptable to the group to add that wording, as we have been talking about
> solutions that would work with any set of providers, not just with a single
> provider or a cooperating set of providers.
> >
> > It is true that the charter does not explicitly say that we will develop an
> “over-the-top” solution. It is, however, the scope of the IETF to do IP-based,
> link-layer-independent work (except in unusual cases), and this work is no
> exception.  We don’t usually use the term OTT in the IETF, but I wouldn’t
> object to explicitly saying that this work will be an IP-based and
> link-layer-independent, if it would make things clearer for people who are not
> familiar with the usual scope of IETF work.  If there is some other property of
> OTT solutions that you think is important here, please let me know what it is.
> >
> > "The Bandwidth Aggregation solutions developed in this group will work in true
> multi-provider scenarios (i.e. they will not depend on all of the aggregated links
> being provided by a single Internet access provider nor by a group of cooperating
> providers).  Any protocols defined by this group will be IP-based,
> link-layer-independent solutions, and they will be designed to work across NATs
> and other middle boxes, as needed."
> >
> > Would this change address your concerns?  If so, I will post it in another thread
> for feedback from the list.  If not, could you suggest a specific change that
> would address your concerns?
> >
> >> You also wrote:
> >> Subsequent to the concerns noted above, the BBF has embarked on a project
> in cooperation with 3GPP to converge fixed access and the 5G core which would
> > Prior to the announcement of the 5G work, it had been stated openly that the
> BBF planned to produce a requirements document for “hybrid access”, and then
> the BBF would look at protocols from other standards bodies, such as the IETF, to
> determine how to meet those requirements.  The BANANA effort was blocked
> (for over a year, with no discussion on the BANANA list or even a statement on
> the list explaining why) until the BBF’s requirements were published, so that our
> work would not interfere (in some unspecified way) with the BBF requirements
> effort.  After the BBF requirements document was published, the BBF withdrew
> its objection to the BANANA work proceeding in the IETF.
> 
> Can you please elaborate on the things you mention in this paragraph?
> Specifically "The BANANA effort was blocked (for over a year, with no discussion
> on the BANANA list or even a statement on the list explaining why)”.
> 
> >
> >
> > Now, the BBF is starting a _new_ standardization effort (started _much later_
> than the BANANA work), and you would like the IETF to further block or limit the
> BANANA work to avoid conflicting with this new effort.   It seems to me that
> the BBF is the one expanding the scope of their work, not the BANANA group.
> >
> > Personally, I object strongly to the notion that the IETF should block or reject
> work on end-user-controlled, multi-provider, IP-based, link-layer-independent
> solutions
> 
> > because they “conflict" with provider-controlled (single-provider or
> cooperating-provider) solutions, or with solutions that are only applicable to
> specific link layer technologies (i.e. BBF/5G) thus only serving a subset of Internet
> users.
> 
> In my view the conflict is about whether the the "provider-controlled
> (single-provider or cooperating-provider)" part is in scope of the charter or not. I
> don’t think we can have a meaningful conversation unless we state this in the
> charter one way or another.
> 
> Thanks
> Suresh
>

---

From: Margaret Cullen <[email-masked]>
Date: Fri, 22 Sep 2017 08:57:23 -0400

Hi Suresh,

> Can you please elaborate on the things you mention in this paragraph? Specifically "The BANANA effort was blocked (for over a year, with no discussion on the BANANA list or even a statement on the…

I don’t know a lot more about this than I said, but I can give you a bit more of the history, I guess…  A couple of years ago, when Brian Haberman was still an Internet AD, we first approached him about bringing the BANANA work into the IETF.  We were told at the time that we couldn’t bring this work into the IETF because of a conflict with the BBF.  Since it was my understanding that the BBF was doing requirements work, and we were planning to do solutions work, I didn’t understand why there was a conflict.  Brian set up a meeting between me and Dave Sincrope that Brian did not attend.  I think David Allan was there, too?  At that meeting, I was told that the IETF and BBF had agreed that the IETF would not do work in this area until the BBF had completed their requirements work, after which the BBF would be looking to the IETF and others to provide solutions in this space, and our work would be unblocked.  I wasn’t very happy with that agreement, but it was clear that I was being informed of the decision not being asked what I thought of it.  I don’t know what group of people made that decision or where it was documented, if anywhere.  The bottom line was that our AD had told us we couldn’t charter this work in the IETF until the BBF finished their requirements document, so we waited.  After the requirements work was published, the work seemed to be unblocked and we have been working with you, Suresh, to charter this work in the IETF.

[Somewhat tangentially, I was also working on an Independent Stream RFC documenting Huawei’s deployed solution in this space, and that document was also blocked until the BBF completed it’s requirements work.  Once the requirements work was completed, that work was unblocked, and the document has been published as RFC 8157.]

I know that the IAB and IESG have advised us to focus the BANANA work on multi-provider solutions in response to our initial BOF request, and the group has agreed with that advice.  I personally think that the multi-provider case is a much more interesting problem space, and more central to the IETF’s scope.  Our charter currently states (as it has for the past 6+ months) the our solutions will be designed for multi-provider scenarios.  So, it is not that I object, at all, to that scope.  I have suggested text to Dave to make the scope of the work clearer, and if he says that it will address his concerns, I will call it out in a separate thread to see if the rest of the group has any comments.

In Dave Sincrope’s recent message, though, it seemed to me that he was claiming that we have a new agreement with the BBF to only do multi-provider, OTT work in this space that does not overlap with the new BBF/5G work?  Am I misunderstanding something there?  If there is such an agreement, when and how was that agreement made, who was involved in making it, and where was it recorded?  

Dave is also clearly suggesting that we should seek BBF review and acceptance of the BANANA charter before we charter this work in the IETF.  Was the IETF asked to review and accept the charter for the BBF/5G cooperation before that work was officially started in the BBF?  If so, where was that discussed, and what was our reply?

> I think this is a strawman. Is there anybody suggesting this? 

Sorry, this was probably an overreaction on my part.  While I still agree with what I said, please allow me to withdraw the implication that anyone might be trying to do otherwise.  
> In my view the conflict is about whether the the "provider-controlled (single-provider or cooperating-provider)" part is in scope of the charter or not. I don’t think we can have a meaningful conve…

The charter currently says that we will work on solutions for multi-provider scenarios, and it has said that for the past 6+ months.  I have proposed some wording to make that clearer and stronger, and I am waiting to see if that wording addresses Dave’s concern before asking for feedback on that change from the entire BANANA group.  If this is really all that is necessary to resolve a possible conflict with the BBF, then there should be no problem, at all.

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Fri, 22 Sep 2017 15:53:20 +0000

Hi Suresh,

>(single-provider or cooperating-provider)" part is in scope of the
>charter or not. I don¹t think we can have a meaningful conversation
>unless we state this in the charter one way or another.

Even if the charter restricts the group from working on only
multi-provider solution (or vice-versa), what does it mean from the IETF
protocol point of view. We don¹t model provider relations in protocols
(other than some path attributes in BGP). Will ³Banana solution² defined
for multi-provider stop working for single provider? Or, we will put an
artificial restriction that this solution MUST NOT be used for single
provider? How do I read that?

IF we step back, the motivation for this work is DSL + LTE aggregation. If
BBF and 3GPP have a work item, they will possibly define a initiator
function which can collocate with the RG and a terminating/aggregating
function on either PGW or on the BNG. Within these entities they have the
choice to use GTP (most likely), PMIPv6, L2TP, MPTCP, MPQUIC, or a Static
tunnel with a controller based interface. If their evaluation of all IETF
protocols result in the conclusion that they need a new signaling
protocol, they will initiate that request to IETF and then Banana WG can
go into some rapid action. However, if they decide to use one of the
existing protocol and need a protocol extension, they will request the
same from that respective protocol group. They may also choose not to go
with an overlay approach/IETF protocols, as they will argue they can
leverage inband QoS signaling for leveraging transport QoS and map between
EPC QoS bearers and service flows on the BBF access; they can provide
single QoS control point. What would be our argument on why any overlap
approach is better in this context?

So, I am wondering who is the customer for this work? Lets also not forget
the 7 other solution approaches that are out there. If the primary stake
holders are asking "not to" start this work, why would we charter
³solution² work? Why would we do that and who uses this standard?

Sri

On 9/21/17, 8:47 PM, "Suresh Krishnan" <[email-masked]> wrote:

>> It is not our intent to overlap with work in the BBF or the 3GPP.  It
>>is also quite explicitly our intention to develop a multi-provider,
>>³over-the-top² solution.  If that is unclear in the charter, we should
>>make it clearer.
>> 
>> You wrote:
>>>BANANA = multi provider, OTT, not in conflict with the BBF) seems to
>>>have been increasingly diluted, resulting in the existing charter
>>>which, as noted below, gives license to deal with the entire space and
>>>seemingly casts aside the previous scope positioning and concerns.
>> 
>> I¹m not sure how this message has been ³diluted", so I will need your
>>help in un-diluting it.  Is there something in particular that was
>>included in previous descriptions of this work that has been (perhaps
>>unintentionally) removed from the current charter?  This is essentially
>>the same charter text we have been discussing for 6+ months, and the
>>same work we have been discussing for multiple years.
>> 
>> "The Bandwidth Aggregation solutions developed in this group will be
>>access provider)."
>> How would you change that sentence to make it clearer that these
>>solutions will be multi-provider?  Would it help to say something about
>>the BANANA solutions not requiring explicit cooperation between
>>providers?  I think it would be acceptable to the group to add that
>>wording, as we have been talking about solutions that would work with
>>any set of providers, not just with a single provider or a cooperating
>>set of providers.
>> 
>> It is true that the charter does not explicitly say that we will
>>develop an ³over-the-top² solution. It is, however, the scope of the
>>IETF to do IP-based, link-layer-independent work (except in unusual
>>cases), and this work is no exception.  We don¹t usually use the term
>>OTT in the IETF, but I wouldn¹t object to explicitly saying that this
>>work will be an IP-based and link-layer-independent, if it would make
>>things clearer for people who are not familiar with the usual scope of
>>IETF work.  If there is some other property of OTT solutions that you
>>think is important here, please let me know what it is.
>> 
>> Putting these together, along with the NAT change discussed earlier, I
>>think we would end up with the following text:
>> 
>> "The Bandwidth Aggregation solutions developed in this group will work
>>in true multi-provider scenarios (i.e. they will not depend on all of
>>the aggregated links being provided by a single Internet access provider
>>nor by a group of cooperating providers).  Any protocols defined by this
>>group will be IP-based, link-layer-independent solutions, and they will
>>be designed to work across NATs and other middle boxes, as needed."
>> 
>> Would this change address your concerns?  If so, I will post it in
>>another thread for feedback from the list.  If not, could you suggest a
>>specific change that would address your concerns?
>> 
>>> You also wrote:
>>> Subsequent to the concerns noted above, the BBF has embarked on a
>>>project in cooperation with 3GPP to converge fixed access and the 5G
>>>core which would include hybrid access scenarios and fixed wireless
>>>access.  BBF and 3GPP currently have work underway and are looking to
>>>produce documents in the 3GPP Rel 16 timeframe.
>> 
>> Prior to the announcement of the 5G work, it had been stated openly
>>that the BBF planned to produce a requirements document for ³hybrid
>>access², and then the BBF would look at protocols from other standards
>>bodies, such as the IETF, to determine how to meet those requirements.
>>The BANANA effort was blocked (for over a year, with no discussion on
>>the BANANA list or even a statement on the list explaining why) until
>>the BBF¹s requirements were published, so that our work would not
>>interfere (in some unspecified way) with the BBF requirements effort.
>>After the BBF requirements document was published, the BBF withdrew its
>>objection to the BANANA work proceeding in the IETF.
>
>Specifically "The BANANA effort was blocked (for over a year, with no
>discussion on the BANANA list or even a statement on the list explaining
>why)².
>
>>  
>> 
>> Now, the BBF is starting a _new_ standardization effort (started _much
>>later_ than the BANANA work), and you would like the IETF to further
>>block or limit the BANANA work to avoid conflicting with this new
>>effort.   It seems to me that the BBF is the one expanding the scope of
>>their work, not the BANANA group.
>> 
>> Personally, I object strongly to the notion that the IETF should block
>>or reject work on end-user-controlled, multi-provider, IP-based,
>>link-layer-independent solutions
>
>> because they ³conflict" with provider-controlled (single-provider or
>>cooperating-provider) solutions, or with solutions that are only
>>applicable to specific link layer technologies (i.e. BBF/5G) thus only
>>serving a subset of Internet users.
>
>(single-provider or cooperating-provider)" part is in scope of the
>charter or not. I don¹t think we can have a meaningful conversation
>unless we state this in the charter one way or another.
>
>Thanks
>Suresh
>

---

From: Margaret Cullen <[email-masked]>
Date: Fri, 22 Sep 2017 13:09:37 -0400

> On Sep 22, 2017, at 11:53 AM, Sri Gundavelli (sgundave) <[email-masked]> wrote:
> 
> IF we step back, the motivation for this work is DSL + LTE aggregation.

Actually, there are quite a few people who have indicated that they would like to aggregate other link types.  There was a presentation at IETF-97 about aggregating LTE and Wifi, one person on this thread mentioned wanted to aggregate two WiFi links, another person has talked about aggregating VPN links. etc.

Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Fri, 22 Sep 2017 17:20:31 +0000

On 9/22/17, 10:09 AM, "Margaret Cullen" <[email-masked]> wrote:

>Actually, there are quite a few people who have indicated that they would
>like to aggregate other link types.  There was a presentation at IETF-97
>about aggregating LTE and Wifi, one person on this thread mentioned
>wanted to aggregate two WiFi links, another person has talked about
>aggregating VPN links. etc.
>

So, LTE and DSL are out;  WiFi as a WAN link to home is not a known
service; SATRAN access to home is unheard of.

So, what are the two remaining links that are not WiFi, LTE, DSL, SATRAN
and not aggregation?

---

From: Margaret Cullen <[email-masked]>
Date: Fri, 22 Sep 2017 13:24:08 -0400

> So, what are the two remaining links that are not WiFi, LTE, DSL, SATRAN

I don’t understand this message…

You made a statement that DSL/LTE was the only motivator for this work, and it would be solved by a BBF/5G layer-two solution.  I indicated that there also people who have stated that they want to aggregate non-BBF, or non-5G link types.  I didn’t say anything was out, and I don’t know what “two remaining links” you are talking about.  

The solutions we have been discussing in BANANA are IP-based link-layer-independent solutions.  We wouldn’t be ruling any link types in or out…

Margaret

---

From: David Sinicrope <[email-masked]>
Date: Fri, 22 Sep 2017 17:23:55 +0000

Margaret,
Please let me address a few of your points. 

First let me address the accusation of “using the threatening tone”. This is a serious accusation and one I believe to be unfounded. At no point is the language I am using making threats. I am simply stating the potential negative impact of overlap to the BBF-IETF relationship and suggest a corrective course of action.        

I do not understand your statement/accusations about me or the BBF blocking work or the BBF releasing objection to the BANANA work. All work in the IETF is done according to the IETF process and I’m not aware of any part of the IETF process that lets another organization or individual block or release work. 

As far as being willing to help and the “hats” I wear, I believe I have made it clear that I have been participating in my role as the IETF Liaison Manager to the Broadband Forum.  At each BANANA BoF I have attended (all but Yokohama), I have tried to help the work progress by advising that the work scope not overlap with the BBF.    Scoping the work to avoid overlap will provide clearer direction, and in turn avoid debate and lead to faster progression of the work. 

As for the proposed alternative wording, I appreciate your attempt to accommodate the concerns I raised and avoid the scope overlap I highlighted.  I think the changed text is a marked improvement toward avoiding overlap.  I stand by the suggestions I have made in my previous email, including liaising a draft of the charter, once stable, to the BBF and requesting their input.  I cannot advise what the BBF position would be on a charter they have not seen.  I believe liaising the draft charter would not only provide BANANA with important feedback, but also would be seen as a gesture of goodwill helping foster a cooperative IETF-BBF relationship on this topic.

I look forward to finalizing the BANANA charter and a cooperative and productive work relationship between the IETF and BBF on this topic.

Regards,
Dave Sinicrope
BBF/IETF Liaison Manager 

On 9/21/17, 11:54 AM, "Margaret Cullen" <[email-masked]> wrote:

    Hi Dave,

    

    

    

    

    

    Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Fri, 22 Sep 2017 13:36:59 -0400

Hi Dave,

> On Sep 22, 2017, at 1:23 PM, David Sinicrope <[email-masked]> wrote:

I am sorry if I have misinterpreted some of your statements in the past, and I will attempt to adjust my input filters in the future to understand that you are making suggestions, rather than trying to lay down rules.  
> As far as being willing to help and the “hats” I wear, I believe I have made it clear that I have been participating in my role as the IETF Liaison Manager to the Broadband Forum.  At each BANANA B…

This sounds like a good path.  Since I am not a member of the BBF or the 3GPP, I am not familiar with what work the BBF currently has chartered in cooperation with 3GPP.  Is there a public charter for that work that I could see, or could you share one with us?  It is easier not to conflict with something, if you actually know what it is.
> 
> As for the proposed alternative wording, I appreciate your attempt to accommodate the concerns I raised and avoid the scope overlap I highlighted.  I think the changed text is a marked improvement toward avoiding overlap.  I stand by the suggestions I have made in my previous email, including liaising a draft of the charter, once stable, to the BBF and requesting their input.  

This was suggested in the BOF, as well, and I think that Suresh said that this sort thing would be considered as part of the IAB/IESG review process — that it isn’t something the BOF group would do, directly.  I will let Suresh correct me if I am wrong about this.

> I cannot advise what the BBF position would be on a charter they have not seen.  I believe liaising the draft charter would not only provide BANANA with important feedback, but also would be seen as a gesture of goodwill helping foster a cooperative IETF-BBF relationship on this topic.

Fostering a cooperative relationship is very important both in general, and in this case in particular.  I would support giving the BBF some sort of heads-up about the charter if/when the IESG actually accepts it for consideration, and I would welcome feedback from BBF members.  If there are BBF members who are also active in the IETF, though, I hope they will provide their feedback here, now, and not wait for the BBF to be asked for feedback.
> I look forward to finalizing the BANANA charter and a cooperative and productive work relationship between the IETF and BBF on this topic.

Me, too.  Thank you for your understanding and your help with this.

Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Fri, 22 Sep 2017 14:08:22 -0400

Hi Sri,

Thinking about this message, I think the real answer to your question lies at a much higher level than the answers I gave…

We are talking about two different things here, one that is being done in the BBF, and the other of which is proposed in the BANANA BOF:

(1)  The BBF is (as I understand it) working on a DSL/5G-specific solution that requires cooperation from a customer's Internet service provider(s).

and

(2) The BANANA group is talking about an IP-based, link-layer-independent solution that does not require explicit cooperation from any Internet service provider.

These things both sound awesome to me, and I think the world should have both of them!  They are very different things, and people will use them in different situations for different reasons.  

#1 lets providers, or cooperating groups of providers, give their BBF/5G customers faster and better service, without the need for the customer to understand the solution or contract for a third-party service.

#2 allows an end user to gain, or a third-party aggregation service provider to provide, faster and better service by aggregating whatever links the customer has available. without the explicit cooperation of the customer’s Internet service providers.

In BANANA, we would be defining a solution that can be used by whoever chooses to run it (or whoever contracts with a third-party to run it).  It is not desirable (or even technically practical) for a BANANA implementation to determine if it is being run by an end-user, a third-party service provider, or an Internet service provider, and there is certainly no good way to tell whether it is running over two links from the same service provider.  You also don’t want to detect the link types and refuse to run if one the links supports 5G, or if one of the links is DSL…  I don’t think any of us, including the BBF or 3GPP, would want that.

What we should do is coordinate with the BBF to ensure that customers will not run into problems if they do try to run the BANANA solution over a set of links that includes links with the BBF/5G solution enabled.  We can’t know what the best approach would be to avoid any problems, until we know a lot more about both solutions, but pertinent question might be whether the BANANA software can detect that the BBF/5G bonding is in place, and if so, whether BANANA would treat a set of bonded BBF/5G links as separate links for the purposes of BANANA, or as a single link.  These are things I think it would be important for the BANANA WG and the folks working on the BBF/5G solution to keep in mind going forward, and to communicate about in more detail as we know more about the two solutions.

Margaret

> On Sep 22, 2017, at 1:24 PM, Margaret Cullen <[email-masked]> wrote:
>

---

From: Margaret Cullen <[email-masked]>
Date: Fri, 22 Sep 2017 14:14:38 -0400

Oh, I forgot to ask…  Are there any further charter edits that would make you feel better how this work relates to the work in the BBF?

Would it make sense to explicitly state, in the charter, that the BANANA WG will work with the BBF/5G group to ensure that our solutions will not interfere with each other if used on the same networks?  Is there still time to get any wording to that effect added to the charter of the BBF/5G group, as well?

Margaret

> On Sep 22, 2017, at 1:36 PM, Margaret Cullen <[email-masked]> wrote:
> 
> Hi Dave,
>

---

From: Joerg Deutschmann <[email-masked]>
Date: Fri, 22 Sep 2017 21:19:43 +0200

Hi,

I'd like to emphasize that there is a actual demand for the bonding of 
satellite and terrestrial links (DSL, cable, LTE, ...). We must go for a 
provider-independent solution, because satellite and terrestrial 
networks are usually run by different operators. Especially branch 
offices or home offices (exchanging all data via a single IPsec tunnel) 
would benefit from a BANANA-like solution. Despite satellite 
connections, I see benefits in the combination of any link type (e.g. 
DSL + cable).

We have dedicated manpower for working on this topic and definitely 
would like to contribute to a BANANA WG (also not only in the context of 
satellite communication).

Best regards,
Joerg

---

From: <[email-masked]>
Date: Mon, 25 Sep 2017 08:42:43 +0000

Hi Sri,

I do not agree here. Yes, DLS+LTE is a very valid use case with a lot of momentum, but there are other use cases which were also mentioned during the BoF. E.g. Bundling of SAT + DSL (and in fact there are offerings, providing Internet over Satellite for home users as well with - very small bandwidth DSL - as a backup). Why do you assume that WiFi means WAN? Most of the home routers today provide WLAN and there are scenarios where the WLAN of different customers is being aggregated. 

I think we should not rule out any technology for the BANANA work.

Regards

Nic

-----Ursprüngliche Nachricht-----
Von: Banana [mailto:[email-masked]] Im Auftrag von Sri Gundavelli (sgundave)
Gesendet: Freitag, 22. September 2017 19:21
An: Margaret Cullen <[email-masked]>
Cc: David Allan I <[email-masked]>; Muley, Praveen (Nokia - US/Mountain View) <[email-masked]>; Alexandre Petrescu <[email-masked]>; Suresh Krishnan <[email-masked]>; David Sinicrope <[email-masked]>; Henderickx, Wim (Nokia - BE/Antwerp) <[email-masked]>; [email-masked]
Betreff: Re: [Banana] Charter

On 9/22/17, 10:09 AM, "Margaret Cullen" <[email-masked]> wrote:

>Actually, there are quite a few people who have indicated that they 
>would like to aggregate other link types.  There was a presentation at 
>IETF-97 about aggregating LTE and Wifi, one person on this thread 
>mentioned wanted to aggregate two WiFi links, another person has talked 
>about aggregating VPN links. etc.
>

So, LTE and DSL are out;  WiFi as a WAN link to home is not a known service; SATRAN access to home is unheard of.

So, what are the two remaining links that are not WiFi, LTE, DSL, SATRAN and not aggregation?

---

From: Mirja Kühlewind <[email-masked]>
Date: Mon, 25 Sep 2017 12:29:35 +0200

Hi Sri, hi all,

I didn’t speak up so far because I think most things have already been said, regarding scope and problem. To me the knowledge that there are (proprietary) already deployed solutions is a good sign that there is a need for this work and the role of standardization here is to document these approaches to make things interoperable between different vendors and different operations. 

However, I would like to add one more point. Sri, you said several times that there are already groups in the IETF that could standardize these extensions, when needed. That is true for some groups but also not for all. MPTCP is chartered to document the use of MPTCP proxy mechanisms but it is not charter to work on protocol extensions that e.g. provide additional signaling for this use case. MPTCP is an TCP extension and TCP is an end-to-end protocol not a tunneling protocol. 

Moreover, seeing that the same problem is considered in various different groups in the IEFT, is another point that indicated to me that we probably need an own working group, otherwise we will end up doing the same work multiple times. What we need is to get all the people working on the same problem in the same room and talk to each other. Of course if extensions are considered to existing protocols and we have an active working group that maintains that protocol, we need close cooperation with that group. However, that is not a new problem and we have that in many groups. What we usually do is a common wgcl in both/multiple groups or even start the doc in one group and move it over to another if we figure out that the needed expertise is there. It’s the job of the ADs to make sure that this inter-wg or even inter-area awareness happens and I’m not worried about this.

Mirja

> Am 22.09.2017 um 17:53 schrieb Sri Gundavelli (sgundave) <[email-masked]>:
> 
> Hi Suresh,
> 
> 
>

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 25 Sep 2017 10:25:39 -0400

Hi Dave,

I am still trying to find a charter or other description of the BBF/5G protocol work, so that we can potentially reference it in our charter, and so that I can make a recommendation to the IESG regarding what group(s) should be apprised of this work.

As I mentioned, I am not a member of the BBF, but I have asked some members to help me find a pointer to this charter, and they couldn’t find it either.  They only found the charter for the requirements work.  They were also under the impression that the BBF doesn’t do this sort of protocol work?  Has the protocol work actually been chartered in the 3GPP?  If so, do you have a pointer to that charter or a description of that work?

I could use some help understanding the scope and state of the protocol work you have referenced.

Thanks,
Margaret

---

From: "Sri Gundavelli (sgundave)" <[email-masked]>
Date: Mon, 25 Sep 2017 16:44:14 +0000

Hi Mirja,

Yes, vendors have been deploying multipath solutions and I have personally
defined and deployed such architectures. There is lot of known-art in this
area and that includes standardization right from multi-link PPP,
DSMIP-IFOM, MPTCP and in other protocols areas. So, this is not a new
problem as being presented. Now, the issue is not with bringing all
multipath discussions to one common place, about the need to document all
known approaches, or asking for a specific extension in a specific
protocol.  But, the idea of Banana WG defining “a solution” and calling it
a IETF Banana solution is an issue. The idea of departing from protocol
work to solution work is an issue. I tend to think that is outside the
IETF charter and falls in the vendor/SDO territory. As a case in point,
lets go back to topic of IPv6 transitioning and the contentious debates
around IETF picking one approach for transitioning, among DSLite,
Gi-DSLit, NAT64 and the zillion other variants. At that time, Jari Arkko,
was very clear, we are only providing tools to the community and we will
not pick one approach as that amounts to helping a vendor.  So, I, not as
an individual, but representing my employer, have a problem giving such
authority to the chairs of this WG. If the counter argument is that the
chairs will not pick a solution, but will standardize many solutions based
on all known protocols offering such promise/capability, then the charter
clear should state that, Ex:”Banana WG will extend protocol X, Y and Z for
multipath support”. But, there has to be explicit statement that the WG is
not going to define new “signaling” or “user plane” protocols. That is the
core issue here.

The bar for defining new protocols must be very high; Besides the few
interested individuals there has to be support from across the community
when chartering such work. Its not just about support from dozen people (+
/ -), that makes it in to the charter. We don’t want IETF protocols to be
diluted with alternatives that do the same thing with a different
name/brand. The need for new signaling protocol must be backed with the
data that existing protocols do not fit the bill and so we need this new
approach. Again, case and point, when IETF went from client-based mobility
to network-based mobility, there was industry wide participation with
100’s participants/companies, support from WiMAX, 3GPP and with a strong
technical backing of basing it on a new architectural approach. There was
no such data presented here for Banana, and in the absence of such
data/analysis, the charter must explicitly state that WG is not allowed to
define a new signaling/user-plane protocol.

It must be noted that there is also no SDO Ask for this work.  If I
understand correctly they are in fact asking IETF to relax for some more
time till they make progress.  It may be for the AD to measure the support
for his work, but I do not see such support for this work beyond a small
group of people.  If you review the number of technical contributions to
Banana WG in 2015, 2016 and 2017, one can can clearly see that its mostly
the authors of 1 or 2 drafts, potential chairs and few individuals who
have contributed any thing. But, I realize its AD has the option charter
WG at their discretion, but the above issues have to be addressed, IMO.

Sri

On 9/25/17, 3:29 AM, "Mirja Kühlewind" <[email-masked]>
wrote:

>Hi Sri, hi all,
>
>I didn’t speak up so far because I think most things have already been
>said, regarding scope and problem. To me the knowledge that there are
>(proprietary) already deployed solutions is a good sign that there is a
>need for this work and the role of standardization here is to document
>these approaches to make things interoperable between different vendors
>and different operations.
>
>However, I would like to add one more point. Sri, you said several times
>that there are already groups in the IETF that could standardize these
>extensions, when needed. That is true for some groups but also not for
>all. MPTCP is chartered to document the use of MPTCP proxy mechanisms but
>it is not charter to work on protocol extensions that e.g. provide
>additional signaling for this use case. MPTCP is an TCP extension and TCP
>is an end-to-end protocol not a tunneling protocol.
>
>Moreover, seeing that the same problem is considered in various different
>groups in the IEFT, is another point that indicated to me that we
>probably need an own working group, otherwise we will end up doing the
>same work multiple times. What we need is to get all the people working
>on the same problem in the same room and talk to each other. Of course if
>extensions are considered to existing protocols and we have an active
>working group that maintains that protocol, we need close cooperation
>with that group. However, that is not a new problem and we have that in
>many groups. What we usually do is a common wgcl in both/multiple groups
>or even start the doc in one group and move it over to another if we
>figure out that the needed expertise is there. It’s the job of the ADs to
>make sure that this inter-wg or even inter-area awareness happens and I’m
>not worried about this.
>
>Mirja
>
>
>
>>If
>>>> (1) The charter is unclear that this is intended to be a
>>>>multi-provider
>>>> solution.  The charter says:
>>>>

---

From: David Sinicrope <[email-masked]>
Date: Mon, 25 Sep 2017 18:12:36 +0000

Margaret,

https://www.broadband-forum.org/standards-and-software/downloads/technical-work-in-progress

I believe you may be looking for WT-378 The solutions document for TR-348 - Hybrid Access Broadband Network Architecture for the home and business.  See the public link above. (Note: the video is home centric, but my understanding is that the work is broader in scope.)  This is the only public description I’m aware of.  (Admittedly, it could say more.)

To get an accurate description and the latest information on their work and its status, I recommend liaising the BANANA draft charter to the BBF and ask them for more information on their work in this space, including their project with 3GPP.  In my opinion, asking them directly would get the most accurate information possible and would be the most help in crafting charter text to avoid overlap.  Given the BBF history with focus on requirements, solutions and architecture rather than protocol design, I can understand the impression you mention below.

I support recommending to the IESG that BBF be kept apprised of all ongoing BANANA work and output.

----------------

I took another look at the Google Doc version of the charter (as of 11:30 EDT today).  Below are some comments regarding the BBF overlap concern:

(Note: These are my personal suggestions i.e., they don’t reflect a representation, opinion or view of the Broadband Forum or its members)

From the Google Doc: “The BANdwidth Aggregation for Network Access (BANANA) Working Group is chartered to develop solution(s) to support dynamic path selection on a per-packet basis in networks that have more than one point of attachment to the Internet.”

- instead of “…support dynamic path selection on a per-packet basis in networks that have more than one point of attachment to the Internet”,  I suggest: “…support dynamic path selection on a per-packet basis for user equipment with more than one point of access to the Internet without operator involvement in the aggregation. (i.e., ‘Over The Top’ (OTT) environment).”  This, in my opinion, describes the “OTT” case. I’d be interested to hear from others whether this is sufficient to describe OTT.

To further support the distinction, anywhere “Internet links” or “link” shows up, it should be change to “Internet access services”.

Also, I recommend changing: “The Bandwidth Aggregation solutions developed in this group will work in true multi-provider scenarios (i.e. they will not depend on all of the aggregated links being provided by a single Internet access provider nor by a group of cooperating providers). “

To

“The Bandwidth Aggregation solutions developed in this working group are designed for scenarios where a user or customer wishes to aggregate multiple independent Internet access services that are agnostic of each other. (i.e. the aggregated Internet access services are not provided by a single Internet access provider nor by multiple cooperating providers).”

Also regarding other SDOs:  From the Google Doc: “The BANANA WG will also work with other IETF WGs (and other SDOs, as requested) that define additional Bandwidth Aggregation mechanisms (if any)  to ensure that the protocols defined by the BANANA WG will meet the needs of those additional mechanisms.”

- What is meant here by “mechanisms”?  Thinking in the BBF context, I would have assumed “mechanism” to be a protocol or protocol extension and that these would prefer to be done in IETF rather than elsewhere.  For the BBF context, it seems like the word “mechanism” could/should be substituted for “solution”.  Are there other SDOs that would define aggregation protocols that BANANA would work with?  (e.g., MP-TCP?)   Assuming yes, perhaps two points to be made in this paragraph then: A. work with other SDOs defining solutions to ensure no conflict; B. work with other WGs & SDOs defining protocol mechanisms to consider their needs.

I suggest changing the text to:

“The BANANA WG will also work with other IETF WGs, and other SDOs, as requested, that: A. define Bandwidth Aggregation solutions (e.g., Broadband Forum) to ensure that the solutions defined by the BANANA WG avoid conflict.  B. define additional Bandwidth Aggregation mechanisms in a way that ensures that the protocols adopted and possibly enhanced by the BANANA WG will consider the needs of the other organizations.”

Please let me know if you have questions.

Regards,

Dave

BBF/IETF Liaison Manager

On 9/25/17, 10:25 AM, "Margaret Cullen" <[email-masked]> wrote:

    Hi Dave,

    Thanks,

    Margaret

---

From: Margaret Cullen <[email-masked]>
Date: Mon, 25 Sep 2017 15:53:12 -0400

Hi Dave,

> On Sep 25, 2017, at 2:12 PM, David Sinicrope <[email-masked]> wrote:
> 
> Margaret,
>  

Thank you.  Yes, this is the sort of thing I was looking for.  I will look at the videos and try to better familiarize myself with this work.

At this point, I think we are in (at least rough) agreement about the spaces in which the proposed BANANA group and the BBF/5G group could work in ways that are not in conflict with one another.  Fortunately for us, I think those are also the spaces in which we both wanted and intended to work!  The difficulty lies in putting the concepts into words that will make this clear to everyone.

We may differ on one point, which is that if the only two solutions that exist in the world are the BBF/5G solution and our generic, OTT solution, non-5G providers may choose to use our solution to aggregate their customers links (e.g. a Cable/LTE combination).  While that isn’t the main scenario for which we would be designing a BANANA solution, I don’t think we should do anything to stop that from working.  Do you?

> “The Bandwidth Aggregation solutions developed in this working group are designed for scenarios where a user or customer wishes to aggregate multiple independent Internet access services that are a…

Terms like “internet access services” are not widely used in the IETF, and I am not sure I want to introduce that term in this charter.

More importantly, we don't prescribe nor proscribe a business model for our protocols (e.g. say that something will run on user equipment vs. equipment owned by someone else, etc.) — we make all of our protocols safe to run over the Internet, and we enable end-users, operators, system integrators, vendors, service providers, and other SDOs to specify, implement and use them wherever they are useful.  

I do think I understand what you are driving at, though, so let’s see if I can come up with a way to say it without making it sound like the IETF is trying to dictate where (in the Internet topology) the resulting protocols can and cannot be used.  

How about this?:

OLD:

	The BANdwidth Aggregation for Network Access (BANANA) Working 
	Group is chartered to develop solution(s) to support dynamic path selection 
	on a per-packet basis in networks that have more than one point of 
	attachment to the Internet.

NEW:

	The BANdwidth Aggregation for Network Access (BANANA) Working 
	Group is chartered to develop solution(s) to support dynamic path selection 
	on a per-packet basis in networks that have more than one point of 
	attachment to the Internet.  These solutions will be designed to work	
	in situations where Internet access is provided by more than Internet 
	Service Provider, and without cooperation between a set of Internet Service 
	Providers (i.e. these will be Over-The-Top (OTT) solutions).

And this?:

OLD:

	The Bandwidth Aggregation solutions developed in this group will work 
	in true multi-provider scenarios (i.e. they will not depend on all of the 
	aggregated links being provided by a single Internet access provider nor
	by a group of cooperating providers).  Any protocols defined by this
	group will be IP-based, link-layer-independent solutions, and they will
	be designed to work across NATs and other middle boxes, as needed.

NEW:

	The Bandwidth Aggregation solution(s) developed in this working group will 
	be designed to work in scenarios where Internet access links from multiple,
	independent Internet access providers are being aggregated  (i.e. where 
	all of the aggregated Internet access links are not provided by a single Internet 
	access provider, nor by a set of cooperating providers).  Any protocols 
	defined by this group will be IP-based, link-layer-independent solutions, and 
	they will be designed to work across NATs and other middle boxes, as needed.

Would those changes capture your points?  Or am I still missing something important?

> - What is meant here by “mechanisms”?  Thinking in the BBF context, I would have assumed “mechanism” to be a protocol or protocol extension and that these would prefer to be done in IETF rather tha…

“Mechanisms" is the word I used to replace “Encapsulations”, because some people thought that “Encapsulations” automatically implied tunneling.  I will try to be clearer, as this change definitely made things less clear in this particular paragraph.

> “The BANANA WG will also work with other IETF WGs, and other SDOs, as requested, that: A. define Bandwidth Aggregation solutions (e.g., Broadband Forum) to ensure that the solutions defined by the…

I agree with what you are sayingthere, and will attempt to incorporate this edit.  I am also thinking that just being more direct here might be better — referencing the actual BBF/5G work by name, and saying that we will work with that group, as needed, to resolve any conflicts or concerns, perhaps.  Is there a 3GPP moniker for this group that I should also include?

How about this?:

OLD:

	The BANANA WG will also work with other IETF WGs (and other SDOs, 
	as requested) that define additional Bandwidth Aggregation mechanisms 
	(if any)  to ensure that the protocols defined by the BANANA WG will meet 
	the needs of those additional mechanisms.

NEW:

	The BANANA WG will work with other IETF WGs that define "Bandwidth
	Aggregation mechanisms" (as defined above), to ensure that the
	provisioning and signaling protocols selected or specified by the BANANA 
	WG will work for those Bandwidth Aggregation mechanisms.  The BANANA 
	WG will also work with other SDO(s) that are defining Bandwidth Aggregation 
	solutions (e.g. Broadband Forum TR-378) to ensure that there are
	no conflicts between our work, and to ensure that there are no negative 
	consequences for users when multiple Bandwidth Aggregation solutions
	are available in a single environment.

How do those changes look to you?  What do other people think?

Margaret

---

From: David Sinicrope <[email-masked]>
Date: Mon, 25 Sep 2017 21:00:53 +0000

Hi Margaret,

Before I go through your new proposed text I’m hoping you can help me with some questions.

Can you elaborate on what you mean by “non-5G providers” below?  This and the “/” in your term “BBF/5G solution” seems to indicate a perception that the BBF solution (TR-348, WT-378) and the BBF work with 3GPP are inseparable and only apply to 5G.  Maybe I’m reading too much into this, but just to be clear, I’m pretty sure, e.g., given the timeframe for the projects, that the TR-348 and WT-348 work applies to previous generations of mobile and fixed networks and can be used in a 5G environment as well.

Another thing I/we may be disconnected on is the word “solution” used in the charter.  You say below that “More importantly, we don't prescribe nor proscribe a business model for our protocols (e.g. say that something will run on user equipment vs. equipment owned by someone else, etc.) — we make all of our protocols safe to run over the Internet, and we enable end-users, operators, system integrators, vendors, service providers, and other SDOs to specify, implement and use them wherever they are useful.  “

The solutions and solution architectures that are normally produced by the BBF also do not pre/proscribe business model (how a service is packaged and offered), but in providing an architecture they do place function in the network, and determine who has ownership & control of it (management, signaling, etc).  This seems to imply that in the charter when you use the word solution, you are talking more about protocols and not what I would consider solution architecture.  This seems to be supported by the work items listed in the draft charter, with the possible exception of the usage scenarios. Yet I would think defining and scoping BANANA to an OTT environment would involve both, the definition of the architecture (even if only a reference architecture), including intended placement of the function, and also the definition/extension of whatever protocols are used.  I’m having trouble seeing how this reconciles with your statement above.  Can you help clarify?

The one thing I see missing from your reply is where a liaison will be sent to the BBF asking for the information you are requesting.

What are your plans for progressing this liaison?

Thanks,

Dave

BBF/IETF Liaison Manager

On 9/25/17, 3:53 PM, "Margaret Cullen" <[email-masked]> wrote:

    Hi Dave,

    How about this?:

    OLD:

    NEW:

    And this?:

    OLD:

    NEW:

    How about this?:

    OLD:

    NEW:

    Margaret

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 25 Sep 2017 22:31:02 -0400

Hi Dave,

> On Sep 25, 2017, at 5:00 PM, David Sinicrope <[email-masked]> wrote:
> 
> Hi Margaret,
> Can you elaborate on what you mean by “non-5G providers” below?  This and the “/” in your term “BBF/5G solution” seems to indicate a perception that the BBF solution (TR-348, WT-378) and the BBF work with 3GPP are inseparable and only apply to 5G. 

I had been told they were specific to 5G, but maybe they are more generic DSL/LTE solutions?  My point is that if the solutions you provide assume the presence of LTE and/or DSL interfaces, they won't work for people who don't have those types of interfaces, right?  So, those people might use BANANA instead.

Obviously,  our charter will not discuss the technical details of your solutions.  I want to understand them, but the BANANA charter doesn't have to block  on me understanding them.

> Maybe I’m reading too much into this, but just to be clear, I’m pretty sure, e.g., given the timeframe for the projects, that the TR-348 and WT-348 work applies to previous generations of mobile and fixed networks and can be used in a 5G environment as well. 
>  
> Another thing I/we may be disconnected on is the word “solution” used in the charter. 

I do not think we use the term solution in the same way that you do.  Even when we do an architecture document, it is a "boxes and arrows" document that shows the major logical elements, message flow, etc.  It doesn't talk about who owns what, where in the Internet topology the boxes will be situated, who owns them, etc.  I think everything we do falls into what you call "protocols".

Our usage scenarios will be for reference and common understanding only.  They won't be a normative part of the protocols, nor will they be published in an RFC or other archival form.

>  Yet I would think defining and scoping BANANA to an OTT environment would involve both, the definition of the architecture (even if only a reference architecture), including intended placement of the function, and also the definition/extension of whatever protocols are used.  I’m having trouble seeing how this reconciles with your statement above.  Can you help clarify?

When you write a protocol, you design it based on assumptions about the environment in which it will be used.  We would write protocols that are IP-based, link-layer-technology-independent, and that don't not require any cooperation between the providers of the aggregated links to function.  Our assumption will be that the remote and local BANANA boxes are across an arbitrary span of Internet from each other.  We don't have any means to control where the BANANA protocols will actually be implemented or used, though.  

> What are your plans for progressing this liaison? 

As I thought I said earlier, it is my understanding that Suresh will handle whatever communication is needed with other standards bodies as part of the IESG/IAB review.  So, I don't think that this will happen until after the charter is submitted to the IESG.  Suresh can correct me if I am wrong.

Margaret