Subject: Re: [manet] Benoit Claise's Discuss on draft-ietf-manet-olsrv2-multitopology-05: (with DISCUSS)

From: "Alvaro Retana (aretana)" <[email-masked]>
Date: Mon, 15 Jun 2015 19:02:43 +0000

On 5/28/15, 9:26 AM, "Benoit Claise (bclaise)" <[email-masked]> wrote:

[Because the authors didn¹t have time to review the ops-dir comments
before the telechat, or before they became a DISCUSS, I¹m explicitly
cc¹ing it here as well as Sue.]

Benoit:

Hi!

Sorry it took me a while to get to this..

>----------------------------------------------------------------------
>DISCUSS:
>----------------------------------------------------------------------
>
>The multiple points, brought up by Sue part of her OPS-DIR review,
>deserve a DISCUSS. Let's engage in the discussion.
. . .

>Summary of Comments:
>
>My comments have 6 major issues, and a set of editorial changes.  Five of
>my major points have to do with adding more details to the draft to judge
>the experiment valuable.  One way to resolve these comments is to create
>document providing details on the test that will be run.  A second way to
>resolve these comments on experiment is to provide additional high-level
>guidance in this document.

Note that section 1.1 (Motivation and Experimentation) already provides
high-level guidance of the type of information to be evaluated.

   While general experiences with this protocol extension, including
   interoperability of implementations, are encouraged, specific
   information would be particularly appreciated on the following areas:

   o  Operation in a network that contains both routers implementing
      this extension, and routers implementing only [RFC7181], in
      particular are there any unexpected interactions that can break
      the network?

   o  Operation in realistic deployments, and details thereof, including
      in particular indicating how many concurrent topologies were
      required.

   A broader issue that applies to unextended [RFC7181] as well as this
   extension (and potentially to other MANET routing protocols) is which
   link metric types are useful in a MANET, and how to establish the
   metrics to associate with a given link.  While this issue is not only
   related to this extension, the ability for an OLSRv2 network to
   maintain different concurrent link metrics may facilitate both
   experiments with different link metric types, ways to measure them,
   etc. and may also allow experimentation with link metric types that
   are not compromises to handle multiple traffic types.

Clearly the focus is on ³running code²: operation in real deployments and
mixed environments.

The suggestions for tests made in the Ops-Dir review are great!  However,
I don¹t believe that such detail belongs in this document, where the main
purpose is to document the extension.  In fact, I think the suggestions
belong more as part of a test plan (used by implementors ‹ similar maybe
to the work done in bmwg, for example), which seems to be in line with the
comments: (Sue wrote) ³The recommended tests in major concern 1-4 could be
created in a separate draft.²

It is not in the manet WGs charter to produce test plans.  In order to not
loose Sue¹s valuable input, I suggest we keep them in the WG¹s wiki ‹
which will allow for other test cases to be added, details included,
results reflected, etc.

. . .
>Major concern 5:  Experiments should drive to create operational
>guidelines for deployment, configuration knobs, and use cases (ADOV-2,
>OLSR-v2, MT-OLSR-v2)

Completely agree!  Implementation and operational experience (not just
experiments) should in fact result in that type of guidelines.

Again, the details are not within the scope of this document..and guidance
is already given in 1.1 about the use in real networks.

Note that this comment mentions not just the extensions proposed, but the
base protocol and even AODVv2.  All these guidelines are important from an
operations point of view, but shouldn¹t be tied to this document.

Aside: the MANET WG is in process of rechartering.  This type of
suggestions should be presented for discussion.

. . .
>Major 6: The IANA section does not answer all the IANA questions.
>
>It has most of the information, but I think it is not up to the latest
>IANA format and information.   Barry Leiba and others have noted that the
>RFC 7181 and RFC7188 do not match this IANA section.  Rather than repeat
>these comments, I will simple state the data needs to be consistent and
>the format match IANA¹s comments.

Barry¹s comment has been solved and will be reflected in an update.

The authors have been talking to IANA directly.  We are now waiting for
review from the MANET registry experts (which is the one open item with
IANA).

Thanks!

Alvaro.

---

From: "Alvaro Retana (aretana)" <[email-masked]>
Date: Thu, 25 Jun 2015 15:38:38 +0000

Benoit/Sue:

Any comments?

Thanks!

Alvaro.

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 25 Jun 2015 13:21:59 -0400

To the Manet author of the comments: 

To help with the comments:  I've pulled them up. 

>>The suggestions for tests made in the Ops-Dir review are great!  

>>However, I don¹t believe that such detail belongs in this document, 

>>where the main purpose is to document the extension.  In fact, I think 

>>the suggestions belong more as part of a test plan (used by 

>>implementors ‹ similar maybe to the work done in bmwg, for example), 

>>which seems to be in line with the

>>comments: (Sue wrote) ³The recommended tests in major concern 1-4 could 

>>be created in a separate draft.²

Let's go to the higher level purpose rather than the "not my WG charter" reasoning.  

If you are running an experiment with this protocol, you need to have specific details enough to determine if this experimental protocol is a success.  Otherwise, you will continue to have the OLSR vs. AODV-v2, or vague review on RFC5444 additions to the protocol.  This ends up in emotional debate without substantial experimental results to back it up.  Emotional debates recycle. My suggestions aim at providing enough detail to settle these arguments with experimental results.  MANET has lots of exciting work to do in the advent of 5G, 802.11ac, and other  mobile network changes. 

As you noted, the reviews on operational portion of this work points out how these experimental results are critical to designing an appropriate operational interface. 

So.. bottom line... Staying "not in my charter" and asking to pass this document on without settling on a mechanism to fix it - is a mistake.  Decide how the MANET WG is going to determine this is a success and put together a plan.  BMWG usually does device compliance.  If you want aid on a network-compliance test case, they will work with  and review the document. However, the real expertise is still in MANET. 

ON IANA ... That's covered by Barry's discuss. 

Sue 

PS – If you are tired of the document and debate, as a WG chair, I understand.  However, the value of the reviews is a fresh set of eyes and emotions. 

-----Original Message-----
From: Alvaro Retana (aretana) [mailto:[email-masked]] 
Sent: Thursday, June 25, 2015 11:39 AM
To: Benoit Claise (bclaise); Susan Hares
Cc: [email-masked]; [email-masked]; [email-masked]; The IESG; [email-masked]
Subject: Re: Benoit Claise's Discuss on draft-ietf-manet-olsrv2-multitopology-05: (with DISCUSS)

Benoit/Sue:

Any comments?

Thanks!

Alvaro.

On 6/15/15, 4:02 PM, "Alvaro Retana (aretana)" < <mailto:[email-masked]> [email-masked]> wrote:

>On 5/28/15, 9:26 AM, "Benoit Claise (bclaise)" < <mailto:[email-masked]> [email-masked]> wrote:

>>My comments have 6 major issues, and a set of editorial changes.  Five 

>>of my major points have to do with adding more details to the draft to 

>>judge the experiment valuable.  One way to resolve these comments is 

>>to create document providing details on the test that will be run.  A 

>>second way to resolve these comments on experiment is to provide 

>>additional high-level guidance in this document.

>Clearly the focus is on ³running code²: operation in real deployments 

>and mixed environments.

>The suggestions for tests made in the Ops-Dir review are great!  

>However, I don¹t believe that such detail belongs in this document, 

>where the main purpose is to document the extension.  In fact, I think 

>the suggestions belong more as part of a test plan (used by 

>implementors ‹ similar maybe to the work done in bmwg, for example), 

>which seems to be in line with the

>comments: (Sue wrote) ³The recommended tests in major concern 1-4 could 

>be created in a separate draft.²

>It is not in the manet WGs charter to produce test plans.  In order to 

>not loose Sue¹s valuable input, I suggest we keep them in the WG¹s wiki 

>‹ which will allow for other test cases to be added, details included, 

>Again, the details are not within the scope of this document..and 

>guidance is already given in 1.1 about the use in real networks.

>Note that this comment mentions not just the extensions proposed, but 

>the base protocol and even AODVv2.  All these guidelines are important 

>from an operations point of view, but shouldn¹t be tied to this document.

>>RFC 7181 and RFC7188 do not match this IANA section.  Rather than 

>>repeat these comments, I will simple state the data needs to be 

>>consistent and the format match IANA¹s comments.

---

From: Henning Rogge <[email-masked]>
Date: Sun, 28 Jun 2015 15:14:47 +0200

On Thu, Jun 25, 2015 at 7:21 PM, Susan Hares <[email-masked]> wrote:
> If you are running an experiment with this protocol, you need to have
> specific details enough to determine if this experimental protocol is a
> success.  Otherwise, you will continue to have the OLSR vs. AODV-v2, or
> vague review on RFC5444 additions to the protocol.  This ends up in
> emotional debate without substantial experimental results to back it up.
> Emotional debates recycle. My suggestions aim at providing enough detail to
> settle these arguments with experimental results.  MANET has lots of
> exciting work to do in the advent of 5G, 802.11ac, and other  mobile network
> changes.

We (Fraunhofer FKIE) need multitopology for our heterogeneous
networks... what use is it to talk about "good routing metric" if your
video traffic is put onto the VHF link with 20 kBit/s? Different
service classes have different needs... and some network/radio
technologies would only serve a sub-group of them. Thats why I
implemented the multitopology draft while it was in an early draft
state.

Henning Rogge