Subject: RE: [Ospf-manet] evaluation plan for OSPF MANET

From: "Henderson, Thomas R" <[email-masked]>
Date: Wed, 30 Nov 2005 07:17:23 -0800

> -----Original Message-----
> From: Russ White [mailto:[email-masked]] 
> Sent: Wednesday, November 30, 2005 5:15 AM
> To: Henderson, Thomas R
> Cc: [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [Ospf-manet] evaluation plan for OSPF MANET
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> 
> > A second step should be to agree on evaluation criteria and 
> scenarios, 
> > to address the comments at the WG meeting that the previously used 
> > scenarios were too narrow.  This should be agreed upon 
> within the next 
> > month or so, based on list discussion.  The 
> scenarios/criteria need to 
> > be feasible to achieve in the timeframe of this evaluation.
> 
> This presupposes emulations are "the" way to evaulate the 
> proposals. In fact, I believe we need to do more traditional 
> work here, such as examining pathological cases. For instance:

Russ, I don't think my message presupposed that emulations were the only
criteria.  I strongly agree with you that the underlying algorithms need
to be examined outside of simulation or experiments.

Other than the below, are there other OSPF benchmarking metrics that you
believe should be considered (I know that Sue Hares has repeatedly
suggested that we consider the work done on benchmarking)?

Tom

> 
> o Is there a point in the mobility graph where the protocol 
> mechanism simply won't converge when real rachability should 
> be available in the network?
> 
> o Is there a specific event that will cause a long outage in 
> reachability when physical reachability still exists?
> 
> o How much change to existing code do the proposed mechanisms 
> cause? One of the reasons we're using OSPF here is to reuse 
> as much of a known working protocol as possible in a 
> different environment. If the proposed changes change entire 
> sections of the OSPF code (1,000 lines sounds trivial, until 
> you realize that most OSPF implementations are probably only 
> 12,000 lines of code, and 1/12th doesn't sound so trivial any 
> longer), then we don't want to mess with it--better to use a 
> new protocol.
> 
> o How extensible, into the future, are the proposed changes? 
> Are they currently "at the end of the line," as optimal as 
> they will ever get, or are there other areas of exploration 
> to be done? If one mechanism has optimized all it can, and 
> another shows similar improvements, but has many more 
> possible optimizations, it's better to go with one that can 
> be further optimized.
> 
> These are the sorts of things we normally look at when 
> examining a protocol change, rather than basing the change 
> simply on simulation results. We all realize implementations 
> have a huge impact on performance (you can implement SPF as 
> n^2, or n(log n), for instance), and that you learn how to 
> improve performance over time. Basing an initial modification 
> based on simple performance in a simulator is the wrong way 
> to work at the problem. Imagine the IETF choosing between 
> IS-IS and OSPF based on simulation work (!).
> 
> Simulations are interesting, but they don't indicate much of 
> anything, until you get to simulating on real routers, and 
> possibly even real radios (although I'm not so certain about 
> that part, for various reasons).
> 
> :-)
> 
> Russ
> 

_______________________________________________
Ospf-manet mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ospf-manet