Subject: Re: [icnrg] Revisiting Information-centric Networking: Evaluation Methodology

From: Christian Esteve Rothenberg <[email-masked]>
Date: Thu, 2 Apr 2015 18:04:27 -0300

Dear Kostas et al.,

I would like to suggest two pieces of comparing existing simulators
and experimentation tools in general:

Tortelli, Michele, Rossi, Dario, Boggia, Gennaro and Grieco, Luigi
Alfredo, CCN Simulators: Analysis and Cross-Comparison . In 1st ACM
SIGCOMM Conference on Information-Centric Networking (ICN-2014), Demo
Session, pages 197-198, Paris, France, September 2014.

http://perso.telecom-paristech.fr/~drossi/paper/rossi14icn-e.pdf

and the related technical report "Cross-comparison of ICN software
tools" by the same authors:
http://telematics.poliba.it/images/file/tortelli/icn-2014.pdf

One aspect I like about the latter work is Figure 1 on Realism vs
Complexity in ICN software. I think it would be nice to capture its
essence into the draft.

Furthermore, when reviewing draft-irtf-icnrg-evaluation-methodology,
it may be good approach to follow a similar organization by presenting
Prototypes/Testbeds, Emulators and simulators as independent
subsections under "Evaluation Methodology".

Regarding the work on emulators I would like to plug our work on the
Mini-CCNx code base (https://github.com/chesteve/mn-ccnx/wiki)  and
the associated publications below.

On an ongoing effort call mini-ndn we are extending mn-ccnx to run
also NDN code:
http://redmine.named-data.net/projects/mini-ndn

Our ongoing work on advancing the mini-ccnx/ndn is to have a common
container virtualization based emulator (that can run both native CCNx
and NDN) that allows running reproducible experiments (same
topologies, links, workloads) and provides the user with means to
collect the experiment results presenting the user with the desired
metrics like metrics (e.g., cache utilization, interface usage,
observed RTT, etc.) on a per node, network-wide or path-specific
manner. Adding automatic analysis and gnuplotting of results is also
on the roadmap. Support of emulated wireless links and mobility are on
the mid-term agenda.

My vision is having a common "high-fidelity"emulation platform that
supports multiple ICN prototypes natively, ie without requiring code
modifications (as long as they are Linux-based implementations) adding
a common abstraction layer that allow reproducible experiments of not
only a single ICN architecture but also fair comparative evaluations
of alternative ICN designs.

[1] Carlos Cabral, Christian Esteve Rothenberg, Mauricio Magalhães.
"High Fidelity Content-Centric Experiments with Mini-CCNx". In 19th
IEEE Symposium on Computers and Communications (ISCC), June 2014.

[2] Carlos Cabral, Christian Esteve Rothenberg, Mauricio Magalhães.
"Mini-CCNx: Fast Prototyping for Named Data Networking." In ACM
SIGCOMM 2013 - ICN'13 Workshop (Poster), Hong Kong, August 2013.

[3] Carlos Cabral, Christian Esteve Rothenberg, Mauricio Magalhães.
"Reproducing Real NDN Experiments using Mini-CCNx." In ACM SIGCOMM
2013 - ICN'13 Workshop (Demo), Hong Kong, August 2013.

On Mon, Mar 16, 2015 at 1:37 PM, Kostas Pentikousis
<[email-masked]> wrote:
> Dear All,
>
> With RFC 7476 published, it's time to revisit our evaluation methodology draft which expired in January. We will edit text that has aged and bring the document up to date, and hopefully ready for publication before Prague.
>
> I have received new text contributions from Hitoshi Asaeda which will be integrated in the upcoming -02 version.
>
> This is an open call for all ICNers to review, comment and propose text for the next version of  http://datatracker.ietf.org/doc/draft-irtf-icnrg-evaluation-methodology/. Feel free to propose text to the list, or send it to me directly.
>
> References to published work on ICN evaluation are also very much appreciated.
>
> Best regards,
>
> Kostas
>
>
> _______________________________________________
> icnrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/icnrg

---

From: Nikos Fotiou <[email-masked]>
Date: Fri, 3 Apr 2015 14:15:03 +0300

Dear all,
Regarding section 3.3, I would like to suggest an ICN privacy evaluation
framework we have created. It specifies various privacy threats,
adversaries, and a methodology for estimating the impact of a privacy
attack.

N. Fotiou, S. Arianfar, M. Sarela G.C. Polyzos, "A framework for privacy
analysis of ICN architectures," Proc. Second Annual Privacy Forum, APF
2014, Athens, Greece, 2014
http://link.springer.com/chapter/10.1007/978-3-319-06749-0_8

Best,
Nikos

On Mon, Mar 16, 2015 at 6:37 PM, Kostas Pentikousis <[email-masked]>
wrote:

> Dear All,
>
> With RFC 7476 published, it's time to revisit our evaluation methodology
> draft which expired in January. We will edit text that has aged and bring
> the document up to date, and hopefully ready for publication before Prague.
>
> This is an open call for all ICNers to review, comment and propose text
> for the next version of
> http://datatracker.ietf.org/doc/draft-irtf-icnrg-evaluation-methodology/.
> Feel free to propose text to the list, or send it to me directly.
>

---

From: Nikos Fotiou <[email-masked]>
Date: Sun, 12 Apr 2015 20:15:39 +0300

You can find a pdf version of this paper here
http://mm.aueb.gr/publications/2014-ICN_Privacy.pdf

Best,
Nikos
On Apr 3, 2015 2:15 PM, "Nikos Fotiou" <[email-masked]> wrote:

> On Mon, Mar 16, 2015 at 6:37 PM, Kostas Pentikousis <[email-masked]
> > wrote:
>
>> Dear All,
>>

---

From: Gennaro Boggia <[email-masked]>
Date: Wed, 15 Apr 2015 16:46:02 +0200

Dear Christian, Kostas at all,

as one of the coauthors of the papers that Christian suggested and one
of the coauthor of the draft, I could try to integrate some of the
ideas of our papers (in particular, the essence of Fig. 1)

Best Regards

Gennaro

--
Gennaro Boggia, PhD
Associate Professor
DEI - Politecnico di Bari
v. Orabona 4
70125 Bari (Italy)
Tel. +39 080 5963913
Fax +39 080 5963410
Skype: g.boggia
e-mail: [email-masked]; [email-masked]
web: http://telematics.poliba.it/boggia

---

From: Dario Rossi <[email-masked]>
Date: Wed, 15 Apr 2015 17:44:45 +0200

Dear all,

thanks Christian for the advertisement ;)

what I believe is also an important message is to define candidates for 
benchmarks that are not easy-to-beat naive solutions  (the paper offer 
some from the caching perspective, but the same principle may hold for 
aspects related  security, mobility, etc.)

other scientific communities run yearly challenges to defeat their 
champions:  keeping the draft up-to-date with new champions can possibly 
be a valuable contribution, and raising the bar higher can assist the 
community progress in the right direction

of course, the icnrg community can rely on e.g., sigcomm icn, to sponsor 
these (admittedly hard to put in practice) challenges

best,
D.

-- 

  Oo    Professor
   >    Telecom ParisTech
  ~     Ecole Polytechnique

mail:  [email-masked]
phone: +33.1.4581.7563
fax:   +33.1.4581.7158
web:   http://www.enst.fr/~drossi