Subject: [icnrg] ICN evaluations on publicly-available testbeds

From: Kostas Pentikousis <[email-masked]>
Date: Mon, 16 Mar 2015 17:57:27 +0100

Dear All,

After reviewing Sec. 2.1 of the methodology draft (https://tools.ietf.org/html/draft-irtf-icnrg-evaluation-methodology-01#section-2.1) it becomes clear that we're missing (peer-reviewed) references for ICN evaluations that have been performed on testbeds. Since this is one of the older parts of the draft, it needs to desperately be brought up to date :)

In particular, the draft mentions the following testbeds that _could_ be used to evaluate ICN solutions:

1. Open Network Lab
2. PlanetLab
3. Emulab
4. Deter
5. NEPI
6. PURSUIT Blackadder testbed
7. GENI and FP7 OFELIA and ALIEN
8. JGN-X

Please kindly let me (and the list) know of any papers that _have actually used_ the aforementioned testbeds, and whether the test scripts/data are available for re-use by others in the community. Of particular interest imo is user experience and advice by the folks who have used the aforementioned testbeds in their evaluations. If we don't actually identify seminal literature, I think we can remove the section as the interest in this line of work may no longer be there. (please prove me wrong :)

Best regards,

Kostas

---

From: Damien Saucez <[email-masked]>
Date: Mon, 16 Mar 2015 23:36:21 +0100

Hello Kostas,

NEPI by itself is not a testbed but a programming interface that facilitates
the deployment of experiments in various testing environment, among them
PlanetLab and ns-3.

To illustrate this programming interface, we have experimented examples in
PlanetLab and ns-3 so there is at least one paper that deployed CCN in
PlanetLab :)

- Demonstrating a unified ICN development and evaluation framework
(http://dl.acm.org/citation.cfm?id=2660132).

All the code is available to test.

More seriously, a brief check on scholar gave me at least that references:

- Video streaming over content centric networking: Experimental studies on
PlanetLab (http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6533602)

- Socially-aware caching strategy for content centric networking
(http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6857093)

- Multipath forwarding strategies in Information Centric Networks with AIMD
congestion control (http://arxiv.org/abs/1412.2204)

…

I dont think a draft is the right place to make a survey though.  I think the
draft can just explain the different options that exist to _potentially_
experiment ICN technologies.  After all the draft is about evaluation
methodology, isn’t it?

Cheers,

Damien Saucez 

On 16 Mar 2015, at 17:57, Kostas Pentikousis <[email-masked]> wrote:

> _______________________________________________
> icnrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/icnrg

---

From: "Trossen, Dirk" <[email-masked]>
Date: Tue, 17 Mar 2015 02:35:59 +0000

Dear Kostas,

The PURSUIT overlay testbed was mainly used for validation and demonstration in the project (and we are about to rekindle the test bed in the context of two new EU projects). We used a dedicated 1Gb/s test bed for performance measurements of different kinds, published in, e.g., the IEEE Comms Mag Special Issue on ICN and other venues (with different foci in the evaluations).

Best,

Dirk

Dirk Trossen
Principal Engineer
InterDigital UK, Inc.
Shoreditch Business Center
64 Great Eastern Street
London,  EC2A 3QR
T: +44 20 7749 9178
[email-masked]
www.InterDigital.com<http://www.interdigital.com>

This e-mail is intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential and/or otherwise protected from disclosure to anyone other than its intended recipient. Unintended transmission shall not constitute waiver of any privilege or confidentiality obligation. If you received this communication in error, please do not review, copy or distribute it, notify me immediately by email, and delete the original message and any attachments. Unless expressly stated in this e-mail, nothing in this message or any attachment should be construed as a digital or electronic signature.

-----Original Message-----
From: icnrg [mailto:[email-masked]] On Behalf Of Kostas Pentikousis
Sent: 16 March 2015 16:57
To: [email-masked]
Subject: [icnrg] ICN evaluations on publicly-available testbeds

Dear All,

1. Open Network Lab
2. PlanetLab
3. Emulab
4. Deter
5. NEPI

Best regards,

Kostas

---

From: Hitoshi Asaeda <[email-masked]>
Date: Tue, 17 Mar 2015 13:48:16 +0900

Hi Kostas,

JGN-X is a major testbed in Japan (and connected to several countries).
(see http://www.jgn.nict.go.jp/english/index.html)
It has been used various experiments as a new generation network testbed, not only for ICN/CCN/etc.

On the other hand, Container-based unified testbed for ICN (CUTEi), published in the last Nov. issue of IEEE Network, is an open testbed dedicated to ICN experiments.
(see http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6963806)
We are open to anyone who is interested in ICN experiments to connect to this testbed.

We'll also publish a paper describing a tool for ICN measurement named Contrace in this Mar. issue of IEEE ComMag. The detail of the contrace tool would be specified in a separate draft, though.

Regards,
--
Hitoshi Asaeda

---

From: Emmanuel Baccelli <[email-masked]>
Date: Tue, 17 Mar 2015 10:50:13 +0100

Hi Kostas,

last year, we have made experiments with ICN on IoT testbeds, using
CCN-lite on top of the RIOT operating system (www.riot-os.org).
See for example this publication in ACM ICN:
http://conferences2.sigcomm.org/acm-icn/2014/papers/p77.pdf

In this context you can also add the following testbeds to your list:
- IoT-LAB (www.iot-lab.info) which is the testbed we are currently using
for our next ICN experiments on IoT (still using RIOT).
- DES testbed (http://www.des-testbed.net) which we used for the
experiments in the above publication.

Best,

Emmanuel

On Mon, Mar 16, 2015 at 5:57 PM, Kostas Pentikousis <[email-masked]>
wrote:

> Dear All,
>
> After reviewing Sec. 2.1 of the methodology draft (
> https://tools.ietf.org/html/draft-irtf-icnrg-evaluation-methodology-01#section-2.1)
> it becomes clear that we're missing (peer-reviewed) references for ICN
> evaluations that have been performed on testbeds. Since this is one of the
> older parts of the draft, it needs to desperately be brought up to date :)
>
> Please kindly let me (and the list) know of any papers that _have actually
> used_ the aforementioned testbeds, and whether the test scripts/data are
> available for re-use by others in the community. Of particular interest imo
> is user experience and advice by the folks who have used the aforementioned
> testbeds in their evaluations. If we don't actually identify seminal
> literature, I think we can remove the section as the interest in this line
> of work may no longer be there. (please prove me wrong :)
>
> Best regards,
>
> Kostas
>
>

---

From: Stefano Salsano <[email-masked]>
Date: Thu, 19 Mar 2015 01:52:27 +0100

Dear Kostas, all,

we have performed experiments about ICN over SDN on the OFELIA testbed:

S. Salsano, N. Blefari-Melazzi, A. Detti, G. Morabito, L. Veltri,
"Information Centric Networking over SDN and OpenFlow: Architectural 
Aspects and Experiments on the OFELIA Testbed"

Computer Networks, Volume 57, Issue 16, 13 November 2013, Pages 
3207-3221, ISSN 1389-1286

http://dx.doi.org/10.1016/j.comnet.2013.07.031

in the Software Download section of http://netgroup.uniroma2.it/CONET/ 
the software we used in the experiments is available

best regards
Stefano

-- 
*******************************************************************
Stefano Salsano
Professore Associato
Dipartimento Ingegneria Elettronica
Universita' di Roma Tor Vergata
Via del Politecnico, 1 - 00133 Roma - ITALY

http://netgroup.uniroma2.it/Stefano_Salsano/

E-mail  : [email-masked]
Cell.   : +39 320 4307310
Office  : (Tel.) +39 06 72597770  (Fax.) +39 06 72597435
*******************************************************************