Subject: Re: [P2Prg] CORE subgroup problem statement

From: [email-masked]
Date: Tue, 24 Jan 2006 02:00:08 -0500

Dear Luca,
Thanks very much for the detailed and useful comments.  Responses below.
Best regards,
John

> Dear John,
> I found the draft interesting. Please, find in this mail same comments.
> I am reporting part of the text and inserting my ideas between the <LC> </LC> tags.
> 
> 
> Best Regards,
> Luca
> 
> 
> Pag. 1:
> 
> However existing designs have limitations in one or more areas of scalability, security, interoperability, or generality.
> 
> <LC>
> I am not quite sure there are generality problems. At least, the concept of generality should be defined. What I think is valuable of p2p, is its ability of building overlays, and then allowing some kind of network independency (BTW, citing JXTA is something, that recalls this concept). Then, you should clarify what is generality for you and why you think p2p can change something. Is generality something related to "re-usability" or a more "smart" concept, such as the network independent deisgn?
> </LC>.

<JB>
Yes, I agree that p2p overlay and index mechanisms are quite general. 
But some p2p systems are application-specific (e.g., file sharing, voip)
so not being general-purpose is meant from service-offering standpoint.  
So generality has to do, in my opinion, with supporting any type of service.
</JB>

> 
> 
> Pag 2:
> 
> The presented literature is quite comprehensive. However, I would consider as follows.
> 
> <LC>
> Since we are talking about IETF, you can cite some good technologies, which are something related to service discovery. Mostly, they are suitable for local-small scale scenarios, but someone is thinking about enabling them for the global Internet. For instance:
> 
> o) RFC 2782 "A DNS RR for specifying the location of services (DNS SRV)" http://www.ietf.org/rfc/rfc2782.txt
> 
> o) The DNSService Discovery (http://www.dns-sd.org/). Besides, the DNS service discovery can be jointly used with the Multicast DNS (http://www.multicastdns.org/ and http://files.dns-sd.org/draft- cheshire-dnsext-nbp.txt). And of course, even if "human readable" and "rough", the DNS is related to "overlays". (http://files.dns-sd.org/ draft-cheshire-dnsext-dns-sd.txt)
> 
> 
> o) You can cite also Zeroconf (http://www.zeroconf.org/).
> 
> o) And of course, one can exploit some mapping with underlying technologies, such as IPv6.
> </LC>

<JB>
Yes, thanks for these suggestions.  Definitely RFC2782 and multicastdns and zeroconf.
However isn't IPv6 an enabling technology?  
</JB>

> 
> Further there are various peer-to-peer overlay networks and proposed designs, both unstructured and structured by which large numbers of peers share content and services.
> 
> <LC>
> From a strict p2p point of view, there are also hybrid architectural flavors, such as BitTorrent. In hybrid frameworks, the centralized component plays a role (not being solely devoted to bootstrap). You know, people out there have used client-server for decades, hence I would be a little pragmatic.
> </LC>

<JB>
Yes agree.  In fact the P2P overlay survey we posted had hybrid in the taxonomy.
Good point.
</JB>

> 
> Typically large-scale unstructured peer-to-peer systems use limited scope flooding or random walk to reach other peers with requests...
> 
> <LC>
> In hybrid systems, centralized component are used to locate peers and then directly route requests.
> 
> For instance, in eMule (I am citing file-sharing since is quite popular), you can use 3 different architectural flavors:
> 1) use server to locate via e2DK
> 2) use Kad (XOR-based Chord DHT ring)
> 3) Exchange info via hording.
> 
> 
> Or in OpenNap
> 1)Use a OpenNap Server
> 2)Use a chained facility of OpenNap servers to route requests.
> 
> However, the basic under such remarks is that "the more the unstructured p2p system is large, the worse the limited scope flooding performs". For this reason, supernodes, centralized components and so on have been introduced.
> 
> Lastly, since it is a problem statement draft, I would mention that "limited scope flooding" does not allow to perform _exhaustive_ searches. Then we have to be honest: p2p is great for what you propose, but still need to be fixed. And in a problem statement draft, probably will be a nicer phrase.
> 
> "Are you going to migrate DNS to an architecture that could be unable to resolve some URL, because peers responsible are too distant?"
> </LC>

<JB>
Good points, this can be cited, do you suggest citations for eMule and OpenNap
technical details?
Also, we kept the overlay discussion short in the problem statement since 
we have proposed in the charter to prepare a separate survey document.
So some of this discussion might go there.

Regarding the chance of not finding something that is stored somewhere in the overlay,
yes it doesn't seem like a desirable scenario. 

What would be nice from a research standpoint is to have a probabilistic formula
as a function of size, architecture, churn, average node lifetime, replication rate,
and index pollution, etc. of finding an item stored in the overlay.
</JB>

> 
> Search: a discovery mechanism in which content or resources stored in a peer-to-peer network can be searched using various methods including text search, keyword search, content-based retrieval or meta-data search
> 
> 
> <LC>
> Are they stored in the p2p network? Or are they stored in peers?
> 
> 
> My 2 cents:
> 
> -)how the overlay is "built" states intrinsically _how to retrieve_ (e.g., A DNS based Overlay roughly corresponds to text search).
> 
> 
> -)peers belonging to a given overlay _store_ the content or resources.
> 
> 
> (??? I am not fond of CDN, but CDN-literature should be inspiring)
> </LC>

<JB>
Why is the distinction between storage in the peer vs p2p network important?
Consider a system like OpenDHT where some node which is not part of the
p2p system can insert and lookup into a p2p index formed by another set of 
nodes.  from the client node standpoint, the p2p index could just as well
be a single server, and the storage is somewhere in the collection of nodes,
hence "p2p network".  Also there is the case of replicated entries, and
cases where we might split up an object into parts before indexing
each part.

Regarding "how the overlay is built ... how to retrieve".  I am not sure.
A number of proposals develop complex mappings and lookups on to a simple
hash-base index.

Agree that CDN is relevant.  Seems like a lot of CDN is push-oriented
model anticipating future requests.
</JB>

> 
> Service: a computational function packaged for use by other nodes in a networked environment .
> 
> <LC>
> It sounds like "RPCs" or "Stored Procedure" stuff. It is quite clear the idea, but for my personal style I would avoid "computational"... "a portion of peer resource" is more general (maybe when you'll define your architectures, you'll have storage-peers, computing- peers... hence, it is more general).
> </LC>

<JB>
How about:
Service: an interface to a resource packaged for use by other nodes in a networked environments.
</JB>

> 
> 
> Pag 3:
> 
> Service description: Information about a networked service such as type of service, name of service, attributes of service, location of service, invocation of service. May be stored in a document or at a service repository
> 
> <LC>
> Confusion here (for me, it is not enough clear). I can't convince myself about what a "Service Repository" is:
> 
> o) Service Repository: a peer containing related infos.
> o) Service Repository: a boot strap server which allows to locate and exploit a p2p-based service.
> o) A portion of the overlay.
> o) A centralized component (see previous comment).
> 
> But basically: "Is Service Repository, inside or outside the p2p- framework?"
> </LC>

<JB>
Service repository is intended to cover architectures which store descriptions at a server
(e.g., SLP, UDDI, LDAP) or proxy node.
</JB>

> 
> The expected volume of transactions expected in a global service discovery mechanism is in the range of DNS lookup volume.
> 
> <LC>
> How can justify such bound? "The expected volume of transactions expected in a global service discovery mechanism is in the range of DNS lookup volume."
> </LC>

<JB>
I think it is an upper bound.  It seems that service invocation frequency should be much
higher than service discovery frequency.  But service discovery < DNS lookups, in that
hostname resolves due to say web browsing should occur at a higher frequency then
service discovery requests.  

</JB>

> 
> 
> Pag. 4
> 
> Use of multiple service description formats which might be tailored by application or level of semantic description
> 
> 
> <LC>
> I appreciate this point, really.
> </LC>
> 
> Access to services, service descriptions, and service advertisements dependent on authorization rights
> 
> 
> <LC>
> I can see HIP at the horizon...
> </LC>
> 
> The mechanism can be used in isolation by peers in PANs, ad hoc networks, or other limited networking environments
> 
> 
> <LC>
> Not clear what isolation is...
> </LC>
> 

<JB>
That is, if say 5 peers are connected to each other but not to the Internet, they
should still be able to do service discovery among each other.
An example is if I have 5 devices in my PAN and I roaming disconnected from
other networks.
</JB>

> 
> Pag. 5
> 
> 
> There are no new security considerations in this document.
> 
> <LC>
> I agree. In this incarnation the security considerations are not pertinent.
> </LC>
> 

_______________________________________________
P2prg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/p2prg

---

From: Arnaud Legout <[email-masked]>
Date: Tue, 24 Jan 2006 10:01:57 +0100

Hello,

> Yes agree.  In fact the P2P overlay survey we posted had hybrid in the taxonomy.
The taxonomy based on structured, unstructured, and hybrid is focused on 
the content localization of P2P.
However, P2P is also about efficient content delivery.
In particular, it is an error to classify BitTorrent as an hybrid 
protocol. BitTorrent does not have any content localization feature.
You find your content on a web site. BitTorrent can be possibly 
classified, from a content localization point of view,
as centralized.

 From my point of view, a better taxonomy for P2P is:
*Content Localization
  -Structured
  -Unstructured
  -Centralized
  -Hybrid (?)
*Content delivery
  -Parallel download
  -File splitting  (swarming)
  -Piece selection
  -Peer selection

This taxonomy must be refined, but at least, it takes into account both 
content localization and content delivery.
With this taxonomy, BitTorrent can now be correctly classified:
Content Localization: centralized
Content Delivery: Parallel download, File splitting, rarest first piece 
selection, choke algorithm peer selection.

Regards,
Arnaud Legout.

-- 
Arnaud Legout, Ph.D.

INRIA Sophia Antipolis - Planète  Phone : 00.33.4.92.38.78.15
2004 route des lucioles - BP 93   Fax   : 00.33.4.92.38.79.78
06902 Sophia Antipolis CEDEX      E-mail: [email-masked]
FRANCE                            Web   : http://www-sop.inria.fr/planete/Arnaud.Legout/index.html

---

From: Luca Caviglione <[email-masked]>
Date: Tue, 24 Jan 2006 10:29:04 +0100

Dear John,
than you for the good discussion. Some additional comments are below.

Best Regards,
Luca

Il giorno 24/gen/06, alle ore 08:00, [email-masked] ha  
scritto:

>>
>> <LC>
>> I am not quite sure there are generality problems. At least, the  
>> concept of generality should be defined. What I think is valuable  
>> of p2p, is its ability of building overlays, and then allowing  
>> some kind of network independency (BTW, citing JXTA is something,  
>> that recalls this concept). Then, you should clarify what is  
>> generality for you and why you think p2p can change something. Is  
>> generality something related to "re-usability" or a more "smart"  
>> concept, such as the network independent deisgn?
>> </LC>.
>
> <JB>

<LC>
Ok: supporting any type of service is a good concept.
</LC>

>> Since we are talking about IETF, you can cite some good  
>> technologies, which are something related to service discovery.  
>> Mostly, they are suitable for local-small scale scenarios, but  
>> someone is thinking about enabling them for the global Internet.  
>> For instance:
>>
>> o) RFC 2782 "A DNS RR for specifying the location of services (DNS  
>> SRV)" http://www.ietf.org/rfc/rfc2782.txt
>>
>> o) The DNSService Discovery (http://www.dns-sd.org/). Besides, the  
>> DNS service discovery can be jointly used with the Multicast DNS  
>> (http://www.multicastdns.org/ and http://files.dns-sd.org/draft-  
>> cheshire-dnsext-nbp.txt). And of course, even if "human readable"  
>> and "rough", the DNS is related to "overlays". (http://files.dns- 
>> sd.org/ draft-cheshire-dnsext-dns-sd.txt)
>>
>>

<LC>
Absolutely. I think IPv6 is a must for P2P. Anyway, let's see what  
happens about P2P and IPv6 :-)
</LC>
>
>
>>
>> Typically large-scale unstructured peer-to-peer systems use  
>> limited scope flooding or random walk to reach other peers with  
>> requests...
>>
>> <LC>
>> For instance, in eMule (I am citing file-sharing since is quite  
>> popular), you can use 3 different architectural flavors:
>> However, the basic under such remarks is that "the more the  
>> unstructured p2p system is large, the worse the limited scope  
>> flooding performs". For this reason, supernodes, centralized  
>> components and so on have been introduced.
>>
>> Lastly, since it is a problem statement draft, I would mention  
>> that "limited scope flooding" does not allow to perform  
>> _exhaustive_ searches. Then we have to be honest: p2p is great for  
>> what you propose, but still need to be fixed. And in a problem  
>> statement draft, probably will be a nicer phrase.
>>
>> "Are you going to migrate DNS to an architecture that could be  
>> unable to resolve some URL, because peers responsible are too  
>> distant?"
>> </LC>
>
> <JB>

<LC>
Personally, I think you can avoid citing eMule and OpenNap. I mean,  
it is not mandatory, but many guys use them and they are well-known.  
You can just introduce that some p2p systems have problem in searches  
(and of course expanding this in the survey as you suggested).   
Besides, I think that the probabilistic formula is the right and  
better answer for convincing/describing p2p - search characteristics.
Let me know if you need help in the survey...
</LC>

>
>>
>> Search: a discovery mechanism in which content or resources stored  
>> in a peer-to-peer network can be searched using various methods  
>> including text search, keyword search, content-based retrieval or  
>> meta-data search
>>
>>
>> <LC>
>> -)how the overlay is "built" states intrinsically _how to  
>> retrieve_ (e.g., A DNS based Overlay roughly corresponds to text  
>> search).
>>
>>

<LC>
Good point. I agree.
</LC>

>> It sounds like "RPCs" or "Stored Procedure" stuff. It is quite  
>> clear the idea, but for my personal style I would avoid  
>> "computational"... "a portion of peer resource" is more general  
>> (maybe when you'll define your architectures, you'll have storage- 
>> peers, computing- peers... hence, it is more general).
>> </LC>
>
> <JB>
> How about:

<LC>
Perfect, I like it very much.
</LC>

>
>>
>>
>> Pag 3:
>>
>> Service description: Information about a networked service such as  
>> type of service, name of service, attributes of service, location  
>> of service, invocation of service. May be stored in a document or  
>> at a service repository
>>
>> <LC>
>> Confusion here (for me, it is not enough clear). I can't convince  
>> myself about what a "Service Repository" is:
>>

<LC>
Clear, now.
</LC>

>
>>
>> The expected volume of transactions expected in a global service  
>> discovery mechanism is in the range of DNS lookup volume.
>>
>> <LC>
>> How can justify such bound? "The expected volume of transactions  
>> expected in a global service discovery mechanism is in the range  
>> of DNS lookup volume."
>> </LC>
>
>
> <JB>

> An example is if I have 5 devices in my PAN and I roaming  

<LC>
Good points.
</LC>

---

From: Luca Caviglione <[email-masked]>
Date: Tue, 24 Jan 2006 10:55:17 +0100

Hi Arnaud,
some comments in-line.

Best Regards,
Luca

Il giorno 24/gen/06, alle ore 10:01, Arnaud Legout ha scritto:

> The taxonomy based on structured, unstructured, and hybrid is  
> focused on the content localization of P2P.

<LC>
Yes, of course.
</LC>

> classified, from a content localization point of view,

<LC>
I would not start a flame about "terminology" :-). However, hybrid  
protocol is something I don't like in this context. In fact, I used  
"architectural", rather than protocol. So, I think there are not  
errors in defying BitTorrent as an hybrid architecture. Actually,  
there are two architectural flavors overlapped: centralized, from "a  
content localization point of view" and p2p for the swarm.
Hybrid Protocol suggests functionalities and does not lead to a  
better understanding of "how the architecture is arranged".
</LC>

>  -File splitting  (swarming)

<LC>
In my opinion, what you suggest is somewhat misleading. I mean,  
parallel download, file-splitting, piece selection... are techniques  
for delivering a file over a network and are not strictly related  
with p2p. The only requirement is that content is replicated among  
several entities.
Then with your taxonomy, I can describe as p2p "GetRight", the FTP  
client for Windows. With GetRight you can set mirrors and  
concurrently download from different FTP servers. Ok, this is an  
exasperation of the concept, but I think that architectures must be  
separated from peculiar tricks.
</LC>

>
> This taxonomy must be refined, but at least, it takes into account  
> both content localization and content delivery.
> Content Delivery: Parallel download, File splitting, rarest first  
> piece selection, choke algorithm peer selection.

<LC>
I think you did not classified BitTorrent. You are only listing some  
techniques used for achieving a better efficiency on the networked  
environment.
</LC>

Regards,
Luca

---

From: Arnaud Legout <[email-masked]>
Date: Tue, 24 Jan 2006 18:32:21 +0100

Hi Luca,

Luca Caviglione wrote:
> Hybrid Protocol suggests functionalities and does not lead to a better 
> understanding of "how the architecture is arranged".
> </LC>
<AL>
I don't agree. When you classify architectures as structured and 
unstructured, hybrid should mean that you have a combination of 
structured and unstructured.
Otherwise, hybrid is for everything you don't know how to classify. 
"Other" would have been a better name, but still a weak classification.

It is not correct to classify some protocol based on their localization 
architecture and some other protocols based on their content replication 
architecture,
without making a clear distinction in the taxonomy.
If you consider BitTorrent to be an hybrid architecture because there is 
a central content localization and p2p file replication, then
any other P2P protocol should be hybrid. Instead, using your taxonomy BT 
should be considered as centralized.

I do not believe I am raising a flame about terminology.
I simply want to point out that the way P2P is considered for now is 
content localization oriented.
This is only one side of P2P, and content localization and content 
replication should be uncorrelated. 
BitTorrent, which is undoubtedly the most efficient general purpose P2P 
protocol, cannot be classified
using the current taxonomy.
Thus, this taxonomy do not cover all the P2P architectures.
</AL>
> for delivering a file over a network and are not strictly related with 
> p2p. The only requirement is that content is replicated among several 
> entities.
> client for Windows. With GetRight you can set mirrors and concurrently 
> download from different FTP servers. Ok, this is an exasperation of 
> the concept, but I think that architectures must be separated from 
> peculiar tricks.
> </LC>
<AL>
I am not sure it is possible to abstract the architecture from the 
technique..Unstructured is not specific to P2P as well. Multicast has 
been using this kind of architectures and techniques (flooding, 
expanding ring search, etc.) for a long time.
Structured is not specific to P2P as well, unless you mean 
Structured=DHT, but in this case you are no more making a distinction 
between architecture
and technique.
Using the current taxonomy you can already classify protocols that are 
not P2P.

You are right that my taxonomy is technique oriented rather architecture 
oriented.
For all the P2P protocols I know, structured means DHT, and unstructured 
means flooding, expanding ring search, random walk, etc.
I would really prefer a taxonomy using DHT, flooding, expanding ring 
search, etc. than structured and unstructured.
</AL>
> techniques used for achieving a better efficiency on the networked 
When you classify Chord as structured, don't you mean it is a DHT? Do 
you have an example of structured P2P architecture that is not a DHT?

 From my point of view, DHT is at the same semantic level than piece 
selection, and Chord is at the same semantic level than rarest first.

I don't know which definition of structured and unstructured 
architecture you consider, but the one presented in [1] is technique 
dependent and vague.
Structured is for DHT, unstructured is for other localization techniques.

Regards,
Arnaud.

[1] Lua et al. "A Survey and Comparison of Peer-to-Peer Overlay Network 
Schemes" IEEE Communication s survey and tutorial, March 2004

---

From: Bill Yeager <[email-masked]>
Date: Tue, 24 Jan 2006 10:00:34 -0800

Bonjour Arnold,

Your taxonomy seems to be restricted to file sharing or at least the  
delivery of content. While the latter is general I think you mean  
reasonably sized files. What about communities, and messaging in  
general? One can easily do text messaging (SMS or MMS like), play  
games, and email within a P2P taxonomy for example.

In particular with a community structure imposed for communication,  
ie, all communication is intra-community, the yield is a virtual  
private network if one adds TLS, and public key authentication as a  
participation in a community requirement. This quickly gets us to  
SPAM less communication, etc.

P2P is quite rich and the application/services one can map onto the  
right system diverse.

Just a thought,

Bill

---

From: Luca Caviglione <[email-masked]>
Date: Tue, 24 Jan 2006 20:19:44 +0100

Hi Arnaud ...

> "Other" would have been a better name, but still a weak classification.

I do agree here. Other is a better name and also a weak classification. However,
I think that correct classification (better, the one I would prefer) must rely
on architectures, rather than functionalities.

> should be considered as centralized.

Actually, BitTorrent is a superposition of two protocols. The BT-Tracker
protocol and the BT protocol for the swarm. Then, the protocols reflect the
architecture where they act. However, this is my perspective and it doesn't
mean it is the best one or better than others.

> I do not believe I am raising a flame about terminology.

Yes, sorry. I was suggesting (for my answer) that it is not solely a debate for
"terms".

> expanding ring search, etc.) for a long time.

I partially agree. However, I think that the first step, in order to produce a
taxonomy, is to separate causes with effects. You can use swarming, hording (or
flooding and expanding ring) since the architectural blueprints allow such
techniques.

> search, etc. than structured and unstructured.

For me, structured means that there is an enforcement in the organization of
peers. Having the peers forced to arrange in a ring (as an example) allows to
exploit some search strategies. On the contrary, having peers connected
randomly, brings you in the case where you don't know where the content is. So
you must flood or something else.

> selection, and Chord is at the same semantic level than rarest first.

This is an interesting point.

> Structured is for DHT, unstructured is for other localization techniques.

...and probably is there the main knot. For me, unstructured and structured is
how "the peers are organized" and not related to localization techniques. I
mean, if I am going crazy, I can use a flood based technique selecting randomly
peers in Chord's finger tables (and probably I can patent this as the useless
search system ever).

Regards,
Luca

> Schemes" IEEE Communication s survey and tutorial, March 2004

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

---

From: Arnaud Legout <[email-masked]>
Date: Tue, 24 Jan 2006 22:32:29 +0100

Hi Luca,

> on architectures, rather than functionalities.
I am not sure I understand why you want to focus on architecture. As you 
said below, that only gives you information
on how peers are organized. They what can you do with that?
A taxonomy should have an intent, when it is based on functionality it 
offers a classification of building blocks that can be
used to design new protocols.
At least, I would have understood a refined taxonomy based on 
architecture. But the taxonomy structured/unstructured
is too coarse to be useful.
> mean it is the best one or better than others.
Now, I see why you classify it as hybrid, but here again I do not see 
how this classification is useful.
I expect from a taxonomy to be able to compare protocols within the same 
classification.
But, protocols classified as structured, unstructured and hybrid are so 
different, that I cannot really compare them.

> flooding and expanding ring) since the architectural blueprints allow such
I am not sure what you  mean by cause and effect. If you mean 
architecture and functionality, I am not sure
that starting with architecture is the right first step.
My argument is that in the context of P2P the architecture, i.e., how 
peers are arranged, does not give you much information.
I agree it is better than nothing, but a taxonomy based on functionality 
would be more useful.

> you must flood or something else.
I see, but you can have peer structured in a ring for content 
localization or for file replication (I agree this is not the best
architecture for file replication, this is just for the example).
The case of unstructured is more realistic. You can have a random graph 
connectivity for content localization or for
content replication.
Thus, knowing that your protocol is classified as structured or 
unstructured is not of much help.

> peers in Chord's finger tables (and probably I can patent this as the useless
and that is my point (I am starting to be very redundant), the taxonomy 
based on structured and unstructured does not give you
a very useful information.

Regards,
Arnaud.

---

From: Michael Rogers <[email-masked]>
Date: Wed, 25 Jan 2006 10:05:15 +0000

Hi Arnaud,

> peers are arranged, does not give you much information.

On the other hand there are some situations (P2P over wireless, NATs and 
firewalls, F2F) where you have to work with the topology you're given, 
and it's useful to know which protocols are suitable for those (maybe 
not "unstructured" but "unstructurable") networks.

> I agree it is better than nothing, but a taxonomy based on functionality 

Perhaps the two are orthogonal - if not, it might be enlightening to 
find out why not.

Cheers,
Michael