Subject: [p2pi] Thoughts on how IETF standards can help P2P/ISPs

From: Laird Popkin <[email-masked]>
Date: Mon, 2 Jun 2008 12:06:06 -0400 (EDT)

As we discussed last week at the IETF P2Pi Workshop, here are a list of some of the ways that IETF standards can potentially help P2P applications make better network decisions, and thus help ISPs. This isn't a complete list, but I hope that it's at least a useful starting point for a discussion.

- Expose more information about TCP streams. For example, expose whether the stream is congested. Currently some p2p applications use a variety of techniques to attempt to guess whether a user's internet connection is congested (e.g. monitoring ping times), and measures stream throughput which could be viewed as a very rough measure of congestion, but it would be more accurate and efficient if TCP told the application when a stream was congested. Even better would be to get some indication of where the congestion occurred. For example, if the congestion is in the 'last mile', or the other peer's 'last mile', or somewhere in between, different responses would be appropriate. For example, if my internet connection is fine, but the peer's connection is congested, I should slow down transfers from that peer, and pull more from other peers. This is, of course, easier said than done.

- Expose some measure of 'communications cost' so that p2p apps could select the lowest cost available data sources. (This is what the P4P protocol does, for example). If using this guidance benefits both P2Ps and ISPs, it is a win-win. This is a fairly broad area that might include a number of steps/messages/protocols in order to address service location, mapping IPs to location, and "cost". The P4PWG is researching this area, and while it feels a little early to propose a standard, it's certainly an area (IMO) that could benefit from attention.

- ISP's could expose usage metrics. For example, if a user has a link that is capable of 1 Mbps, has a monthly transfer capacity of 500 GB, and has used 350 GB so far, then the application can make better choices, and application users could make informed choices. If the ISP could tell applications when the user's internet connection is congested, that would be extremely helpful.

- Allow applications to tag some traffic as 'not time sensitive' so that ISPs can manage bulk traffic as lower priority than time-sensitive traffic. There would need to be a benefit to the application for doing so, such as excluding such traffic from ISP capacity limits.

- Laird Popkin, CTO, Pando Networks
  mobile: 646/465-0570
_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 2 Jun 2008 09:38:26 -0700

On Jun 2, 2008, at 9:06 AM, Laird Popkin wrote:
>
>
> - Allow applications to tag some traffic as 'not time sensitive' so  
> that ISPs can manage bulk traffic as lower priority than time- 
> sensitive traffic. There would need to be a benefit to the  
> application for doing so, such as excluding such traffic from ISP  
> capacity limits.

I actually think the opposite ("tag as priority") is better, because  
its easy to do "tag & cap" and the incentives match up better.

Here's why...

There is no benefit to saying "I'm worse than best effort", because  
any application who's worse-than-best effort can always use a  
friendlier than TCP congestion control ( http://www.icir.org/floyd/tcp_small.html 
  has a list), have the same effect on the network, and suddenly go  
unnoticed in moments of congestion without requiring network support  
for QoS to be enabled.

Given the problems users are experiencing with BitTorrent just on  
their own network ( http://www.formortals.com/Home/tabid/36/EntryID/57/Default.aspx 
  , http://tech.slashdot.org/article.pl?sid=08/05/24/1859218 ), there  
actually seems to be some incentive in place for the bulk-data P2P  
developers to use one of the existing off-the-shelf solutions for  
friendlier-than-TCP behavior (note, this is something I was WRONG on  
earlier, as I thought there was no incentive), and if they don't have  
incentive to use friendlier-than-TCP, why would they have incentive to  
mark as 'not time sensitive', as they would accomplish the same  
thing.  [1]

However there is a benefit to saying "I'm better than best effort",  
for applications like VoIP or online games, or even the non-image  
portions of your HTTP requests.  ISPs can spend a lot of effort using  
packet analysis to determine these flows, or just rely on the  
endpoints to get it right.

There is no benefit to cheating, IF the network enforces a cap on the  
number of (packets/bytes) sent with priority per (ms/sec/minute/hour),  
as the measurement is relatively easy to enforce (1 memory lookup/ 
update per packet, and because its only high-volume users which have  
an incentive to cheat, you can easily use a small LRU cache for  
addresses).

[1] It might be the role of the OS to implement some of this.  EG,  
coupling AIMD across all flows of an application, combined with some  
more detailed RTT-based congestion control a'la TCP Vegas, again  
correlating across multiple flows)

---

From: Salman Abdul Baset <[email-masked]>
Date: Mon, 2 Jun 2008 12:50:41 -0400 (EDT)

> - Expose more information about TCP streams. For example, expose whether 
> the stream is congested. Currently some p2p applications use a variety 
> of techniques to attempt to guess whether a user's internet connection 
> is congested (e.g. monitoring ping times), and measures stream 
> throughput which could be viewed as a very rough measure of congestion, 
> but it would be more accurate and efficient if TCP told the application 
> when a stream was congested. Even better would be to get some indication 
> of where the congestion occurred. For example, if the congestion is in 
> the 'last mile', or the other peer's 'last mile', or somewhere in 
> between, different responses would be appropriate. For example, if my 
> internet connection is fine, but the peer's connection is congested, I 
> should slow down transfers from that peer, and pull more from other 
> peers. This is, of course, easier said than done.

It is possible to query the congestion state of a TCP connection in Linux 
using getsockopt() and tcp_info structure. It gives an indication whether 
a connection is in timeout, fast retransmit, fast recovery or congestion 
avoidance. It is also possible to query the TCP transmit and receive 
queue size (in addition to buffer size). I think Windows Vista implements 
a similar mechanism but not sure about it.

> - Expose some measure of 'communications cost' so that p2p apps could 
> select the lowest cost available data sources. (This is what the P4P 
> protocol does, for example). If using this guidance benefits both P2Ps 
> and ISPs, it is a win-win. This is a fairly broad area that might 
> include a number of steps/messages/protocols in order to address service 
> location, mapping IPs to location, and "cost". The P4PWG is researching 
> this area, and while it feels a little early to propose a standard, it's 
> certainly an area (IMO) that could benefit from attention.

Both short-term and long term solutions may be needed here. Mechanisms for 
delivering this cost need to be different than the ways in which cost can 
be specified. Solution spectrum ranges from passing the complete BGP state 
to end nodes to end-host driven computations for communication costs.

> - ISP's could expose usage metrics. For example, if a user has a link 
> that is capable of 1 Mbps, has a monthly transfer capacity of 500 GB, 
> and has used 350 GB so far, then the application can make better 
> choices, and application users could make informed choices. If the ISP 
> could tell applications when the user's internet connection is 
> congested, that would be extremely helpful.

Definitely. Perhaps, need to borrow ideas from the mobile networks/apps 
such as 'view my minutes' option or allow a user to see the usage 
statistics (iPhone).

-s

---

From: Joe Touch <[email-masked]>
Date: Mon, 02 Jun 2008 10:37:25 -0700

Laird Popkin wrote:
> - Expose more information about TCP streams. For example, expose
> whether the stream is congested. Currently some p2p applications use a
> variety of techniques to attempt to guess whether a user's internet
> connection is congested (e.g. monitoring ping times), and measures
> stream throughput which could be viewed as a very rough measure of
> congestion, but it would be more accurate and efficient if TCP told the
> application when a stream was congested. 

As others have posted, this information should be accessible.

> Even better would be to get
> some indication of where the congestion occurred. For example, if the
> congestion is in the 'last mile', or the other peer's 'last mile', or
> somewhere in between, different responses would be appropriate. For
> example, if my internet connection is fine, but the peer's connection is
> congested, I should slow down transfers from that peer, and pull more
> from other peers. This is, of course, easier said than done.

This is not feasible for a number of reasons, including the use of 
tunnels and VPNs. It's possible to know whether the end systems are 
loaded (applications can tell each other), but path limits are hard to 
pin down. It's more useful to just try a few peers and see what works 
than to assume this kind of information is (or ever will be) available.

Joe

---

From: Nicholas Weaver <[email-masked]>
Date: Mon, 2 Jun 2008 10:51:12 -0700

On Jun 2, 2008, at 10:37 AM, Joe Touch wrote:
> loaded (applications can tell each other), but path limits are hard  
> to pin down. It's more useful to just try a few peers and see what  
> works than to assume this kind of information is (or ever will be)  
> available.

Bulk data P2P can actually tell this if the TCP access API is  
suitable, by looking for correlations.  If its just one flow that is  
seeing congestion, assume it is the other side.  If multiple flows are  
seeing congestion, then the problem probably is at your end.

---

From: Laird Popkin <[email-masked]>
Date: Mon, 2 Jun 2008 13:58:23 -0400 (EDT)

I certainly understand why it's somewhere between hard and impossible to give precise congestion status in all cases, due to VPN's, etc.. That being said, even getting less precise guidance, and only in the more common cases, would help us make better decisions for large numbers of users. For example, being told when we're causing congestion of home consumer internet connections with single-NAT networks would help us "play nicely" for a large number of users.

---

From: Joe Touch <[email-masked]>
Date: Mon, 02 Jun 2008 12:50:01 -0700

Nicholas Weaver wrote:
> Bulk data P2P can actually tell this if the TCP access API is suitable, 
> by looking for correlations.  If its just one flow that is seeing 
> congestion, assume it is the other side.  If multiple flows are seeing 
> congestion, then the problem probably is at your end.

It depends on where the multiple flows go; they could as easily all 
experience congestion over a tunnel/VPN, even if they diverge and go to 
different destinations. A single connection that experiences congestion 
(out of multiples) does so where the connections diverge, but again that 
could be anywhere.

It would be useful not to over-tune P2P based on these sort of 
assumptions, as a result. The best you can do is split things as follows:

Congestion in the uphill path (you to the core):
	- all connections more likley to experience similar congestion

Congestion on the downhill path (core to the servers):
	- connections more likely to diverge

Other inferences are, AFAICT, assuming too much.

Joe

---

From: Enrico Marocco <[email-masked]>
Date: Tue, 3 Jun 2008 00:15:13 +0200

Laird Popkin wrote:
> As we discussed last week at the IETF P2Pi Workshop, here are a list of
> some of the ways that IETF standards can potentially help P2P
> applications make better network decisions, and thus help ISPs. This
> isn't a complete list, but I hope that it's at least a useful starting
> point for a discussion.

It seems to me that this turned out to be a two-sided problem.  On the
one hand, it is clear that applications on the endpoints are in the
worst position to make optimal network decisions; a first part of the
solution should thus consist of a mechanism to defer such decisions to
external entities (not necessarily controlled by ISPs, as e.g. in the
case of Ono, where the redirection service is provided by Akamai
servers). The need for such a mechanism was claimed by several papers
[4,26,28,29] and, AFAIR, received little objection (the one that I
recall was the "hot potato" thing, where the external entity providing
the peer selection service would be able to place sort of DDOS attacks
against specific targets/networks).

On the other hand, how to compute best paths is a separate part of the
problem; topology/cost/congestion notification, cache identification,
usage metrics, and the like all fall in this category and, at this time,
are probably more research topics than engineering issues.

Right now, I think the IETF should focus on the first part of the
problem, the front-end of a generic third-party peer selection service
p2p applications can query in order to make better choices; innovation
and competition will then happen on the back-end of such a service
without requiring standardization in the first place.

[*] References at
http://www.ietf.org/mail-archive/web/p2pi/current/msg00005.html

-- 
Ciao,
Enrico

---

From: Ted Hardie <[email-masked]>
Date: Mon, 2 Jun 2008 16:06:03 -0700

At 3:15 PM -0700 6/2/08, Enrico Marocco wrote:
>external entities

Of course you can recast this  as "So we should make sure that
there are mechanisms that allow them to make more optimal decisions.
The facilities which provide this information need not be controlled
by the ISPs, but might be third party services which indicate
cost/topology/congestion or some profile mix of the three."

To my way of thinking, that makes it easier to consider the API
design, as it is not so much an API that says "What's the best
choice right now?"  but "What's the best X (flow pattern, topology,
price)".  The second set of questions is sufficiently more modular
that it is both easier to get right and, at least in my opinion,
more interesting.

Just  my two cents,
				Ted

>(not necessarily controlled by ISPs, as e.g. in the
>Content-Type: application/x-pkcs7-signature; name="smime.p7s"
>Content-Description: S/MIME Cryptographic Signature.p7s
>Content-Disposition: attachment; filename="smime.p7s"; size=3504;
>	creation-date="Mon, 02 Jun 2008 15:26:25 GMT";
>	modification-date="Mon, 02 Jun 2008 15:26:25 GMT"
>
>Attachment converted: Macintosh HD:smime 1480.p7s (    /    ) (007B339F)
>Content-Type: text/plain; name="ATT00001.txt"
>Content-Description: ATT00001.txt
>Content-Disposition: attachment; filename="ATT00001.txt"; size=191;
>	creation-date="Mon, 02 Jun 2008 15:26:25 GMT";
>	modification-date="Mon, 02 Jun 2008 15:26:25 GMT"
>
>Attachment converted: Macintosh HD:ATT00001 1846.txt (TEXT/ttxt) (007B33A0)

---

From: Song Haibin <[email-masked]>
Date: Tue, 03 Jun 2008 14:18:51 +0800

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On Behalf Of Ted
>Hardie
>Sent: Tuesday, June 03, 2008 7:06 AM
>To: Enrico Marocco; Laird Popkin
>Cc: [email-masked]; [email-masked]; [email-masked]

I think it depends on whether the P2P application or the party that provides
peer selection information service makes the final choice. IMO, the peer
selection information service should provide information to help the p2p
application to determine best paths or peers, but not itself makes the final
choice.

-Song Haibin

---

From: Joe Touch <[email-masked]>
Date: Mon, 02 Jun 2008 23:30:53 -0700

>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Ted
...
> application to determine best paths or peers, but not itself makes the final

FWIW, there is a lot of work in peer selection in web cache hierarchies 
that may be applicable here. Keith Moore's work (SONAR) comes to mind, 
but there are others.

Come to think of it, there are a lot of analogies between P2P - and P4P 
- and web cache hierarchies. Has this been explored??

Joe

---

From: Enrico Marocco <[email-masked]>
Date: Tue, 3 Jun 2008 10:38:07 +0200

> cost/topology/congestion or some profile mix of the three."

Existing systems today used (or proposed) for optimizing peer selection
use different methods to make their choices.  While it would be easy to
define a common interface such systems could expose to p2p applications,
it would be probably hard to standardize cost/topology/congestion/policy
notification mechanisms in order to accommodate all different approaches
adopted by existing and foreseeable solutions.

Examples of existing systems are Vivaldi and Ono, embedded in 10^5~10^6
Azureus clients out there, P4P, iPlane, PIC and many others, simulated
and emulated in friendly environments.

> that it is both easier to get right and, at least in my opinion,

Well, I'd probably prefer a compromise between the two, something like
"What's the best choice for downloading X Mb of delay-sensitive data?".
And, just to be clear, available options must be provided by the
application which, at the end of the day, will be also in charge of
making the final decision (including ignoring the answer if for some
reason it doesn't like it).

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Wed, 04 Jun 2008 11:39:11 -0500

> - and web cache hierarchies. Has this been explored??

Not to toot our horn here, but our position paper [1] provided
what we think is a good literature review on application layer
topology estimation -- from synthetic coordinate systems like
Vivaldi, PIC, and GNP to the more recent work on Ono, which uses
Akamai CDN servers to fine tune peer selection.

By and large, the conclusions we arrived at following the
literature review were summarized aptly by Enrico in an earlier
email:

   I think the IETF should focus on the first part of the
   problem, the front-end of a generic third-party peer selection
   service p2p applications can query in order to make better
   choices; innovation and competition will then happen on the
   back-end of such a service without requiring standardization
   in the first place.

[1] Enrico Marocco, Vijay K. Gurbani, Volker Hilt, Ivica Rimac,
  Marco Tomsu - Peer-to-Peer Infrastructure: A Survey of Research on
  the Application-Layer Traffic Optimization Problem and the Need for
  Layer Cooperation

Thanks,

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
2701 Lucent Lane, Rm. 9F-546, Lisle, Illinois 60532 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
WWW:   http://www.alcatel-lucent.com/bell-labs
_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: "Wei Gengyu" <[email-masked]>
Date: Thu, 5 Jun 2008 10:49:41 +0800

Hi, Vijay,

I have a question.

As your paper is about "Application-Layer Traffic Optimization Problem" and 
CDN service,
does it mean that different appoaches are required for P2P file share, P2P 
voice, and P2P video applications.

Regards,

Gengyu
School of Computer Science and Technology
Beijing University of Posts and Telecommunications

---

From: Enrico Marocco <[email-masked]>
Date: Thu, 5 Jun 2008 09:00:14 +0200

> voice, and P2P video applications.

Actually our goal was to focus the survey on solutions applicable to
file-sharing, because that is the main cause of congestion in our
networks.  However, the same solution would be also beneficial to other
applications; we tried to list some in section 3 of the problem
statement (http://tools.ietf.org/id/*-alto-problem-statement), but we
certainly have missed many.  Needless to say, contributions and
suggestions are welcome.

---

From: stefano previdi <[email-masked]>
Date: Mon, 09 Jun 2008 16:47:22 +0200

> From: Enrico Marocco <[email-masked]>
> 
> Ted Hardie wrote:

I agree with the addition that ietf could work on standardization of
extensions/enhancements of existing routing protocols that could be
leveraged by any localization/proximity implementation.

I know at least one implementation that goes towards that direction...

s.