Subject: [P2PSIP] supporting large messages in reload

From: "Bruce Lowekamp" <[email-masked]>
Date: Tue, 4 Nov 2008 15:57:03 -0500

Reload supports large messages, but it is missing a number of
components required to make this work.  In particular, there is no
end-to-end congestion control, and a large message could essentially
block a link between two peers long enough to force other messages
(and the large message itself) to time out.

Solving this properly is likely to be quite complicated.  Since our
primary usage requires only the exchange of URIs and certificates, the
authors would like to avoid adding the complexity necessary to handle
arbitrarily large messages to the base protocol.  We propose the
following:

the base protocol will support:
- an overlay-configured (policy) maximum message size for non-direct messages
 * an error when a peer attempts to relay a message larger than this
 * a response code of some sort indicating that the reply would be
larger than this size, so a direct connection is required
- will not specify what the max msg size is, but will suggest that
since we have an all-or-nothing fragment approach, that the ideal
message size is not large.
- continue to support ice & turn (+ tcp variants)

the base protocol will not support, but will be designed to be
extensible to include:
- end-to-end reliable, congestion-controlled transport
- traffic prioritization via sctp/multiple tcp's to prioritize overlay
control and maintenance over bulk data transfer
- such extensibility support will likely include at least enough of a
sketch of how these mechanisms should work that we can be sure the
basic headers will be compatible

So goal is that the base protocol will be designed to require a direct
connection for the exchange of large messages and will be extensible
enough to support more complete solutions to this problem in future
extensions.

Even with these changes, there are some more elements of the base
protocol that need to be cleared up, such as timeouts for large
messages (even sent directly) and message fragmentation, but I think
this captures the general intention of the changes.

Bruce
_______________________________________________
P2PSIP mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2psip

---

From: Henning Schulzrinne <[email-masked]>
Date: Sat, 8 Nov 2008 12:02:24 -0500

> (and the large message itself) to time out.

This assumes that there is only a single "link" between nodes.

> primary usage requires only the exchange of URIs and certificates, the

This isn't quite true. Even for the SIP usage, we'll have to worry  
about voicemail messages, for example.

> message size is not large.

I don't think that's helpful. For a relatively low-usage overlay or an  
overlay running in a single data center or LAN, blocking won't matter  
even with a single logical link, so I see no reason that an  
implementation (rather than an instance) restrict the message size.

I think it's important to distinguish between implementation limits  
(which should be as close to the maximum given by length fields) and  
the policy enforced for a specific overlay instance.

> control and maintenance over bulk data transfer

It should support the use of multiple links between nodes. I don't see  
any great difficulty in allowing an implementation to open multiple  
connections to another node. Whether to actually do this is then a  
policy decision.

> this captures the general intention of the changes.

Timeouts are going to be challenging no matter what. Even a small  
message may take several seconds if links are crappy or the overlay is  
large. (Somebody mentioned that search times in existing overlays,  
such as Azereus, can routinely exceed 10 seconds and call setup  
latencies in Skype are pretty long.)

I suspect we'll need to recommend a more-or-less random value and high- 
performance implementations will need to implement algorithms to tune  
the timeout based on the measured properties of the overlay.  
Unfortunately, time outs will matter mostly in less-than-ideal  
networks, rather than a data center LAN.

---

From: jiangxingfeng <[email-masked]>
Date: Mon, 10 Nov 2008 10:05:43 +0800

> networks, rather than a data center LAN.
I agree. Maybe let the peer do some diagnostic actions to gather the measured data of the overlay and based on the data, the timeout value could be adjusted. For example, a peer could use PING or other messages to calculate the RTT for a request, even there is a late response for the request, i,e, the response could not find the corresponding request transactions, because timer fires and transaction are destroyed.

 By maintaining the maximum RTT may be helpful to choose a reasonable timeout. However, in some cases, the timeout value will be limited by the upper layer application, for instance, the user hope to get the information from the overlay within a fixed time.

---

From: "Bruce Lowekamp" <[email-masked]>
Date: Sun, 9 Nov 2008 21:07:44 -0500

On Sat, Nov 8, 2008 at 12:02 PM, Henning Schulzrinne
<[email-masked]> wrote:
> This isn't quite true. Even for the SIP usage, we'll have to worry about
> voicemail messages, for example.

The SIP usage does not contain the word "voicemail."  While I would
love to see a draft proposing how to manage voicemail, there is not
one at present.  There are certainly some deployments that would be
incomplete without a way of storing voicemail in the overlay, but
there are also lots that have other ways to handle voicemail.  I don't
know if it's even true that a majority of deployments is likely to
need or want voicemail in the overlay.

> overlay running in a single data center or LAN, blocking won't matter even
> with a single logical link, so I see no reason that an implementation
> (rather than an instance) restrict the message size.
>
> I think it's important to distinguish between implementation limits (which
> should be as close to the maximum given by length fields) and the policy
> enforced for a specific overlay instance.

I'm unclear what you're objecting to.

> It should support the use of multiple links between nodes. I don't see any
> great difficulty in allowing an implementation to open multiple connections
> to another node. Whether to actually do this is then a policy decision.
>

First, I think the udp link protocol will have problems well before
that.  Second, for stream-oriented transports, I'm not sure that the
draft currently prohibits multiple connections between peers.  I'm not
convinced it has enough support to make it work (and I would really
not want to complicate it to add this feature), but I don't see why it
wouldn't, offhand.

> Timeouts are going to be challenging no matter what. Even a small message
> may take several seconds if links are crappy or the overlay is large.
> (Somebody mentioned that search times in existing overlays, such as Azereus,
> can routinely exceed 10 seconds and call setup latencies in Skype are pretty
> long.)
>
> I suspect we'll need to recommend a more-or-less random value and
> high-performance implementations will need to implement algorithms to tune
> the timeout based on the measured properties of the overlay. Unfortunately,
> time outs will matter mostly in less-than-ideal networks, rather than a data
> center LAN.
>

I agree.  I'm not convinced that the hard-coded 3 second timeout in
5.1.1 is what we want regardless of the large message issue.

Bruce

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 09 Nov 2008 18:50:30 -0800

At Sun, 9 Nov 2008 21:07:44 -0500,
> need or want voicemail in the overlay.

This isn't an accident, either, I don't think.

My memory of the early WG and pre-WG discussions was that voicemail
was going to be out of scope. Indeed the charter seems pretty clear on
this point:

  The work focuses on collections of nodes called "P2PSIP peers" and
  "P2PSIP clients". P2PSIP peers manifest a distributed namespace in
  which overlay users are identified and provides mechanisms for
  locating users or resources within the P2PSIP overlay. P2PSIP clients
  differ from P2PSIP peers primarily in that they do not store
  information in the overlay, but only use it to locate users and
  resources. P2PSIP clients and peers use the resolution services of the
  peers as an alternative to the SIP discovery process of RFC 3263. In
  this way, P2PSIP offers an alternative mechanism for determining the
  correct destination for SIP requests. The working group's initial
  charter scope will be to produce protocols to enable this alternate
  mechanism for RFC 3263 functionality. Session management, messaging,
  and presence functions are performed using conventional SIP.

Speaking only for myself, I've been deliberately 
avoiding thinking about the voice mail problem based on the
above understanding.

-Ekr

---

From: Henning Schulzrinne <[email-masked]>
Date: Sun, 9 Nov 2008 23:14:26 -0500

The e2e timeout serves two purposes, at least:

- Retrying if the request or response got blackholed somewhere, hoping  
that the second attempt uses a different set of nodes (e.g., after the  
overlay has stabilized after churn) and is luckier;

- Giving up when a response is unlikely, so that the calling  
application doesn't hang forever.

The latter is more of a local API design issue, the former requires  
specification advice, to avoid too many spurious retransmissions.

The ping can help, but high variance of the response time is likely in  
"challenged" networks, so it may not be all *that* helpful. I don't  
think we need to specify the precise algorithm, as this seems like a  
good opportunity for implementor cleverness and deployment experience.

I suspect that trying to make the per-hop behavior as reliable as  
possible and providing error feedback when a request can't be routed  
further for some reason (e.g., a node finds that none of its next-hop  
links are working due to a NAT issue) is likely to help more than  
elaborate timer guessing.

Henning
(sitting on a hotel wireless LAN that went from almost useless to  
lossless within a minute)

>>
>>
> I agree. Maybe let the peer do some diagnostic actions to gather the  
> measured data of the overlay and based on the data, the timeout  
> value could be adjusted. For example, a peer could use PING or other  
> messages to calculate the RTT for a request, even there is a late  
> response for the request, i,e, the response could not find the  
> corresponding request transactions, because timer fires and  
> transaction are destroyed.
>
> By maintaining the maximum RTT may be helpful to choose a reasonable  
> timeout. However, in some cases, the timeout value will be limited  
> by the upper layer application, for instance, the user hope to get  
> the information from the overlay within a fixed time.
>
>
>
>