Subject: RE: [BEHAVE] Re: removing application milestone

From: "Dan Wing" <[email-masked]>
Date: Mon, 16 Jul 2007 12:05:22 -0700

For several of those NAT types, the SDP answer needs to be received before
the UDP hole-punching is successful.  It's important to also know the (SIP?)
signaling delay through your rendezvous servers (SIP prox(y|ies)), between
the two RTP peers?  For example, if it takes 250ms for the SDP offer to get
to the called party, and another 250ms for their answer to get back there is
built-in overhead of 500ms due to the signaling system -- ignoring all the
STUN candidate gathering and ignoring all of the ICE connectivity checks.

-d

> -----Original Message-----
> From: David Barrett [mailto:[email-masked]] 
> Sent: Monday, July 16, 2007 11:44 AM
> To: 'Henry Sinnreich'; [email-masked]
> Cc: [email-masked]; [email-masked]; 'Dan Wing'; [email-masked]
> Subject: RE: [BEHAVE] Re: removing application milestone
> 
> > -----Original Message-----
> > From: Henry Sinnreich [mailto:[email-masked]]
> > Subject: RE: [BEHAVE] Re: removing application milestone
> > 
> > Given that ICE has now gained the status of a creed and maybe I have
> > missed it: Are there any published results on the 
> effectiveness of ICE,
> > such as % of success in various NAT scenarios and call 
> setup delay when
> > used with SIP?
> > I have seen one private report and it does not look that great.
> 
> Incidentally, I'm dealing with measuring peer connectivity of 
> my own system
> (non-ICE), and here are some of the results for various NAT types:
> 
> Type        Count       Connectivity Open Time
> Fullcone    150 (9%)    92%          1056ms
> Fullstatic  397 (25%)   94%          867ms
> Fw          68  (4%)    2%           374ms
> Rstrcone    598 (37%)   89%          706ms
> Rstrstatic  82 ( 5%)    91%          769ms
> symmetric	293 (18%)   94%          906ms
> 
> This is read as "598 tested clients had a restricted cone NAT 
> (making up 37%
> of all clients tested), with an average peer connectivity 
> success rate of
> 89%".  ("FullStatic" and "Rstrstatic" means a non-NAT'd 
> client with endpoint
> independent/dependent filtering, respectively; "Fw" means a 
> client that
> failed our STUN operation and thus might just have UDP 
> blocked altogether;
> "Open Time" is the time it took to establish the connection and begin
> sending actual connection data.)  
> 
>            fullcone fullstatic fw      rstrcone rstrstatic symmetric
> fullcone   98%      96%        0%      96%      94%        98%    
> fullstatic 97%      97%        5%      97%      90%        98%    
> fw         0%       5%         0%      0%       0%         0%    
> rstrcone   96%      97%        0%      16%      66%        27%    
> rstrstatic 94%      92%        0%      66%      100%    
> symmetric  98%      98%        0%      12%      100%    
> 
> Here are the pairwise peer connectivities.  As you can see, 
> this is a pretty
> small test (only maybe 1000 total clients), an thus not all pairwise
> scenarios got tested (ie, no rstrstatic:symmetric or 
> symmetric:symmetric
> tests were attempted).
> 
> Thus I wouldn't put too much faith in those numbers 
> (especially anything
> claiming 100%) until I have numbers on a broader scale.  For 
> some reason I'm
> seeing peer connectivity problems at larger scales (average peer
> connectivity across the board is 90% in the above case, but 
> drops to 40%
> when tested on 10x the clients), and I'm still ironing them out (there
> appears to be a problem in my test infrastructure).
> 
> Basically, I'd love to see some results on ICE in the above format, as
> that'll let me do an apples-to-apples comparison of it versus mine.
> 
> -david

_______________________________________________
Behave mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/behave

---

From: "David Barrett" <[email-masked]>
Date: Mon, 16 Jul 2007 12:24:51 -0700

To be clear, I'm not using SIP (or ICE, or standard STUN).  So, I don't have
the SDP exchange step -- can you remind me why that is necessary?

The connection-establishment protocol in my architecture is:

0) Clients centrally register their addresses on startup
...
1) Client A asks the server how to contact B
2) The server returns B's address to A
3) Simultaneously, the server instructs B to connect to A
4) A and B connect to each other at the same time over all routes
5) The connection is established once a single route is identified
6) If a faster route is identified, it switches to that

I don't have a detailed breakdown of the latencies of each step (though that
would be interesting).

-david

> From: Dan Wing [mailto:[email-masked]]
> Sent: Monday, July 16, 2007 12:05 PM
> To: 'David Barrett'; 'Henry Sinnreich'; [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Tue, 17 Jul 2007 09:44:15 +0300

On Monday 16 July 2007 22:24:51 ext David Barrett wrote:
> To be clear, I'm not using SIP (or ICE, or standard STUN).  So, I don't
> have the SDP exchange step -- can you remind me why that is necessary?
>

I expect such a design increase ease of implementation a lot at the cost of 
success rate and battery lifetime. With ICE, the candidates may (it's up to 
the implementation) be generated right before sending the invitation.

When using centralized registration of your binding, there is a much higher 
risk that bindings on the middleboxes have vanished (reboot?) or changed 
between (I am not willing to guesstimate how high is higher) registration and 
use.

Also, it supposedly means you have to keep refreshing the binding, which is a 
well-known battery killing process for mobile devices. In SIP+ICE case, I 
figure you'd merely keep a TCP connection to your SIP proxy, which 
(*according to research*) requires much fewer keep-alives packet than UDP.

Nevertheless, your approach might be better suited for peer-to-peer 
server-poor environments where even signaling (proxies) might be expensive.

-- 
Rémi Denis-Courmont

---

From: "David Barrett" <[email-masked]>
Date: Tue, 17 Jul 2007 00:56:59 -0700

If anybody could provide *any* real world data on *any* ICE implementation,
of *any* draft version, used at *any* scale, that'd be really helpful.  It's
really hard to do a real world analysis against something that only exists
in theory.  (A friend of mine once said "Everything works in theory.  I want
to move there someday.")  But I'm happy to try.  

Re: Peer connectivity
The system I show achieves real-world peer connectivity over 90%.  I'm open
to ICE achieving greater than that, but I don't see how that would be the
case.  Are you saying ICE will try additional routes, or that the routes it
tries will somehow be more likely to work?  (I should mention that peers in
my system keep their bindings alive; when they die, new bindings are
discovered and the server is updated.)

Re: Battery life
This isn't a concern to me, but I'd love to see the data on this.

Re: Server-poor
I didn't follow the last bit -- I'm not sure what a "peer-to-peer
server-poor environment" is.  This is designed for and proven in the real
world internet at a very large scale around the globe.  Are you suggesting
that somehow ICE is better adapted for reality than this system, or that
this system is somehow better suited for some other environment than the one
it's currently being used in?

-david

> From: Rémi Denis-Courmont [mailto:[email-masked]]
> Sent: Monday, July 16, 2007 11:44 PM
> To: [email-masked]
> Subject: Re: [BEHAVE] Re: removing application milestone
>