Subject: Re: [alto] Cost Maps, Endpoint Cost Maps and Coarse Grained Searches

From: Wendy Roome <[email-masked]>
Date: Tue, 15 Sep 2015 10:08:54 -0400

Lyle,

Here is my opinion on Endpoint Cost Service (ECS) vs. Network Maps & Cost
Maps (NM/CM): They are two different ways to present the same data. They
are different viewports into the same data, if you will, with different
levels of detail, and different performance and ease-of-use.

To start, I believe most clients will prefer ECS. Clients really care
about costs between endpoints; PIDs and Network Maps are irrelevant. For
example, consider a Bittorrent peer which gets 50 new peers every 15
minutes or so. That ALTO client would prefer to send an ECS request to the
ALTO server to rank those peers, rather than fetching full maps,
converting endpoints to PIDs and looking up the costs between PIDs.

For a client, ECS is simple & easy to use: the client does not have to
maintain any maps, and the client gets the latest cost data.

Now consider a busy Bittorrent tracker. It gets 10+ requests/second, and
for each request, it must select 50 out of (say) 5,000 peers. If the
tracker used ECS, it would have to send 10 ECS requests a second, each
with 5,000 endpoints. That would add too much latency to the tracker
response, and would overload the ALTO server.

So a tracker client would prefer to download the full NM/CM and evaluate
costs locally, without involving the ALTO server each time. That requires
more programming effort, but is more efficient.

Here is another piece of historical motivation that is not clear in the
RFC: we expect ALTO Servers will be offered by network service providers.
Some providers will be (understandably!) reluctant to publish a detailed
breakdown of their internal network. The NM/CM model allows a service
provider to control the level of detail it wants to make available to the
outside world.

Incidentally, the RFC says an ALTO Server MUST provide an NM/CM, while ECS
is optional. Personally, I think that is a mistake. I think NM/CM should
be optional, and ECS should be required.

So if you are writing a server, I recommend you build an internal cost
map, with as high a level of detail as practical. Offer an ECS that uses
that underlying detailed map. Then to satisfy the RFC, use some form of
cluster analysis to partition the endpoints in the underlying map into
(say) 100 to 200 PIDs, and offer a Network Map & Cost Map based on that
partitioning.

I hope that helps.

	- Wendy Roome

>Date: Tue, 8 Sep 2015 20:53:37 -0500
>From: Lyle Bertz <[email-masked]>
>To: [email-masked]
>Subject: [alto] Cost Maps, Endpoint Cost Maps and Coarse Grained
>	Searches
>Message-ID:
>	<CAC5bAiYH5N2tQpr3CEZn+eO8C9kYgWttyqNrwKKe=[email-masked]>
>Content-Type: text/plain; charset="utf-8"
>
>All,
>
>I am implementing an Erlang based ALTO server and had a couple of
>questions
>based upon an observation of 7285.
>
>The Cost Map is assumed to be coarse grained and one cannot make a
>determination about whether an endpoint cost measure is fine or coarse per
>the RFC.
>
>If i am to search for a cost between two endpoints (1 source and 1
>destination) and 'miss' on the first endpoint map I am looking at the
>other
>endpoint costs responses I may have available for an answer.  In such a
>case I can just look for the two endpoints and, if present, I have a hit
>and I am good to go.
>
>However, if I am looking to Cost Maps the map dependency assumes that both
>endpoints are members of the same map.   This implies that only endpoint
>cost maps can contain metrics between two endpoints that are not in the
>same map.   I find this interesting in that as a designer if you want all
>data in Network Cost Maps you have to model the entire internet or you can
>just rely on endpoint cost maps.
>
>What was the intent in this relationship?  I like the open ended option
>the
>endpoint cost maps provide but I am a bit reluctant to begin coding
>something that may have not been an intentional feature in ALTO.
>
>Thanks.
>
>Lyle
>
>PS - Code for Erlang ALTO server (very Alpha) can be found at
>https://github.com/lylebe/e__alto
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: 
><https://mailarchive.ietf.org/arch/browse/alto/attachments/20150908/d08924
>af/attachment.html>

---

From: "Y. Richard Yang" <[email-masked]>
Date: Tue, 15 Sep 2015 17:04:45 -0400

Wendy, Lyle,

On Tue, Sep 15, 2015 at 10:08 AM, Wendy Roome <[email-masked]>
wrote:

> To start, I believe most clients will prefer ECS.

 I agree that most (simple) clients prefer ECS. One may consider NM/CM as a
batch/summary query for a set of ECS queries.

Clients really care
> be optional, and ECS should be required.
There have been very strong supporters on ECS from the very beginning. In
retrospect, it will indeed help if we make ECS required.

Richard

> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lylebe_e-5F-5Falto&d=AwICAg&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=4G36iiEVb2m_v-0RnP2gx9KZJjYQgfvrOCE3789JGIA&m=bsmcTVcdu8qOJFBpCCyYNy8FyoE7j1RI0_OP-y3fGDg&s=R5Ss1SQJ0WXr0U5o1CYjG-Uq4QVHmG2HLXUq1hpRawQ&e=
> ><
> https://urldefense.proofpoint.com/v2/url?u=https-3A__mailarchive.ietf.org_arch_browse_alto_attachments_20150908_d08924-0A&d=AwICAg&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=4G36iiEVb2m_v-0RnP2gx9KZJjYQgfvrOCE3789JGIA&m=bsmcTVcdu8qOJFBpCCyYNy8FyoE7j1RI0_OP-y3fGDg&s=Lqfyuj4wkDXxJZKF9RdJ9O9tZdOKDck31prF5OPt9tQ&e=
> >af/attachment.html>
>
>
> _______________________________________________
> alto mailing list
> [email-masked]
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_alto&d=AwICAg&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=4G36iiEVb2m_v-0RnP2gx9KZJjYQgfvrOCE3789JGIA&m=bsmcTVcdu8qOJFBpCCyYNy8FyoE7j1RI0_OP-y3fGDg&s=-xMi5SISranYqJCfRPuFeb4DqNkbGfjkUNVLyh3aS7o&e=
>

---

From: Lyle Bertz <[email-masked]>
Date: Tue, 15 Sep 2015 18:12:49 -0500

Wendy,

This is quite helpful - thank you.

I also agree with your assessment,  IMO, I am most interested in ECS and
even assume there will be multiple Clients that optimize using internal
ALTO information while exporting public versions of their map - especially
for security concerns.

Lyle

On Tue, Sep 15, 2015 at 9:08 AM, Wendy Roome <[email-masked]>
wrote:

> https://mailarchive.ietf.org/arch/browse/alto/attachments/20150908/d08924
> >af/attachment.html>
>
>
> https://www.ietf.org/mailman/listinfo/alto
>

---

From: Chenguohai <[email-masked]>
Date: Wed, 16 Sep 2015 01:06:35 +0000

Hi Wendy and Lyle,

See below for comments.

BR
G.Robert Chen
----------------------------------------------------------------------------------------------------------------------------------------

>>>> -----Original Message-----
>>>> From: alto [mailto:[email-masked]] On Behalf Of Wendy Roome
>>>> Sent: Tuesday, September 15, 2015 10:09 PM
>>>> To: [email-masked]
>>>> Subject: Re: [alto] Cost Maps, Endpoint Cost Maps and Coarse Grained
>>>> Searches
>>>> 
>>>> Lyle,
>>>> 
>>>> are different viewports into the same data, if you will, with different levels of
>>>> detail, and different performance and ease-of-use.
>>>> 
>>>> To start, I believe most clients will prefer ECS. Clients really care about costs
>>>> between endpoints; PIDs and Network Maps are irrelevant. For example,
>>>> consider a Bittorrent peer which gets 50 new peers every 15 minutes or so.
>>>> That ALTO client would prefer to send an ECS request to the ALTO server to
>>>> rank those peers, rather than fetching full maps, converting endpoints to PIDs
>>>> and looking up the costs between PIDs.
>>>> 
>>>> For a client, ECS is simple & easy to use: the client does not have to maintain
>>>> any maps, and the client gets the latest cost data.
>>>> 

I agree on this.

>>>> Now consider a busy Bittorrent tracker. It gets 10+ requests/second, and for
>>>> each request, it must select 50 out of (say) 5,000 peers. If the tracker used ECS,
>>>> it would have to send 10 ECS requests a second, each with 5,000 endpoints.
>>>> That would add too much latency to the tracker response, and would
>>>> overload the ALTO server.
>>>> 
>>>> costs locally, without involving the ALTO server each time. That requires more
>>>> programming effort, but is more efficient.
>>>> 
>>>> provider to control the level of detail it wants to make available to the outside
>>>> world.
>>>> 
>>>> is optional. Personally, I think that is a mistake. I think NM/CM should be
>>>> optional, and ECS should be required.

I agree on this

>>>> So if you are writing a server, I recommend you build an internal cost map,
>>>> with as high a level of detail as practical. Offer an ECS that uses that
>>>> underlying detailed map. Then to satisfy the RFC, use some form of cluster
>>>> analysis to partition the endpoints in the underlying map into
>>>> >questions based upon an observation of 7285.
>>>> >
>>>> >determination about whether an endpoint cost measure is fine or coarse
>>>> >per the RFC.
>>>> >
>>>> >other endpoint costs responses I may have available for an answer.  In
>>>> >such a case I can just look for the two endpoints and, if present, I
>>>> >have a hit and I am good to go.
>>>> >
>>>> >data in Network Cost Maps you have to model the entire internet or you
>>>> >can just rely on endpoint cost maps.
>>>> >
>>>> >the endpoint cost maps provide but I am a bit reluctant to begin coding
>>>> >-------------- next part -------------- An HTML attachment was
>>>> >scrubbed...
>>>> >URL: