Subject: [apps-discuss] How to find a service?

From: Patrik Fältström <[email-masked]>
Date: Thu, 22 Jul 2010 10:20:56 +0200

We have had various discussions during the years on "how to find a service". And we have many ways of doing it at the moment. For example:

- Just use a well known algorithm
- Look up A or AAAA record, and then use well known port number
- Look up MX, and then A/AAAA and use well known port number
- Look up SRV record, and use that hostname and port number
- Use a DDDS algorithm as defined somewhere
- Use mdns like services to "broadcast" information (or respond to broadcast queries)
- :
- :

...and many more...

Now we have other ways as well, like the two currently proposed mechanisms that some people have seen discussed on the IETF list:

- What is described in draft-daboo-srv-caldav, which I in a sloppy way describe as "use SRV records and then a well known path that should result in an HTTP redirect"

- What is described in draft-faltstrom-uri, which I in a sloppy way describe as "use URI records like we use SRV records, but get a full URI back"

As we are so close to Maastricht, I do not find we have time for a doodle regarding interested parties that want to talk about this.

I have therefore decided we meet on Wednesday July 28 at 08:00 (yes, for a breakfast meeting). It will be by the IETF registration/announcement board, OR, where a note on that very same announcement board says. I guess it will be near or even where breakfast is served...

I am deliberately sending this only to Apps Discuss as I think we should start this discussion from the point of view of "the need applications has". That said, please spread the word to other interested parties. It is not supposed to be a "secret cabal meeting".

I envision a mailing list discussion after this first meeting.

    Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Thu, 22 Jul 2010 17:44:57 +0200

Ok, there is a confusion regarding the breakfast...no breakfast at the MECC says the hotel info, but on the agenda I saw:

> 0800-0900 Beverages - Trajectum Lobby

I interpreted that as "breakfast".

My call is still to have the meeting 8-9AM Wednesday, but there will not be breakfast...

   Patrik

---

From: Joe Hildebrand <[email-masked]>
Date: Fri, 23 Jul 2010 15:28:07 -0600

Sounds like an interesting conversation.  Here's another pattern that I've
seen:

- Look up a TXT record containing the URI.  Example:
http://xmpp.org/extensions/xep-0156.html

Same basic idea as your URI I-D, with all of the TXT trade-offs.

On 7/22/10 8:44 AM, "Patrik Fältström" <[email-masked]> wrote:

> Ok, there is a confusion regarding the breakfast...no breakfast at the MECC
> says the hotel info, but on the agenda I saw:
> 
>> We have had various discussions during the years on "how to find a service".
>> And we have many ways of doing it at the moment. For example:
>> 
>> Now we have other ways as well, like the two currently proposed mechanisms
>> that some people have seen discussed on the IETF list:
>> 
>> - What is described in draft-daboo-srv-caldav, which I in a sloppy way
>> describe as "use SRV records and then a well known path that should result in
>> an HTTP redirect"
>> 
>> - What is described in draft-faltstrom-uri, which I in a sloppy way describe
>> as "use URI records like we use SRV records, but get a full URI back"
>> 
>> 
>> As we are so close to Maastricht, I do not find we have time for a doodle
>> regarding interested parties that want to talk about this.
>> 
>> I have therefore decided we meet on Wednesday July 28 at 08:00 (yes, for a
>> breakfast meeting). It will be by the IETF registration/announcement board,
>> OR, where a note on that very same announcement board says. I guess it will
>> be near or even where breakfast is served...
>> 
>> I am deliberately sending this only to Apps Discuss as I think we should
>> start this discussion from the point of view of "the need applications has".
>> That said, please spread the word to other interested parties. It is not
>> supposed to be a "secret cabal meeting".
>> 
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

-- 
Joe Hildebrand

---

From: Blaine Cook <[email-masked]>
Date: Sat, 24 Jul 2010 01:09:10 +0100

I won't be able to make the Wednesday meeting, but many of us
implementing stuff are using draft-hammer-hostmeta, FWIW. Webfinger (a
related higher-level approach that is more pattern and less spec) is
often described as "DNS for People".

Those considering DNS-based solutions are well-advised to consider the
relative programmability of the deployed HTTP and DNS infrastructures,
and the ever-increasing need for new record types. Also, consider that
browser-based JavaScript environments are unable to make DNS requests
without a suitable proxy, but that they are absolutely able to make
HTTP requests.

b.

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 24 Jul 2010 06:10:58 +0200

Yes, have seen those as well...

  Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 24 Jul 2010 06:12:50 +0200

On 24 jul 2010, at 02.09, Blaine Cook wrote:

> and the ever-increasing need for new record types.

I want people to remember that there is a difference between thinking of good architecture (that drives how libraries etc are designed) and how people might do things based on how an existing library might be written.

   Patrik

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Fri, 23 Jul 2010 23:31:39 -0700

SRV and NAPTR records are a great example of where standards fails when it comes to practical deployment. In many cases, we need to discover a service endpoint, not just a service host (and port). SRV excels in its simplicity but because it cannot store a URI, it fails to cover these use cases. This leaves the problem largely unsolved (leading to the use of a complimentary TXT record or well-known URI).

NAPTR is supposed to solve this by allowing a more complex value, but it suffers from being overly complicated and generally unavailable in most hosting environments.

When it comes to social web protocols, we are forced to use what's practical, not what is architecturally correct - or fail to gain any meaningful adoption. So while I agree with you when it comes to developing protocols like OAuth, where it is legitimate to raise the bar beyond what existing code can do, it is not when dealing with web protocols that largely live on a pretty limited/restrictive environment.

EHL

> -----Original Message-----
> From: [email-masked] [mailto:apps-discuss-
> [email-masked]] On Behalf Of Patrik Fältström
> Sent: Friday, July 23, 2010 9:13 PM
> To: Blaine Cook
> Cc: Apps Discuss
> Subject: Re: [apps-discuss] How to find a service?
> 
> 
> I want people to remember that there is a difference between thinking of
> good architecture (that drives how libraries etc are designed) and how
> people might do things based on how an existing library might be written.
> 
>    Patrik

---

From: Jan Algermissen <[email-masked]>
Date: Sat, 24 Jul 2010 11:12:44 +0200

> SRV and NAPTR records are a great example of where standards fails when it comes to practical deployment. In many cases, we need to discover a service endpoint, not just a service host (and port).…

Maybe this is of interest in this context: http://www.infoq.com/articles/rest-discovery-dns

Jan

>> people might do things based on how an existing library might be written.

-----------------------------------
Jan Algermissen, Consultant
NORD Software Consulting

Mail: [email-masked]
Blog: http://www.nordsc.com/blog/
Work: http://www.nordsc.com/
-----------------------------------

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 27 Jul 2010 14:21:38 +0200

In the land of Dijkstra, and with the topic of "how to find a service", I first asked myself whether I had to announce the room name, but, here it is anyway:

The meeting will be held in room 2.1.

   Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Wed, 28 Jul 2010 08:51:49 +0200

Problems with unknown resource record types.

Service discovery is one piece.

Service provisioning is what the apps people want.
ACAP did not work.

1. Do not end up with an arbitrary number of these objects -- makes things more complicated

2. You want to retrieve a single blob from somewhere and then use it, instead of retrieving large number of records from DNS repeatedly

3. We do not want a lot of repeated lookups for similar things in the same zone

4. We do not want to be forced to pull to see if there have been changes

There are a number of requirements that we should understand...

---
Stewart from Apple give short explanation of what Apple is doing:

2 Layers

Service discovery, regardless of transport

Uses DNS records, either in a dns server, or multicast dns in a p2p fashion

Uses PTR records using the SRV prefix syntax, to get the SRV record
SRV + TXT record in the next step (same owner for the two)

Asynchronous queries ("tell me if the answer changes") - subscriptions
---

Desire of creation of a "marketplace" of similar services, and the user then chooses/selects which one of the services to use. First and later "lookups" could create different responses (later responses could "remember" what selection was done the first time).

What is "service"?

One example: Humans think "ink on paper" is a service, but computers think "printers using a specific protocol (and paper size etc)" is a service?

Are we now ("where is my fragments of data?") now talking about resource discovery instead of service discovery?

We have some data inbound in the protocol, and some data outbound. Depends a lot on what protocol you have on what you can do inbound and what you have to do outbound.

We do not want overlap between you do in the discovery part of the resolution and what you can negotiate inbound.

Bootstrap might be hard (specifically for DNS dynamic updates) due to lack of tools/applications for new users (i.e. key management).

Slightly different topic, how do one know whether a service works? How do we know whether something is running? UDP and TCP combinations, where UDP is only used for "I am alive" systems. Not very optimal.

"Who do I talk with?"
"What port do I use?"

Discussion about distribution of the bonjour code, which Apple said was under Apache license from Apple, and GPL from others.

Lightweight protocol needed to find services, so we do not use unique port numbers for each service (we must decrease the consumption of ports).

What we need is:

Code to implement the whole thing
Application registering in some shared port/service
Protocol to query whether service is there

HTTP based services are doing lookups etc over http, because possibility to use DNS in new ways is indeed hard (and other new non-existing lookup protocols).

Having the local printer is one thing, but the hard thing might be to know what to query for. Can I get the Alto server in a specific environment?

   Patrik

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 28 Jul 2010 02:15:10 -0500

On Wed, Jul 28, 2010 at 08:51:49AM +0200, Patrik Fältström wrote:
> Lightweight protocol needed to find services, so we do not use unique
> port numbers for each service (we must decrease the consumption of
> ports).

Nice thought.  Guess what?  NFS doesn't work across overzealous packet
firewalls because of the use of dynamic ports.  :(  (Linux has options
to run all the relevant services on registered port numbers; port
squatting ensues.)

And yet there is a well-known service registration/discovery protocol
for ONC RPC (on which NFS is built), and there have been firewalls that
know how to use it.  But as we see, that's not good enough because
firewalls get built that without regard to dynamic port registration.

Some of us are looking at making NFS more firewall-friendly by using a
combination of registered ports and indirect RPC calls (i.e., via the
portmapper).  Indirect RPC calls are not always appropriate, though they
are an interesting pattern to consider (though one runs the risk of
inviting firewalls to filter indirection services, and/or do deep
inspection).  (Some Apple engineers might remember NetInfo, an ONC
RPC-based protocol that made extensive use of indirect RPC calls.)

Indirection is nothing new...  Remember TCP port 1, tcpmux?

The problem is meddling middle boxes.  The solution is...?

In short, good luck,

Nico

PS: While researching the NFS problem mentioned above I discovered at
    least one port number conflict with an IANA registration that might
    well be obsolete.  I can't find a procedure for obsoleting port
    number registrations, something one might think would be desirable.

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 28 Jul 2010 02:18:11 -0500

> The problem is meddling middle boxes.  The solution is...?

...always to tunnel everything through HTTP, or port 80.