Subject: Escaping of attr tags in SLP API

From: Mikael Pahmp <[email-masked]>
Date: Mon, 23 Aug 1999 14:25:17 +0200

I see a problem in the 'findAttributes' method of the Java API in RFC 2614. In the describing text for the 'attributeIds' argument it says:

attributeIds
	A Vector of String patterns identifying the desired attributes. An empty vector means return all attributes. As described in [7], the patterns may include wildcards to match substrings. The strings may include SLP reserved characters, they will be escaped by the API before transmission. 

The last sentence is problematic. An attributeId string may contain '*' (star) characters indicating wild card matching. If these are escaped by the API, they will not be wild chards any more when processed by the SA or DA.

To discern between '*' chars that are part of the attribute ID and those that are wild chars, the 'attributeIds' must be correctly escaped by the caller, not within the API.

How do we handle this? What is an RFC-compliant implementation supposed to do until this has been corrected?

/Mikael

---

From: James Kempf <[email-masked]>
Date: Mon, 23 Aug 1999 07:46:08 -0700 (PDT)

Mikael,

Thanx for pointing this out:

>The last sentence is problematic. An attributeId string may contain '*' (star) 
characters indicating wild card matching. If these are escaped by the API, they 
will not be wild chards any more when processed by the SA or DA.
>
>To discern between '*' chars that are part of the attribute ID and those that 
are wild chars, the 'attributeIds' must be correctly escaped by the caller, not 
within the API.

Requiring the caller to escape a '*' character is the right way to go.
The others can be escaped by the API. 

		jak

---

From: Mikael Pahmp <[email-masked]>
Date: Tue, 24 Aug 1999 08:59:50 +0200

By the way, I find this feature of selecting attribute tags using wild chars of little use. It's a feature I would consider removing when SLP eventually goes to draft standard.

Disagree?

/MiP

> -----Original Message-----
> From: James Kempf [mailto:[email-masked]]
> Sent: den 23 augusti 1999 4:46
> To: [email-masked]; [email-masked]
> Subject: Re: Escaping of attr tags in SLP API
> 
> 
> Mikael,
>

---

From: Mikael Pahmp <[email-masked]>
Date: Tue, 24 Aug 1999 20:59:23 +0200

> >
> >By the way, I find this feature of selecting attribute tags 
> using wild chars of 
> little use. It's a feature I would consider removing when SLP 
> eventually goes to 
> draft standard.
> >
> >
> 
> Why do you think that?
> 
> It's been useful in some projects we're working on, which are more
> in the area of research. Also, I can see it being useful for
> obtaining all attributes as a way of displaying them for browsing,
> particularly if you don't know a priori what the tags are.
> 

An empty tag list does the same thing. Having a tag list so you can choose the attributes you are interested in and thereby keep packet size below MTU is good. But the burden put on the SA/DA of processing wild chars should be avoided to keep them small, especially SAs. This kind of processing is better done on the UA side.

While we're at it. I think attribute requests with only service type specified (not a complete service URL), where the reply should be a collection of all attribute values from all services of that service type, should be removed. This is costly to implement in an SA. Attribute requests are optional to implement but is mostly needed. It should at least be allowed to support AttrRqst with complete URLs without having to support the service type case. Again this can be handled on the UA side and I think this should be removed in the future.

/Mikael

---

From: James Kempf <[email-masked]>
Date: Tue, 24 Aug 1999 14:45:23 -0700 (PDT)

>
>
>An empty tag list does the same thing. Having a tag list so you can choose the 
attributes you are interested in and thereby keep packet size below MTU is good. 
But the burden put on the SA/DA of processing wild chars should be avoided to 
keep them small, especially SAs. This kind of processing is better done on the 
UA side.
>
>While we're at it. I think attribute requests with only service type specified 
(not a complete service URL), where the reply should be a collection of all 
attribute values from all services of that service type, should be removed. This 
is costly to implement in an SA. Attribute requests are optional to implement 
but is mostly needed. It should at least be allowed to support AttrRqst with 
complete URLs without having to support the service type case. Again this can be 
handled on the UA side and I think this should be removed in the future.

OK, I think I understand the drift of your thinking. The idea is to just
get rid of the attribute wildcard specifier, `*`, but not prohibit
using the empty list for getting all attributes. The UA side would 
therefore be required to sort through the attributes, rather than
have the SA do it. At the API level, the attributes would be passed
back to the client to sort through.

Also, removing the attribute request by service type would remove a
complicated feature of AttrRqst. This feature is costly to implement
because of the duplicate elimination feature on the UA side as well.
Duplicate elimination can't just be done on the SA side, because the
UA may get back duplicate attributes from a multicast.

On the other hand, attribute request by URL is relatively simple for the SA, 
since the SA only needs to determine whether it has the URL or not, and, if
so, just get those attributes in the request. Similarly, the UA doesn`t
need to do any duplicate elimination, except to drop multiple replies
(in the multicast case).

Yes, I agree that dropping these would simplify both the UA and the SA.
I think these features are ones that will be seldom used but can
complicate the implementation.

I wonder if someone out there has some examples of applications
where these features were so handy that they would be sorely missed.
Otherwise, I agree we should consider dropping them as we go to standard.

		jak

---

From: "Roger Holm" <[email-masked]>
Date: Wed, 25 Aug 1999 17:24:03 -0600

All,

I strongly disagree with the notion of eliminating the wildcarding capabilities in SLP.  At Novell we have some experience now with implementing SLP in very large scale network installations.  The idea of always retrieving everything and tossing what you don't need at the receiving end is just opposite of what customers want out of SLP.  That kind of thinking is almost okay for a pure high speed LAN environment where network bandwidth is perceived as cheap, but in networks that include WAN technologies the focus must be in minmizing the number of packets on the wire and maximizing the work at the endpoints.  Otherwise you can bring a respectable network to its knees with SLP (believe me we know from experience).  Our use of SLP makes extensive use of wild carding to minimize the number of responses we receive for each request and we plan to leverage it even more as we proceed.  What has been suggested is just the way SAP worked in the old IPX world, gather everything and toss !
what you don't want .  You will also remember that SAP didn't scale too well in large LAN network environments and was never effective in WAN networks.  With SAP you didn't have a choice, and if you remove wildcarding from SLP as it goes standard you will end up with a similar situation.

At Novell we have implemented UAs, SAs, and DAs so we know what a pain the wildcarding can be
(I personally have implemented all three), but remember that you code once but you retrieve data always at the network's expense.  I think the better direction would be to define how a lightweight SA could be incorporated into the SLP envrionment.  By lightweight I mean an SA that perhaps would not implement full wildcarding.  But I also believe that lightweight SAs should be the execption and used mostly for embedded devices and the like.    The one great advantage we (Novell) saw in SLP was the ability to send "smart" requests and only retrieve the wanted information.  So again, philosophically and from raw experience I strongly disagree with the notion of doing away with wildcarding to ease the burden of implementing an SA and DA.

Roger Holm
Novell, Inc.
[email-masked]

>>> James Kempf <[email-masked]> 08/24/99 03:45PM >>>

		jak

---

From: Mikael Pahmp <[email-masked]>
Date: Thu, 26 Aug 1999 08:09:29 +0200

> From: Roger Holm [mailto:[email-masked]]
> 
> All,
> 
> I strongly disagree with the notion of eliminating the 
> wildcarding capabilities in SLP.  At Novell we have some 
> experience now with implementing SLP in very large scale 
> network installations.  The idea of always retrieving 
> everything and tossing what you don't need at the receiving 
> end is just opposite of what customers want out of SLP.  That 
> kind of thinking is almost okay for a pure high speed LAN 
> environment where network bandwidth is perceived as cheap, 
> but in networks that include WAN technologies the focus must 
> be in minmizing the number of packets on the wire and 
> maximizing the work at the endpoints.  Otherwise you can 
> bring a respectable network to its knees with SLP (believe me 
> we know from experience).  Our use of SLP makes extensive use 
> of wild carding to minimize the number of responses we 
> receive for each request and we plan to leverage it even more 
> as we proceed.  What has been suggested is just the way SAP 
> worked in the old IPX world, gather everything and toss what 
> you don't want .  You will also remember that SAP didn't 
> scale too well in large LAN network environments and was 
> never effective in WAN networks.  With SAP you didn't have a 
> choice, and if you remove wildcarding from SLP as it goes 
> standard you will end up with a similar situation.
> 
> (I personally have implemented all three), but remember that 
> you code once but you retrieve data always at the network's 
> expense.

Are you referring to wildcards in SrvRqst predicate filters only? In that case I agree with you completely. My suggestion about skipping wilcards was not about the predicate filter but the tag-list of AttrRqsts and SrvDeRegs. Predicate filters is an excellent way of reducing the scope of your request and network load. I'm however not so sure that the wildcards in tag-lists is worth it. Are you using this feature to select attributes in attribute requests and service deregistrations?

About the predicate filter. Maybe we could reduce the grammar used by SLP to be only a subset of LDAP's? Is that something you've thought of?
I believe the grammar in SLPv1 is smaller than LDAP's used in v2.

How about attribute requests by service type? Is that useful to you?

/Mikael

---

From: James Kempf <[email-masked]>
Date: Thu, 26 Aug 1999 07:39:00 -0700 (PDT)

>
>About the predicate filter. Maybe we could reduce the grammar used by SLP to be 
only a subset of LDAP's? Is that something you've thought of?
>How about attribute requests by service type? Is that useful to you?

The grammer of the predicate filter is a subset of LDAPv3, but only insofar
as extensible matching rules are not handled. The predicate filter matches
the LDAPv3 base set.

I also think it would not be a good idea to reduce compatibility with
LDAP any further, as it would make LDAP integration more difficult.

		jak

---

From: "Roger Holm" <[email-masked]>
Date: Fri, 27 Aug 1999 13:00:16 -0600

Mikael,

Sorry for the delay in responding.  The reaction to my post has been interesting, so let me tell you where I was coming from in general.  Perhaps my response was somewhat knee jerk in the sense of generalizing about wildcarding in SLP rather than limiting my discussion to the original question about wildcarding attribute tags.   But my experience in networking  leaves me more convinced than ever that network bandwidth is a very precious commodity, especially in large enterprise networks, and that design decisions should fall towards being network friendly when a tradeoff must be made.  

When I first started in this business (over twenty years ago), 9600 baud leased trunk lines between mainfraime and remote computer sites were the norm and so every data bit on the line was precious.  Of course network speeds have changed quite a bit since then, but so has network use.  The number of attachments to the networks of today are so much larger than in the early days that they can't even be compared.  So while network bandwidth has exploded, so has the number of network users.  When LANs came along it appeared that network bandwidth was no longer an issue, and as a result many products were developed that freely used bandwidth, some to advantage and others due to laziness on the part of the designer/developer.  NetWare used a model in which the decision making was pushed out to the workstation so the server could be kept simple and fast in order to service many users.  For a LAN based network this was great, but over WANs where bandwidth is not plentiful, it becomes !
problematic.   When many products with this processing model are utilized on a WAN network, the fast server is soon hidden due to the network congestion caused by the many requests and many responses generated by every user.   Thus, in my experience (when it comes to WAN networks such as those deployed by many large enterprises  or the Internet itself), the realities concerning network bandwidth haven't changed much, its just that the numbers are larger, both  in bandwith and number of users.  So when I read the previous e-mails (and if I have misunderstood them I apologize, just tell me and I will be quiet)  that seemed to advocate making the client (UA) sophisticated and the SA simple (i.e., pull all information and filter out what you requesting user doesn't want), I felt that it was just the wrong model and had to speak up.

With respect to wildcarding I agree that SrvRqst predicate filters are the most valuable and the ones I have seen most used.  But I am not ready to decommit the wildcarding of attributes tags quite yet based on my experience within Novell and the evolution that our utlization of SLP has taken.  In the way of background, Novell included SLPv1 in  NetWare 5 as one of the connectivity technologies that clients could use to find NDS and Bindery servers and printers.  SLP use is optional in NetWare 5 and the standard DNS, DHCP and static configuration methods for IP connectivity are provided.  The use of SLP was initially intended to allow clients to locate an NDS server, after which the client uses the directory for further service discovery and use.  For all intents and purposes that is how things work, although since the introduction of NetWare 5, some interesting uses of SLP have emerged, but I digress.  Our first use of SLP was to advertise every server that held a directory p!
artion.  In this way the clients could query for the directory partition they were a part of and get URLs indicating the address of the servers holding the partition.  After that the client would authenticate to the directory and SLP was out of the picture.  What we quickly realized was that the number of URLs was large when you consider that a separate URL had to be registered for every interface of a server holding a partion.  That is, the number of service registrations equalled the number of servers holding partitions mulitplied by the number of network interfaces per server.   And so it went for every service on the server that registered with SLP.  At Novell we have thousands of servers, most of which are multihomed so the sheer number of URLs that could be returned was enourmous depending on how smart the client was  in making its SLP requests.  At first the clients used the same model they used for IPX/SAP, find all servers in the tree and then toss the uninteresting o!
nes.  What a disaster that turned out to be, so that was fixed ver
(perhaps not typical of many installations),  the number of clients was even larger, so as each client came online, or the user started browsing for servers and directory trees, the number of SLP requests was huge with each response being potentially large.  I think you start to get the picture.    We were not being very network friendly (much to the dismay of our IS department). 

Of course you say that the scaling issue calls for a DA.  We have one and it was deployed.  But the sheer number of clients making requests and the resulting size of responses was still impacting the network too much.  Additionally, the DA now had to deal with its own scaling issues in terms of the number of service registrations it was holding and servicing.  As you attempt to resolve that issue by using scopes and deploying more DAs, the potential for generating more messages goes up because clients may now have to deal with multiple DAs to find the desired information (this is sounding much more grim than it really is now).  Because network bandwidth utilization has to do with not only the number of messages but the amount of data they carry as well, we went to work on ways to better utilize SLP to minimize the number of messages and the amount of data exchanged between DAs and UAs.  To reduce the number of service registrations for directory partitions we removed the IP ad!
dress(es) of each server from the URL and made them an attribute of the service registration for a directory partition.  The result was a single URL for each directory partition.  Since every server holding a partition replica would register the same partition URL, the address attribute value would collect all addresses of servers holding a partition replica.  This greatly reduced the number of service registrations that were present in the network.  But it now required the client to issue an AttributeRequest to get the NDS server address(es)  Although we issue two requests to get the NDS server address, the overall effect of these changes has brought the impact on network bandwidth utilization to an acceptable level for finding NDS partitions.  We are now experimenting with the clients being able to compose the URL directly and issuing only an AttrRqst to get all addressesof servers holding replicas of a directory partition.  And the research continues ....

I would like to make several points from this:

1) When you are dealing with large network environments (our large customers have literally 10s of thousands of servers and workstations), the sheer number of nodes in the network can quickly cause the number of service registrations to beome unwieldy if you are not smart about how you register and locate service information.  Granted that a good SLP architecture and not allowing full access to every service in the network is helpful, but interestingly enough, one of our most common questions or requests from customers is for their users to see every server in the company.  This is what people want to do, the question is can we deliver?  The one who does it best is in line to make a lot of money.

2) We quickly had to revert to using attributes to hold and convey information in order to deal with the problems we initially had.  In my opinion, this does not bode well for the SA that is implemented to RFC 2608 Appendix B guidelines.  Also,  we don't just store server addresses as attributes, but other useful information to pare down the number of URLs returned and to find the closest server holding a partition replica (but we do basically limit it to information that helps clients make smarter connectivity decisions).  This saves further bandwidth later on in the login/authentication process.  So wildcarding in SrvRqst predicate strings and possibly attribute tags is very important from our experience.

3) Another concept that is being suggested and experimented with is to register more general/abstract service types and allow specific services to just add their own set of attributes to a single service registration.  Perhaps the easiest example of this concept to  understand is a network server of some sort that hosts multiple unique services.  Instead of each specific service registering its own service URL(s) with an SA, the server's service registration is augmented with a set of attributes that belong to the specific service.  If the server's service template defined unique naming schemes for each service supported,  wildcarding of attribute tags could be useful in obtaining all of the attributes of a specific service but not all attributes of all services active on the server.  This is something that couldn't be done with an empty select list.  I am not sure I like this approach the best, but I bring it up because it is being actively discussed as a way to further minim!
ize the number of service registrations that are active in the network and it does bring relevance to the notion of wildcarding attribute tags.

4) Finally, there are lots of ways to make all of this work and our experience is that customers are liking what they are seeing with SLP.  They realize this is a new technology and we are still working out the kinks.  When we hit issues like we have been discussing we can either constrain the customer into our way of implementation or we can move the technology along towards the customers thinking and desires.  Customers seem to want SLP to scale in a major way, so constraining them to a model that makes our implementatioin efforts easier at the expense of the network may not cut it with the "big boys".  The question is who do want to market your products to?  If your implementation of SLP does not scale, your product's potential customer base may be less than you thought.  Just some food for thought.

The notion has been brought up that perhaps SA requirements should be minimal and the DAs be sophisticated in terms of attribute support.  I don't favor that approach because then the user experience is different dependent on the presence of a DA.  I view DAs as a network administrator's tool to deal with scaling issues and network performance, not an experience altering entity.  After my first e-mail response on topic of discussion, I reviewed the difficulty of support wildcarding and it wasn't that great once you decide to support attributes.  The wildcarding scheme defined is simple minded and involves using standard substring location functions.  Then again, that's my implementation and not yours and perhaps I haven't figured out all the problems yet. And I do understand the issues of implementing these features in a small device; but it seems to me that the decision point is not whether we support wildcarding on attributes and their tags,  but whether or not attributes ar!
e supported at all as suggested by Appendix B of RFC 2608.  Leave the wildcarding capability in the protocol and then decide, based on your SLP user's needs, whether or not attributes are supported.  I have previously given my concerns about not supporting attributes.

Lastly, you asked if we had found the Attribute Request by service type useful: not to my knowlege.  What would have been really useful is an Attribute request by service type with both a where clause and select list.  In that way a single "smart" query could return only the attributes from the selected set of registrations thus eliminating the need for an initial SrvRqst/SrvReply sequence.

This probably is a lot more than you were expecting as a response, but I hope you better understand my reaction and first e-mail.  I am open to new ideas and perspectives, but I thought all could learn from our (Novell's) first experience with SLP and I hope this will stimulate new thinking for you.  SLP has great potential but some of these kinds of issues could still use some work.  I want to thank everyone that has provided input for the SLP protocol and especially those who are members of the working group for SLP.  You have done a great job!!!!

I have addressed this reply to all even though this is a reply to Mikael's e-mail so I welcome any other input.

Regards,

Roger Holm

>>> Mikael Pahmp <[email-masked]> 08/26/99 12:09AM >>>

/Mikael

---

From: James Kempf <[email-masked]>
Date: Fri, 27 Aug 1999 13:06:46 -0700 (PDT)

Roger,

These are absolutely spectacular comments. Thanx for the detail.

>Of course you say that the scaling issue calls for a DA.  We have one and it 
was deployed.  But the sheer number of clients making requests and the resulting 
size of responses was still impacting the network too much.  Additionally, the 
DA now had to deal with its own scaling issues in terms of the number of service 
registrations it was holding and servicing.  As you attempt to resolve that 
issue by using scopes and deploying more DAs, the potential for generating more 
messages goes up because clients may now have to deal with multiple DAs to find 
the desired information (this is sounding much more grim than it really is now).

I don't understand why this should be the case. A UA should only need
to deal with one DA since all DAs having the same scopes as the UA
should have the same registrations. Naturally, SAs do have to register
with more than one DA.

>3) Another concept that is being suggested and experimented with is to register 
more general/abstract service types and allow specific services to just add 
their own set of attributes to a single service registration.  Perhaps the 
easiest example of this concept to  understand is a network server of some sort 
that hosts multiple unique services.  Instead of each specific service 
registering its own service URL(s) with an SA, the server's service registration 
is augmented with a set of attributes that belong to the specific service.  If 
the server's service template defined unique naming schemes for each service 
supported,  wildcarding of attribute tags could be useful in obtaining all of 
the attributes of a specific service but not all attributes of all services 
active on the server.  This is something that couldn't be done with an empty 
select list.  I am not sure I like this approach the best, but I bring it up 
because it is being actively discussed as a way to further minim!

That's pretty cool. :-)

> And I do understand the issues of implementing these features in a small 
device; but it seems to me that the decision point is not whether we support 
wildcarding on attributes and their tags,  but whether or not attributes ar!
>!
>!
>!
>e supported at all as suggested by Appendix B of RFC 2608.  Leave the 
wildcarding capability in the protocol and then decide, based on your SLP user's 
needs, whether or not attributes are supported.  I have previously given my 
concerns about not supporting attributes.

My feeling is that Appendix B devices probably won't show up in regular
enterprise networks very often, if at all. I think they are more likely
to be seen in home networks.

>
>Lastly, you asked if we had found the Attribute Request by service type useful: 
not to my knowlege.  What would have been really useful is an Attribute request 
by service type with both a where clause and select list.  In that way a single 
"smart" query could return only the attributes from the selected set of 
registrations thus eliminating the need for an initial SrvRqst/SrvReply 
sequence.

Can you give an example of how this might be used? 

		jak

---

From: Jonathan Wood <[email-masked]>
Date: Mon, 30 Aug 1999 11:21:34 -0700 (PDT)

Mikael and Jim,

I just noticed that the grammer for attributes in section 5 of RFC 2608 states 
that '*' cannot be used in tags:

   attr-tag = 1*safe-tag
   safe-tag = ; Any character except reserved, star and bad-tag.

The exception to this is when '*' is to be used as a wildcard (section 9.4).

The API RFC states that only reserved characters will be escaped; '*' is not a 
reserved character, so it will not be escaped. This is the correct behavior. If 
you try to include an escaped '*' in the tag, the API should reject the tag.

So I don't think there is a problem here.

Jon

---

From: Mikael Pahmp <[email-masked]>
Date: Mon, 30 Aug 1999 20:43:17 +0200

> From: Jonathan Wood [mailto:[email-masked]]
> 
> Mikael and Jim,
> 

You're right. Thank you for correcting me.
I who often brag about my excellence in logic thinking ....

/MiP

---

From: James Kempf <[email-masked]>
Date: Mon, 30 Aug 1999 12:45:40 -0700 (PDT)

Thanx for pointing this out, Jon.

		jak