Subject: Re: [paws] JSON-RPC vs. JSON-REST

From: Michael Head <[email-masked]>
Date: Thu, 29 Aug 2013 10:45:17 -0700

Thanks for the detailed response Dan,

I still believe the idea of separating metadata from data in the protocol
and defining version and capability negotiation in as would be enabled by a
RESTful architecture would yield significant benefits down the road.

However, given that nobody has voiced any support for my proposal, and in
the interest of moving the draft forward, I'll be happy to retract it.

Perhaps we may revisit it when we draft future versions of PAWS.

-- mike

On Mon, Jul 22, 2013 at 9:49 AM, Harasty, Daniel J
<[email-masked]>wrote:

> In considering Michaels input, these are my thoughts:****
>
> ** **
>
> **·         **I  agree that the current draft’s ruleset concept and the
> number of optional messages (especially *server optional* message) does
> not lead to a particularly clean “negotiation”.  In fact, while it looks
> workable, I’m trying to implement it now, and it “feels” rather clumsy.***
> *
>
> **o   **To remedy this, I was  on the verge of proposing a required set
> of “feature negotiations” in the INIT_REQ/RESP messages.****
>
> **o   **This would also be a good place for servers to initialize any
> authentication schemes (if required by the server, and until a standard
> certificate-based scheme is specified and adopted). ****
>
> ** **
>
> **·         **Moving this negotiation to the HTTP headers (as Michael
> proposes) is – I suspect – workable.****
>
> **o   **However, it is my personal opinion that that ties the protocol
> MORE TIGHTLY to HTTP.  ****
>
> **§  **Keeping it “in band” – inside the JSON-RPC message itself – allows
> for alternate transports: say directly over UDP, or some future
> wireless-packet scheme, for example.****
>
> **o   **And, as and implementer, I’d rather just deal with the options of
> the protocol “in band”, in the protocol itself. ****
>
> **§  **As a server process implementing this protocol, what is the
> benefit to me to have to “go looking” for “how to handle this message” by
> looking a bit in the message, and a bit in the HTTP headers?****
>
> ** **
>
> **·         **Yes, interworking of new protocols (or a new version of the
> “same” protocol) with old ones is always a bit tricky; it is difficult to
> know NOW everything we’ll need in the future to make interworking work
> smoothly.****
>
> **o   **However, we already have a perfectly good “escape hatch” in our
> current JSON-RPC-oriented approach: at anytime in the future, and new PAWS
> version can define a new method outside of the scope of PAWS 1.0. ****
>
> **§  **To my reading, the old PAWS server would reply with a JSON-RCP
> “method not implemented”… and thus interworking was handled smoothly.  (It
> would be the client’s prerogative to “step back” through older protocols
> that it does implement.)****
>
> **o   **Also, in some cases, new features/extensions could be “added” to
> PAWS (and safely interwork old and new versions) due to the fact that
> “ignoring unexpected fields” is REQUIRED.****
>
> **o   **While I think Michael’s proposal would offer a different way to
> handle version negotiation, I don’t see that it is “better”… just “a
> different way”. ****
>
> ** **
>
> **·         **I agree that the ability to be “transparently redirected”
> to a peer (“federated”) server is a good idea…****
>
> **o   **… but, to my reading, that is already accommodated by the Section
> 7 text regarding the handling of HTTP Redirects.****
>
> ** **
>
> **·         **As for additional “mappings” from browsers: nothing in our
> current spec precludes a Database from replying one way to PAWS messages
> (in HTTP POST encoded as JSON-RPC), and to HTTP GETs another way.****
>
> **o   **In fact, we’re already implementing that, too.****
>
> ** **
>
> I have other minor comments (maybe even quibbles) with some of Michaels
> statements… but there is no particular need to go in to all of that now.**
> **
>
> ** **
>
> I propose the bigger questions is one of general direction.  At this point
> in time:****
>
> **1.       **If the rest of the group finds the current “feature/ruleset
> negotiation” scheme is just fine, then perhaps no action is needed.****
>
> **2.       **If others feel that it is somewhat clunky, do we attempt to
> address it by:****
>
> **o   **“Fixing it” in the current “style”: introduce a more clear –
> possibly REQUIRED –  “negotiation” to the existing JSON-RPC.****
>
> **o   **Changing direction now to Michaels proposed use of HTTP headers.**
> **
>
> **o   **Pursuing BOTH to a sufficient degree of clarity until we can then
> pick between them.****
>
> ** **
>
> Thoughts?****
>
> ** **
>
> Dan Harasty****
>
> ** **
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Michael Head
> *Sent:* Monday, July 22, 2013 11:07 AM
> *To:* [email-masked]
> *Subject:* [paws] JSON-RPC vs. JSON-REST****
>
> ** **
>
> I suppose this will be controversial, but I’m finding that relying on
> JSON-RPC is driving unnecessary coupling among the device, database and
> regulatory authority. What I mean by this is that each device/client has to
> know (or detect) something about the database/server it's talking to as
> well as the ruleset under which the server will operate.****
>
> ** **
>
> For example, it is allowed for a device to be implemented so that it only
> makes the spectrum.paws.GetSpectrum RPC. However, in order for this to
> work, the database it is configured to communicate with must be built to
> allow for this: it won’t work with a database that requires
> spectrum.paws.Init and spectrum.paws.Register to be called. The device can
> only detect that the database requires these calls by inspecting the error
> returned from the GetSpectrum RPC. Furthermore, registration rules
> (including which fields are required for proper registration) may be
> governed by the regulatory authority. The device needs to know these
> details a priori (or possibly detect them by attempting a request and
> inspecting error results in an underspecified manner). Much of those
> details are determined by the governing ruleset ID, and its precise meaning
> must be baked into both the database and the device, and they must assume
> that their respective semantics will agree.****
>
> ** **
>
> Some of these issues could be solved by reducing the amount of optionality
> (and perhaps some of the optional calls) and requiring a standard sequence
> of operations. This may work, but I feel the coupling would still be high.
> Protocol version upgrades (and version negotiation) won’t be particularly
> graceful: the client might send a version 1.1 message to the database, and
> even though the client might also support 1.0, it would get an error
> response if the database didn’t support 1.0.****
>
> ** **
>
> As a secondary issue, it’s less natural to expose the service via the web.
> JSON-RPC is really only built to allow for programmatic clients.****
>
> ** **
>
> So.... I’d rather see a RESTful API that develops application state
> through the use of hyperlinked, content-typed message bodies. I suppose
> REST-vs-RPC has been discussed quite a bit in general, but I haven’t seen
> it brought here.****
>
> ** **
>
> A sketch of the protocol might look like this (I’m not a RESTful API
> expert, so I’m sure it could be better):****
>
> ** **
>
> Database publishes a single URL entry point: http://example.com/paws****
>
> ** **
>
> Device POSTs an application/vnd.paws.init.request-v1+json message to
> http://example.com/paws with header****
>
> Accept:
> application/vnd.paws.init.response-v2+json;application/vnd.paws.init.response-v1+json
> ****
>
> ** **
>
> Database (let’s say it only supports v1) responds with a
> application/paws.init.response-v1+json message which includes the details
> in 4.2.2 of the draft -- with, say a cache-control: max-age=86400 header --
> but contain an additional field: spectrumResource:
> {acceptable-content-type: application/paws.getspectrum.request-v1+json;
> required-fields: device/fccId; uri:
> http://example.com/paws/available-spectrum}****
>
> ** **
>
> Device POSTs application/paws.getspectrum.request-v1+json a message with
> the FCC id to http://example.com/paws/available-spectrum and header ****
>
> Accept: application/vnd.paws.notifyuse-v1+json****
>
> ** **
>
> Database response with an application/paws.getspectrum.response-v1+json
> response containing the details specified in 4.4.2 with an additional field
> (if it is required for spectrum use to be notified by the device):
> notifyUseToResource {acceptable-content-type:
> application/vnd.paws.notifyuse-v1+json, uri:
> http://example.com/paws/used-spectrum}.****
>
> ** **
>
> Device POSTs the selected frequency ranges to the URL and is done.****
>
> ** **
>
> In this way, the device only needs to know how to negotiate and handle the
> content types and be configured with the initial URL of the service. Some
> interesting benefits:****
>
> ** **
>
> * The device can connect to the database to ask about any location and --
> if the original database doesn’t support a given country -- be
> transparently directed in the response to another database to file the get
> spectrum request. It’s feasible for any database to federate with other
> databases and provide seamless database discovery anywhere whitespace
> databases are in use (though this doesn’t speak to the protocol by which
> the databases would discover each other)****
>
> * At each POST, the device declares the response version it can handle
> which indicates to the server that it can safely use that version and
> assume it will be handled by the client according to that particular
> version of the spec. Further, it allows the client and server to easily
> agree on an upgraded version using the HTTP headers alone without needing
> to design such a feature into the protocol itself.****
>
> * There’s a very natural mapping to a web-browser accessible site for
> viewing/querying the database contents: visit http://example.com/paws in
> a browser, and it sends back a page with a map allowing the user to pick a
> location. URLs like
> http://example.com/paws/available-spectrum/latitude=45N/longitude=75Wmight be bookmarked for repeated viewings for a particular location.
> ****
>
> * The initialization response could be augmented with ruleset definitions
> that could be cachable, GETtable resources that (perhaps in a future
> version of the spec when the breadth of rulesets is better known) allow
> devices to seamlessly upgrade their rulesets from the database.****
>
> ** **
>
> ** **
>
> -- ****
>
> ----------------------------------****
>
> Michael R Head <[email-masked]>****
>
> http://www.cs.binghamton.edu/~mike****
>
> +1-201-BLISTER****
>
> _______________________________________________
> paws mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/paws
>
>

-- 
----------------------------------
Michael R Head <[email-masked]>
http://www.cs.binghamton.edu/~mike
+1-201-BLISTER

---

From: "Stine, John A." <[email-masked]>
Date: Thu, 29 Aug 2013 18:32:56 +0000

Mike,

                I very much agree with your goal.  Back when we were putting together the use cases and requirements document, I had recommended separating the business process data from the modeling of spectrum use.  The following lines did make the document as a guideline but not as a requirement.

The data model SHOULD provide a modular design separating
messaging-specific, administrative-specific, and spectrum-
specific parts into distinct modules.

My vision was that any one of these parts may need to evolve and if the IETF PAWS were designed to support this modularity that it would very useful beyond the TV Whitespace.  The less specific the protocol was to the TV Whitespace problem the better.  It would be better if data modules were designed to be specific for the particular spectrum and regulatory domain in which it is applicable as opposed to confounding the requirements of multiple domains into a single data model. and then making it integral to the exchanges of the protocol.

I am involved with another standardization effort through the IEEE DySPAN SC called Spectrum Consumption Modeling.   We are developing an approach that can be used to model and use of spectrum and then to arbitrate compatibility.  It is very relevant to PAWS since it could open up the concepts of database managed sharing as envisioned by the PCAST report in the US.  Owners of spectrum can use these models to specify the contours that allow other users to share.  Models can be used for the trading of spectrum.  Models can enable database administrators to manage coexistence.  It would be nice if the design of the IETF PAWS would seek to support the data modularity and separation from the communications.

John

From: [email-masked] [mailto:[email-masked]] On Behalf Of Michael Head
Sent: Thursday, August 29, 2013 1:45 PM
To: Harasty, Daniel J
Cc: [email-masked]
Subject: Re: [paws] JSON-RPC vs. JSON-REST

I still believe the idea of separating metadata from data in the protocol and defining version and capability negotiation in as would be enabled by a RESTful architecture would yield significant benefits down the road.

However, given that nobody has voiced any support for my proposal, and in the interest of moving the draft forward, I'll be happy to retract it.

-- mike

On Mon, Jul 22, 2013 at 9:49 AM, Harasty, Daniel J <[email-masked]<mailto:[email-masked]>> wrote:
In considering Michaels input, these are my thoughts:

*         I  agree that the current draft's ruleset concept and the number of optional messages (especially server optional message) does not lead to a particularly clean "negotiation".  In fact, while it looks workable, I'm trying to implement it now, and it "feels" rather clumsy.

o   To remedy this, I was  on the verge of proposing a required set of "feature negotiations" in the INIT_REQ/RESP messages.

o   This would also be a good place for servers to initialize any authentication schemes (if required by the server, and until a standard certificate-based scheme is specified and adopted).

*         Moving this negotiation to the HTTP headers (as Michael proposes) is - I suspect - workable.

o   However, it is my personal opinion that that ties the protocol MORE TIGHTLY to HTTP.

*  Keeping it "in band" - inside the JSON-RPC message itself - allows for alternate transports: say directly over UDP, or some future wireless-packet scheme, for example.

o   And, as and implementer, I'd rather just deal with the options of the protocol "in band", in the protocol itself.

*  As a server process implementing this protocol, what is the benefit to me to have to "go looking" for "how to handle this message" by looking a bit in the message, and a bit in the HTTP headers?

*         Yes, interworking of new protocols (or a new version of the "same" protocol) with old ones is always a bit tricky; it is difficult to know NOW everything we'll need in the future to make interworking work smoothly.

o   However, we already have a perfectly good "escape hatch" in our current JSON-RPC-oriented approach: at anytime in the future, and new PAWS version can define a new method outside of the scope of PAWS 1.0.

*  To my reading, the old PAWS server would reply with a JSON-RCP "method not implemented"... and thus interworking was handled smoothly.  (It would be the client's prerogative to "step back" through older protocols that it does implement.)

o   Also, in some cases, new features/extensions could be "added" to PAWS (and safely interwork old and new versions) due to the fact that "ignoring unexpected fields" is REQUIRED.

o   While I think Michael's proposal would offer a different way to handle version negotiation, I don't see that it is "better"... just "a different way".

*         I agree that the ability to be "transparently redirected" to a peer ("federated") server is a good idea...

o   ... but, to my reading, that is already accommodated by the Section 7 text regarding the handling of HTTP Redirects.

*         As for additional "mappings" from browsers: nothing in our current spec precludes a Database from replying one way to PAWS messages (in HTTP POST encoded as JSON-RPC), and to HTTP GETs another way.

o   In fact, we're already implementing that, too.

I have other minor comments (maybe even quibbles) with some of Michaels statements... but there is no particular need to go in to all of that now.

I propose the bigger questions is one of general direction.  At this point in time:

1.       If the rest of the group finds the current "feature/ruleset negotiation" scheme is just fine, then perhaps no action is needed.

2.       If others feel that it is somewhat clunky, do we attempt to address it by:

o   "Fixing it" in the current "style": introduce a more clear - possibly REQUIRED -  "negotiation" to the existing JSON-RPC.

o   Changing direction now to Michaels proposed use of HTTP headers.

o   Pursuing BOTH to a sufficient degree of clarity until we can then pick between them.

Thoughts?

Dan Harasty

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Michael Head
Sent: Monday, July 22, 2013 11:07 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [paws] JSON-RPC vs. JSON-REST

I suppose this will be controversial, but I'm finding that relying on JSON-RPC is driving unnecessary coupling among the device, database and regulatory authority. What I mean by this is that each device/client has to know (or detect) something about the database/server it's talking to as well as the ruleset under which the server will operate.

For example, it is allowed for a device to be implemented so that it only makes the spectrum.paws.GetSpectrum RPC. However, in order for this to work, the database it is configured to communicate with must be built to allow for this: it won't work with a database that requires spectrum.paws.Init and spectrum.paws.Register to be called. The device can only detect that the database requires these calls by inspecting the error returned from the GetSpectrum RPC. Furthermore, registration rules (including which fields are required for proper registration) may be governed by the regulatory authority. The device needs to know these details a priori (or possibly detect them by attempting a request and inspecting error results in an underspecified manner). Much of those details are determined by the governing ruleset ID, and its precise meaning must be baked into both the database and the device, and they must assume that their respective semantics will agree.

Some of these issues could be solved by reducing the amount of optionality (and perhaps some of the optional calls) and requiring a standard sequence of operations. This may work, but I feel the coupling would still be high. Protocol version upgrades (and version negotiation) won't be particularly graceful: the client might send a version 1.1 message to the database, and even though the client might also support 1.0, it would get an error response if the database didn't support 1.0.

As a secondary issue, it's less natural to expose the service via the web. JSON-RPC is really only built to allow for programmatic clients.

So.... I'd rather see a RESTful API that develops application state through the use of hyperlinked, content-typed message bodies. I suppose REST-vs-RPC has been discussed quite a bit in general, but I haven't seen it brought here.

A sketch of the protocol might look like this (I'm not a RESTful API expert, so I'm sure it could be better):

Database publishes a single URL entry point: http://example.com/paws

Device POSTs an application/vnd.paws.init.request-v1+json message to http://example.com/paws with header

Accept: application/vnd.paws.init.response-v2+json;application/vnd.paws.init.response-v1+json

Database (let's say it only supports v1) responds with a application/paws.init.response-v1+json message which includes the details in 4.2.2 of the draft -- with, say a cache-control: max-age=86400 header -- but contain an additional field: spectrumResource: {acceptable-content-type: application/paws.getspectrum.request-v1+json; required-fields: device/fccId; uri: http://example.com/paws/available-spectrum}

Device POSTs application/paws.getspectrum.request-v1+json a message with the FCC id to http://example.com/paws/available-spectrum and header

Accept: application/vnd.paws.notifyuse-v1+json

Database response with an application/paws.getspectrum.response-v1+json response containing the details specified in 4.4.2 with an additional field (if it is required for spectrum use to be notified by the device): notifyUseToResource {acceptable-content-type: application/vnd.paws.notifyuse-v1+json, uri: http://example.com/paws/used-spectrum}.

Device POSTs the selected frequency ranges to the URL and is done.

In this way, the device only needs to know how to negotiate and handle the content types and be configured with the initial URL of the service. Some interesting benefits:

* The device can connect to the database to ask about any location and -- if the original database doesn't support a given country -- be transparently directed in the response to another database to file the get spectrum request. It's feasible for any database to federate with other databases and provide seamless database discovery anywhere whitespace databases are in use (though this doesn't speak to the protocol by which the databases would discover each other)

* At each POST, the device declares the response version it can handle which indicates to the server that it can safely use that version and assume it will be handled by the client according to that particular version of the spec. Further, it allows the client and server to easily agree on an upgraded version using the HTTP headers alone without needing to design such a feature into the protocol itself.

* There's a very natural mapping to a web-browser accessible site for viewing/querying the database contents: visit http://example.com/paws in a browser, and it sends back a page with a map allowing the user to pick a location. URLs like http://example.com/paws/available-spectrum/latitude=45N/longitude=75W might be bookmarked for repeated viewings for a particular location.

* The initialization response could be augmented with ruleset definitions that could be cachable, GETtable resources that (perhaps in a future version of the spec when the breadth of rulesets is better known) allow devices to seamlessly upgrade their rulesets from the database.

--
----------------------------------
Michael R Head <[email-masked]<mailto:[email-masked]>>
http://www.cs.binghamton.edu/~mike
+1-201-BLISTER

_______________________________________________
paws mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/paws

--
----------------------------------
Michael R Head <[email-masked]<mailto:[email-masked]>>
http://www.cs.binghamton.edu/~mike
+1-201-BLISTER