Subject: [T2TRG] 2. WoT Thing needs to have istate/ostate Bands

From: David Janes <[email-masked]>
Date: Wed, 27 Jan 2016 04:33:01 -0500

****
Formatted Version:
https://github.com/dpjanes/homestar-plugfest/blob/master/docs/2016-01%20W3C%20IETF/2.%20WoT%20Thing%20needs%20to%20have%20istate:ostate%20Bands.md
****

2. WoT Thing needs to have istate/ostate Bands

Points:

   - WeMo Model
   - Interstitial State
   - What do the properties return?

<https://github.com/dpjanes/homestar-plugfest/blob/master/docs/2016-01%20W3C%20IETF/2.%20WoT%20Thing%20needs%20to%20have%20istate:ostate%20Bands.md#wemo>
WeMo

Consider the WeMo Switch, a popular "stater" IoT device. Turning a light
off to on looks like this.

Stages:

   1. the switch is *off*
   2. the user sends *turn on* command / the WeMo receives it
   3. about 300ms (guestimate) passes
   4. the switch is *on*

It's step (3) we're considering.
<https://github.com/dpjanes/homestar-plugfest/blob/master/docs/2016-01%20W3C%20IETF/2.%20WoT%20Thing%20needs%20to%20have%20istate:ostate%20Bands.md#wot-thing>WoT
Thing

So let's look at this as a WoT property manipulation. Note that I'm just
pseudocoding this, I don't know if there's a property called exactly "on"
but I'm sure there's something like it.

thing.getProperty("on")         ## (stage 1) a promise that yields false
thing.setProperty("on", true)   ## (stage 2)
thing.getProperty("on")     ## (stage 3) WHAAA?
thing.getProperty("on")     ## (stage 4)

<https://github.com/dpjanes/homestar-plugfest/blob/master/docs/2016-01%20W3C%20IETF/2.%20WoT%20Thing%20needs%20to%20have%20istate:ostate%20Bands.md#the-contradiction>The
contradiction

It's Stage 3 that's the concern. We don't have a clean way of knowing that
we're turning on the light, but the light is currently off.

This is called the "Interstitial State" and it's important that this is
indicated in the UI.

Now, the immediate objection might be "Well, thing.setProperty('on', true)"
returns a Promise that will tell when the the light turns on. But *this is
not sufficient* because *thing.getProperty("on")* might be called *somewhere
else in the code* or *by someone else entirely in a different process*
<https://github.com/dpjanes/homestar-plugfest/blob/master/docs/2016-01%20W3C%20IETF/2.%20WoT%20Thing%20needs%20to%20have%20istate:ostate%20Bands.md#the-solution>The
solution

Break up the properties into two states:

   - the *istate*, the "input state", the actual state of the thing
   - the *ostate*, the "output state", the state we're trying to transition
   to.

These are independent

Now our code looks like this

thing.istate.getProperty("on")      ## (stage 1) a promise that yields false
thing.istate.setProperty("on", true)    ## (stage 2)
thing.istate.getProperty("on")      ## (stage 3) FALSE
thing.ostate.getProperty("on")      ## (stage 3) TRUE
thing.istate.getProperty("on")      ## (stage 4)

Or even more simply if we make the assumption that the istate and ostate
are basically views of the properties

thing.istate.get("on")      ## (stage 1) a promise that yields false
thing.istate.set("on", true)    ## (stage 2)
thing.istate.get("on")      ## (stage 3) FALSE
thing.ostate.get("on")      ## (stage 3) TRUE
thing.istate.get("on")      ## (stage 4)
thing.ostate.get("on")      ## (stage 4) NULL - "nothing happening"

---

From: Kerry Lynn <[email-masked]>
Date: Wed, 27 Jan 2016 11:56:24 -0500

David,

I think you're hitting on something important with the notion of
"interstitial state",
but I wonder if it's a special case of a more general concept - the
freshness
(or fidelity?) of istate?

I've seen systems that assign timestamps to istate and ostate.  If a command
is issued through the API then ostate timestamp is modified.  Reading istate
through the API returns ostate and istate timestamps.  If the relation
istate_ts
< ostate_ts is true then the returned istate is unreliable.

There is also the problem of devices that can be manually as well as
program-
matically controlled.  To detect an out-of-band state change, the device
either
needs to push the new state to the authoritative data store, be observed by
the data store, or be polled by the data store.

Observation exists in some transports but not others and push requires
additional configuration, therefore polling seems like the lowest common
denominator.  Polling raises the question "how frequently should I poll?"

These questions seem to touch on the philosophy of state transfer in REST.
The "real" state resides in the device and any representation of that state
that resides elsewhere would seem to be stale within some epsilon.

Kerry

---

From: Michael Koster <[email-masked]>
Date: Thu, 28 Jan 2016 02:35:26 +0700

" These questions seem to touch on the philosophy of state transfer in REST.
The "real" state resides in the device and any representation of that state
that resides elsewhere would seem to be stale within some epsilon. "

Kerry has it right, but let me elaborate some. Assuming REST properties
as defined by Fielding in The Document for the purpose of this discussion.

http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm <http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm>

It seems like we're trying to add a notion of temporality to a state machine, or 
more likely confusing the delays between state transitions with meta-states in
the server.

The state machine is the REST interaction between client and server.

The client applies a method to the server, the server changes state, the client
receives a resolution of the Promise indicating the new state.

Between these actions, there are many delays, like network transmission
and buffering, the server processing time, time waiting for resources in the
client or server, then there could be network switches, satellite links...
It's the internet.

These delays aren't part of the state machine, and they are occurring in
the signals between the state transition. The server doesn't know when 
the client initiated the request with the deferred operation. Only the 
client has a meta-state while it is waiting for the Promise to resolve.

The rest of the system is not in a meta-state. The resource has no meta-state.
Resource state will either be observed in the "before" state or the "after" state,
or some other state from a request from some other client. A resource in 
REST has exactly one state.

Does a client observing the state of a resource need to know that some other 
client's request may have reached the sever first? No, that is not the nature of
REST. REST has no global consistency model. 

If operations from >1 client need to be serialized in some global consistency 
model, another serialization control needs to be added to the system. This
will most assuredly break the scalability property of REST.

Observing some intended state that may or may not ever be observed in
the resource state doesn't seem to add anything to REST.

Long running actions are different. Long running actions are cases where the 
resource state may change multiple times during the processing of the 
request. You may be tempted to call this a meta-state but since the state
observed by each client is a separate self-contained representation and 
we don't try to introduce a global consisteny model again, it still seems 
to satisfy the REST constraints as defined by Fielding.

The design of WoT.Action is to allow the server to make multiple state changes 
to a request and transmit those as needed in representations as needed. Another
client may observe these state transitions and may be allowed to add another 
action if those are still in progress. A client state machine may also be advanced
by intermediate states in the server as it executes the action. If we want to use
Deferred/Promise pattern for this we need to add something between request
and resolution of the promise. An update or multiple fulfillments or something.

Note that from the client's point of view, a long running Action is still a state 
machine. No temporality is assumed. It's no different from e.g. CoAP Observe.

So I would conclude that IoT using REST does not benefit by adding an istate/osate
construct.

Best regards,

Michael

> On Jan 27, 2016, at 11:56 PM, Kerry Lynn <[email-masked]> wrote:
> 
> David,
> 
> I think you're hitting on something important with the notion of "interstitial state",
> but I wonder if it's a special case of a more general concept - the freshness
> through the API returns ostate and istate timestamps.  If the relation istate_ts
> There is also the problem of devices that can be manually as well as program-
> matically controlled.  To detect an out-of-band state change, the device either
> _______________________________________________
> T2TRG mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/t2trg

---

From: David Janes <[email-masked]>
Date: Wed, 27 Jan 2016 15:25:57 -0500

(1)

There could be a more general concept under istate/ostate, but in all my
experience with Things I haven't seen it. So it easily makes the 80/20 cut,
and probably even the 99.9/0.1 cut.

If there's need for really more detailed analytics on what's happening, I
suggest other approaches be taken altogether, e.g. Actions. But as written
about elsewhere, I don't believe this is necessary

(2)

I do use timestamps to coordinate what's happening with istate / ostate.
You can see this here

http://homestar.io:20000/api/things/urn:iotdb:thing:REST:6acd18449fe571d3a1d12ef66aa443dd:rest-dimmer-light/istate
http://homestar.io:20000/api/things/urn:iotdb:thing:REST:6acd18449fe571d3a1d12ef66aa443dd:rest-dimmer-light/ostate

(3)

I don't believe necessarily "unreliable" is the right word. For example,
the light could be reported as on and the light could actually be on: the
command just hasn't executed yet, for whatever reason. Once again I will
note that this is not a theoretical concern: it happens all the time and
IoT UX people talk about dealing with it.

Let's say it's in a "transitional" state.

(4)

I don't think polling is necessary on modern devices. I do do it on some
legacy devices, but only at the 30s level.

Notifications that the state has changed can be sent via MQTT or CoAP. For
example, on my reference server you can do

$ mqtt subscribe --host 184.107.137.235 --port 20001 --topic '#' --verbose
$ curl --location '
http://homestar.io:20000/api/things/urn:iotdb:thing:REST:20b2c3ca712e8b0031debf3453cc75d1:rest-color-light/ostate
' -H 'Content-Type: application/json' -X PUT -d '{ "on": true }'

And watch the transitions happen (assuming the light is off).

(5)

MQTT is quite reliable. If you have a couple of computers running nearby,
play with the interface here. All the interfaces (dynamically made via
introspection!) will update as you transition state.

http://homestar.io:20000/

It's not quite as exciting as it could because the device simulators I
wrote were written before I really got a good sense how real world things
work, and the state transitions immediately.

(6)

Now that I've played with CoAP I'm quite excited by the possibilities of a
technology that will do this "all in one"

D.

---

From: David Janes <[email-masked]>
Date: Wed, 27 Jan 2016 15:55:36 -0500

(1)

My contention is that a real physical Thing will have multiple states which
are very closely modelled by "istate" / "ostate". Furthermore, these
provide a simple, elegant and understandable representation of what's going
on in the Thing.

(2)

A Thing doesn't correspond correspond to a single document / "resource".
There's a bunch of things that are basically orthogonal to each other (e.g.
istate / ostate / meta / model) and can be cleanly thought of as having
their own URLs, as being separate resources.

(3)

Yes, you can assume that there's only a single state (corresponding to the
ostate) with a Thing and everything else involves applying Actions to a
Thing. At the cost of complexity on the server, loss of reliability (when
do you get rid of Action state?), and additional work for consumers of the
state (I now have to start observing _other people's actions_).

If you look at e.g. the Hue, or the WeMo, or TCP Lighting, or a Denon AVR,
or KNX, or the Sonos, or the Chromecast, or any other thing I've actually
picked up and work with: they all do stuff by (sometimes in odd ways I
admit) PUT some value to some place, not "create an Action, and track it
through to completion".

SmartThing's device (code) API works by "find the device, call a function
(e.g. PUT), time passes, a event is triggered, and you read (GET) the new
state". Right here:
https://github.com/dpjanes/iotdb-smartthings/blob/master/smartthings.groovy

(5)

There is a lot of danger in creating a  _prescriptive_ IoT standard, e.g.
"here's what you have to do", not least of all that there's so many of them
right now.

I'm a strong proponent of "let's describe what Things actually do" and try
to create interoperability and compatibility from there, e.g. _descriptive_
standard.

Regards,
D.

On Wed, Jan 27, 2016 at 2:35 PM, Michael Koster <
[email-masked]> wrote:

> http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
>

---

From: Michael Richardson <[email-masked]>
Date: Thu, 28 Jan 2016 15:12:07 -0500

David Janes <[email-masked]> wrote:
    > 1 the switch is off
    > 2 the user sends turn on command / the WeMo receives it
    > 3 about 300ms (guestimate) passes
    > 4 the switch is on

    > WoT Thing

    > pseudocoding this, I don't know if there's a property called exactly "on" but
    > I'm sure there's something like it.

    > The contradiction

    > This is called the "Interstitial State" and it's important that this is

okay... so the RESTful analysis would tend to look at the problem
differently.  (The example I recall was about rebooting a virtual machine)

One would do:
    POST /light/XXX      state: on
one would receive a 201 Location: /light/XXX/activity/YYY

A GET to /light/XXX/activity/YYY  would return the Interstitial State, with
the current value retured, and possibly some other status, such as maybe,
some kind of encoding: "power is on, but no current flowing, is bulb probably absent?"

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: David Janes <[email-masked]>
Date: Thu, 28 Jan 2016 20:10:00 -0500

Well this is interesting and got me thinking

(1)

First, I'd like to make sure it's clear that I think we should not be
thinking "the state of a Thing is the Thing". Rather, the Thing is its
state(s), its model, and meta; which all function together but orthogonal

(2)

So (I am arguing)

   POST /light/1 { "on": true }
   PUT /light/1 { "on": true }

Are better expressed as

   POST /light/1/ostate { "on": true }
   PUT /light/1/ostate { "on": true }

Because you might want to also work with /light/1/model and /light/1/meta

Don't get caught up in me using "ostate" here - this will become clear why
below. Also if you believe POST should be on /light/1 I'm not overly caught
up about it

(3)

So let's examine the POST

   POST /light/1/ostate { "on": true }

which returns some new thing (in Michael's argument)

   /light/1/activity/X

The reason I've been arguing against this is because:

   - it doesn't model usually what happens in things, which is much simpler
   - everyone has to start looking for activities if they want to see if
   the light is in an interstitial state
   - the server has no idea when to dispose of the
   resource /light/1/activity/X
   - it's more complicated to implement

(4)

Here's my AHA moment. What is?

   /light/1/activity/123

*** it's _an_ ostate band ***, specific to the invoked POST.

(5)

So now this gives us an interesting unification

   POST /light/1/ostate { "on": true }

can redirect to the resource

   /light/1/ostate/123

or just (if the server doesn't want to be smart and just track all the
actions at once, which is IMO the usual case)

   /light/1/ostate

Note that I don't feel like you have to add a layer of hierarchy
(/activity/( - the band model lets you add as much as you want below the
thing itself.

(6)

Then this returns the "actual" state of the thing

   GET /light/1/istate

This returns the _unity of all actions_, to detect interstitial state

   GET /light/1/ostate

This returns the ostate of some particular action (if used)

   GET /light/1/ostate/123

(7)

The IOTDB model for ostate completion works here: everything transitions to
null in the ostate when the action is complete. I.e.

    GET /light1/ostate/123
    { "on": null }

Says we're done.

(8)

As a big bonus, you no longer need to differentiate between Properties and
Actions. They are invoked the same way and are "dealers choice" as far as
the server.

D.

---

From: David Janes <[email-masked]>
Date: Thu, 28 Jan 2016 20:20:45 -0500

Hi John,

I am, thank you - and I've been tracking it for a while here!
https://iotdb.org/social/imadeit/tagged/hypercat

I think the kind of models / meta (etc) infrastructure I've been proposing
could easily be "HyperCat transported", and it's been on my mind to do this
with IOTDB for a while.

HyperCat doesn't semantically define terms for sensors, actuators, etc.
which also makes it dovetail nicely with things I've been proposing.

The reason I'm not necessarily saying "start with the HyperCat model" is I
think there's something much nicer / cleaner in terms of representation -
the band model - as a starting point.

D.

On Thu, Jan 28, 2016 at 5:33 PM, <[email-masked]> wrote:

> David,
>
>
>
> This is new to me and I have just checked out your Slideshare overview
> presentation – looks interesting.
>
>
>
> Are you aware of the Hypercat specification: a lightweight, JSON-based
> spec for describing IoT resources (actually could be used in principle to
> describe anything you care to ascribe a URI to). As you advocate, it is
> based on JSON and RESTful tech for low barrier to participation.
>
>
>
> Hypercat has a small mandatory core of metadata elements and can be
> arbitrarily extended as required with additional elements.
>
>
>
> If interested, take a look at: http://www.hypercat.io/standard.html
>
>
>
> There is also a serialisation of Hypercat in RDF.
>
>
>
> Regards,
>
> John Davies.
>
>
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *David
> Janes
> *Sent:* 27 January 2016 09:33
> *To:* [email-masked]; [email-masked]
> *Subject:* 2. WoT Thing needs to have istate/ostate Bands
>
>
>
> ****
> Formatted Version:
>
> Now, the immediate objection might be "Well, thing.setProperty('on',
> true)" returns a Promise that will tell when the the light turns on. But *this
> is not sufficient* because *thing.getProperty("on")* might be called *somewhere
> The solution
>    - the *ostate*, the "output state", the state we're trying to
>    transition to.
>
> These are independent
>

---

From: <[email-masked]>
Date: Thu, 28 Jan 2016 22:33:34 +0000

David,

This is new to me and I have just checked out your Slideshare overview presentation – looks interesting.

Are you aware of the Hypercat specification: a lightweight, JSON-based spec for describing IoT resources (actually could be used in principle to describe anything you care to ascribe a URI to). As you advocate, it is based on JSON and RESTful tech for low barrier to participation.

Hypercat has a small mandatory core of metadata elements and can be arbitrarily extended as required with additional elements.

Regards,
John Davies.

From: [email-masked] [mailto:[email-masked]] On Behalf Of David Janes
Sent: 27 January 2016 09:33
To: [email-masked]; [email-masked]
Subject: 2. WoT Thing needs to have istate/ostate Bands

****
Formatted Version:

Points:

  *   WeMo Model
  *   Interstitial State
  *   What do the properties return?

WeMo

Consider the WeMo Switch, a popular "stater" IoT device. Turning a light off to on looks like this.

Stages:

  1.  the switch is off
  2.  the user sends turn on command / the WeMo receives it
  3.  about 300ms (guestimate) passes
  4.  the switch is on

WoT Thing

So let's look at this as a WoT property manipulation. Note that I'm just pseudocoding this, I don't know if there's a property called exactly "on" but I'm sure there's something like it.

The contradiction

It's Stage 3 that's the concern. We don't have a clean way of knowing that we're turning on the light, but the light is currently off.

This is called the "Interstitial State" and it's important that this is indicated in the UI.

Now, the immediate objection might be "Well, thing.setProperty('on', true)" returns a Promise that will tell when the the light turns on. But this is not sufficient because thing.getProperty("on") might be called somewhere else in the code or by someone else entirely in a different process

The solution

  *   the istate, the "input state", the actual state of the thing
  *   the ostate, the "output state", the state we're trying to transition to.

These are independent

Or even more simply if we make the assumption that the istate and ostate are basically views of the properties