Subject: [mmox] <integer> tags

From: Catherine Pfeffer <[email-masked]>
Date: Wed, 25 Feb 2009 13:48:14 +0100

Infinity said:
> about sizes of ints as attributes.
>
> let me pose the question... what is the difference in terms of
> processing expectations if one sends
>
>  '<integer size="8">12</integer>'
>
> vs. sending
>
> '<integer size="32">12</integer>' ?

Honestly, having a <integer> tag is so ... unusual in the XML world, that I
won't even attempt to answer this detail question and the next ones. We
should simply abandon the idea of having <integer>, <float>, <string> and so
on in the data flow.

In a binary serialization, you need to say :  the next four bytes will be an
integer. Or the next 25 bytes will be a 25-bytes long string. Fine. But
there is *no* reason to do the same in an XML file. The angle brackets < >
or the quotes " " are delimiters. They tell you where the data starts and
where it ends, so there is no need to say it an integer or whatever else.

When you want to pass a key-value pair like
    Taper_X = 0.5
then if we want to stick to XML paradigm, it should be a
    ... Taper_X="0.5" ...
attribute of some prim-related tag (14 bytes), and not
           ...
          <key>Taper_X</key>
          <float>0.5</float>
          ...
in some <map> (36 bytes plus the whitespace).

The quotes " " tell where "0.5" starts and where it ends. The information
"Taper_X is a float" belongs to the XML schema or DTD, not to the data sent
over the wire.

What I am trying to say is that the current xxSD XML serialization might
look like XML, but it definitely belongs to another world from a conceptual
point of view. It's like Canada Dry, it looks like beer, but it's no beer.

What you have come up with is just a rewriting of the binary serialization
that is well-formed XML. That does not make it something that fits naturally
into XML:
 - LLIDL plays the same role as XML schemas or DTDs and could simply be
dropped.
 - currently, accessing the value of Taper_X from XPath is convoluted and
probably slow, compare:
   map/key[. = 'Taper_X']/following-sibling::float[position() = 1]  with:
   @Taper_X
 - I also already mentioned the higher (250%) bandwidth footprint
 - etc

What is the point of having a XML serialization just to have it "look like"
some XML, while it's not really XML?

OK, let's try to be positive (for once ;-) ):

I am currently wondering if there is not an automated way to map the LLSD
abstract view into a real XML-ish serialization. Perhaps that would make
everyone happy. That would keep the very high flexibility of LLSD, have a
lower angle bracket tax, and allow "natural" use of all XML tools. That
would keep the possibility of having three different serializations, even
though I still think that it's a false good idea.

The technical problem I see with this approach is that the current LLSD
draft does not recognize the special role played by the maps in storing
name-value pairs used by the upper layers. It does not either say that
arrays are made for vectors and quaternions, and so on. There is a lot of
semantics that are evacuated from this draft and that I only know for having
worked with SL client source code as an Open Source contributor. And I don't
see how I could specify such an "intelligent" XML serialization if such
context information is missing.

(I apologize in advance if this context information is in the LLSD draft and
I did not see it.)

-- 
Cathy

---

From: "Meadhbh Hamrick (Infinity)" <[email-masked]>
Date: Wed, 25 Feb 2009 09:33:29 -0800

so. again. let me reiterate the issues we have with "the XML way."

first off, XML is for us, not a first order representational system.  
we do this because:

	* we have an external requirement to expose a JSON interface and an  
internal requirement to export a Binary interface.
	* we have an internal requirement to not co-mingle application  
semantics and the presentation logic (this means we cannot have a  
system that requires us to deploy new XML to data structure parsers to  
our system components when new versions of PDUs are defined.)

so in short, we would not deploy a protocol that mandated we abandon  
these requirements.

that being said, i suspect at some point we will be discussing  
application layer objects (user, avatar, location, prim, texture,  
mesh, etc.) if there was an "XML way" PDU defined for each protocol  
interaction, i don't think we would care one whit as long as  
supporting it was not mandatory and application layer PDUs were  
defined in a way that allowed us to continue to use LLSD.

to paraphrase... we think the "XML way" does not buy us anything. we  
respect that other people might have a different preference or set of  
requirements related to XML. but you have to respect that we're not  
going to abandon a key portion of what makes our architecture  
resilient in the face of version slew. so why not just agree to  
disagree about it, allow virtual world deployers make up their own  
minds about which presentation tier to adopt.

i would also argue that if we could develop a mechanical scheme to  
convert between LLIDL and "the XML way" XML DTD, we could interoperate  
with such systems will less ambiguity.

On Feb 25, 2009, at 4:48 AM, Catherine Pfeffer wrote:

> Honestly, having a <integer> tag is so ... unusual in the XML world,  
> that I won't even attempt to answer this detail question and the  
> next ones. We should simply abandon the idea of having <integer>,  
> <float>, <string> and so on in the data flow.
>
> In a binary serialization, you need to say :  the next four bytes  
> will be an integer. Or the next 25 bytes will be a 25-bytes long  
> string. Fine. But there is *no* reason to do the same in an XML  
> file. The angle brackets < > or the quotes " " are delimiters. They  
> tell you where the data starts and where it ends, so there is no  
> need to say it an integer or whatever else.
>
> The quotes " " tell where "0.5" starts and where it ends. The  
> information "Taper_X is a float" belongs to the XML schema or DTD,  
> not to the data sent over the wire.
>
> What I am trying to say is that the current xxSD XML serialization  
> might look like XML, but it definitely belongs to another world from  
> a conceptual point of view. It's like Canada Dry, it looks like  
> beer, but it's no beer.
>
> What you have come up with is just a rewriting of the binary  
> serialization that is well-formed XML. That does not make it  
> something that fits naturally into XML:
>  - LLIDL plays the same role as XML schemas or DTDs and could simply  
> be dropped.
>  - currently, accessing the value of Taper_X from XPath is  
> convoluted and probably slow, compare:
> What is the point of having a XML serialization just to have it  
> "look like" some XML, while it's not really XML?
>
>
> I am currently wondering if there is not an automated way to map the  
> LLSD abstract view into a real XML-ish serialization. Perhaps that  
> would make everyone happy. That would keep the very high flexibility  
> of LLSD, have a lower angle bracket tax, and allow "natural" use of  
> all XML tools. That would keep the possibility of having three  
> different serializations, even though I still think that it's a  
> false good idea.
>
> The technical problem I see with this approach is that the current  
> LLSD draft does not recognize the special role played by the maps in  
> storing name-value pairs used by the upper layers. It does not  
> either say that arrays are made for vectors and quaternions, and so  
> on. There is a lot of semantics that are evacuated from this draft  
> and that I only know for having worked with SL client source code as  
> an Open Source contributor. And I don't see how I could specify such  
> an "intelligent" XML serialization if such context information is  
> missing.
>
> (I apologize in advance if this context information is in the LLSD  
> draft and I did not see it.)
>
> -- 
> Cathy
> _______________________________________________
> mmox mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmox

---

From: Morgaine <[email-masked]>
Date: Thu, 26 Feb 2009 00:17:10 +0000

> context information is missing.

Catherine, this is excellent advice that you're giving us about XML, but the
underlying hint to "do everything in XML to do it properly" is not a
worldview that I can subscribe to everywhere, for very practical and
somewhat insurmountable RL engineering reasons.

Using XML for communication with Limited Capability Clients can easily be a
deal breaker in the application domain that we're talking about, which is
characterized by very high rate streaming of object updates over limited
bandwidth.  Removing a few angle brackets and producing better XML is nice,
but a good binary serialization is orders of magnitude faster and at least
one order of magnitude more compact, so such improvements are not even in
the same ballpark.

Perhaps your advice could work at a higher level instead.  Replacing LLSD
and LLIDL with a pure XML-based abstraction layer would be fine in
principle, allowing us to use XML power tools to generate our programming
APIs.  However, at run time, the interface between wire API and programs
needs to be fast and tiny, both to save LCCs from collapse and to reduce the
load on servers, so binary serializations are more than slightly important.

Even power desktops working over broadband struggle with the object update
stream, and that is mostly binary-encoded at the moment in SL so it would be
even worse with XML.  But this is not an SL-only issue.  No systems anywhere
benefit from adding orders of magnitude of overhead.

The solution might be to use an XML-based IDL to generate an efficient
binary encoding, devoid of DTDs and without the performance-destroying
string comparisons that are completely unnecessary when the two endpoints
have agreed on an interchange spec.  Call it XML Protocol Buffers and you'd
be onto a winner. ;-)

Morgaine.

On Wed, Feb 25, 2009 at 12:48 PM, Catherine Pfeffer
<[email-masked]>wrote:

> In a binary serialization, you need to say :  the next four bytes will be
> an integer. Or the next 25 bytes will be a 25-bytes long string. Fine. But
> (I apologize in advance if this context information is in the LLSD draft
> and I did not see it.)
>
> --
> Cathy
>

---

From: Jon Watte <[email-masked]>
Date: Wed, 25 Feb 2009 16:55:13 -0800

Morgaine wrote:
> Even power desktops working over broadband struggle with the object 
> update stream, and that is mostly binary-encoded at the moment in SL 
> so it would be even worse with XML.  But this is not an SL-only 
> issue.  No systems anywhere benefit from adding orders of magnitude of 
> overhead.

Does the current binary encoding encode the actual names and types of 
properties before sending the values? If so, you could probably cut out 
at least half, if not more, using the separate schema method.

There are other ways to further compact the amount of data sent. In 
OLIVE, we use an input-synchronous, deterministic simulation model, 
which means we do about 9-12 users per T1 connection, including 
voice-over-IP (our min spec says 128 kilobits per user). I would perhaps 
submit this technology for standardization (with some IP disclosures), 
but I don't think it's actually appropriate for standardization, because 
it takes advantage of the fact that the client and the server run 
*exactly* the same simulations, which probably is not a great approach 
for an open standard ;-)

Sincerely,

jw