Subject: [mmox] xxSD working group?

From: Catherine Pfeffer <[email-masked]>
Date: Tue, 24 Feb 2009 15:40:58 +0100

Infinity wrote:
> Catherine... Jon...
>
> can you create a proposal for a next generation XML serialization?

Yes I can.

> the existing serialization is in current use by SL, OpenSim and PyOGP,

Really? I am surprised to hear that, because I had once the curiosity to run
Wireshark to sniff packets exchanged by my Second Life client with the
simulators, and I saw that the binary serialization was used all the time,
even when using the new capacities system. Could you please point me to a
situation where I could test the existing XML serialization?

> so there's probably going to be a lot of resistance to changing
> something that currently works and is deployed.

Infinity, we need to make once for all clear whether this is just a Linden
Lab existing technologies rubber stamping work group, or if we are designing
vendor independent protocols for the 3 dimensional web of the future.

While I agree there is no reason to throw away existing stuff (like LLSD's
simplicity) when it is cool, I think that every bit of the technology should
be evaluated, and replaced when it has conceptual flaws in it.

If you disagree with this position, please state it clearly, and I would be
happy to withdraw from this mailing list and to go to other projects where I
might be more useful. Similarly, if Linden Lab's position is "let them speak
and then enforce our own stuff at the end", please also spare my time.

> keep in mind, however, that OGP does not use XML to represent it's
> PDUs, it uses LLSD. XML is one of three defined serializations of
> LLSD. it's a subtle difference, but important.

I have it in mind, even though the rationale for using three different
serializations is far from being clear to me.

> for instance... is there a benefit to explicitly adding support for
> namespaces to the XML serialization?

That's a very good question. What would be the purposes of such namespaces?

> there was a bit of discussion
> about this amongst some AWG members, and the consensus was... "why
> bother? the XML (presentation layer) is not the place where you want
> to extend the PDU... you want to extend it in the LLSD / LLIDL
> definition of the interaction."

Makes sense.

But this reasoning apparently assumes that the namespaces would be used to
extend the interoperability standard.

Normally, namespaces are rather used to mix data from different standards.

That could be *very useful* if we want to mix LLSD for example with a mesh
description expressed in another standard. Or to be able to include SVG
graphics. Etc. While this would be very powerful, that would not scale back
to the two other serializations, because it's pure XML mechanisms.

(That's not the only point where needing support for three serializations
has drawbacks. For example, LLIDL would become unnecessary if using only
XML, because the names and types of the fields could be stored in schema or
DTD fragments, or even in a WSDL dictionary. But never mind.)

> but, i still think it's an interesting idea as it might allow systems
> that use XML serialization exclusively to extend display and
> distribution options of various PDUs.

-- 
Cathy

---

From: Christian Scholz <[email-masked]>
Date: Tue, 24 Feb 2009 15:57:30 +0100

Catherine Pfeffer wrote:
> Really? I am surprised to hear that, because I had once the curiosity to
> run Wireshark to sniff packets exchanged by my Second Life client with
> the simulators, and I saw that the binary serialization was used all the
> time, even when using the new capacities system. Could you please point
> me to a situation where I could test the existing XML serialization?
> 
> Infinity, we need to make once for all clear whether this is just a
> Linden Lab existing technologies rubber stamping work group, or if we
> are designing vendor independent protocols for the 3 dimensional web of
> the future.
> 
> While I agree there is no reason to throw away existing stuff (like
> LLSD's simplicity) when it is cool, I think that every bit of the
> technology should be evaluated, and replaced when it has conceptual
> flaws in it.

I also wonder if we not first should think about a general architecture
and then check all the fields if we can use existing formats or not and
if not, what the requirements are our inventions need to fulfil.

> If you disagree with this position, please state it clearly, and I would
> be happy to withdraw from this mailing list and to go to other projects
> where I might be more useful. Similarly, if Linden Lab's position is
> "let them speak and then enforce our own stuff at the end", please also
> spare my time.
> 

I also wonder how much the "normal" programmer will understand this.
Most likely they will skip the LLSD step and directly output the XML or
parse that. It might not be good but if this thing gets widespread it
will certainly happen.

> That's a very good question. What would be the purposes of such namespaces?

I wonder this, too. Especially if this is just a serialization layer.
What do you do with namespaces if you deserialize to LLSD? What happens
to them if you then serialize again into a different format?

Maybe some XML representation is really all which is needed. One might
also look at OpenSocial where they have two formats, ATOM and JSON and
rules for translating one to the other. Maybe even ATOM is usable as a
base here.

Also remember that adding new concepts/terms/names to something will
make it more difficulty to understand. If it's just XML or JSON or
whatever is in use already, it's easier to understand and it might get
more adoption.

> But this reasoning apparently assumes that the namespaces would be used
> to extend the interoperability standard.
> 

Well, I know a lot of places where it's used to add additional
attributes etc.

> That could be *very useful* if we want to mix LLSD for example with a
> mesh description expressed in another standard. Or to be able to include
> SVG graphics. Etc. While this would be very powerful, that would not
> scale back to the two other serializations, because it's pure XML
> mechanisms.

Right. But you might say that such a mesh is stored in one field (tag
then) and you can simply copy the whole XML part over to this field in
e.g. a JSON structure. But that might be somewhat messy ;-)

But actually I really would first think about a more global approach
towards a general architecture and then defined the more details
protocols and formats needed.

-- Christian

-- 
Christian Scholz                          Homepage: http://comlounge.net
COM.lounge                                   blog: http://mrtopf.de/blog
Luetticher Strasse 10                                    Skype: HerrTopf
52064 Aachen                             Video Blog: http://comlounge.tv
Tel: +49 241 400 730 0                           E-Mail [email-masked]
Fax: +49 241 979 00 850                               IRC: MrTopf, Tao_T

neuer Podcast: Der OpenWeb-Podcast (http://openweb-podcast.de)
new podcast: Data Without Borders (http://datawithoutborders.net)

---

From: "Meadhbh Hamrick (Infinity)" <[email-masked]>
Date: Tue, 24 Feb 2009 09:13:30 -0800

1. can you please submit your proposal?

2. yes, LLSD is used extensively in the currently deployed viewer and  
server. here are a few references to code:

	https://svn.secondlife.com/svn/linden/branches/2008/Branch_1-20-Viewer/indra/llcommon/llsd.cpp 
  (the linden C++ implementation)
	https://svn.secondlife.com/svn/linden/branches/2008/Branch_1-20-Viewer/indra/llcommon/llsd.h 
  (the linden C++ interface)
	http://svn.secondlife.com/trac/linden/browser/release/indra/lib/python/indra/base/llsd.py?rev=170&format=txt 
  (the linden python implementation)
	http://www.openmetaverse.org/svn/libsl/libopenmetaverse/tags/0.3.2.2/libsecondlife/LLSD/XmlLLSD.cs 
  (libsecondlife's XML LLSD serialization file)

what you're seeing that looks like binary encoding is actually our  
"gMessage system" (aka "that pile of UDP packets we don't want to get  
standardized cause the last thing the internet needs is another half  
implemented version of TFTP.) most of the XML LLSD is sent over HTTPS,  
which might explain why you can't see it in wireshark. (yes, it  
irritates me too... but then again, it's not too hard to run wireshark  
as root, have it open /dev/mem, walk the process space, find the  
OpenSSL variables, tease out the private (or session) key and decrypt  
the SSL. if you try that, i think you'll see a lot more XML.) and the  
XML serialized LLSD that is not going over HTTPS is generally sent  
compressed, so again, if you're not uncompressing it before you look  
at it, you won't see angle-brackets.

3. the charter of this list is rather constrained... it is to attempt  
to standardize a protocol that is in wide use (for some definition of  
the term "wide".) it has yet been altered to be "think about a general  
architecture for virtual worlds." don't get me wrong, this is a very  
good idea (thinking about general architectures and so forth.) but you  
might have a MUCH better audience with the VWR and MPEG-V.

4. normal programmers understand LLSD, abstraction, and separating the  
internal representation of a data structure from it's serialization.  
do a google search for "rpc", "xml-rpc", "iiop", "dbus" and  
"marshalling".

5. namespace support for LLSD XML serialization isn't what i would  
call "a big win," but i can imagine a use where someone might want to  
extend the LLSD XML serialization DTD / schema to include additional  
information like ("don't ever let this serialized blob out of  
example.com's domain" or "if this blob goes out of this domain, please  
apply this XSLT stylesheet to it first.") For us at LL, these aren't  
"big wins," but i brought it up cause if there's a strong need for it,  
we could add an "improved" XML serialization to the list of  
serialization techniques, and i would undergo the ritual vegetable  
pelting that Linden employees subject themselves to when they  
volunteer other groups to support their "crazy" ideas.

6. the reason we separate the serialization from the protocol  
definition is we want to avoid having to add "binary serialization",  
"json serialization" and "xml serialization" sections to each defined  
protocol interaction. LLSD serialization was conceived as a way to  
avoid that. i think there's a history of people defining protocols  
using relatively "high level" abstractions, then providing mechanistic  
technique for generating the "octets over the wire." ASN.1 / BER  
encoding is an example, *shudder* CORBA / IIOP is another (though i'm  
definitely not recommending either for use here.) so... to recap... if  
you define your protocol interactions in a "high level" abstraction  
like LLSD/LLIDL, then have a separate section that says "how to  
convert LLSD blobs into {binary | json | xml} octets over the wire",  
you don't have to keep repeating yourself in each and every section.  
this limits the size of the spec, and may actually lead to fewer  
errors if you believe in murphy's law. it also makes the interaction  
more understandable, IMHO, because you're dealing with an appropriate  
level of abstraction. honestly, i could care less about XML when i'm  
thinking about application layer interaction.

7. i really like the idea of using LLSD to represent meshes, link-sets  
of prims, or even 2d SVG representations of avatars and objects.

On Feb 24, 2009, at 6:57 AM, Christian Scholz wrote:

> _______________________________________________
> mmox mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmox

---

From: Jon Watte <[email-masked]>
Date: Tue, 24 Feb 2009 12:21:40 -0800

Meadhbh Hamrick (Infinity) wrote:
> the term "wide".) it 

I thought we all agreed that the charter was *not* to standardize any 
particular protocol currently in use, because that's an explicit 
technology bias.
If that is actually the charter, then I know that a lot of people on 
this list will have to withdraw.

Sincerely,

jw

---

From: Lawson English <[email-masked]>
Date: Tue, 24 Feb 2009 14:01:40 -0700

> this list will have to withdraw.
Does the IETF make up new protocols from scratch, generally? Thought 
"working/implemented code" was part of the deal.

L

---

From: Jon Watte <[email-masked]>
Date: Tue, 24 Feb 2009 13:20:33 -0800

> "working/implemented code" was part of the deal.

Sounds like I should formally propose our internal interoperability 
protocol, then. I'd rather not propose our implementation details as a 
standard, but instead work towards something that is more vendor 
neutral, though.

Sincerely,

jw