More musings on XDM 3.0

Note: these are musings and should not be taken as a concrete proposal!

Balisage’s tag line is « there is nothing so practical as a good theory ». I love Balisage but my brain doesn’t really work like that and its tag line would rather be « there is nothing as theoretical as a good exercise ».

χίμαιραλ (chimeral) is the kind of exercise I needed to get into XDM 3.0 (and probably even 2.0 that I had been using without really thinking about it) and I’d like to share the musings that this exercise has inspired.

For XDM, the most generic piece of information is called an « item ».

XDM 2.0 has introduced a distinction between two kind of items:

• Nodes which come directly from the XML infoset and borrow some properties to the PSVI.
• Atomic values that are « pure » values living their own lives independently of their occurrences in XML documents as text or attribute node values.

Coming straight from the XML infoset, nodes are rather concrete, and you can still smell the (electronic) ink of their tags… Each node is unique and lives in its own context: when you write:

<?xml version="1.0" encoding="UTF-8"?>
<root>
  <foo>5</foo>
  <foo>5</foo>
</root>

The two <foo/> elements are two different nodes. They may look identical and have the same content, they are still two different nodes like identical twins are different persons.

Their value, by contrast, is the same value: a 5 is a 5, the fact that it’s the value of the first or of the second <foo/> (or of a @bar attribute) doesn’t make any difference.

XDM 2.0 also introduced the notion of « sequences ». Sequences are really special. They are not considered as a model item and are just a kind of invisible bag to package items together and are useful because you can store them in variables and use them as parameters.

Sequences have some interesting magics…

They disappear by themselves when they pack only one item and their is no way to differentiate a sequence of one item from the item itself. XDM has invented a perfectly biodegradable bag!

They also disappear when you put them within another sequence and you can’t have sequences of sequences. You’d better think twice if you want to separate apple from oranges (or meat and cheese) before packaging them in a sequence!

<xsl:variable name="nodeset" select="/root/foo"/>

<xsl:variable name="rtf">
  <xsl:copy-of select="/root/foo"/>
</xsl:variable>

XDM 3.0 adds a third type of items: functions that become first class objects transforming XPath/XQuery/XSLT into functional languages.

Michael Kay’s proposal adds as a fourth item type: maps (and arrays considered as maps with integer keys). Maps being a brand new type of item, the choice is open: they could have been considered similar to XML nodes but the current proposal is to consider them as pure values.

In my previous post I have explained the consequences of this important design decision and I’d like to take a step backward and analyze the differences between values and nodes.

Still before discovering Fortran IV, I did study mathematics and geometry and I was fascinated by the dual approach for solving problem in geometry using either Euclidean vectors or points and segments.

In the current proposal, a map is like a vector: you can reuse it as many time as you like in other map entries and it will remain unique. On the contrary, a node is like a segment. It has an identity,  is bound to a parent and if you want to reuse it you need to copy and paste it.

In geometry, vectors and segments are both useful and complementary. I can understand that there is a need for both nodes and values, but why should maps always be values and nodes always be « concrete » nodes? Both are useful data structures, why should they be treated so differently?

I understand that there are (at least) to use cases for maps: to support JSON and to provide lightweight data structures. These two use cases look very different to me. Wanting to meet them both with a single feature, won’t we miss both points? Why should lightweight structures be limited to maps and why should maps always been lightweight?

In geometry the processes by which you create a vector from a segment or a segment from a vector by pinning one of its extremities are well known. Can’t we define similar processes to transform a « concrete » data structure (either node tree or map) into a pure value and vice versa?

This is not so uncommon for programmers: values can also be seen as classes (with class properties) and concrete structures as instantiations of these classes.

Now, what about chimeras mixing pure values and concrete nodes?

A handy feature of sequences and maps as currently proposed is that they can include nodes. This « inclusion » is done « by reference » and the nodes keep they identity and context. Maps using this feature are « pure values » with references to concrete nodes.

When a map entry is a node, the node keeps its context within a document and its parent remains its parent in this document. How can you do that if you want to also represent the reverse relation between the node and the map entry in which it has been added?

One option could be to define a mechanism similar to symbolic links on Linux: the link is a kind of shortcut and when you follow it you can’t always guarantee that you’ll be able to come back in the same directory. Adapted to the XDM, we wouldn’t store the parent/child relation between map entries and nodes. This would be a limitation, but we wouldn’t meet this limitation when dealing with maps de-serialized from JSON objects (JSON objects do not contain XML nodes).

Another option could be to « decorate » or « re-instantiate » the node so that the new instance has two parents, probably not with the same kind of parent/child relation but with a new kind that could be followed with a different axis. This decoration  would add new context information to a tree and would be very similar to the process by which values are turned into concrete nodes. Now, would that be a practical thing to do? What about adding a map with a node as a new entry on another map? Don’t we end up with nodes that can contain an indefinite number of ancestors?

I hope that these musings can be helpful, but I should probably stick to my role of XDM user rather than giving suggestions!

To summarize: I think that we need both lightweight map structures and the full set of XPath axis on maps de-serialized from JSON objects.

Having only lightweight map structures means that users (and probably other specs and technologies) will have to continue to define custom mappings between JSON and XML to perform serious work.