Catégorie : English

Yesterday, I had the opportunity to read more carefully the press release made back in January 2005 to announce QuarkXPress Markup Language (QXML).

My first guess, before clicking on the link, was that QXML would be an XML vocabulary.

Wrong guess!

QXML appears to be an XML schema of the World Wide Web Consortium (W3C) Document Object Model (DOM).

Although the press release doesn’t give a definition of this term (new to me), its benefits are detailed:

With QXML, the new DOM schema for QuarkXPress, developers can dynamically access and update the content, structure, and style of a QuarkXPress project using a DOM interface. XTensions modules can be more versatile because they can use a project’s complete content, including all formatting, style sheets, hyphenation, and justification specifications.

A PDF white paper further explains:

One of the goals of an open-standard DOM is to produce common methods for accessing, modifying, creating, and deleting content within a DOM schema. If you are familiar with one particular DOM, understanding and working with another DOM is easy because you are already familiar with the common methods and metaphors applicable to all DOMs. This commonality gives the DOM a distinct advantage over traditional C/C++ application programming interfaces (APIs).

I did quite a lot of searches both on the Quark’s web site and on the Internet and did not see any reference to the possibility of directly using the XML document or any description of the XML vocabulary itself.

To me, it looks like Quark is just missing the point of XML.

XML is about letting anyone read documents in a text editors and write them through print statements…

Whether you like it or not, you just can’t publish documents in XML and still constrain developers to use your own API, only available to your official partners on your company web site and working only on the platforms and languages that you support!

And since you can’t avoid that people use their XML format directly, you should rather document it…

See also:

• InDesign and XML: no better than its competition
• The transform source effect

There is a tendency, among XML gurus, to deprecate everything from the XML recommendation that is not element or attribute and XML constructions such as comments or processing instructions have been deprecated de facto by specifications such as W3C XML Schema that have reinvented their own element based replacements.

Many people also think that DTDs are an archaism that should be removed from the spec.

Lacking of SGML culture, I am not a big fan of DTDs, but there are cases where they can be very useful.

I came upon one of these cases this afternoon while implementing multi-level grouping in XSLT 1.0 following the so-called Muenchian method.

At the fourth level, I ended up with XPath expressions that looked like these ones:

        <xsl:when test="key('path4', concat(@name, '¤', ../path/step[1], '¤', ../path/step[2], '¤',
          ../path/step[3], '¤', ../path/step[4]))[../path/step[5]] ">
          <xs:complexType>
            <xsl:if test="key('path4', concat(@name, '¤', ../path/step[1], '¤', ../path/step[2],
              '¤', ../path/step[3], '¤', ../path/step[4]))[../path/step[5][starts-with(., '@')]]">
              <xsl:attribute name="mixed">true</xsl:attribute>
            </xsl:if>
            <xs:sequence>
              <xsl:apply-templates select="key('path4', concat(@name, '¤', ../path/step[1], '¤',
                ../path/step[2], '¤', ../path/step[3], '¤', ../path/step[4]))[ count( . |
                key('path5', concat(@name, '¤', ../path/step[1], '¤', ../path/step[2], '¤',
                ../path/step[3], '¤', ../path/step[4], '¤', ../path/step[4]))[1]  )
                = 1 ]" mode="path5"/>
            </xs:sequence>
            <xsl:apply-templates select="key('path4', concat(@name, '¤', ../path/step[1], '¤',
              ../path/step[2], '¤', ../path/step[3], '¤', ../path/step[4]))[ count( . | key('path5',
              concat(@name, '¤', ../path/step[1], '¤', ../path/step[2], '¤', ../path/step[3], '¤',
              ../path/step[4], '¤', ../path/step[4]))[1]  ) = 1                             ]"
              mode="path5Attributes"/>
          </xs:complexType>
        </xsl:when>

Isn’t it cute?

If you have to write such repetitive expressions, XML entities are your friends. Just write:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xsl:stylesheet [
<!ENTITY path1 "concat(@name, '¤', ../path/step[1])">
<!ENTITY path2 "concat(@name, '¤', ../path/step[1], '¤', ../path/step[2])">
<!ENTITY path3 "concat(@name, '¤', ../path/step[1], '¤', ../path/step[2], '¤', ../path/step[3])">
<!ENTITY path4 "concat(@name, '¤', ../path/step[1], '¤', ../path/step[2], '¤', ../path/step[3], '¤', ../path/step[4])">
<!ENTITY path5 "concat(@name, '¤', ../path/step[1], '¤', ../path/step[2], '¤', ../path/step[3], '¤', ../path/step[4], '¤', ../path/step[4])">
<!ENTITY kCase "key('case', @name)">
<!ENTITY kPath1 "key('path1', &path1;)">
<!ENTITY kPath2 "key('path2', &path2;)">
<!ENTITY kPath3 "key('path3', &path3;)">
<!ENTITY kPath4 "key('path4', &path4;)">
<!ENTITY kPath5 "key('path5', &path5;)">

]>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    version="1.0">
    <xsl:import href="excel2model.xsl"/>
    <xsl:output media-type="xml" indent="yes"/>
    <xsl:key name="case" match="case" use="@name"/>
    <xsl:key name="path1" match="case" use="&path1;"/>
    <xsl:key name="path2" match="case[../path/step[2]]" use="&path2;"/>
    <xsl:key name="path3" match="case[../path/step[3]]" use="&path3;"/>
    <xsl:key name="path4" match="case[../path/step[4]]" use="&path4;"/>
    <xsl:key name="path5" match="case[../path/step[5]]" use="&path5;"/>
            ...

And you’ll be able to simplify the previous snippet to:

                <xsl:when test="&kPath4;[../path/step[5]] ">
                    <xs:complexType>
                        <xsl:if test="&kPath4;[../path/step[5][starts-with(., '@')]]">
                            <xsl:attribute name="mixed">true</xsl:attribute>
                        </xsl:if>
                        <xs:sequence>
                            <xsl:apply-templates select="&kPath4;[ count( . | &kPath5;[1] = 1 ]"
                                  mode="path5"/>
                        </xs:sequence>
                        <xsl:apply-templates select="&kPath4;[ count( . | &kPath5;[1]  ) = 1]"
                                  mode="path5Attributes"/>
                    </xs:complexType>
            </xsl:when>

Doesn’t that look better?

Spreadsheet tables are full of holes and spreadsheet processors such as OpenOffice or Excel have implemented hacks to avoid having to store empty cells.

In the case of Excel, that’s done using ss:Index and ss:MergeAcross attributes.

While these attributes are easy enough to understand, they add a great deal of complexity to XSLT transformations that need to access to a specific cell since you can’t any longer index directly your target.

The traditional way to work around this kind of issue is to pre-process your spreadsheet document to get an intermediary result that lets you index your target cells.

Having already encountered this issue with OpenOffice, I needed something to do the same with Excel when Google led me to a blog entry proposing a transformation something similar.

The transformation needed some adaptation to be usable as I wanted to use it, ie as a transformation that does not modify your SpreadsheetML document except for inserting an ss:Index attribute to every cell.

Here is the result of this adaptation:

This version is buggy. An updated one is available here

<?xml version="1.0"?>
<!--

Adapted from http://ewbi.blogs.com/develops/2004/12/normalize_excel.html

This product may incorporate intellectual property owned by Microsoft Corporation. The terms
and conditions upon which Microsoft is licensing such intellectual property may be found at
http://msdn.microsoft.com/library/en-us/odcXMLRef/html/odcXMLRefLegalNotice.asp.
-->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns="urn:schemas-microsoft-com:office:spreadsheet"
    xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet">
    <xsl:output method="xml" indent="no" encoding="UTF-8"/>
    <xsl:template match="@*|node()">
        <xsl:copy>
            <xsl:apply-templates select="@*|node()"/>
        </xsl:copy>
    </xsl:template>
    <xsl:template match="ss:Cell/@ss:Index"/>
    <xsl:template match="ss:Cell">
        <xsl:copy>
            <xsl:apply-templates select="@*"/>
            <xsl:variable name="prevCells" select="preceding-sibling::ss:Cell"/>
            <xsl:attribute name="ss:Index">
                <xsl:choose>
                    <xsl:when test="@ss:Index">
                        <xsl:value-of select="@ss:Index"/>
                    </xsl:when>
                    <xsl:when test="count($prevCells) = 0">
                        <xsl:value-of select="1"/>
                    </xsl:when>
                    <xsl:when test="count($prevCells[@ss:Index]) > 0">
                        <xsl:value-of select="($prevCells[@ss:Index][1]/@ss:Index) +
                            ((count($prevCells) + 1) -
                            (count($prevCells[@ss:Index][1]/preceding-sibling::ss:Cell)
                            + 1)) + sum($prevCells/@ss:MergeAcross)"/>
                    </xsl:when>
                    <xsl:otherwise>
                        <xsl:value-of select="count($prevCells) + 1 +
                            sum($prevCells/@ss:MergeAcross)"/>
                    </xsl:otherwise>
                </xsl:choose>
            </xsl:attribute>
            <xsl:apply-templates select="node()"/>
        </xsl:copy>
    </xsl:template>
</xsl:stylesheet>
            

This version is buggy. An updated one is available here

Uche Ogbuji asks me on the IRC:

Eric, I looked in your book for a general discussion of <data type="string"/> versus <text/>, but I didn’t find one.

It’s a question that I’ve come across a few times. I might have missed something or perhaps I and others over-think the distinction :-)

From my reading of the specs, it seems to me there is no lexical difference but that there are differences with respect to pattern restrictions (the infamous data typing restrictions in RNG), so it seems to me one should always use <text/> unless they’re sure they know what they’re doing.

Also, any thoughts about the « simple string type considered harmful » note in WXS? It seems to raise a respectable point that not allowing child elements in elements to contain prose can lead to unexpected restrictions.

That’s a common question, or if not, it’s a question that anyone writing RELAX NG or X3C XML Schema schemas should ask himself and I think that it might be useful to publish my thoughts on the subject.

The make it short, I fully agree with Uche’s analysis and here are my reasons:

The names of the elements of the RELAX NG XML syntax have been chosen very carefully and basically, <data/> has been designed for data oriented applications (or part of applications) and <text/> has been designed for text (ie document) oriented applications.

The first and best thumb rule is thus probably: if you think of the content of an element or attribute as « a piece of text » you should use <text/> and if you think of it as « data » you should use <data/>.

As mentioned by Uche, there is no lexical differences and element foo {text} and element foo {string} or element foo {token} (ie the notation of an element foo containing either text or string or token data in the RNG compact syntax) validate exactly the same set of elements. The difference is in the restrictions attached to these two patterns. The main restrictions are that:

• With the « text oriented » pattern (element foo {text}) , it is possible and even easy to extend the model to add sub elements and make it mixed content but you can’t add facets that will restrict your text nodes.
• With the « data oriented » pattern (element foo {string}) you can add restrictions to your datatype if the datatype library that you are using does support it which is the case of the W3C XML Schema datatype library, such as in element foo {xsd:token {maxLength="10"}} but you can’t extend the content model into mixed content.

Now, why do I consider string datatypes harmful?

This datatype is harmful because it doesn’t behave like any other datatype in that spaces are normalized for any datatype except string and xsd:normalizedString before text nodes reach the datatype library and left unchanged for string datatypes.

This is very different if you are applying restrictions to your datatypes.

Consider for instance: element foo {xsd:boolean "true"} and element foo {xsd:string "true"}.

The first one will accept: <foo> true </foo> while the second one will reject it and there is nothing you can do about it because the white spaces are handled before they reach your datatype library.

That’s why I strongly advise, in 99% of the cases, to use token (or xsd:token) instead of string (or xsd:string).

And don’t think that this advise applies only to cases like the preceding example where we have actual « tokens » (such as « true »). The name of the token datatype is very misleading and its lexical space is the same than the lexical space of string. In other words,  » this is a string  » is a valid token (or xsd:token) type. The difference is that if it’s a token datatype , the spaces will be normalized and what will reach the datatype library is « this is a string » .

How are these two questions linked?

The fact that <data/> is meant to be used for data oriented applications while <text/> is meant to be used for document oriented applications reduces the need to ever use the string datatype: if your spaces are meaningful, such as within an HTML <pre/> element, chances are that you’re dealing with a document oriented applications and that you should be using a <text/> pattern. On the other hand, if you are designing a data oriented applications, chances are that spaces are not significant, should better be normalized to be coherent with the other datatypes that you are probably using.

The opportunities to use string or xsd:string datatypes are thus very infrequent.

How do that translate into W3C XML Schema?

A good way to answer the question is to see what James Clark’s converter, trang, thinks about it.

If you convert a pattern such as « element foo {attribute bar {text}, xsd:token} » into W3C XML Schema, you get:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
  <xs:element name="foo">
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base="xs:token">
          <xs:attribute name="bar" use="required"/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>
</xs:schema>

Sounds logical: a RELAX NG <data/> pattern is clearly a close match for W3C XML Schema simple types.

Now, what does happen if we convert « element foo {attribute bar {text}, text}« ? We get:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
  <xs:element name="foo">
    <xs:complexType mixed="true">
      <xs:attribute name="bar" use="required"/>
    </xs:complexType>
  </xs:element>
</xs:schema>

And that clearly shows that the semantic of a <text/> pattern is to be a text node that has the potential of becoming mixed content model, or in other words a mixed content without sub elements!

It’s also showing a smart (and probably not used enough) option; for those of us who are using W3C XML Schema, to represent text nodes as mixed content models rather than simple string or token types in document oriented applications.

One of the things which are really hard when you specialize in XML schema languages is to play the classical « elevator story » and explain in the time it takes to go from the ground floor to your 7th floor what you are doing to own your life to your neighbor.

That’s already tough to play the elevator story explaining what is XML, just try to imagine how difficult it is with XML schema languages!

In fact, I have found that the shorter amount of time it takes me to really initiate people (that is, not my neighbors but people already knowing XML quite well) to XML Schema languages is half a day and I have made a tutorial to perform that initiation!

The next times I’ll be giving this tutorial will be on April 18th in Amsterdam for XML Europe and then on July 27th in Portland for the Open Source Conference.

I have already been giving that tutorial quite a few times at various conferences including previous issues of XML Europe, XML 2003, OSCON and even SD West 2004 no later than last month but I am still enjoying giving that talk as much as I did the first time.

As far as I know, its agenda is unique and no other tutorials or training cover all three big schema languages (Schematron, RELAX NG and W3C XML Schema).

Most of the attendees seem to enjoy this tutorial as well and to be astonished by the scope that is hiding beyond the term of « XML schema languages ». Of course, many of the concepts seem familiar to those of them who are DTD experts, but the approach taken by these « modern » schema languages is quite different and people tend to underestimate the amount of work that has been done in these areas.

I have been asked why people should want to learn about three different schema languages if they are going to use only one or two of them.

Of course, people can always learn no more than what they will immediately need, but I think that it’s a good thing to know your environment before stepping into a new discipline.

When you start learning computer science, you can start learning only a single programming language, let’s say Visual Basic, if you think that it’s the one that you’ll be using later on. I would argue that you’ll become a better programmer (even in Visual Basic) if you have an insight of other programming languages (let’s say C and Java or Python). Furthermore, you may also find out that Visual Basic isn’t always the best choice for what you’ll have to do later on.

That’s the same with schema languages. Even if right now you think that you’ll be using only one of them (let’s say W3C XML Schema), you’ll get a better understanding of W3C XML Schema if you learn about other approaches: that will help you to understand its most basic principles and also its limitations. And eventually, you may find out that W3C XML Schema isn’t always the best choice for what you’ll have to do…

See you in Amsterdam!