Dataset versioning

The first feature complete version of my MarkLogic persistence layer for Orbeon Form Runner is available but as far as I know nobody is using it yet and it’s still time to think about what will happen for new versions.

Versioning dataset “schemes” is always hard.

I am using the term “scheme” to refer to both structures and conventions used to store and retrieve information.

I would have used  the term “schema” if it hadn’t different meaning depending on the technology  (SQL database schemas, XML schemas, RDF Schema, …) which are all part of what I am calling a “scheme” in this post.

By “dataset” I mean a set of data isolated by any mean one from each other for the application. Datasets is a logical concept which overlaps the concept of databases as defined by the different database implementations.

It’s a generic issue and if you take the example of WordPress which is powering this blog, most of its plugins include a mechanism to check that their database schemes is up to date with their versions and perform automatic upgrades when needed (see for instance the section called “Store the Plugin Version for Upgrades” and the following one in this article by Dave Donaldson).

This is most needed for applications such as WordPress where there the end user is often the administrator of his blog and enterprise applications usually have system administrators to deal with upgrades but still there might be something to borrow in this approach.

Like in a WordPress plugin, in most database application like my persistence layer, you have on one side the “program” (implemented in XQuery, pipelines or whatever for XML databases) and on the other side one or several “datasets”.

This persistence layer supports storing multiple “datasets” in a single database.

A use case for that could be for a provider to define a common dataset as a persistence layer for form definitions shared by different customers, a dataset per customer, a dataset for demonstration purposes, … All these datasets could be either in separate MarkLogic databases or in a single database.

There is currently a single version of the persistence layer and there is no possible mismatch by the XQuery modules which implement the REST API and the dataset schemes.

The situation might be different in a few months.

There might be for instance a version 2 implementing the optional features which haven’t been implemented yet. These features will likely rely on new documents properties added to the current ones and thus rely on a version 2 of the dataset scheme.

The current scheme relies on URIs to store application and form names and document types in a directory fashion. This seems the natural thing to do because that mirrors the URIs structures in the persistence layer specification. Other options could be to use MarkLogic collections or document properties to store this information and we might have decided to go this way in a version 3.

Our provider could then be in a situation where the dataset with the form definitions would still be in v1 (if it doesn’t use features added in v2, why should he migrate), the datasets with customer data would be in v2 and the dataset used for demo purposes would be testing v3.

Traditional approaches rely on upgrade scripts executed by system administrators which can be run through Roxy  which is the preferred deployment tool for MarkLogic applications such as this persistence layer.

In that case, system administrators need to carefully keep modules and dataset schemes versions synchronized and supporting multiple versions of modules for a single database can be tough even if the modules URIs can be versioned.

This is where the WordPress plugin approach may be useful.

What about adding some kind of metadata to each dataset to determine its version?

The URL rewriter would retrieve this information and choose the REST library options corresponding to a specific version to execute the right version of the modules.

An administrative REST API could easily be added to list datasets, display their metadata and perform datasets upgrades (and downgrades if available) and this REST API could be used by Roxy.

The idea of adding dataset metadata seems really powerful but what should such metadata include?

The minimum would be to identify applications using the datasets and their versions but what about adding more information, such as some kind of user readable documentation, URIs to additional documentation and resources, …

Proposing a vocabulary to define such information is an interesting exercise that I’ll be happy to do if needed but I can’t believe it has never been done…

If you are aware of something similar, please tell us!

Many thanks to Peter Kester (MarkLogic) for sharing his thoughts on the subject.

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites

First steps with MarkLogic

[Edited to take into account Dave Cassel’s comments]

To get started with MarkLogic I have chosen to develop a persistence layer for Orbeon Form Runner.

This is the kind of projects I like, small enough to be done in a few days yet technical enough to see advanced topics and potentially useful to other people.

The project is available on my community site and I’d like to share in this post my feelings during this first contact with MarkLogic.

The first contact with a new product is the installation and I have been really surprised by the simplicity of MarkLogic installation  process. My laptop is running Ubuntu which is not a supported platform but the install went very smoothly after converting the RPM package as documented everywhere on the web and it didn’t took me more than a few minutes to get MarkLogic up and running.

The second contact with the admin interface has been less obvious: MarkLogic comes with a series of different generations of web UIs (admin, configuration manager, monitoring, information studio, application builder and query console) and it’s not always obvious to find your way between these tools.

I must also say that I am an old school administrator who prefers configuration files rather than point and click administration windows!

Fortunately this is well documented and I have rapidly been able to create a new database and servers for my project. The interface with my favorite XML tool, oXygen XML editor has also been very easy to setup.

The feeling that hasn’t left me all over this project is a feeling of stability and robustness: I have never needed to restart the server, all the modifications of configuration have been made while the server was up and running, I have never seen any crash nor any non understandable error message.

In other words, MarkLogic is the kind of software which makes you feel secure and comfortable!

A Form Runner persistence layer is a REST API and such APIs are reasonably easy to implement in MarkLogic thanks to their REST library. I think I have found a bug (I am pretty good for that, all the products I have worked with will tell you that) but that was in a minor function and nothing really blocking.

Something to note if you want to try it by yourself is that paths to documents in a database does not always start with a “/” and “foo/bar” is a different directory than “/foo/bar”. To search all the documents under “foo/bar/” you’ll write something such as:

cts:search(/, cts:directory-query('foo/bar/', "infinity"))

If you forget the trailing slash in (foo/bar) MarkLogic will raise an error with a self-explanatory message but if you add a leading slash (/foo/bar/) like you’d do for any decent file system you will search in a different directory and your search may  silently result in an empty sequence!

In fact, as pointed out by Dave Cassel, Marklogic considers that “foo/” is a root directory like “/” and “/foo/” is a subdirectory of the root directory “/”. A database can thus have as many root directory as you want but you need to be careful and if you insert a document as “foo/bar/bat.xml” you won’t be able to find it as “/foo/bar/bat.xml”!

And as you’ve noticed with this simple snipet you’ll have to use many proprietary functions to develop XQuery applications in MarkLogic. This is not really a problem specific to MarkLogic but XQuery has been defined to be generic and we use it for things which are well beyond its original scope.

The good news is that MarkLogic comes with a very extensive library and that you won’t be blocked in your developments. The bad news is of course that what you’ll develop in MarkLogic won’t be easily portable to other XML databases.

The last thing I want to report is the quality of the online documentation, on MarkLogic Community but also on the web at large and on stackoverflow in particular : during my development I have always been able to find answers for the many questions I had in a very reasonable amount of time.

To summarize, I haven’t had the opportunity to test the support of big data yet but this first contact leaves me with a very positive feeling of a product which is mature, stable, rich of features and well documented and supported by its community.

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites

How to customize GitLab to support OpenID authentication

Rational

While setting my GitLab servers I had to perform a number of customizations, the main one being to support OpenID and the method described here could thus be used to do any kind of customization.

If like me you try to use the software packages that come with your preferred Linux based operating system you’ll find the “official” installation guide for GitLab rather untypical. This installation guide seems to totally ignore Linux good practices such as separating configuration, programs and data into /etc, /usr and /var and installs everything under a standard user directory.

A benefit of this “simplification” is that the whole GitLab installation is a git clone of the GitLab source code (that would probably be more difficult if stuff were spread into multiple locations). And the beauty of using git clones is that updates are basically a matter of pulling a new version from the remote repository.

And if you need to “customize” the code you can take advantage of git and create you own local branch(es) which you can push into you own remote repository if you want to share the same customization between multiple GitLab installations.

Disclaimers

  1. I have been following these principles for an initial installation of GitLab 6.1 and upgrades to 6.2 and 6.3 without any major issues but of course I can’t guarantee that it will be the case for you: use at your own risk!
  2. Many thanks to Stephen Rees-Carter for a most helpful post that got me started.

In Practice

Initial installation

Start by a full installation following the “official” installation guide for GitLab and check that everything is running as expected. You could probably save some time by modifying the source during this initial installation but if anything went wrong you’d be in trouble to determine if this is a consequence of your modifications or anything else.

Stop the server

$ sudo service gitlab stop

Create a local branch

It’s a matter of taste but I tend to prefer running commands as the GitLab user (git if you’ve followed the installation guide to the letter) than prefixing them with sudo -u git -H as shown in the installation guide. Of course gitlab-shell will block you if you try to ssh directly as the GitLab user but you can  become the git-lab user using sudo and su:

$ sudo su - git

To create a local branch called openid and switch to this branch:

~$ cd gitlab
~/gitlab$ git checkout -b openid

 Add the omniauth-openid gem

Edit Gemfile to add gem ‘omniauth-openid‘ after gem ‘omniauth-github‘ so that it looks like:

# Auth
gem "devise", '~> 2.2'
gem "devise-async"
gem 'omniauth', "~> 1.1.3"
gem 'omniauth-google-oauth2'
gem 'omniauth-twitter'
gem 'omniauth-github'
gem 'omniauth-openid'

After this update you’ll need to create a new bundle. To do so you have to run the command bundle install as root (running sudo as a user which has enough rights). This will update the Gemfile.lock file and other resources which should belong to the GitLab:

~/gitlab$sudo bundle install
~/gitlab$sudo chown -R  gitlab.gitlab .

At that point, git status should tell you that you’ve updated both Gemfile and Gemfile.lock and you can commit this first step:

~/gitlab$ git commit -a -m "omniauth-openid gem installed"

Configuration

Update config/gitlab.yml to enable omniauth:

  ## OmniAuth settings
  omniauth:
    # Allow login via Twitter, Google, etc. using OmniAuth providers
    enabled: true

In a perfect world you should be able to configure OpenID as an omniauth provider here but unfortunately, the code that handles these definition (located in config/initializers/devise.rb) requires mandatory app_id and app_secret parameters used by proprietary protocols to lock users. OpenID doesn’t use these parameters and we’ll define OpenID providers directly in the code.

Defining OpenID providers

Update config/initializers/devise.rb to add the definition of the OpenID provider(s) so that it looks like:

...
      name_proc: email_stripping_proc
  end
# Update starts here
#  require "openid/fetchers"
  OpenID.fetcher.ca_file = "/etc/ssl/certs/ca-certificates.crt"

  config.omniauth :open_id,
    :name => 'google',
    :identifier => 'https://www.google.com/accounts/o8/id'

  config.omniauth :open_id,
    :name => 'openid'
# Update ends here
  Gitlab.config.omniauth.providers.each do |provider|
...

(Add the first declaration only if you want to offer OpenID authentication to Google users)

Define how these providers should be handled

Update app/controllers/omniauth_callbacks_controller.rb to include these definitions:

# Update starts here
  def google
     handle_omniauth
  end

  def openid
     handle_omniauth
  end
# Update ends here

  private

  def handle_omniauth

Declare these providers as “enabled social providers”

At that point, users should be able to login using OpenID if the relevant information was available in the database and we need to enable the user interface which will put these info in the database.

Update app/helpers/oauth_helper.rb to add OpenID (and google if you’ve defined it) to the list of “enabled_social_providers“:

  def enabled_social_providers
    enabled_oauth_providers.select do |name|
      [:openid, :google, :twitter, :github, :google_oauth2].include?(name.to_sym)
    end
  end

This list was initially limited to [:twitter, :github, :google_oauth2].

Disable protect_from_forgery in omniauth_callbacks_controller.rb

At that point, profile/account pages should present a list of “social accounts” including Google and OpenID. Unfortunately if you click on one of these buttons the authentication will succeed but the database update will be blocked by the protect_from_forgery feature.

This issue is documented in stackoverflow and GitHub and a workaround is to switch this feature in the controller that handles the authentication.

Update app/controllers/omniauth_callbacks_controller.rb to comment the third line and require to skip this:

class OmniauthCallbacksController < Devise::OmniauthCallbacksController
  # Update starts here
  #protect_from_forgery :except => :create
  skip_before_filter :verify_authenticity_token
  # Update ends here

  Gitlab.config.omniauth.providers.each do |provider|

Start the server and test

$ sudo service gitlab start
$ sudo service nginx restart # or sudo service apache2 restart

OpenID authentication should work fine at that point.

Commit

~/gitlab$ git commit -a -m "Ready for OpenID..."

Upgrades

To upgrade your GitLab installation you’ll need to merge the new version into your openid branch before you can follow the upgrade guide. For instance, to upgrade to version 6.3 I have typed:

~/gitlab$ git fetch
~/gitlab$ git checkout 6-3-stable # to have a look at the new version
~/gitlab$ git checkout openid # back to the local branch
~/gitlab$ git merge 6-3-stable

You may run into merge conflicts. During the upgrade to 6.3, the Gemfile had been updated and as a result of this update, Gemfile.lock was in conflict.

To fix this specific issue I have run the bundle install command again:

~/gitlab$sudo bundle install
~/gitlab$sudo chown -R  gitlab.gitlab .

When you’ve fixed your conflicts, you need to add the files in conflict and commit:

~/gitlab$ git add Gemfile.lock 
~/gitlab$ git commit -m "Merging 6.3"

From that point you should be able to follow standards upgrade instructions.

Using your own remote directory

If you need to install several servers with the same customization, you may want to push your branch to a remote directory.

To do so, you must define a new remote and specify whenever you push to that directory, for example:

~/gitlab$ git remote add myremote git@gitlab.example.com:gitlab/custom.git
~/gitlab$ git push myremote openid

Note that of course if you want to pull/push from the GitLab server you are working on it would need to be up and running ;) !

To install new servers you can now follow the standard installation guide just replacing the initial git clone by a clone of your own remote directory.

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites

From Trac to GitLab

I have been using Trac since ages and I like its simplicity compared to systems such as Jira but I must say it looks like a dinosaur compared to GitHub

I like GitHub but I am also a strong believer that the web should stay decentralized and have been searching an open source alternative that I could host on my servers for a while.

The latest version of GitLab is a perfect solution for my needs: previous versions were focused on private projects but version 6.2 introduces the support for public projects that was missing in previous versions.

I have installed GitLab at https://gitlab.dyomedea.com/ and I have started using it to publish my presentation at XML Amsterdam.

My plan is now to slowly migrate my open source projects between Trac and GitLab and I have found a couple of Python scripts that may help for this process:

I have consolidated these two scripts and the result is available as trac-to-gitlab which is of course… published on GitLab!

One of the things I really like with git is that at the bottom of the network of changes you can see the initial history of the two scripts with the commit dates and names of their authors.

I hope that trac-to-gitlab may be useful to others and be similarly cloned into other projects!

Of course, comments, issues and merge requests are welcome.

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites

XForms Unit

Update: After this post I have launched http://xformsunit.org.

XForms forms can be incredibly complex and I have been thinking of an XForms unit test environment for a while. Before I start coding here are some of my thoughts on the topic…

XForms test cases should be native

A common way of testing XForms application is to use a generic purpose web testing tool such as Selenium.

The benefits of such tools is that you can test exactly what is displayed in the browser and simulate user actions. It’s downside is that the tests are expressed in browser related terms rather than using XForms concepts. To write these tests you need to know how XForms will be transformed into HTML and this transformation depends on the XForms implementation being used and may vary between versions.

By contrast a native XForms test environment would allow to express the tests using XForms concepts such as binds, controls and events.

 Principles

The basic idea is to describe a test suite and to add actions to the form to test so that the tests are executed by the XForms engine.

Let’s say we have the following form which is a slightly adapted version of the traditional XForms “hello world”:

<?xml version="1.0" encoding="UTF-8"?>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:xf="http://www.w3.org/2002/xforms">
    <head>
        <title>Hello World in XForms</title>
        <xf:model id="model">
            <xf:instance id="instance" xmlns="">
                <data>
                    <PersonGivenName></PersonGivenName>
                    <Greetings></Greetings>
                </data>
            </xf:instance>
            <xf:bind id="greetings" nodeset="/data/Greetings"
                calculate="concat('Hello ', ../PersonGivenName, '. We hope you like XForms!')"/>
        </xf:model>
    </head>
    <body>
        <p>Type your first name in the input box. <br /> If you are running XForms, the output should be displayed in
            the output area.</p>
        <xf:input ref="PersonGivenName" incremental="true">
            <xf:label>Please enter your first name: </xf:label>
        </xf:input>
        <br />
        <xf:output value="Greetings">
            <xf:label>Output: </xf:label>
        </xf:output>
    </body>
</html>

To test that the greeting are what you’d be expecting you would enter a value in the name and check the greetings.

This test can be expressed as:

<?xml version="1.0" encoding="UTF-8"?>
<suite xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:xf="http://www.w3.org/2002/xforms"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
    <form src="hello-world.xhtml"/>

    <!-- The test cases -->
    <case id="test-greetings">
        <action>
            <setvalue ref="instance('instance')/PersonGivenName">Eric</setvalue>
        </action>
        <assertEqual>
            <actual ref="instance('instance')/Greetings"/>
            <expected>Hello Eric. We hope you like XForms!</expected>
            <message>The greetings should be the concatenation of "Hello ", the given name and ". We hope you like
                XForms!".</message>
        </assertEqual>
    </case>
</suite>

The generation of the augmented form which will perform the tests can be done in XSLT.

To implement these tests in XForms, we can add an instance initialized with the description of the test suite and updated by XForms actions to reflect the result of the tests:

<xf:instance id="xfu:instance">
    <suite xmlns="">
        <case id="test-greetings">
            <action id="d8e9">
                <setvalue ref="instance('instance')/PersonGivenName" id="d8e11">Eric</setvalue>
            </action>
            <assertEqual id="d8e15" passed="">
                <actual ref="instance('instance')/Greetings" id="d8e17"></actual>
                <expected id="d8e19">Hello Eric. We hope you like XForms!</expected>
                <message id="d8e22">The greetings should be the concatenation of "Hello ", the given name
                    and ". We hope you like XForms!".</message>
            </assertEqual>
        </case>
    </suite>
</xf:instance>

Note how the transformation has added id attributes to the elements of this instance which did not already carry one. These ids will be helpful while writing the actions.

These actions themselves can be coded as:

<xf:dispatch targetid="model" name="xfu:d8e9-action"/>
<xf:action ev:event="xfu:d8e9-action">
    <xf:setvalue ref="instance('instance')/PersonGivenName">Eric</xf:setvalue>
    <xf:recalculate/>
    <xf:dispatch targetid="model" name="xfu:d8e15-action"/>
</xf:action>
<xf:action ev:event="xfu:d8e15-action">
    <xf:setvalue ref="instance('xfu:instance')//*[@id = 'd8e17']" value="instance('instance')/Greetings"/>
    <xf:setvalue ref="instance('xfu:instance')//*[@id = 'd8e15']/@passed" value="(instance('instance')/Greetings) = 'Hello Eric. We hope you like XForms!'"/>
</xf:action>

In this version I have chosen to encapsulate each action and test in its own XForms action and use custom events to trigger these actions one after the other. Also note the xf:recalculate which is necessary to make sure that the greetings are recalculated after we’ve changed the  name.

Now that xfu:instance contains the result of the tests, it could be saved on a server and can also be displayed using XForms controls such as:

<xf:group xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xh="http://www.w3.org/1999/xhtml"
    xmlns:ev="http://www.w3.org/2001/xml-events" xmlns:xfu="http://xformsunit.org/"
    ref="instance('xfu:instance')">
    <xh:h3>Test results</xh:h3>
    <xh:dl>
        <xf:repeat nodeset="case">
            <xh:dt><xh:dfn><xf:output ref="@id"></xf:output></xh:dfn></xh:dt>
            <xh:dd><xh:ul>
                    <xf:repeat nodeset="*[@passed]">
                        <xh:li><xf:group ref=".[@passed = 'true']">
                                <xh:span>passed</xh:span>
                            </xf:group><xf:group ref=".[@passed = 'false']">
                                <xh:span>failed</xh:span>
                                <xf:output ref="actual">
                                    <xf:label>Actual : </xf:label>
                                </xf:output>
                                <xf:output ref="expected">
                                    <xf:label>Expected : </xf:label>
                                </xf:output>
                            </xf:group></xh:li>
                    </xf:repeat>
                </xh:ul></xh:dd>
        </xf:repeat>
    </xh:dl>
</xf:group>

The resulting forms works fine with Orbeon Forms:

Screenshot of Orbeon Forms running the tests.
Screenshot of Orbeon Forms running the tests.

It does also work with betterForm:

Screenshot of betterFORM running the tests
Screenshot of betterFORM running the tests

But it fails when run on XSLTForms:

Screenshot of XSLTForms running the tests.
Screenshot of XSLTForms running the tests.

The error raised by XSLTForms is caused by an issue with the support of the xf:recalculate action. A possible workaround is to add a delay in the dispatch action that triggers the test:

<xf:dispatch targetid="model" name="xfu:d8e15-action" delay="1">

Note that this delay causes additional client/server exchanges for server based implementations such as Orbeon Forms and the workaround should be used specifically with XSLTForms.

What you test and what you see

In our test we check that an instance value is what we expect, but can we be sure that this instance value is displayed? This is pretty obvious in our example, but what if there was a typo in the control:

        <xf:output value="greetings">
            <xf:label>Output: </xf:label>
        </xf:output>

Here  the name of the element is lower case and the xf:output will display an empty string but the test will still pass.

The xf:output could also be embedded in more complex xf:group elements, such as:

         <xf:group ref=".[starts-with(PersonGivenName, 'J')]">
            <xf:output value="Greetings">
                <xf:label>Output: </xf:label>
            </xf:output>
        </xf:group>

Here the greetings are displayed only when the name starts with a “J” but, again, the test still passes.

The visibility of the greetings could also be affected by class or style attributes using AVTs:

        <div style="{if (starts-with(PersonGivenName, 'J')) then '' else 'display: none' }">
            <xf:output value="Greetings">
                <xf:label>Output: </xf:label>
            </xf:output>
        </div>

The effect would, again, be to display the greetings only if the name starts with a “J” but this would be much more complex to detect.

What’s missing here is the ability of functions to check the status of the controls themselves rather than instance values.

Orbeon Forms does provide extension functions working on controls which may be useful but are specific to this implementation.

 Introspection

More generally speaking there is a lack of standard introspection functions, not only for controls but also on instances and testing if an instance node is relevant, readonly, required or valid is not possible without using extension functions. Some (but not all) these needs are covered by EXForms functions but their support by actual implementations has to be checked.

Conclusion

Despite some limitations there seems to be a nice potential in a native XForms test framework and of course your feedback is most welcome.

 

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites

When MVC becomes a burden for XForms – XML London 2013

This paper, presented at XML London 2013, is also available as a presentation and as a screencast:

Abstract

XForms is gaining traction and is being used to develop complex forms, revealing its strengths but also its weaknesses.

One of the latest is not specific to XForms but inherent to the MVC (Model View Controller) architecture which is one of the bases of XForms.

In this talk we see how the MVC architecture dramatically affect the modularity and reusabilty of XForms developments and some of the solutions used to work around this flaw.

Practice: a quiz

Let’s start with a quiz…

Basic XForms

Question

Given the following instance:

<figures>
   <line>
      <length>
         <value>10</value>
         <unit>in</unit>
      </length>
   </line>
</figures>

implement a standard XForms 1.1 form displaying the following user interface:

Answer

Model:

<xf:model>
   <xf:instance>
      <figures>
         <line>
            <length>
               <value>10</value>
               <unit>in</unit>
            </length>
         </line>
      </figures>
   </xf:instance>
</xf:model>

View:

<xf:group ref="line/length">
   <xf:input ref="value">
      <xf:label>Length: </xf:label>
   </xf:input>
   <xf:select1 ref="unit">
      <xf:label></xf:label>
      <xf:item>
         <xf:label>pixels</xf:label>
         <xf:value>px</xf:value>
      </xf:item>
      <xf:item>
         <xf:label>font size</xf:label>
         <xf:value>em</xf:value>
      </xf:item>
      <xf:item>
         <xf:label>font height</xf:label>
         <xf:value>ex</xf:value>
      </xf:item>
      <xf:item>
         <xf:label>inches</xf:label>
         <xf:value>in</xf:value>
      </xf:item>
      <xf:item>
         <xf:label>centimeters</xf:label>
         <xf:value>cm</xf:value>
      </xf:item>
      <xf:item>
         <xf:label>millimeters</xf:label>
         <xf:value>mm</xf:value>
      </xf:item>
      <xf:item>
         <xf:label>points</xf:label>
         <xf:value>pt</xf:value>
      </xf:item>
      <xf:item>
         <xf:label>picas</xf:label>
         <xf:value>pc</xf:value>
      </xf:item>
      <xf:item>
         <xf:label>%</xf:label>
         <xf:value>%</xf:value>
      </xf:item>
   </xf:select1>
</xf:group>

Using instances and actions

Question

Implement the same user interface if the instance uses the CSS2 / SVG 1.1 conventions for sizes:

<xf:instance id="main">
   <figures>
      <line length="10in"/>
   </figures>
</xf:instance>

Answer

Model:

<xf:model>
   <xf:instance id="main">
      <figures>
         <line length="10in"/>
      </figures>
   </xf:instance>
   <xf:instance id="split">
      <line>
         <length>
            <value/>
            <unit/>
         </length>
      </line>
   </xf:instance>
   .../...
</xf:model>

View:

<xf:group ref="instance('split')/length">
   <xf:input ref="value" id="length-control">
      <xf:label>Length: </xf:label>
   </xf:input>
   <xf:select1 ref="unit" id="unit-control">
      <xf:label/>
      <xf:item>
         <xf:label>pixels</xf:label>
         <xf:value>px</xf:value>
      </xf:item>
      .../...
      <xf:item>
         <xf:label>%</xf:label>
         <xf:value>%</xf:value>
      </xf:item>
   </xf:select1>
</xf:group>

Controller:

<xf:model>
   .../...
   <xf:action ev:event="xforms-ready">
      <xf:setvalue ref="instance('split')/length/value" value="translate(instance('main')/line/@length, '%incmptxe', '')"/>
      <xf:setvalue ref="instance('split')/length/unit" value="translate(instance('main')/line/@length, '0123456789', '')"/>
   </xf:action>
   <xf:action ev:event="xforms-value-changed" ev:observer="length-control">
      <xf:setvalue ref="instance('main')/line/@length" value="concat(instance('split')/length/value, instance('split')/length/unit)"/>
   </xf:action>
   <xf:action ev:event="xforms-value-changed" ev:observer="unit-control">
      <xf:setvalue ref="instance('main')/line/@length" value="concat(instance('split')/length/value, instance('split')/length/unit)"/>
   </xf:action>
</xf:model>

Modularity

Question

Still using XForms 1.1 standard features, extend this user interface to edit the height and width of a rectangle:

<xf:instance id="main">
   <figures>
      <rectangle height="10in" width="4em"/>
   </figures>
</xf:instance>

Hint: copy/paste is your friend!

Answer

Model:

<xf:model>
   <xf:instance id="main">
      <figures>
         <rectangle height="10in" width="4em"/>
      </figures>
   </xf:instance>
   <xf:instance id="height">
      <height>
         <value/>
         <unit/>
      </height>
   </xf:instance>
   .../...
   <xf:instance id="width">
      <width>
         <value/>
         <unit/>
      </width>
   </xf:instance>
   .../...
</xf:model>

View:

<xf:group ref="instance('height')">
   <xf:input ref="value" id="height-value-control">
      <xf:label>Height: </xf:label>
   </xf:input>
   <xf:select1 ref="unit" id="height-unit-control">
      <xf:label/>
      <xf:item>
         <xf:label>pixels</xf:label>
         <xf:value>px</xf:value>
      </xf:item>
      .../...
   </xf:select1>
</xf:group>
<xh:br/>
<xf:group ref="instance('width')">
   <xf:input ref="value" id="width-value-control">
      <xf:label>Width: </xf:label>
   </xf:input>
   <xf:select1 ref="unit" id="width-unit-control">
      <xf:label/>
      <xf:item>
         <xf:label>pixels</xf:label>
         <xf:value>px</xf:value>
      </xf:item>
      .../...
   </xf:select1>
</xf:group>

Controller:

<xf:model>
   .../...
   <xf:action ev:event="xforms-ready">
      <xf:setvalue ref="instance('height')/value" value="translate(instance('main')/rectangle/@height, '%incmptxe', '')"/>
      <xf:setvalue ref="instance('height')/unit" value="translate(instance('main')/rectangle/@height, '0123456789', '')"/>
   </xf:action>
   <xf:action ev:event="xforms-value-changed" ev:observer="height-value-control">
      <xf:setvalue ref="instance('main')/rectangle/@height" value="concat(instance('height')/value, instance('height')/unit)"/>
   </xf:action>
   <xf:action ev:event="xforms-value-changed" ev:observer="height-unit-control">
      <xf:setvalue ref="instance('main')/rectangle/@height" value="concat(instance('height')/value, instance('height')/unit)"/>
   </xf:action>
   .../...
   <xf:action ev:event="xforms-ready">
      <xf:setvalue ref="instance('width')/value" value="translate(instance('main')/rectangle/@width, '%incmptxe', '')"/>
      <xf:setvalue ref="instance('width')/unit" value="translate(instance('main')/rectangle/@width, '0123456789', '')"/>
   </xf:action>
   <xf:action ev:event="xforms-value-changed" ev:observer="width-value-control">
      <xf:setvalue ref="instance('main')/rectangle/@width" value="concat(instance('width')/value, instance('width')/unit)"/>
   </xf:action>
   <xf:action ev:event="xforms-value-changed" ev:observer="width-unit-control">
      <xf:setvalue ref="instance('main')/rectangle/@width" value="concat(instance('width')/value, instance('width')/unit)"/>
   </xf:action>
</xf:model>

Homework: repeated content

Still using standard XForms features, extend this form to support any number of rectangles in the instance.

Hint: you will not be able to stick to atomic instances for the width and height but act more globally and maintain instances with a set of dimensions which you’ll have to keep synchronized with the main instance when rectangles are inserted or deleted.

What’s the problem?

XForms lacks a feature to define and use “components” that would package a group of controls together with their associated model and actions.

Theory: the MVC design pattern

XForms describes itself as a MVC architecture:

An XForm allows processing of data to occur using three mechanisms:

  • a declarative model composed of formulae for data
    calculations and constraints, data type and other property declarations, and data submission
    parameters
  • a view layer composed of intent-based user interface
    controls
  • an imperative controller for orchestrating data
    manipulations, interactions between the model and view layers, and data submissions.

Micah Dubinko argues that the mapping is more obvious with Model-view-presenter
(MVP)
, a derivative of the MVC software pattern but that’s not the point I’d like to make and
I’ll stick to the MVC terminology where:

  • The model is composed of XForms instances and binds
  • The view is composed of the XForms controls together with the HTML elements and CSS stylesheets
  • The controller is composed of the actions

Orbeon Form Builder/Form Runner go one step forward and add a fourth concern for localization and we get a model/view/localization/controller pattern.

This separation of concerns is great to differentiate different roles and split work between specialists but doesn’t play well with modularity and reusability.

I am currently working on a project to develop big and complex forms and this is becoming one of the biggest issues: these forms share a number of common fields and group of fields and, not even speaking of sharing these definitions, this separation of concerns adds a significant burden when copying these definitions from one form to another.

To copy a field from one form to another you need to copy definitions from the model, the view, the localization and the controller and can’t just copy a “component”.

And of course, there is no easy way to reuse common components instead of copying them.

This kind of issue is common with the MVC design pattern and the Hierarchical model–view–controller (HMVC) has been introduced for this purpose, but how can we use such a pattern with XForms?

Solutions

A number of solutions are being used to work around this issue with XForms.

Copy/Paste

This is what we’ve done for our quiz and we’ve seen that this is easy -but very verbose and hard to maintain- until we start to deal with repeated content.

I would guess that this is the most common practice when fields (or group of fields) are being reused in XForms though!

XForms generation or templating

We’re XML developers, aren’t we? When something is verbose we can use XSLT or any other tool to generate it and XForms is no exception.

XForms can be generated from any kind of model including annotated schemas or other vocabularies such as DDI (we’ll be presenting this option at the Balisage International Symposium on Native XML User Interfaces in August.

Projects without any obvious model formats in mind often chose to transform simplified versions of XForms into plain XForms. In that case the approach may tends toward a templating system where placeholders are inserted into XForms documents to be transformed into proper XForms.

We may want for instance to define <my:dimension/> placeholders which would look like XForms controls and generate the whole model, view and controller XForms definitions.

The source form would then be something as simple as:

<xh:html xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:xf="http://www.w3.org/2002/xforms"
   xmlns:my="http://ns.dyomedea.com/my-components/">
   <xh:head>
      <xh:title>Template</xh:title>
      <xf:model>
         <xf:instance id="main">
            <figures>
               <rectangle height="10in" width="4em"/>
            </figures>
         </xf:instance>
      </xf:model>
   </xh:head>
   <xh:body>
      <my:dimension ref="rectangle/@height">
         <xf:label>Height</xf:label>
      </my:dimension>
      <br/>
      <my:dimension ref="rectangle/@width">
         <xf:label>Width</xf:label>
      </my:dimension>
   </xh:body>
</xh:html>

A simplistic version of a transformation to process this example is not overly complex. The controls are quite easy to generate from the placeholders:

<xsl:template match="my:dimension">
    <xsl:variable name="id" select="if (@id) then @id else generate-id()"/>
    <xf:group ref="instance('{$id}-instance')">
        <xf:input ref="value" id="{$id}-value-control">
            <xsl:apply-templates/>
        </xf:input>
        <xf:select1 ref="unit" id="{$id}-unit-control">
            <xf:label/>
            <xf:item>
                <xf:label>pixels</xf:label>
                <xf:value>px</xf:value>
            </xf:item>
            .../...
        </xf:select1>
    </xf:group>
</xsl:template>

A model can be appended to the <xh:head/> element:

<xsl:template match="xh:head">
    <xsl:copy>
        <xsl:apply-templates select="@*|node()" mode="#current"/>
        <xf:model>
            <xsl:apply-templates select="//my:dimension" mode="model"/>
        </xf:model>
    </xsl:copy>
</xsl:template>

And the instances and actions can be generated similarly:

<xsl:template match="my:dimension" mode="model">
    <xsl:variable name="id" select="if (@id) then @id else generate-id()"/>
    <xf:instance id="{$id}-instance">
        <height>
            <value/>
            <unit/>
        </height>
    </xf:instance>
    <xf:action ev:event="xforms-ready">
        <xf:setvalue ref="instance('{$id}-instance')/value"
            value="translate(instance('main')/{@ref}, '%incmptxe', '')"/>
        <xf:setvalue ref="instance('{$id}-instance')/unit"
            value="translate(instance('main')/{@ref}, '0123456789', '')"/>
    </xf:action>
    <xf:action ev:event="xforms-value-changed" ev:observer="{$id}-value-control">
        <xf:setvalue ref="instance('main')/{@ref}"
            value="concat(instance('{$id}-instance')/value, instance('{$id}-instance')/unit)"/>
    </xf:action>
    <xf:action ev:event="xforms-value-changed" ev:observer="{$id}-unit-control">
        <xf:setvalue ref="instance('main')/{@ref}"
            value="concat(instance('{$id}-instance')/value, instance('{$id}-instance')/unit)"/>
    </xf:action>
</xsl:template>

As always, the devil is in details and this would be far from perfect:

  • In actions, references to the main instance do not take into account the context node under which the <my:dimension/> placeholder is defined (paths are therefore
    expected to be relative to the default instance). Mimicking the behavior of an XForms control and its support of the context node would be much more challenging.
  • Supporting repetitions would be another challenge.

Orbeon Forms’ XBL implementation

Orbeon’s
component architecture
is inspired by XBL 2.0 which describes itself as:

XBL (the Xenogamous Binding Language) describes the ability to associate elements in a document with script, event handlers, CSS, and more complex content models, which can be stored in another document. This can be used to re-order and wrap content so that, for instance, simple HTML or XHTML markup can have complex CSS styles applied without requiring that the markup be polluted with multiple semantically neutral div elements.It can also be used to implement new DOM interfaces, and, in conjunction with other specifications, enables arbitrary tag sets to be implemented as widgets. For example, XBL could be used to implement the form controls in XForms or HTML.
XBL 2.0

Even if this specification is no longer maintained by the W3C Web Applications Working Group, the concepts described in XBL 2.0 fit very nicely in the context of XForms documents even though the syntax may sometimes look strange, such as when CSS selectors are used where XPath patterns would look more natural in XForms documents.

Note

The syntax of XBL declarations has been changed between Orbeon Forms version 3 and 4. The syntax shown in this paper is the syntax of version 4.

The definition of an XBL component to implement our dimension widget would be composed of three parts: handlers, implementation and template:

<xbl:binding id="my-dimension" element="my|dimension" xxbl:mode="lhha binding value">
   <xbl:handlers>
      .../...
   </xbl:handlers>
   <xbl:implementation>
      .../...
   </xbl:implementation>
   <xbl:template>
      .../...
   </xbl:template>
</xbl:binding>

A fourth element could be added to define component specific resources such as CSS stylesheets.

The XForms component’s model goes into the implementation:

<xbl:implementation>
   <xf:model id="my-dimension-model">
      <xf:instance id="my-dimension-instance">
         <dimension>
            <value/>
            <unit/>
         </dimension>
      </xf:instance>
      .../...
</xbl:implementation>

The XForms component’s controls are defined into the template:

<xbl:template>
   <xf:input ref="value" id="my-dimension-value-control"/>
   <xf:select1 ref="unit" id="my-dimension-unit-control">
      <xf:label/>
      <xf:item>
         <xf:label>pixels</xf:label>
         <xf:value>px</xf:value>
      </xf:item>
      .../...
   </xf:select1>
</xbl:template>

The XForms actions are split between the handlers and the implementation (or the template): handlers are used to define actions triggered by events which are external to the component (such as in our case xforms-ready) while traditional XForms actions are used to handle events “internal” to the component such as user actions.

The handlers would thus be:

<xbl:handlers>
   <xbl:handler event="xforms-enabled xforms-value-changed">
      <xf:setvalue ref="instance('my-dimension-instance')/value" 
          value="translate(xxf:binding('my-dimension'), '%incmptxe', '')"/>
      <xf:setvalue ref="instance('my-dimension-instance')/unit" 
          value="translate(xxf:binding('my-dimension'), '0123456789', '')"/>
   </xbl:handler>
</xbl:handlers>

And the remaining actions:

<xbl:implementation>
   <xf:model id="my-dimension-model">
      .../...
      <xf:setvalue ev:event="xforms-value-changed" ev:observer="my-dimension-value-control" 
          ref="xxf:binding('my-dimension')" 
          value="concat(instance('my-dimension-instance')/value, instance('my-dimension-instance')/unit)"/>
      <xf:setvalue ev:event="xforms-value-changed" ev:observer="my-dimension-unit-control" 
          ref="xxf:binding('my-dimension')" 
          value="concat(instance('my-dimension-instance')/value, instance('my-dimension-instance')/unit)"/>
   </xf:model>
</xbl:implementation>

I won’t go into the details which are described in Orbeon’s XBL – Guide to Using and Writing XBL Components but it is worth noting that there is a strict encapsulation of both the model, the view and the controller of this component that seen from the outside acts as a standard XForms control.

Of course, this component can be used as a standard XForms control:

<xh:body>
   <my:dimension ref="rectangle/@height">
      <xf:label>Height</xf:label>
   </my:dimension>
   <br/>
   <my:dimension ref="rectangle/@width">
      <xf:label>Width</xf:label>
   </my:dimension>
</xh:body>

The complete form with the component definition would be:

<?xml-stylesheet href="xsltforms/xsltforms.xsl" type="text/xsl"?>
<?xsltforms-options debug="yes"?>
<xh:html xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:xf="http://www.w3.org/2002/xforms"
   xmlns:xxf="http://orbeon.org/oxf/xml/xforms" xmlns:ev="http://www.w3.org/2001/xml-events"
   xmlns:xbl="http://www.w3.org/ns/xbl" xmlns:xxbl="http://orbeon.org/oxf/xml/xbl"
   xmlns:fr="http://orbeon.org/oxf/xml/form-runner" xmlns:my="http://ns.dyomedea.com/my-components/">
   <xh:head>
      <xh:title>Simple XBL Component</xh:title>
      <xbl:xbl script-type="application/xhtml+xml">
         <xbl:binding id="my-dimension" element="my|dimension" xxbl:mode="lhha binding value">
            <xbl:handlers>
               <xbl:handler event="xforms-enabled xforms-value-changed">
                  <xf:setvalue ref="instance('my-dimension-instance')/value" value="translate(xxf:binding('my-dimension'), '%incmptxe', '')"/>
                  <xf:setvalue ref="instance('my-dimension-instance')/unit" value="translate(xxf:binding('my-dimension'), '0123456789', '')"/>
               </xbl:handler>
            </xbl:handlers>
            <xbl:implementation>
               <xf:model id="my-dimension-model">
                  <xf:instance id="my-dimension-instance">
                     <dimension>
                        <value/>
                        <unit/>
                     </dimension>
                  </xf:instance>
                  <xf:setvalue ev:event="xforms-value-changed" ev:observer="my-dimension-value-control" ref="xxf:binding('my-dimension')" value="concat(instance('my-dimension-instance')/value, instance('my-dimension-instance')/unit)"/>
                  <xf:setvalue ev:event="xforms-value-changed" ev:observer="my-dimension-unit-control" ref="xxf:binding('my-dimension')" value="concat(instance('my-dimension-instance')/value, instance('my-dimension-instance')/unit)"/>
               </xf:model>
            </xbl:implementation>
            <xbl:template>
               <xf:input ref="value" id="my-dimension-value-control"/>
               <xf:select1 ref="unit" id="my-dimension-unit-control">
                  <xf:label/>
                  <xf:item>
                     <xf:label>pixels</xf:label>
                     <xf:value>px</xf:value>
                  </xf:item>
                  <xf:item>
                     <xf:label>font size</xf:label>
                     <xf:value>em</xf:value>
                  </xf:item>
                  <xf:item>
                     <xf:label>font height</xf:label>
                     <xf:value>ex</xf:value>
                  </xf:item>
                  <xf:item>
                     <xf:label>inches</xf:label>
                     <xf:value>in</xf:value>
                  </xf:item>
                  <xf:item>
                     <xf:label>centimeters</xf:label>
                     <xf:value>cm</xf:value>
                  </xf:item>
                  <xf:item>
                     <xf:label>millimeters</xf:label>
                     <xf:value>mm</xf:value>
                  </xf:item>
                  <xf:item>
                     <xf:label>points</xf:label>
                     <xf:value>pt</xf:value>
                  </xf:item>
                  <xf:item>
                     <xf:label>picas</xf:label>
                     <xf:value>pc</xf:value>
                  </xf:item>
                  <xf:item>
                     <xf:label>%</xf:label>
                     <xf:value>%</xf:value>
                  </xf:item>
               </xf:select1>
            </xbl:template>
         </xbl:binding>
      </xbl:xbl>

      <xf:model>
         <xf:instance id="main">
            <figures>
               <rectangle height="10in" width="4em"/>
            </figures>
         </xf:instance>

      </xf:model>
   </xh:head>
   <xh:body>
      <my:dimension ref="rectangle/@height">
         <xf:label>Height</xf:label>
      </my:dimension>
      <br/>
      <my:dimension ref="rectangle/@width">
         <xf:label>Width</xf:label>
      </my:dimension>

      <fr:xforms-inspector/>
   </xh:body>
</xh:html>

Subforms

Subforms are implemented by XSLTForms and betterFORM. They have been considered for inclusion in XForms 2.0 but no consensus have been reached and they won’t be included in 2.0.

There are a number of differences between the XSLTForms and betterFORM implementations but the principle -and the shortcomings- are the same.

The basic principle behind subforms is to embed (or load) a form within another one. This embedding must be specifically performed using an <xf:load> action with a @show="embed" attribute. Subforms can also be unloaded.

The fact that subforms are explicitly loaded and unloaded in their “master” form is a key feature for big forms where this mechanism reduces the consumption of resources and leads to important performance improvements.

Subforms, betterFORM flavor

Subforms are described, in the betterFORM documentation, as a way to avoid redundancies and keep the documents maintainable:

As XForms follows a MVC architecture the XForms model is the first logical candidate when decomposing larger forms into smaller pieces. Aggregating more complex forms from little snippets of UI (or snippets of a model) is a limited approach as the interesting parts are located on the bind Elements. This is where controls learn about their constraints, states, calculations and data types. Instead of just glueing pieces of markup together the inclusion of complete models allow the reuse of all the semantics defined within them.
betterFORM
“Modularizing forms”

Joern Turner, founder of Chiba and co-founder of betterFORM, makes it clear that subforms haven’t been introduced to implement components, though:

Sorry i need to get a bit philosophic here but subforms are called subforms as they are *not* components ;) I don’t want to dive into academic discussions here but the main difference for us is that from a component you would expect to use it as a part of a form as a means to edit one or probably several values in your form and encapsulate the editing logic inside of it. A subform on the other hand should be designed to be completely standalone. Typically we build our subforms as complete xhtml documents which can be run and tested standalone without being a part of a host document.
Joern Turner on the betterform-users mailing list

A proper solution for components, based on Web Components) should be implemented in betterFORM 6:

We have also considered to implement this [XBL] but decided against it due to the acceptance and future of XBL and due to the fact that we found it overly complex and academic. We will come up with our own component model in betterFORM 6 which will orient at more modern approaches (Web Components).
Joern Turner on the betterform-users mailing list

In the meantime it is still possible to use subforms to design component like features assuming we take into account the following constraints:

  • Communications between the master form and the subform are done using either
    in memory submissions (ContextSubmissionHandler identified by a model:
    pseudo protocol), the instanceOfModel() function which gives access to
    instances from other models or custom events passing context information.
  • There is no id collision between the main form and the subforms which are loaded
    simultaneously.

This second constraint should be released in the future but the current version of the processor doesn’t address it. In practice it means that in our sample we cannot load simultaneously an instance of the subform to edit the width and a second instance to edit the height but we can still take a “master/slave approach” where a single instance of the subform will be used to edit the width and the height separately or mimic an “update in place feature” where an instance of the subform will replace the display of the width or height.

A way to implement our example using these principles could be:

  • In the master form:
    • Define an instance used as an interface with the subform to carry the
      value to edit and identify the active subform instance.
    • Include triggers to load and unload subforms.
    • Define actions to load and unload the subforms and maintain the
      “interface” instance.
    • Control when to display the triggers to avoid that simultaneous loads of
      the subform.
  • In the subforms:
    • Synchronize the local model with the instance used as an interface.
    • Perform all the logic attached to the component.

The master form would then be:

<xh:html xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:ev="http://www.w3.org/2001/xml-events"
    xmlns:xf="http://www.w3.org/2002/xforms">
    <xh:head>
        <xh:title>Subforms</xh:title>
        <xf:model id="master">
            <xf:instance id="main">
                <figures>
                    <rectangle height="10in" width="4em"/>
                </figures>
            </xf:instance>

            <!-- Instance used as an "interface" with the subform -->
            <xf:instance id="dimension-interface">
                <dimension active=""/>
            </xf:instance>
        </xf:model>

        <!-- Dirty hack to style controls inline -->
        <xh:style type="text/css"><![CDATA[

.xfContainer div {
    display: inline !important;
}

.xfContainer span {
    display: inline !important;
}
]]>
        </xh:style>
    </xh:head>
    <xh:body>
        <xf:group ref="rectangle">
            <!-- Height -->
            <xf:group ref="@height">
                <xf:label>Height: </xf:label>
                <!-- This should be displayed when the subform is not editing the height -->
                <xf:group ref=".[instance('dimension-interface')/@active!='height']">
                    <xf:output ref="."/>
                    <!-- Display the trigger when the subform is not loaded anywhere -->
                    <xf:trigger ref=".[instance('dimension-interface')/@active = '']">
                        <xf:label>Edit</xf:label>
                        <xf:action ev:event="DOMActivate">
                            <!-- Set the value of the interface instance -->
                            <xf:setvalue ref="instance('dimension-interface')" value="instance('main')/rectangle/@height"/>
                            <!-- Remember that we are editing the height -->
                            <xf:setvalue ref="instance('dimension-interface')/@active">height</xf:setvalue>
                            <!-- Load the subform -->
                            <xf:load show="embed" targetid="height" resource="subform-embedded.xhtml"/>
                        </xf:action>
                    </xf:trigger>
                </xf:group>
                <xh:div id="height"/>
                <!-- This should be displayed only when we're editing the height -->
                <xf:group ref=".[instance('dimension-interface')/@active='height']">
                    <xf:trigger>
                        <xf:label>Done</xf:label>
                        <xf:action ev:event="DOMActivate">
                            <!-- Copy the value from the interface instance -->
                            <xf:setvalue value="instance('dimension-interface')" ref="instance('main')/rectangle/@height"/>
                            <!-- We're no longer editing any dimension -->
                            <xf:setvalue ref="instance('dimension-interface')/@active"/>
                            <!-- Unload the subform -->
                            <xf:load show="none" targetid="height"/>
                        </xf:action>
                    </xf:trigger>
                </xf:group>
            </xf:group>
            <br/>
            <!-- Width -->
            <xf:group ref="@width">
                <xf:label>Width: </xf:label>
                <xf:group ref=".[instance('dimension-interface')/@active!='width']">
                    <xf:output ref="."/>
                    <xf:trigger ref=".[instance('dimension-interface')/@active = '']">
                        <xf:label>Edit</xf:label>
                        <xf:action ev:event="DOMActivate">
                            <xf:setvalue ref="instance('dimension-interface')" value="instance('main')/rectangle/@width"/>
                            <xf:setvalue ref="instance('dimension-interface')/@active">width</xf:setvalue>
                            <xf:load show="embed" targetid="width" resource="subform-embedded.xhtml"/>
                        </xf:action>
                    </xf:trigger>
                </xf:group>
                <xh:div id="width"/>
                <xf:group ref=".[instance('dimension-interface')/@active='width']">
                    <xf:trigger>
                        <xf:label>Done</xf:label>
                        <xf:action ev:event="DOMActivate">
                            <xf:setvalue value="instance('dimension-interface')" ref="instance('main')/rectangle/@width"/>
                            <xf:setvalue ref="instance('dimension-interface')/@active"/>
                            <xf:load show="none" targetid="width"/>
                        </xf:action>
                    </xf:trigger>
                </xf:group>
            </xf:group>
        </xf:group>
    </xh:body>
</xh:html>

And the subform:

<xh:div xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:ev="http://www.w3.org/2001/xml-events"
    xmlns:xf="http://www.w3.org/2002/xforms">
    <xf:model id="dimension-model">
        <xf:instance id="concat">
            <data/>
        </xf:instance>
        <xf:instance id="split">
            <height>
                <value/>
                <unit/>
            </height>
        </xf:instance>
        <!-- Get the value from the "interface" instance and initialize the   -->
        <xf:submission id="get-dimension-value" resource="model:master#instance('dimension-interface')/*"
            replace="instance" method="get">
            <xf:action ev:event="xforms-submit-done">
                <!--<xf:message level="ephemeral">Subform has updated itself.</xf:message>-->
                <xf:setvalue ref="instance('split')/value" value="translate(instance('concat'), '%incmptxe', '')"/>
                <xf:setvalue ref="instance('split')/unit" value="translate(instance('concat'), '0123456789', '')"/>
            </xf:action>
            <xf:message ev:event="xforms-submit-error" level="ephemeral">Error while subform update.</xf:message>
        </xf:submission>
        <xf:send ev:event="xforms-ready" submission="get-dimension-value"/>
        <xf:submission id="set-dimension-value" resource="model:master#instance('dimension-interface')/*" replace="none"
            method="post">
            <xf:action ev:event="xforms-submit-done">
                <!--<xf:message level="ephemeral">Main form has been updated</xf:message>-->
            </xf:action>
            <xf:message ev:event="xforms-submit-error" level="ephemeral">Error while main form update.</xf:message>
        </xf:submission>
    </xf:model>
    <xf:group ref="instance('split')">
        <xf:input ref="value">
            <xf:action ev:event="xforms-value-changed">
                <xf:setvalue ref="instance('concat')" value="concat(instance('split')/value, instance('split')/unit)"/>
                <xf:send submission="set-dimension-value"/>
            </xf:action>
        </xf:input>
        <xf:select1 ref="unit">
            <xf:action ev:event="xforms-value-changed">
                <xf:setvalue ref="instance('concat')" value="concat(instance('split')/value, instance('split')/unit)"/>
                <xf:send submission="set-dimension-value"/>
            </xf:action>
            <xf:item>
                <xf:label>pixels</xf:label>
                <xf:value>px</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>font size</xf:label>
                <xf:value>em</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>font height</xf:label>
                <xf:value>ex</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>inches</xf:label>
                <xf:value>in</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>centimeters</xf:label>
                <xf:value>cm</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>millimeters</xf:label>
                <xf:value>mm</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>points</xf:label>
                <xf:value>pt</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>picas</xf:label>
                <xf:value>pc</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>%</xf:label>
                <xf:value>%</xf:value>
            </xf:item>
        </xf:select1>
    </xf:group>
</xh:div>

The code for defining the subform has the same level of complexity than the definition of the XBL in Orbeon Forms but a lot of geeky stuff needs to be added around the invocation of the form which becomes tricky.

From a user perspective, the page would initially look like:

When a user clicks on one of the “Edit” buttons, the corresponding subform is loaded (note that all the “Edit” buttons have disappeared):

Once the user is done editing the values in this subform, (s)he can click on “Done” to come back to a state where both the height and width are displayed and can be edited:

The presentation can be improved replacing for instance the buttons by trendy icons but we had to bend our requirements to get something that can be implemented with subforms.

Of course here we are misusing subforms to implement components, something which was not a design goal, and it’s not surprising that the resulting code is more verbose and that we’ve had to accept a different user interface. The future component feature announced by Joern Turner should solve these glitches.

Subforms, XSLTForms flavor

The support of subforms in XSLTForms is illustrated by a sample: a writers.xhtml master form embeds a books.xhtml subform.

The main principle behind this subform implementation appears to be the same than for betterFORM but there are some important differences between these two implementations:

  • XSLTForms doesn’t isolate the models from the master form and its subform and it is
    possible to access directly to any instance of the master form from the subforms.
  • The features to communicate between models implemented by betterFORM are thus not necessary and do not exist in XSLTForms.
  • The context node is not isolated and is available directly from the controls in the
    subform (see the writers/books example for an illustration.
  • A specific action (xf:unload) is used to unload subforms in XSLTForms while an xf:load action with an @show="none" attribute is used in betterFORM for the same purpose.

With these differences, the code developed for betterFORM could be adapted to work with XSLTForms as:

<?xml-stylesheet href="xsltforms/xsltforms.xsl" type="text/xsl"?>
<?xsltforms-options debug="yes"?>
<xh:html xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:ev="http://www.w3.org/2001/xml-events"
   xmlns:xf="http://www.w3.org/2002/xforms">
   <xh:head>
      <xh:title>Subforms</xh:title>
      <xf:model id="master">
         <xf:instance id="main">
            <figures>
               <rectangle height="10in" width="4em"/>
            </figures>
         </xf:instance>

         <!-- Instance used as an "interface" with the subform -->
         <xf:instance id="dimension-interface">
            <dimension active=""/>
         </xf:instance>
      </xf:model>

      <!-- Dirty hack to style controls inline -->
      <xh:style type="text/css"><![CDATA[

.xforms-group-content, .xforms-group, span.xforms-control, .xforms-label {
   display:inline; 
}

]]>
      </xh:style>
   </xh:head>
   <xh:body>
      <xf:group ref="rectangle">
         <!-- Height -->
         <xf:group ref="@height">
            <xf:label>Height: </xf:label>
            <!-- This should be displayed when the subform is not editing the height -->
            <xf:group ref=".[instance('dimension-interface')/@active!='height']">
               <xf:output ref="."/>
               <!-- Display the trigger when the subform is not loaded anywhere -->
               <xf:trigger ref=".[instance('dimension-interface')/@active = '']">
                  <xf:label>Edit</xf:label>
                  <xf:action ev:event="DOMActivate">
                     <!-- Set the value of the interface instance -->
                     <xf:setvalue ref="instance('dimension-interface')" value="instance('main')/rectangle/@height"/>
                     <!-- Remember that we are editing the height -->
                     <xf:setvalue ref="instance('dimension-interface')/@active">height</xf:setvalue>
                     <!-- Load the subform -->
                     <xf:load show="embed" targetid="height" resource="subform-embedded.xml"/>
                  </xf:action>
               </xf:trigger>
            </xf:group>
            <xh:span id="height"/>
            <!-- This should be displayed only when we're editing the height -->
            <xf:group ref=".[instance('dimension-interface')/@active='height']">
               <xf:trigger>
                  <xf:label>Done</xf:label>
                  <xf:action ev:event="DOMActivate">
                     <!-- Copy the value from the interface instance -->
                     <xf:setvalue value="instance('dimension-interface')" ref="instance('main')/rectangle/@height"/>
                     <!-- We're no longer editing any dimension -->
                     <xf:setvalue ref="instance('dimension-interface')/@active"/>
                     <!-- Unload the subform -->
                     <xf:unload targetid="height"/>
                  </xf:action>
               </xf:trigger>
            </xf:group>
         </xf:group>
         <br/>
         <!-- Width -->
         <xf:group ref="@width">
            <xf:label>Width: </xf:label>
            <xf:group ref=".[instance('dimension-interface')/@active!='width']">
               <xf:output ref="."/>
               <xf:trigger ref=".[instance('dimension-interface')/@active = '']">
                  <xf:label>Edit</xf:label>
                  <xf:action ev:event="DOMActivate">
                     <xf:setvalue ref="instance('dimension-interface')" value="instance('main')/rectangle/@width"/>
                     <xf:setvalue ref="instance('dimension-interface')/@active">width</xf:setvalue>
                     <xf:load show="embed" targetid="width" resource="subform-embedded.xml"/>
                  </xf:action>
               </xf:trigger>
            </xf:group>
            <xh:span id="width"/>
            <xf:group ref=".[instance('dimension-interface')/@active='width']">
               <xf:trigger>
                  <xf:label>Done</xf:label>
                  <xf:action ev:event="DOMActivate">
                     <xf:setvalue value="instance('dimension-interface')" ref="instance('main')/rectangle/@width"/>
                     <xf:setvalue ref="instance('dimension-interface')/@active"/>
                     <xf:unload targetid="width"/>
                  </xf:action>
               </xf:trigger>
            </xf:group>
         </xf:group>
      </xf:group>
   </xh:body>
</xh:html>

for the main form and:

<?xml-stylesheet href="xsltforms/xsltforms.xsl" type="text/xsl"?>
<?xsltforms-options debug="yes"?>
<xh:html xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:xf="http://www.w3.org/2002/xforms"
   xmlns:ev="http://www.w3.org/2001/xml-events">
   <xh:head>
      <xh:title>A subform</xh:title>
      <xf:model id="subform-model">
         <xf:instance id="split">
            <height>
               <value/>
               <unit/>
            </height>
         </xf:instance>
         <xf:action ev:event="xforms-subform-ready">
            <xf:setvalue ref="instance('split')/value" value="translate(instance('dimension-interface'), '%incmptxe', '')"/>
            <xf:setvalue ref="instance('split')/unit" value="translate(instance('dimension-interface'), '0123456789', '')"/>
         </xf:action>
      </xf:model>
   </xh:head>
   <xh:body>
      <xf:group ref="instance('split')">
         <xf:input ref="value">
            <xf:label/>
            <xf:setvalue ev:event="xforms-value-changed" ref="instance('dimension-interface')" value="concat(instance('split')/value, instance('split')/unit)"/>
         </xf:input>
         <xf:select1 ref="unit">
            <xf:label/>
            <xf:setvalue ev:event="xforms-value-changed" ref="instance('dimension-interface')" value="concat(instance('split')/value, instance('split')/unit)"/>
            <xf:item>
               <xf:label>pixels</xf:label>
               <xf:value>px</xf:value>
            </xf:item>
            <xf:item>
               <xf:label>font size</xf:label>
               <xf:value>em</xf:value>
            </xf:item>
            <xf:item>
               <xf:label>font height</xf:label>
               <xf:value>ex</xf:value>
            </xf:item>
            <xf:item>
               <xf:label>inches</xf:label>
               <xf:value>in</xf:value>
            </xf:item>
            <xf:item>
               <xf:label>centimeters</xf:label>
               <xf:value>cm</xf:value>
            </xf:item>
            <xf:item>
               <xf:label>millimeters</xf:label>
               <xf:value>mm</xf:value>
            </xf:item>
            <xf:item>
               <xf:label>points</xf:label>
               <xf:value>pt</xf:value>
            </xf:item>
            <xf:item>
               <xf:label>picas</xf:label>
               <xf:value>pc</xf:value>
            </xf:item>
            <xf:item>
               <xf:label>%</xf:label>
               <xf:value>%</xf:value>
            </xf:item>
         </xf:select1>
      </xf:group>
   </xh:body>
</xh:html>

for the subform.

Acknowledging that things could be easier, XSLTForms has introduced a new experimental feature, derived from subforms, to implement simple components:

I have implemented a new component control in XSLTForms. It is named "xf:component” and has two attributes named “@ref” and “@resource“. There are still restrictions within a component: ids cannot be used if the component is to be instantiated more than once. The default instance is local to each instantiated component and the subform-instance() function can be used to get the document element of it. From the main form to the component, a binding with a special mip named “changed” is defined. The subform-context() allows to reference the node bound to the component control in the main form. The corresponding build has been committed to repositories: http://sourceforge.net/p/xsltforms/code/ci/master/tree/build/
Alain Couthures on the Xsltforms-support mailing list

With this new experimental feature and another extension (the @changed MIP implemented in XSLTForms), the master form would be:

<?xml-stylesheet href="xsltforms/xsltforms.xsl" type="text/xsl"?>
<?xsltforms-options debug="yes"?>
<xh:html xmlns:xh="http://www.w3.org/1999/xhtml" xmlns:ev="http://www.w3.org/2001/xml-events"
   xmlns:xf="http://www.w3.org/2002/xforms">
   <xh:head>
      <xh:title>Subforms</xh:title>
      <xf:model>
         <xf:instance id="main">
            <figures>
               <rectangle height="10in" width="4em"/>
            </figures>
         </xf:instance>

      </xf:model>
   </xh:head>
   <xh:body>
      <xf:group ref="rectangle">
         <!-- Height -->
         <xf:group ref="@height">
            <xf:label>Height: </xf:label>
            <xf:component ref="." resource="component-subform.xml"/>
         </xf:group>
         <br/>
         <!-- Width -->
         <xf:group ref="@width">
            <xf:label>Width: </xf:label>
            <xf:component ref="." resource="component-subform.xml"/>
         </xf:group>
      </xf:group>
   </xh:body>
</xh:html>

and the subform (or component):

<?xml-stylesheet href="xsltforms/xsltforms.xsl" type="text/xsl"?>
<xh:html xmlns:xh="http://www.w3.org/1999/xhtml" 
    xmlns:xf="http://www.w3.org/2002/xforms" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xmlns:ev="http://www.w3.org/2001/xml-events">
    <xh:head>
        <xh:title>Size</xh:title>
        <xf:model>
            <xf:instance>
                <size>
                    <value xsi:type="xsd:decimal">2</value>
                    <unit>cm</unit>
                </size>
            </xf:instance>
            <xf:bind ref="subform-instance()/value" 
                changed="translate(subform-context(), '%incmptxe', '')"/>
            <xf:bind ref="subform-instance()/unit" 
                changed="translate(subform-context(), '0123456789', '')"/>
        </xf:model>
    </xh:head>
    <xh:body>
        <xf:input ref="subform-instance()/value">
            <xf:label/>
            <xf:setvalue ev:event="xforms-value-changed" 
                ref="subform-context()" value="concat(subform-instance()/value, 
                subform-instance()/unit)"/>
        </xf:input>
        <xf:select1 ref="subform-instance()/unit">
            <xf:label/>
            <xf:item>
                <xf:label>pixels</xf:label>
                <xf:value>px</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>font size</xf:label>
                <xf:value>em</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>font height</xf:label>
                <xf:value>ex</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>inches</xf:label>
                <xf:value>in</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>centimeters</xf:label>
                <xf:value>cm</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>millimeters</xf:label>
                <xf:value>mm</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>points</xf:label>
                <xf:value>pt</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>picas</xf:label>
                <xf:value>pc</xf:value>
            </xf:item>
            <xf:item>
                <xf:label>%</xf:label>
                <xf:value>%</xf:value>
            </xf:item>
            <xf:setvalue ev:event="xforms-value-changed" 
                ref="subform-context()" value="concat(subform-instance()/value, 
                subform-instance()/unit)"/>
        </xf:select1>
    </xh:body>
</xh:html>

The level of complexity of both the definition of the subform component and its invocation are similar to what we’ve seen with Orbeon’s XBL feature. The main difference is the encapsulation (no encapsulation in XSLTForms and a controlled encapsulation in Orbeon Forms which handles the issue of id collisions).

Note that we are escaping the issue caused by id collision because we are accessing the instance from the master form directly from the subform using the subform-context() function.

This feature allows us to use only one local instance in the subform and we take care of not defining any id for this instance and access it using the subform-instance() function.
This trick wouldn’t work if we needed several instances or if we had to define ids on other elements in the subform.

Conclusion

The lack of modularity has been one of the serious weaknesses in the XForms recommendations so far.

A common solution is to generate or “template” XForms but this can be tricky when dealing with “components” used multiple times in a form and especially within xf:repeat controls.

Different implementation have come up with different solutions to address this issue (XBL for Orbeon, subforms for betterFORM and XSLTForms).

The main differences between these solutions are:

  • The syntax:
    • XBL + XForms for Orbeon Forms
    • XForms with minor extensions for betterFORM and XSLTForms)
  • The encapsulation or isolation and features to communicate between the component and other models:
    • complete for betterFORM with extensions to communicate between models
    • either complete or partial for Orbeon Forms with extension to communicate between models
    • no isolation for XSLTForms with extensions to access to the context node and default instance from a component
  • The support of id collisions between components and the main form:
    • Id collisions are handled by Orbeon Forms
    • They are forbidden by betterFORM and XSLTForms

The lack of interoperability between these implementations will probably not be addressed by the W3C XForms Working Group and it would be very useful if XForms implementers could work together to define interoperable solutions to define reusable components in XForms.

In this paper, generation (or templating) has been presented as an alternative to XML or subforms but they are by no mean exclusive. In real world projects, hybrid approaches mixing XForms generation (or templating) and components (XBL or subforms) are on the contrary very valuable. They have been demonstrated in a number of talks during the pre-conference day at XML Prague.

These hybrid approaches are easy to implement with common XML toolkits. The generation/templating can be static (using tools such as XProc, Ant or classical make files) or dynamic (using XProc or XPL pipelines or plain XQuery or XSLT) and Orbeon Forms XBL implementation even provides a feature to dynamically invoke a transformation on the content of the bound element).

Acknowledgments

I would like to thank Erik Bruchez (Orbeon), Joern Turner (betterFORM) and Alain Couthures (XSLTForms) for the time they’ve spent to answer my questions and review this paper.

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites

When MVC becomes a burden for XForms

XForms describes itself as a MVC architecture:

An XForm allows processing of data to occur using three mechanisms:

  • a declarative model composed of formulae for data calculations and constraints, data type and other property declarations, and data submission parameters
  • a view layer composed of intent-based user interface controls
  • an imperative controller for orchestrating data manipulations, interactions between the model and view layers, and data submissions.

Micah Dubinko argues that the mapping is more obvious with Model-view-presenter (MVP), a derivative of the MVC software pattern but that’s not the point I’d like to make and I’ll stick to the MVC terminology where:

  • The model is composed of XForms instances and binds
  • The view is composed of the XForms controls together with the HTML elements and CSS stylesheets
  • The controller is composed of the actions
Mode-view-controler on wikimedia
Mode-view-controler on wikimedia

Orbeon Form Builder goes one step forward and adds a fourth concern for localization and we get a model/view/localization/controller pattern.

This separation of concerns is great to differentiate different roles and split work between specialists but doesn’t play well with modularity and reusability.

I am currently working on a project to develop big and complex forms and this is becoming one of the biggest issues: these forms share a number of common fields and group of fields and, not even speaking of sharing these definitions, this separation of concerns adds a significant burden when copying these definitions from one form to another.

To copy a field from one form to another you need to copy definitions from the model, the view, the localization and the controller and can’t just copy a “component”.

And of course, there is no easy way to reuse common components instead of copying them.

I am not the first to question this issue with MVC and the Hierarchical model–view–controller (HMVC) has been introduced for this purpose, but how can we use such a pattern with XForms?

Hierarchical model-view-controller in JavaWorld
Hierarchical model-view-controller in JavaWorld

In fact, I m wondering if some of the innovative ways to develop XForms application which have been presented during the XML Prague 2013 pre-conference day were not more or less ways to deal with this issue.

For Orbeon Forms the way to deal with reusability issues is through XBL components.

XBL components cleanly encapsulate their own model/view/controller concerns but their development is highly technical and can’t be defined using Form Builder: you can use XBL components in Form Builder but you have to create them by hand and there is no way to just say “I want to define this group of fields as a new XBL component and use this component in other forms”. Such a feature would be most useful!

Another way to deal with this issue is to use a meta model and generate XForms. I am involved in another project which is following this path and you can expect to hear more about it in these pages!

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites

XDM Maps should be first class citizens

Note: This issue has been submitted to the W3C as #16118.

The XPath/XQuery/XSLT 3.0 Data model distinguishes three types of information items:

  • Nodes that directly relate to the XML Infoset with some information borrowed from the PSVI.
  • Functions
  • Atomic types.

Michael Kay has recently proposed to add maps as a fourth item type derived from functions.

The main motivation for this addition is to support JSON objects that can be considered as a subset of maps items.

However, in the current proposal map items are treated very differently from XML nodes and this has deep practical consequences.

Take for instance the following simple JSON sample borrowed from Wikipedia:

{
     "firstName": "John",
     "lastName" : "Smith",
     "age"      : 25,
     "address"  :
     {
         "streetAddress": "21 2nd Street",
         "city"         : "New York",
         "state"        : "NY",
         "postalCode"   : "10021"
     },
     "phoneNumber":
     [
         {
           "type"  : "home",
           "number": "212 555-1234"
         },
         {
           "type"  : "fax",
           "number": "646 555-4567"
         }
     ]
 }

To get the postalCode from an equivalent structure expressed as XML and stored in the variable $person, one would just use the following XPath expression: $person/address/postalCode.

When the same structure is expressed in JSON and parsed into an XDM map, XPath axes can no longer be used (their purpose is to traverse documents, ie nodes) and we need to use map functions: map:get(map:get($person, 'address'), 'postcalCode').

That’s not as bad as it sounds because maps can be invoked as functions and this can be rewritten as $person('address')('postalCode') but this gives a first idea of the deep differences between maps and nodes and things would become worse if I wanted to get the postal code of persons whose first name are “John”…

Another important difference is that node items are the only ones that have a context or an identity.

When I write <foo><bar>5</bar></foo><bat></bar>5</bar></bat> each of the two bar elements happen to have the same names and values but they are considered as two different elements and even the two text nodes that are their children are two different text nodes.

When I write foo: {bar: 5}, bat: {bar: 5} the two bar entries are actually the same thing and can’t be distinguished.

This difference is important because that means that XPath axes as we know them for nodes could never be implemented on maps: if an entry in a map can’t be distinguished from en identical entry else where in another map there is no hope to be able to determine its parent for instance.

Now, why is it important to be able to define axes on maps and map entries?

I think that this is important for XSLT and XQuery users to be able to traverse maps like they traverse XML fragments (with the same level of flexibility and syntaxes that are kept as close as possible). And yes, that means being able to apply templates over maps and be able to update maps using XQuery update…

But I also think that this will be important to other technologies that rely on XPath such as (to name those I know best) XForms, pipeline languages (XPL, XPROC, …) and Schematron.

Being able to use XForms to edit JSON object is an obvious need that XForms 2.0 is trying to address through a “hack” that has been presented at XML Prague 2012.

In a longer term we can hope that XForms will abandon this hack to rely on XDM maps XForms relies a lot on the notions of nodes and axes. XForms binds controls to instance nodes and the semantics of such bindings would be quite different to be applied to XDM map entries as currently proposed.

XML pipeline languages are also good candidates to support JSON objects. Both XPL and XProc have features to loop over document fragments and choose actions depending on the results of XPath expressions and again the semantics of these features would be affected if they had to support XDM maps as currently proposed.

Schematron could be a nice answer to the issue of validating JSON objects. Schematron relies on XPath at two different levels: its rules are defined as XPath expressions and it is often very convenient to be able to use XPath axes such as ancestor and its processing model is defined in term of traversing a tree. Again, an update of Schematron to support maps would be more difficult is maps are not similar to XML nodes.

Given the place of JSON on the web, I think that it is really important to support maps and the question we have to face is: “do we want a minimal level of support that may require hard work from developers and other standards to support JSON or do we want to make it as painless as possible for them?”.

And obviously, my preference is the later: if we add maps to the XDM, we need to give them full citizenship from the beginning!

Note: The fact that map entries are unordered (and they need to be because the properties of JSON objects are unordered) is less an issue to me. We already have two node types (namespaces nodes and attributes) which relative order are “stable but implementation-dependent”.

 

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites

Owark WordPress plugin v0.1

I am proud to announce that I have installed the very first version of my owark WordPress plugin on this blog.

Note: the plugin is still in an early stage and I wouldn’t recommend to install it on your blog!

Standing for Open Web Archive, owark is a project that I’ll be presenting at OSCON 2011.

This first version is only a small piece in the bigger vision I’ll be presenting at OSCON, but I find it already pretty cool…

The plugin relies on the Broken Link Checker to harvest the links in the blog content and check for broken links and on GNU Wget to perform the archiving itself.

I had been archiving links for a while with a bash script and I have already some stuff in my archive database so that this plugin doesn’t start from scratch and can take advantage of this history.

These are just a couple of simple examples, but I am happy with the progress so far…

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites

XML Prague 2011 : XML à l’attaque du web

Salle de conférence pendant la pause café

Après une période un peu folle entre 2000 et 2008 pendant laquelle j’ai participé à un nombre impressionnant de conférences, je m’étais mis un peu en retrait et n’avais plus participé à aucune conférence depuis XTech 2008.

XML Prague 2011 était donc pour moi l’occasion de rencontrer à nouveau la communauté des experts XML internationaux et j’étais curieux de voir comment elle avait évolué pendant ces trois dernières années.

MURATA Makoto (EPUB3: Global Language and Comic)A côté des aspects plus techniques, je n’oublierai pas l’image de Murata Makoto exprimant sobrement sa peine pour les victimes du tremblement de terre au Japon.

La tagline de XML Prague 2011 était “XML devait être l’espéranto du Web. Pourquoi n’est-ce pas le cas?” (“XML as new lingua franca for the Web. Why did it never happen?”).

Michael Sperberg-McQueen (Closing keynote)Le contenu de la conférence est resté proche de cette ligne mais il a été résumé de manière plus exacte par Michael Sperberg-McQueen lors de sa clôture : “Mettons du XML dans le navigateur, qu’ils le veuillent ou non!”

Norman Walsh (HTML+XML: The W3C HTML/XML Task Force)Le ton a été donné par Norman Walsh dès la toute première présentation: la convergence entre HTML et XML n’aura pas lieu.

XML a tenté d’être un format neutre convenant aussi bien aux documents qu’aux données sur le web. On peut dire aujourd’hui que cet objectif n’a pas été atteint et que les formats les plus populaires sur le web sont HTML pour les documents et JSON pour les données.

Cela ne semble pas préoccuper plus que mesure le public de XML Prague composé d’aficionados des langages à balises : si la “masse des développeurs web” n’est pas intéressée par XML c’est son problème. Les bénéfices liés à XML sont bien connus et cela signifie simplement que la communauté XML devra développer les outils nécessaires pour utiliser XML dasn le navigateur aussi bien que sur le serveur.

Sur ce thème, beaucoup de présentations couvraient le support de XML dans le navigateur ainsi que les passerelles entre JSON et XML :

  • Validation XML Schema côté client par Henry S. Thompson and Aleksejs Goremikins
  • JSON pour XForms par Alain Couthures
  • XSLT dans le navigateur par Michael Kay
  • Traitement de XML efficace dans les navigateurs par Alex Milowski
  • XQuery dans le navigateur par Peter Fischer

Les outils côté serveurs ont fait l’objet de moins de sessions, peut être parce que le sujet est plus ancien :

  • Une façade JSON pour le serveur MarkLogic par Jason Hunter
  • CXAN: étude de cas pour Servlex, un framework XML pour le web par Florent Georges
  • Akara – “Spicy Bean Fritters” et services de données XML par Uche Ogbuji

Bien entendu, les standards étaient aussi au programme :

  • HTML+XML: la task force W3C HTML/XML (déjà mentionnée) par Norman Walsh
  • Standards update: XSLT 3.0 par Michael Kay
  • Standards update: XML, XQuery, XML Processing Profiles, DSDL par Liam Quin, Henry S. Thompson, Jirka Kosek

Ainsi que les applications de XML :

  • Configuration d’équipements réseau avec NETCONF et YANG par Ladislav Lhotka
  • Développements XML – XML Projects par George Bina
  • EPUB3: le langage et les bandes dessinées par Murata Makoto
  • EPUB: Chapitres et versets par Tony Graham
  • DITA NG – une implémentation Relax NG de DITA par George Bina

Sans oublier quelques présentations techniques sur les implémentations elles mêmes :

  • Traduction de SPARQL et SQL en XQuery par Martin Kaufmann
  • Réécritures déclaratives de XQuery pour le profit et le plaisir par John Snelson

Et la séance de clôture par le roi de cet excercice, Michael Sperberg-McQueen.

Ma présentation, “injection XQuery”, était assez atypique dans cet ensemble et il a fallu tout le talent de Michael Sperberg-McQueen pour lui trouver un point commun en faisant remarquer que pour avoir une chance de mettre XML sur le web il faudrait se préoccuper un peu plus de sécurité.

J’avais été impressionné lors des conférences XTech par l’évolution des techniques de présentation, la plupart des intervenants rejetant les traditionnelles présentation powerpoint et leurs “transparents” surchargés pour des alternatives plus légères et beaucoup plus imagées.

John Snelson (Declarative XQuery Rewrites for Profit or Pleasure)Je pensais ce mouvement inéluctable et ai été bien surpris de voir qu’il n’avait guère atteint les intervenants de XML Prague 2011 qui (à l’exception très notable de John Snelson) continuaient à utiliser powerpoint de manière très traditionnelle.

J’avais conçu ma présentation en suivant ce que je croyais être la technique de présentation devenue classique. Utilisant Slidy, j’avais pas moins de 35 pages très concises à présenter en 25 minutes. Chaque page avait une photo différente en arrière plan et ne comprenait que quelques mots.

Les commentaires ont été plutôt positifs bien que certaines photos d’injections aient choqué quelques participants.

Ma présentation étant du HTML standard, j’avais jugé plus sur d’utiliser l’ordinateur mis à disposition par les organisateurs. C’était sans compter sur les 74 Moctets d’images à charger pour les fonds de pages qui ont mis à mal cet ordinateur un peu poussif et les pages étaient un peu lentes à l’affichage (note personnelle : la prochaine fois, utilise ton ordinateur)!

The twitter wall (and Norman Walsh)Le “mur twitter” projeté au moyen d’un second vidéo projecteur a eu également beaucoup de succès.

Ce mur a été bien pratique pour communiquer pendant les sessions et il remplace avantageusement les canaux IRC que nous utilisions auparavant.

Twitter ne permet malheureusement pas de rechercher dans ses archives et, alors que j’écris ces mots, je ne peux déjà plus accéder aux tweets du premier jour de la conférence!

Avec un peu de recul, si j’essaye d’analyser ce qui s’est dit à XML Prague 2011, j’ai des sentiments mitigés à propos de ce fossé qui se creuse entre communautés Web et XML.

Le rêve que XML puisse être accepté par l’ensemble de la communauté des développeurs web était une vision très forte et nous ne devons pas oublier que XML a été conçu pour mettre “SGML sur le web“.

Ceci dit, il faut bien reconnaître que les développeurs web ont toujours été réticents devant la complexité additionnelle (réelle ou perçue) de XHTML. Ce fossé a toujours existé et après que XML ait manqué le virage du Web 2.0 il était trop tard pour espérer le combler.

XML sur le web restera donc une niche et continuera à être utilisé par une minorité, mais la créativité et le dynamisme de la communauté qui s’est manifesté à Prague est impressionnant et encourageant : il y a encore place pour beaucoup d’innovations et XML est, plus que jamais, une technologie de choix pour développer des applications web.

Photos

Share and Enjoy:
  • Identi.ca
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Add to favorites