What is XSpec?

An introduction to XSpec, a unit testing framework for XML technologies

XSpec is an open source unit test and behaviour driven development framework for XSLT and XQuery. XSpec consists of a syntax for describing the behaviour of XSLT and XQuery and code enabling to write tests against those descriptions.

Testing is a fundamental part of writing software that aims to be robust, reliable, and maintainable. In fact, testing can be considered as a promise made to customers and users that the code behaves as intended. Writing tests regularly also improves the code base as it forces developers to write smaller units of code that can be more easily tested, debugged, and maintained. Finally, testing acts as self-documentation and can help other developers to understand and modify existing code.

A unit test is typically written to test an individual unit of code, e.g. a function or a method. In test driven development unit tests are usually written by developers as they write their code in order to make sure that new features work according to specifications and bug fixes do not break other parts of the code base.

Although testing is important for any serious software developer, there aren't many testing tools for XSLT and XQuery when compared to other programming languages. Furthermore, their use is not yet very widespread. XSpec aims to fill this gap by offering a testing framework and raising awareness about testing in the XML community.

XSpec is hosted on GitHub and released v0.5.0 in January 2017. Documentation on how to use XSpec is available in the official wiki which also contains instructions to install and run XSpec on Windows and MacOS/Linux. XSpec is also integrated by default in the Oxygen XML editor.

Testing a template

Let's dive into XSpec and start with a simple example of an XSpec test checking that an element title is always converted into a h1 element in the XSLT code. To follow test driven development practices we start by writing the XSpec test to match these specifications:

<?xml version="1.0" encoding="UTF-8"?>
<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec"
               xmlns:xs="http://www.w3.org/2001/XMLSchema" 
			   xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    		   stylesheet="title.xsl">

    <x:scenario label="When converting a title element">
        <x:context>
            <title>My Title</title>
        </x:context>

        <x:expect label="it should return a h1 element">
            <h1>My Title</h1>
        </x:expect>

    </x:scenario>

</x:description>

The XSpec test is contained in x:scenario and the label attribute provides a human-readable description of the test. The x:context provides a node that is passed to the XSLT when the test is executed. This works like a mock object, i.e. a simple snippet of XML created for testing purposes. According to our specifications we pass to the XSLT a title element with a text node.

Each scenario can have one or more expectations included inside x:expect. These are statements that should be true when running the XSpec test against the XSLT code. Again, the label attribute provides a human-readable description of the expectation. Following our specifications, we expect the title element to be transformed into a h1 element and the text node to be replicated as it is. We link the XSpec test to the relevant XSLT code - which we have not written yet - at the top in the stylesheet attribute of x:description. Here we link the test to the XSLT stored in title.xsl.

We now write the XSLT code in title.xsl. In order to show what happens when a test fails, we write a match template that does not follow the specifications and converts a title element into a h2 element instead of h1:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    		    xmlns:xs="http://www.w3.org/2001/XMLSchema" 
			    exclude-result-prefixes="xs" 
				version="2.0">

    <xsl:template match="title">
        <h2>
            <xsl:value-of select="."/>
        </h2>
    </xsl:template>


</xsl:stylesheet>

Running the XSpec test returns a failing test with the following test report:

Let's fix the XSLT to conform with the specifications so that it converts a title element into a h1 element.

<xsl:template match="title">
    <h1>
        <xsl:value-of select="."/>
    </h1>
</xsl:template>

Re-running the XSpec now returns a passed test:

We can now be sure that the XSLT behaves as expected and future changes breaking the specifications will be reported by the XSpec test.

Note this simple test can be made more modular by embedding a mock XML file as the context - this is particularly useful when the testing data is longer than few lines - and by using the three dot notation for ignoring the value of the text node.

Testing a function

Functions and named scenarios are perfect fits for unit testing as they are self-contained pieces of code that get called and reused several times. XSpec provides specific syntax for testing functions and named templates. This time we start with the XSLT code as it is common to have a library of functions and templates already written but not yet fully tested.

This XSLT stored in capitalize_first.xsl contains a function taken from the FunctX XSLT Function Library that accepts a string as input and returns the string with the first letter capitalized:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xs="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xs" version="2.0">

    <xsl:function name="functx:capitalize-first" as="xs:string?"
        xmlns:functx="http://www.functx.com">
        <xsl:param name="arg" as="xs:string?"/>

        <xsl:sequence select="concat(upper-case(substring($arg, 1, 1)), substring($arg, 2))"/>

    </xsl:function>

</xsl:stylesheet>

Here is the XSpec code testing that the function behaves as expected:

<?xml version="1.0" encoding="UTF-8"?>
<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec" 
    xmlns:xs="http://www.w3.org/2001/XMLSchema" 
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:functx="http://www.functx.com"
    stylesheet="capitalize_first.xsl">
    
    <x:scenario label="When calling function capitalize-first with a string">
        <x:call function="functx:capitalize-first">
            <x:param name="arg" select="'hello world'"/>
        </x:call>
        
        <x:expect label="only the first character of the string is capitalized"
            select="'Hello world'"/>
    </x:scenario>
    
</x:description>

The test is associated with the file capitalize_first.xsl where the function is stored. The x:scenario contains a call to the function functx:capitalize-first using x:call. The parameter arg with the string hello world is passed to the function using x:param. The x:expect block shows the expected result, i.e. a string where only the first letter is capitalized.

Running the XSpec provides the following test report:

Integration with other tools

XSpec plays well with build and continuous integration (CI) tools so that it can be easily integrated into CI workflows. For example, it is possible to run XSpec tests via ant whereas support for maven is provided via plugins. An XProc harness for Saxon is also provided. Finally, configuring XSpec tests to run automatically when changes in the code base occur can be done using CI servers like Jenkins, Travis, and TeamCity.

Conclusion

XSpec is an open source unit test and behaviour driven development framework for XSLT and XQuery available on GitHub. If you are serious about your XSLT and XQuery code, you should definitely check out XSpec and integrate testing into your development workflow.