Using Unit Tests

Introduction

This article introduces a method for writing and using unit tests in the development of o:XML programs. The method and format is suitable for adding test suites to existing software as well as for test driven development.

The article is primarily aimed at people who are using o:XML as a programming language and wish to integrate unit or system testing. It will also be useful for those who are interested in writing or running tests for existing o:XML projects.

Unit Tests

This section should have some information about unit tests, what they are, why and how they should be used etc. Instead it has a link to the JUnit website, which has several articles on the subject. Another good place to look is the Wikipedia entry on Unit Tests.

An o:XML unit test consists of three parts: test definition, input vector and the expected result. Of these, only the test definition is required - it is possible to write tests that have no input and produce no output.

Example 1: Simplest Test


<ut:test>
  <ut:definition>
    <o:assert test="false()"/>
  </ut:definition>
</ut:test>
      

The above test always fails, because the assertion expression will always be false.

The namespace for unit tests, here denoted with the ut: prefix, is http://www.o-xml.org/namespace/unit-test/. Using namespaces ensures that the tests are kept separate from other XML content, and means that they can be easily embedded with documentation and code in o:XML source files.

Example 2: Unit Test Namespace Declaration


<ut:test xmlns:ut="http://www.o-xml.org/namespace/unit-test/">
...
</ut:test>

Next example shows an actual test from the o:XML RegularExpression implementation.

Example 3: RegularExpression Test


<ut:test>
  <ut:definition>
    <o:set pattern="RegularExpression('a*b')"/>
    <o:return select="$pattern.substitute($input, '-')"/>
  </ut:definition>
  <ut:input>aabfooaabfooabfoob</ut:input>
  <ut:result>-foo-foo-foo-</ut:result>
</ut:test>
      

The ut:test element always denotes a complete test, with or without input and result elements. The ut:definition holds the code that will be executed when the test runs. If there is an input vector, it is available in the test definition as variable $input. In the above example, the test creates a regular expression and returns the result of a substitution (regular expression string replacement). With the input string given in ut:input, the value in ut:result is what the output should look like for a successful test.

Embedded Tests

Thanks to the XML namespace feature, unit tests can be easily inlined in o:XML source code. The test can be associated with a type, a particular function or a procedure depending on where you place it in the source file.

For example, the o:XML String type contains this definition of the String.length() function:

Example 4: String.length()


<o:function name="length">
  <doc:p>Get the size of this string</doc:p>
  <doc:return>The number of characters in this String</doc:return>
  <o:do>
    ... function body ...
  </o:do>
  <ut:test>
    <ut:definition>
      <o:return select="string($input).length()"/>
    </ut:definition>
    <ut:input>how long is a piece of string?</ut:input>
    <ut:result>30</ut:result>
  </ut:test>
</o:function>
      

The close association with the source code firstly means that the tests are easy to write and maintain, and secondly that the test and reporting tools know which parts of the code the tests belong to.

Unit tests may be included anywhere in a program, though you probably want to avoid putting tests where they may be interpreted as literal result elements. Good places to include tests are inside type definitions after all the functions, and within function or procedure definitions after the o:do element. It is not generally a good idea to put tests immediately within a o:program element, as the test declarations will be included when running the program normally.

Reusable Elements

When writing more than one test you may want to reuse the same input vectors or result sets. To facilitate this, test data can be declared as named, reusable datasets, and referenced in the test definition. Below is an example taken from the o:XML Element type, where we declare a dataset and use it as the input to two different tests.

Example 5: Element Dataset


<-- first declare a dataset -->
<ut:dataset name="test-element">
  <test>
    <one>one</one>
    <two>two</two>
    <three>three</three>
  </test>
</ut:dataset>
<-- test that the name() function works -->
<ut:test>
  <ut:definition>
    <o:return select="$input.name()"/>
  </ut:definition>
  <ut:input ref="test-element"/>
  <ut:result>test</ut:result>
</ut:test>
<-- test that the string() function works -->
<ut:test>
  <ut:definition>
    <o:return select="$input.string()"/>
  </ut:definition>
  <ut:input ref="test-element"/>
  <ut:result>onetwothree</ut:result>
</ut:test>
      

Note that not only ut:input definitions may reference datasets, but ut:result as well.

You can reuse not only input and output data, but also the test definition itself. Simply declare a named ut:definition and reference it in the same way as datasets. The next example shows how it is done: First we define a test that exercises the function RegularExpression.match(). The test returns the number of words contained in the input vector. We then use the definition in two different tests, providing different test data.

Example 6: Wordcount Test Definition


<ut:definition name="wordcount">
  <o:set regex="RegularExpression('\w+')"/>
  <o:return select="count($regex.match($input))"/>
</ut:definition>
<ut:test>
  <ut:definition ref="wordcount"/>
  <ut:input>how long is a piece of string?</ut:input>
  <ut:result>7</ut:result>
</ut:test>
<ut:test>
  <ut:definition ref="wordcount"/>
  <ut:input>how long... is a-piece-of-string?</ut:input>
  <ut:result>7</ut:result>
</ut:test>
      

Standalone Tests

o:XML unit tests don't have to reside in the same file as the code you are testing. It is possible to write stand-alone test suites without even having access to the source code that you're testing against. In principle any XML file may contain the tests, though to make it easy to schema-validate the file use ut:suite as the document element. A ut:suite may contain any number of tests, datasets or test definitions, along with documentation, references and other XML data.

Running Tests

The easiest way to run a suite of tests is to use the built in functionality of the o:XML compiler/interpreter ObjectBox. Simply invoke the command line tool with the -test option to run the tests and produce an XML report. Adding in the -xsl option will automatically transform the XML test report into browser-friendly HTML format.

If you are using JDK 1.4 or newer and have downloaded objectbox.jar, the following command should work automagically. The example file testsuite.xml may contain either embedded or standalone tests.

Example 7: Running tests with ObjectBox

Run tests and produce XML report:

java -jar objectbox.jar -test testsuite.xml > results.xml

Run tests and produce HTML report:

java -jar objectbox.jar -test -xsl testsuite.xml > results.html

For more information on how to run o:XML programs, see the ObjectBox Usage Instructions or Quickstart Guide.

Manually running o:XML unit tests is generally a three step process: generate tests, execute, process results.

Generate Tests

To run your unit tests you first have to produce an executable test program. The test program is generated by an XSLT stylesheet, generate-tests.xsl. The input to the stylesheet is the file that contains your unit tests. The output is an o:XML program that will execute those tests.

Any XSLT processor can do the transformation. For example, to produce a test program from a file called testsuite.xml using Apache Xalan (an open-source XSLT processor) you could run this command:

java org.apache.xalan.xslt.Process -in testsuite.xml -out testsuite.oml -xsl generate-tests.xsl

Note that if you are using JDK 1.4 or newer you will already have Xalan installed, and the above command will work without even changing your classpath!

If you are running on Linux or Unix and have libxslt installed you might find that using xsltproc is faster and easier:

xsltproc generate-tests.xsl testsuite.xml > testsuite.oml

Execute

Running the test produces not only test results, but also information about the types, functions and procedures in the original file. This output can be saved to file for further processing.

The generated test program can be executed from the command line by the ObjectBox.

java -jar objectbox.jar testsuite.oml > results.xml

Process Results

The XSLT stylesheet report.xsl summarises the information in the test output in viewable HTML format. As with the first transformation, any XSLT processor can run this stylesheet. If you are using Xalan, the command might look like this:

java org.apache.xalan.xslt.Process -in results.xml -out results.html -xsl report.xsl

Or with xsltproc:

xsltproc report.xsl results.xml > results.html

Test Automation

To automate our tests we will use Ant build scripts. For more information about Ant see the Apache Ant website.

Same as with the manual process, automation is done in three steps: generate the tests, run them, and produce the test reports.

Generating Test Files

The first step runs our XSLT stylesheet on our source files. For each file, it produces a program that contains all the unit tests in that file. The example below shows the Ant directives that performs this function.

Example 8: Generating Test Files with Ant


<target name="tests.gen">
  <xslt basedir="src" destdir="build/docs" style="generate-tests.xsl" extension=".oml">
    <include name="**/*.oml"/><-- the source files -->
  </xslt>
</target>
	

Test Execution

As the tests were generated in the previous step, they can now be executed in the same way as any other o:XML program. We will run the tests in an automated way using the o:XML Ant task that is included with the ObjectBox distribution. For more information on running o:XML programs using Ant build scripts, see the relevant section in the ObjectBox Usage Instructions.

The output of this step is a test report in XML format, one for each test that we run.

Example 9: Test Execution with Ant


<target name="tests.run">
  <-- define the o:XML Ant task -->
  <taskdef name="obox" classname="org.oXML.extras.ant.ObjectBoxTask"/>
  <-- run the tests -->
  <obox suffix=".xml" destdir="build/docs">
    <extension name="org.oXML.extras.java.JavaExtensions"/>
    <fileset dir="build/docs"><-- the generated test program files -->
      <include name="**/*.oml"/>
    </fileset>
  </obox>
</target>
	

Test Reports

Lastly we use the XML output of the test run to produce a nicely formatted report for each file.

Example 10: Test Reports with Ant


<target name="tests.report">
  <xslt basedir="build/docs" destdir="build/docs" style="report.xsl" extension=".html">
    <include name="**/*.xml"/><-- the generated test output files -->
  </xslt>
</target>
	

Resources