XML.com: XML From the Inside Out
oreilly.comSafari Bookshelf.Conferences.


Writing and Debugging XQuery Web Apps with Qexo

June 11, 2003

In this article we will show to to write and install a very simple web application written in the XQuery language. We will also show you how errors are handled, and how you can debug them. Our application uses the servlet extension of Qexo (version 1.7beta2 or later), a free software implementation of XQuery.

Our application presents to the user a form like the following:

First screenshot of adder webapp

You can edit either field, perhaps to 10 and 3. When you click Submit, you get an updated form showing the sum of the two fields:

Second screenshot of adder webapp, after Submit

The application uses HTTP parameters to "remember" the sum from one request to the next. A more interesting application might use some kind of permanent storage or a datebase. We'll get to those in a later article; for now let's think of this as a slightly more interesting version of "Hello world!".

An XQuery program to generate a form

The following XQuery program (adder.xql) handles both the "logic" and "presentation" of our web application.

define function num-parameter($name, $default) {
  number(request-parameter($name, $default))

  <head><title>Accumulating Adder</title></head>

          <td>Result so far: </td>
              value="{num-parameter("sum1", 0)
                     +num-parameter("sum2", 0)}" />
          <td>Add to it:</td>
              value="{num-parameter("sum2", 1)}" />
      <input type="submit" value="Submit" />

The main part of our XQuery module is just a big "element constructor expression" that generates the HTML (or rather XHTML) of a single <form> element. The initial values of the <input> fields are given by embedded XQuery expressions inside {curly braces}. Those use the num-parameter function, which is defined in the Query prolog. The num-parameter function uses request-parameter to extract a named HTTP parameter from the URL. (Future Qexo versions may provide alternative ways of getting request parameters, including likely moving request-parameter to a non-default namespace.)

I'll explain the control flow of our "application" after we look at how to install and get it running.

Installing Qexo under Tomcat

Installing this application is very easy, assuming you have a web server that can run Java Servlets. I tested this application using version 4.1.24 of Tomcat, a web server written in Java and released by the Apache Foundation's Jakarta project. I assume you or someone else has already installed Tomcat, and that the environment variable $CATALINA_HOME is where Tomcat is installed.

Qexo is part of the Kawa framework. You will need to install a jar file of Kawa (version 1.7 or later). You can get kawa-1.7.jar from the Kawa ftp site or from a mirror site. Install this as $CATALINA_HOME/shared/lib/kawa-1.7.jar. (If you're using Tomcat 4.0.x, shared/lib doesn't exist. Install as $CATALINA_HOME/lib/kawa-1.7.jar instead.)

If Tomcat isn't already running, start it. For example, under Unix-like systems you can run the script $CATALINA_HOME/bin/bin/startup.sh. You may need to set the environment variable JAVA_HOME to point to the location of Java on your machine. (On Mac OS X 10.2 this is /System/Library/Frameworks/JavaVM.framework/Home.) If you haven't changed any of the defaults, you should now be able to point your browser at http://localhost:8080/ and get the default Tomcat home page.

Installing our web application

A web application is a group of data, servlets, and configuration files to handle a related set of URLs. The servlet specification specifies the directory structure of a web application. Let us install our adder in a new web application called utils. This means that we need to create two directories:

mkdir $CATALINA_HOME/webapps/utils
mkdir $CATALINA_HOME/webapps/utils/WEB-INF

Each web application has a web.xml configuration file. Copy the following web.xml into $CATALINA_HOME/webapps/utils/WEB-INF/web.xml:

<?xml version="1.0" encoding="ISO-8859-1"?>
  <display-name>XQuery Utils</display-name>



The <servlet-mapping> clause tells Tomcat that if it sees a URL that matches the pattern *.xql within the current web application utils, then it should use the servlet named KawaPageServlet to handle it. The <servlet> clause tells Tomcat that the servlet named KawaPageServlet is implemented by the Java class gnu.kawa.servlet.KawaPageServlet. (This class is included in the kawa-1.7.jar that we installed earlier.) In other words, any URL of the form http://localhost:8080/utils/*.xql or http://localhost:8080/utils/*/*.xql will be handled by the class gnu.kawa.servlet.KawaPageServlet.

So far this has all been boiler-plate. Now we need to install our XQuery program adder.xql into the utils web application by copying to the file $CATALINA_HOME/webapps/utils/adder.xql.

Accessing our web application

To access the web application, point you favorite browser at http://localhost:8080/utils/adder.xql. Tomcat will receive this request, use the utils part to determine that it is for the utils web application, and use the utils/WEB-INF/web.xml configuration file. That tells Tomcat to forward the request to the KawaPageServlet.

The first time the KawaPageServlet sees a request for adder.xql, it will compile adder.xql to a Java class named adder. This class will get added to the JVM that is running Tomcat, but by default the class is not written to a file. This means that adder.xql will have to be recompiled the first time you request it each time you restart Tomcat, but the Kawa compiler is fast enough that there is no real gain tp keeping the compiled class around. Kawa also gives you the option of manually compiling to a servlet. (If you're curious about the class that KawaPageServlet creates, add a qexo-save-class parameter to the initial request, as in http://localhost:8080/utils/adder.xql?qexo-save-class. This will write out the compiled class to $CATALINA_HOME/webapps/utils/WEB-INF/classes/adder.class.)

After KawaPageServlet has compiled adder.xql, it will execute the compiled body of adder.xql. This first time there won't be any parameters in the request, so calls to the request-parameter function return the specified default values: 0, 0, and 1 respectively. So the initial values in the <input> fields will be 0 and 1. The result of evaluating adder.xql will be some XHTML, which will be sent as the HTTP response back to your browser, which will display as in the first image above.

Use your browser to edit the input fields, changing them to 10 and 3. When you click Submit those values are used to set the HTTP request parameters sum1 and sum2, with the browser sending the URL http://localhost:8080/utils/adder.xql?sum1=10&sum2=3. Tomcat receives the request, passes it to KawaPageServlet, which forwards the request to the previously-compiled adder class. This time the adder gets the values 10 and 3 for the parameters sum1 and sum2, so when the new form is returned to the browser the initial values of the two fields are 13 and 3, which will display the second image above. If you leave the fields as-is and again click Submit, the updated forms will show 16 and 3. And so on.

Development and Debugging

Qexo provides help for developing and debugging your applications.

XQuery syntax errors

If you request adder.xql after editing it, KawaPageServlet will automatically recompile it. This makes it easy to test changes. If you make a syntax error, the result sent to the browser will include error messages from the compiler. For example, supposed you have the file min-cats.xql with the following semi-nonsense:

define function min-cats($x, $y) {
let $z in " cats" return
  if (x < $y)
    $x else $y
  " cats"
(: returns number of cats
min-cats(3, 4)

If you request this file, you'll see in your browser the following error log instead:

min-cats.xql:2:17: missing ':=' in 'let' clause
min-cats.xql:3:9: node test when focus is undefined
min-cats.xql:4:5: missing 'then'
min-cats.xql:5:9: missing '}' or ','
min-cats.xql:9:1: non-terminated comment starting at line 6

This gives you the filename, line number, and column number each place Qexo found a syntax error. Sometimes an earlier error will cause multiple errors, but in this case each message results from a separate error. (They should all be easy to figure out, except perhaps the second one, where the $x mistyped as x is interpreted as a node-test in a path expression. Qexo can catch this as it is an undefined context for such a node-test.)

Run-time exceptions

If you cause a run-time error, you will get a stack trace which may help you track down the error. Here is an application list-data.xql that looks for a non-existant "data.txt":

define function listing($url) {
  (: doc($url) replaced document($url) in the May'03 spec. :)
  <pre>{ doc($url) }</pre>
listing("data.txt"), ""

If you try to run this file, Tomcat will return a Java execution stack trace. Look for the root cause, which looks like this:

java.io.FileNotFoundException: http://localhost:8080/data.txt
    at sun.net.www.protocol.http.HttpURLConnection
    at gnu.kawa.xml.XMLParser.(XMLParser.java:57)
    at gnu.kawa.xml.XMLParser.(XMLParser.java:49)
    at gnu.kawa.xml.XMLParser.(XMLParser.java:42)
    at gnu.kawa.xml.Document.parse(Document.java:52)
    at gnu.kawa.xml.Document.apply(Document.java:94)
    at gnu.mapping.CallContext.runUntilDone(CallContext.java:258)
    at gnu.mapping.CallContext.runUntilValue(CallContext.java:290)
    at listData.listing$T(list-data.xql:3)
    at listData.apply(list-data.xql)
    at gnu.mapping.CpsMethodProc.apply(CpsMethodProc.java:49)
    at gnu.mapping.CallContext.runUntilDone(CallContext.java:258)
    at gnu.mapping.CallContext.runUntilValue(CallContext.java:290)
    at listData.apply(list-data.xql:5)
    at gnu.kawa.servlet.KawaPageServlet.apply(KawaPageServlet.java:51)
    at gnu.kawa.servlet.KawaServlet.doGet(KawaServlet.java:57)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
    at org.apache.catalina.core.ApplicationFilterChain
    ... lots of Tomcat internals ...
    at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable
    at java.lang.Thread.run(Thread.java:554)

More from Practical XQuery

XQuery Implementation

Interactive Web Applications with XQuery

Processing RSS

The cause is a FileNotFoundException, and the exception message names the URL it was looking for. (The data.txt is a relative URL that gets resolved to that of the web application.) The first few lines are within Kawa runtime routines, but then we get to two lines in the listData class, which is the Java "mangling" of the list-data.xql file. The first one specifies that we're at line 3, in method listing$T, which is the Java mangling of the listing function. And that is indeed where the bad call to doc is. A little further down you will see a call at list-data.xql line 5, which is where listing gets called.

We append an empty (, "") after the call to listing. This is to suppress tail-call optimization, which is an optimization done when the last thing in a function is a call to another function. The optimization allows some kinds of recursion to execute without a stack overflow, but the disadvantage is resulting that stack traces can be confusing. So when debugging, it may sometimes be helpful to append some empty value so that it becomes the last expression in the function.

Adding trace output

Sometimes it is difficult to understand what an application is doing, in which case it is useful to add print statements for debugging purposes. XQuery is a language free of side effects, so it doesn't really have print statements. However, the May 2003 draft specification added a trace which takes two parameters. The first parameter can be an arbitary value that is returned as the result of the trace call. The other parameter is descriptive string. Both values are written to a "trace data set" in an implementation-defined manner. For example you could replace the num-parameter implementation of adder.xql by the following:

define function num-parameter($name, $default) {
  trace (
  number(request-parameter($name, $default))
  , concat("num-parameter of '", $name, "' is:"))

That writes the following output to Qexo's standard error output (System.err). Tomcat redirects the error output to the file $CATALINA_HOME/logs/catalina.out, where you can read it.

XQuery-trace: num-parameter of 'sum1' is: 0
XQuery-trace: num-parameter of 'sum2' is: 0
XQuery-trace: num-parameter of 'sum2' is: 1