Menu

XML Forms, Web Services and Apache Cocoon

January 29, 2003

Ivelin Ivanov

Server side business logic is often invariant with regard to client devices. An email client supports the same basic operations whether it's used from a cellular phone, PDA, or a PC. To address the needs of web developers who build applications for a variety of devices, the W3C has formed the XForms working group. According to the XForms specification,

"XForms" is W3C's name for a specification of Web forms that can be used with a wide variety of platforms including desktop computers, hand helds, information appliances, and even paper. Part of the HTML Activity, XForms started life as a subgroup of the HTML Working Group, but has now been spun off as an independent Working Group.

XForms are comprised of separate sections that describe what the form does and how it looks. This allows for flexible presentation options, including classic XHTML forms, to be attached to an XML form definition. The following illustrates how a single device-independent XML form definition, called the XForms Model, has the capability to work with a variety of standard or proprietary user interfaces:

XForms Conceptual Model
Figure 1. XForms Conceptual Model

In this article we discuss the Cocoon XMLForm framework's separation of the purpose from the presentation of a form, maximizing its reusability for a variety of client devices. We also explain how this technology allows us to extend web applications to Web Services.

Apache Cocoon XMLForm is aligned to a large extent with the W3C XForms standard. While XForms requires that client devices understand the XForms markup, XMLForm can be used with any browser for any client device. The trade-off for this convenience is that XMLForm lacks some of the client side features of XForms, such as events and actions.

XMLForm basics

XMLForm is a middle-tier framework based on the MVC (Model-View-Controller) design, which combines the best from Jakarta Struts, W3C XForms, and Schematron.

Its goals include the following:

  • Ease the development of interactive web applications utilizing complex data types and multi-page transactions
  • Provide an automated 2 way mapping between HTML Forms <-> XML <-> JavaBeans (and XML DOM objects)
  • Use standard XML schema languages like XML Schema, Relax-NG and Schematron for data validation
  • Promote reuse of Cocoon Actions for:
    • Workflow control of UI centric web applications - HTML, WML, UXL, VoiceXML, etc.
    • Web Services (based on the REST paradigm)
    • Remote Portal Forms. An interactive extension of RSS for Web syndication of portal functionality.
    • Utilize XPath tools like Apache Jakarta Commons JXPath for read/write access to JavaBeans, DOM nodes and mixed objects.
    • Allow maximum utilization of XML functional testing frameworks like AntEater

XMLForm allows developers to build and edit an XML document, called the form model or instance -- subject to constraints from some schema: WXS, Schematron, and so on -- by interacting with a series of form pages. The instance is either a DOM-document or a Java bean-structure or a mix. XMLForm consists of three main components:

  • Form -- responsible for the instance and its validation. Form objects are stored in request attributes for one page forms and session attributes for wizards (multipage forms). A Form can be populated from the request parameters. This is the "model" in MVC terms.
  • Transformer (XMLFormTransformer) -- takes a form descriptor document, (similar to XForms) as input and fills it with data and error messages from a Form object that is referred to in the document. This is the "view" in MVC terms.
  • Action (AbstractXMLFormAction) -- creates the Form object if necessary and populates it with data based on the request parameters. It can also take care of workflow handling and special controls like checkbox states. This is the "controller" in MVC terms.

The following data flow diagram is an overview of the XMLForm components interaction.

XForms Dataflow Diagram
Figure 2. XForms Dataflow Diagram

To set the stage for future discussion, we will show a snippet of a typical Cocoon sitemap for an application which uses XMLForm components:

Example  1. Web Application Utilizing XMLForm

<map:pipeline>
  <map:match pattern="wizard*">
    <map:act type="WizardAction">
      <!-- XMLForm parameters  -->
      <map:parameter name="xmlform-validator-schema-ns" 
      value="http://www.ascc.net/xml/schematron"/>
      <map:parameter name="xmlform-validator-schema" 
      value="schematron/wizard-xmlform-sch-report.xml"/>
      <map:parameter name="xmlform-id" value="form-feedback"/>
      <map:parameter name="xmlform-scope" value="session"/>
      <map:parameter name="xmlform-model" 
      value="org.apache.cocoon.samples.xmlform.UserBean"/>
      <!-- Content transformation logic -->
     
      <!-- original XMLForm document -->
      <map:generate src="wizard/{page}.xml"/>
     
      <!-- populating the document with model instance data -->
      <map:transform type="xmlform"  label="debug, xml"/>
      <!-- personalizing the look and feel of the form controls  -->
      <map:transform src="stylesheets/wizard2html.xsl"/>
      <!-- Transforming the XMLForm controls to HTML controls -->
      <map:transform src="context://samples/stylesheets/xmlform/xmlform2html.xsl"/>
     
      <!-- sending the HTML back to the browser -->
      <map:serialize type="html"/>
     
    </map:act>
  </map:match> 
</map:pipeline>

With the Cocoon 2.1 distribution, you can see XMLForm in action by visiting the url: http://localhost:8080/cocoon/samples/xmlform/.

Form and Model

The form is a key concept in XMLForm. A form is always associated with an underlying domain model, which is either a JavaBean or an XML DOM object. A form serves two roles: It describes how the user should enter a certain piece of data, either through a text field, drop down select, check box, radio button or another input control. It also specifies what part of the underlying model does the input data update.

What is the purpose of the Form and the Model?

Let's consider the following web form which collects personal information:

Sample web form
Figure 3. Sample web form

This web form is rendered in HTML, which was derived from the following XMLForm document.

Example 2. XMLForm document

      <xf:form id="form-feedback" view="userIdentity" action="wizard">
        <xf:caption>Personal Information</xf:caption>
           <error>
              <xf:violations class="error"/>
           </error>
           <xf:textbox ref="firstName">
              <xf:caption>First Name</xf:caption>
              <xf:violations class="error"/>
           </xf:textbox>
               . . .
           <xf:selectMany ref="role" selectUIType="listbox">
              <xf:caption>Professional roles</xf:caption>
              <xf:item>
                <xf:caption>Geek</xf:caption>
                <xf:value>Geek</xf:value>
              </xf:item>
              <xf:item>
                <xf:caption>Hacker</xf:caption>
                <xf:value>Hacker</xf:value>
              </xf:item>
               . . . 
           </xf:selectMany>
           <xf:submit id="next" class="button">
              <xf:caption>Next</xf:caption>
           </xf:submit>
         </xf:form>
               

XMLForm, borrowing from W3C XForms, defines a device-neutral, platform-independent set of form controls suitable for general-purpose use. The controls are bound to the XMLForm model via the XMLForm binding mechanism, in this case using the ref attribute on the controls. This markup would be transformed by XSLT to HTML or WML, VoiceXML or another client device markup.

The Form model for this example can be an XML DOM object or a JavaBean. XMLForm uses the Apache Commons JXPath library for its binding mechanism, which in turn relies on JavaBeans introspection.

Notice the following features of the XMLForm design:

  • The user interface is not hard-coded to use HTML check boxes or a select list. Different devices (such as voice browsers) can render the concept of "select many" as appropriate.
  • Form controls always have labels directly associated with them as child elements - this is a key feature designed to enhance accessibility.
  • Markup for specifying form controls has been simplified in comparison with HTML forms.

The fact that you can bind form controls to the model like this simplifies integrating XMLForms into other host languages, since any form control markup may be used to bind to the model.

Providing Model Data

The client browser will collect input from the user and will directly submit the data as HTTP GET or HTTP POST parameters.

XMLForm processing keeps track of the state of the partially filled form through this model data. Initial values for the model may be provided or not. The model essentially holds a JavaBean or a skeleton XML document that gets updated as the user fills out the form. It gives the author full control on the structure of the submitted data. When the form is submitted, the input is serialized as HTTP GET or POST parameters.

This design has features worth noting:

  • There is complete flexibility in the structure of the model data for both JavaBean and XML models, including the use of element attributes.
  • Empty elements roles and hobbies serve as place-holders in the JavaBean structure and will be filled in with form data provided by the user.
  • An initial value for the form control is provided through the model; in this case first name, last name, email, and age. In the submitted data, the initial values will be replaced by the user input, if the user changes the form control displaying that data.

To connect the model with form controls, the ref attributes on the form controls need to point to the proper part of the model data, using binding expressions:

Example 3. Binding Form Controls to Model Nodes with ref

                ...
                <xf:textbox ref="firstName">
                ...
                <xf:textbox ref="/email">
                ...

Binding expressions are based on XPath, including the use of the @ character to refer to attributes, as we will see later. Note that for illustrative purposes, the first two expressions make use of the XPath context node, which defaults to the top-level element. The third and forth expressions show an absolute path.

Form processing

As we discussed earlier, an Action is a controller component responsible for processing user input, possibly updating the domain model, and selecting the next view for the client. An XMLForm Action class is an Action which automatically populates the form model with the input values, as specified by the binding ref attributes on the elements in a form documents.

Participating Classes

Let's look at the structure of an XMLForm Action and its associations.

Form Processing Class Diagram
Figure 4. Form Processing Class Diagram

Following is a short description for each of the key methods of an XML Form:

  • perform() is invoked after the model is populated with request parameters and usually validated by a schema. It's responsible for implementing the navigational logic by interacting with business logic components and inspecting the state of the Form model.
  • getFormView() returns the value of the Form view name. Each XMLForm document specifies the ID of the form it is referring to and the name of the page view it is used in.
  • getCommand() returns the ID of the action button pressed by the user. The Action class can use this method in combination with getFormView() to determine the next page for the user.
  • prepare() can be optionally implemented to handle "Cancel" buttons or unselected check boxes. It is invoked before the form model is populated with request parameters.
  • getFormId() can be optionally implemented to provide the unique ID for an XMLForm instance. Each XMLForm instance must have its unique ID within the application. The form key is used to store the form instance in the Servlet Request or Session. It is referenced by this key also in the id attribute of the XML form elements.
  • getFormScope() can be optionally implemented to specify the storage scope for the XMLForm instance. Each XMLForm instance is stored in either the Servlet Request or Session for reference by the XMLForm view documents or other application modules.
  • getFormValidator() can be optionally implemented to provide the model Validator. Each XMLForm instance can have a Validator which is automatically applied against the model after it has been populated with new data. The result of a validation is held in the Form instance along with the model. Further in the pipeline the XMLForm view documents can access the form instance model and its violations.

Class Interaction

We skimmed the surface of the class structure, now it's time to get acquainted with the interactions between the classes. The following diagram illustrates the sequence of calls which take place when an HTTP request is made to an XMLForm Action.

Form processing sequence
Figure 5. Form processing sequence

A servlet request is first handed to the XMLForm Action. The action may choose to interrupt further processing in its prepare() method, depending on the request parameters or some other business rules. If it decides to continue, the action will pass the servlet request to the Form instance, which in turn automatically populates the associated model. Next the form uses the Validator to test the state of the model corresponding to the current form view. As a result the Form holds the model populated with new data, accompanied by a set of violations, possibly empty.

Control is then given to the perform() method, which is the essence of each XMLForm action. This is where the workflow rules for a Form instance are executed. The implementation depends on the needs of each concrete application. At the end of its work, the method will return either null, prohibiting further pipeline steps within the corresponding <map:act> section, or a Map object, which has at least one key-value pair of the form ("page", nextPageName), where the value of nextPageName is a logical name of the next page for the user.

XMLForm markup

XMLForm implements the majority of Form Controls defined in the W3C XForms specification.

A Form Control is: An XForms user interface control that serves as a point of user interaction.

Form controls enable accessibility by taking a uniform approach to labels, help text, navigation, data input, and so on. Internationalization issues are addressed by following the same design principles as XHTML. Cocoon's i18n markup is a natural match for the XMLForm input markup. All form controls are suitable for styling as aural or visual media.

When rendered, form controls display the underlying data values to which they are bound. While the data presented to the user through a form control must directly correspond to the bound instance data, the display representation is not required to match the lexical value. For example, user agents should apply appropriate conventions to the display of dates, times, durations, and numeric values including separator characters.

Form controls encapsulate high-level semantics without sacrificing the ability to deliver real implementations. For instance, form controls select and select1 enable the user to select one or more items from a set. These form controls distinguish the functional aspects of the underlying control from the presentational and behavioral aspects. This separation enables the expression of the intent without detailing how it will be fulfilled.

For detailed description of each form control, consult the W3C XForms specification

Transforming XMLForm Documents to Presentation Languages

XForms and XMLForm allow the web developer to focus on the workflow and declaratively specify the intent of each input page. Without XForms, developers are both coding the page navigation and dealing with the constantly growing browser-specific issues. There are normally two or more transformations before a raw XMLForm document is rendered into a client hosted language. The first is to populate the form controls with the values of the model properties which they reference. This stage also unrolls repeat-like elements into single elements addressing individual model properties. The second stage is optional and depends on the client host. If it is a typical HTML browser, the second stage will apply an XSLT transformation which converts XMLForm tags into corresponding HTML tags. Additional stages may be involved to resolve internationalization tags or other application specific tags.

XMLFormTransformer: Populating model values in XMLForm tags

When writing an XMLForm document, an author uses static constructs with references to model properties. It is the job of the XMLFormTransformer to populate the actual model values into the author specified form controls. XMLFormTransofmer is a relatively complex component. We will try to provide enough examples to illustrate its work.

Transforming the textbox element

Here is an example of a textbox form control as it appears in the original document, followed by the output of the transformation.

Example 4. textbox element in the original document

 ...
<xf:textbox ref="/firstName">
  <xf:caption>First Name</xf:caption>
  <xf:violations class="error"/>
</xf:textbox>
...

Example 5. textbox element after transformation

...
<xf:textbox ref="/firstName">
  <xf:value>abc</xf:value>
  <xf:caption>First Name</xf:caption>
  <xf:violation class="error">First name should be at 
  least 4 characters.</xf:violation>
</xf:textbox>  
...

From these XML snippets, you can see how an additional value element was added to the textbox. Also the violations tag was replaced by a violation tag with an error message related to the value of the firstName model property.

Transforming the group element

The group element is not transformed itself. Its ref attribute is used as a base for the child form controls. The ref attributes of the children are relative to the reference of the group.

Example 6. group element in the original document.

...
    <xf:group ref="/system" id="sysgroup">
      <xf:textbox ref="@ram">
        <xf:caption>RAM</xf:caption>
        <xf:violations class="error"/>
      </xf:textbox>
      ...
    </xf:group>
...

Example 7. group element after transformation

...
    <xf:group id="sysgroup">
      <xf:textbox ref="/system/@ram">
        <xf:value>512</xf:value>
        <xf:caption>RAM</xf:caption>
      </xf:textbox>
      ...
    </xf:group>
...

Notice how the model reference of the textbox changed from "@ram" to "/system/@ram" after the transformation. This way the reference is absolute in regard to the form model.

Transforming the repeat element

The transformation of the repeat element consists of two phases. In the first phase the content of the element is unrolled into a set of group tags, each with a unique reference to an individual node from the node set. The second phase recursively applies the original transformation algorithm to each of the group elements.

Example 8. repeat element in the original document.

...
      <xf:repeat nodeset="favorite[position() &lt;= 3]" id="favorites">
        <xf:textbox ref="." class="info">
          <xf:caption>URL: </xf:caption>
        </xf:textbox>
      </xf:repeat>
...

Example 9. repeat element after transformation

...
<xf:repeat nodeset="favorite[position() &lt;= 3]" id="favorites">
  <xf:group ref="/favorite[1]">
    <xf:textbox class="info" ref="/favorite[1]/.">
      <xf:value>http://xml.apache.org/cocoon</xf:value>
      <xf:caption>URL: </xf:caption>
    </xf:textbox>
  </xf:group>
  <xf:group ref="/favorite[2]">
    <xf:textbox class="info" ref="/favorite[2]/.">
      <xf:value>http://jakarta.apache.org</xf:value>
      <xf:caption>URL: </xf:caption>
    </xf:textbox>
  </xf:group>
</xf:repeat>
...

Notice that there are multiple group elements referencing the indexes of the favorite node set. Each of the children textbox elements has a absolute reference in regard to the form model.

Transforming XMLForm markup to HTML

The output of the XMLFormTransformer is valid XML which can be fed to any other XML consumer. The consumer can be a web services client or even a Portal Syndication component. This feature of XMLForm is a distinct advantage over HTML forms, which are exclusively targeted to HTML capable browsers.

Accessing Model properties

XMLForms uses Apache JXPath to address instance data nodes in binding expressions, to express constraints, and to specify calculations. JXPath is an implementation of W3C XPath which provides access to nodes in XML documents as well as access to properties of JavaBeans and even mixed objects like JavaBeans containing DOM nodes. XPath expressions which are not syntactically valid, including attempted calls to undefined functions, result in server side exceptions at runtime.

XPath Datatypes

XPath data types are used only in binding expressions. A binding expression is an XPath LocationPath expression that addresses a model property. XMLForm uses XPath datatypes boolean, string, number, and node-set. When addressing JavaBean properties, JXPath automatically applies conversion to and from string, which means that it will support any Java primitive type as well as complex types which provide conversion methods via the Java BeanInfo interface. Additionally JXPath automatically converts a Collection type or a primitive Java array into a node-set.

Extending XMLForm Applications to Web Services

The core functionality of the Web Service is the same as for the GUI application. It collects information about the usage of a Cocoon deployment. Whether submitted directly via browser or through a remote program, the application will receive the message and if the data is valid, it will process it in some fashion then notify the client of the outcome.

To make the Web Service public, we will have to present a WSDL file which defines the contract for the interface for remote users. To maximize reuse we will opt for the HTTP GET binding, instead of XML-RPC or SOAP. This is a very straightforward exercise, which simply lists the names of the request parameters.

Following is a snippet of the WSDL file describing the service:

<definitions>
   . . .
  <message name="UsageFeedbackHttpGetIn">
    <part name="firstName" type="xsd:string"/>
    <part name="lastName" type="xsd:string"/>
    <part name="email" type="xsd:string"/>
      . . .
  </message>
      . . .
  <portType name="UsageFeedbackHttpGet">
    <operation name="UsageFeedback">
      <input message="s0:HelloWorldHttpGetIn"/>
      <output message="s0:HelloWorldHttpGetOut"/>
    </operation>
  </portType>
  <binding name="UsageFeedbackHttpGet" type="s0:UsageFeedbackHttpGet">
    <http:binding verb="GET"/>
    <operation name="WSUsageFeedback">
      <http:operation location="/UsageFeedbackService"/>
      <input>
        <http:urlEncoded/>
      </input>
      <output>
        <mime:mimeXml part="Body"/>
      </output>
    </operation>
  </binding>
  <service name="UsageFeedback">
    <port name="UsageFeedbackHttpGet" binding="s0:UsageFeedbackHttpGet">
      <http:address location="http://localhost:8080/cocoon/samples/xmlform"/>
    </port>
  </service>
</definitions>

Since the data model is exactly the same as the one for the GUI version, we can readily reuse all validation rules.

Since the request mechanism is HTTP GET, we could reuse the existing XMLForm Action class to handle the Web Service requests. Apart from the WSDL file, there is hardly any extra work to do to extend the application to a Web Service. Well designed Cocoon applications can be turned into Web Services with little effort. We capitalized on the clean separation of logic, content, and presentation once again.

Conclusion

This article introduced a new perspective on form handling in web applications, a technique for connecting the business logic and the UI layer, while preserving a thin line which cleanly separates them. Programmers can now focus on the implementation of the application workflow without the burden of tedious HTML coding.

Web page authors on the other hand can work on the presentational aspects of the application without knowing how to code in Java or even run the application server. Usability experts can sketch UML activity diagrams and write the initial XML form documents. And quality assurance professionals can write regression tests against the web pages in their XML stage, rather than manually testing poorly structured HTML.

Additional Information

Look for the author's new book Extreme XML Publishing with Open Source Tools from Manning Publications. You can also follow the CocoonHive.org weblog which posts use cases, best practices and success stories on advanced Cocoon applications.

Resources