XML.com: XML From the Inside Out
oreilly.comSafari Bookshelf.Conferences.

advertisement

Versioning XML Vocabularies

December 03, 2003

Editor's Note: An update to this article has been posted here on 10/27/04.

Introduction

XML is designed for the creation of languages based upon self-describing markup. The inevitable evolution of these languages is called versioning. Versioning means adding, deleting, or changing parts of the language. Making versioning work in practice is one of the most difficult problems in computing, with a long history of failed attempts. Arguably one reason why the Web rose dramatically in popularity is because evolution and versioning were built into HTML and HTTP headers, each of which provides explict extensibility points and rules for understanding extensions that enabled their decentralized extension and versioning.

XML Namespaces provide an ideal mechanism for identifying versions of languages, and all XML schema languages -- such as W3C XML Schema -- provide for controlled extensibility.

This article describes techniques to achieve more effective loose coupling between systems by increasing the possibility for backwards- and forwards-compatible changes to occur when related systems evolve. These techniques are designed for compatible changes with or without schema propagation. A number of rules are described for versioning XML vocabularies, making use of XML Namespaces and XML Schema constructs. It includes rules for working with languages that provide an extensible container model, notably SOAP. The collective set of rules is called the "Must Ignore" pattern of extensibility. Strangely, the "Must Ignore" pattern for HTML tags and HTTP headers that significantly helped their adoption has not been widely adopted by XML practitioners. This article aims to rectify that situation for current Schema validation environments. Follow-on material will explore innovative relaxed schema validation environments.

Defining Compatibility

FOLDOC [1] provides definitions of backwards and forwards compatibility. This article will reprise those definitions and focus on the actions of messages, specifically senders and receivers instead of clients and servers. Backwards compatibility means that a new version of a receiver can be rolled out so it does not break existing senders. This means that a sender can send an old version of a message to a receiver that understands the new version and still have the message successfully processed. Forwards compatibility means that an older version of a receiver can consume newer messages and not break. Of course the older version will not implement any new behavior, but a sender can send a newer version of a message and still have the message successfully processed.

In other words, backwards compatibility means that existing senders can use services that have been updated, and forwards compatibility means that newer senders can continue to use existing services.

Forwards-compatible changes typically involve adding optional element(s) and/or attribute(s). The costs associated with introducing changes that are not backwards- or forwards-compatible are often very high, typically requiring deployed software to be updated to accommodate the newer version. The rules described below optimize for the case of making backwards- and forwards-compatible changes. These article argues that this means not changing the namespace name or changing element names.

Compatibility is defined for the sender and receiver of an individual message. However, most web service specifications provide definitions of inputs and outputs. In these definitions of compatibility, a web service that updates its output schema is considered a newer sender. This simply reverses the sender/receiver terminology of input messages when applying compatibility definitions to output messages. If a receiver updates the schema of the output message, then it is "sending" a newer version of the message, hence it is considered a "sender".

Identifying and Extending Languages

Designing extensibility into languages has typically resulted in systems that are more loosely coupled. Extensibility allows senders to change instances without going through a centralized authority. Thus, the first rule relating to extensibility is

1. Allow Extensibility rule: Languages SHOULD be designed for extensibility.

A fundamental requirement for extensibility is to be able to determine the language of elements and attributes. XML Namespaces [14] provide a mechanism for associating a URI with an XML element or attribute name, thus specifying the language of the name. This also serves to prevent name collisions.

W3C XML Schema [15] provides a mechanism called a wildcard, <xs:any>, for controlling where elements from certain namespaces are allowed. The wildcard indicates that elements in specified namespaces are allowed in instance documents where the wildcard occurs. This allows for later extension of a schema in a well-defined manner. Receivers of extended documents can identify and, depending upon the processing model for the extensions, safely ignore the extensions they don't understand.

<xs:any> uses the namespace attribute to control what namespaces extension elements can come from. The main values for this attribute are ##any, which means one can extend the schema using an element from any possible namespace; ##other, which only allows extension elements from namespaces other than the current one; and ##targetnamespace, which only allows extension elements from the current namespace.

<xs:any>uses the processContents attribute to control how a XML parser validates extended elements. Permissible methods include "lax", allowing validation to occur, "strict", requiring validation, and "skip", to skip validation. This article uses "lax" validation, as it is the most flexible and is the typical choice for web services specifications.

The main goal of the "Must Ignore" pattern of extensibility is to allow backwards- and forwards-compatible changes to documents. At a minimum, this means neither changing nor adding namespace names, nor changing element names. Adding element names in the target namespace can only be done with the ##any namespace or a combination of ##other namespace and the target namespace of the vocabulary.

A number of examples illustrate the application of these rules. Imagine that a Purchase Order is being sent from one machine to another. The processing of the purchase order results in a "shipped" message. But that message could be sent quite some time after the Purchase order is received. It would be undesirable for the sending software to have to wait for a potentially arbitrary amount of time for the response (synchronous messaging). A preferable model is for the receiver to be able to send the "shipped" message under its own control, without the sender waiting. The receiver "calls back" to the initial sender, hence the term "callback". The sender provides an address to the receiver in a Callback address. This indicates the address that the receiver should use for sending any subsequent messages to the sender. In web services, this Callback would typically be sent as a SOAP Header block.

Our preference would be to have an extensible style of ##any. A callback type that uses this model is:


<s:complexType name="CallbackType">
 <s:sequence>
 <s:element name="callbackLocation" type="s:anyURI" 
minOccurs="1" maxOccurs="1"/>
 <s:any processContents="lax" minOccurs="0" 
maxOccurs="unbounded" namespace="##any"/>
 </s:sequence>
 <s:anyAttribute/>
</s:complexType>

Example 1 -- A schema using ##any for extensibility

However, the determinism constraint of W3C XML Schema, described in more detail later, prevents this model from working. The problem arises when a subsequent version of the callback adds an optional element. A timeout is a good example of this. The timeout of a callback is an informative piece of information for a receiver. Receivers can continue processing if they don't understand the timeout. The following schema is roughly what is desired using wildcards, but it is illegal because of the determinism constraint:

<s:complexType name="CallbackType">
 <s:sequence>
 <s:element name="callbackLocation" type="s:anyURI" 
minOccurs="1" maxOccurs="1"/>
 <s:element name="expires" type="s:dateTime" 
minOccurs="0" maxOccurs="1"/>
 <s:any processContents="lax" minOccurs="0" 
maxOccurs="unbounded" namespace="##any"/>
 </s:sequence>
 <s:anyAttribute/>
</s:complexType>

Example 2 -- An illegal schema

Since this pattern does not work, we need to create a design pattern that enables roughly the equivalent in order to achieve the original goals. For allowing new extensions in the same namespace, the author must create an extension type that allows extensions in the same namespace. The extension type should be used only for future compatible extensions in the same namespace. We need two more rules to allow proper versioning of XML language definitions. First the rule for namespaces:

2. Any Namespace rule: The extensibility point SHOULD allow for extension in any namespace. For XML Schema applications, the extensibility point SHOULD be an element that allows for extension in the target namespace and a wildcard that allows for extension in any other namespace.

The rule for allowing extensibility:

3. Full Extensibility rule: All XML Elements SHOULD allow for element extensibility after element definitions, and allow any attributes.

An example of a Callback type that follows these rules:


<s:complexType name="CallbackType">
 <s:sequence>
 <s:element name="callbackLocation" type="s:anyURI" 
minOccurs="1" maxOccurs="1"/>
 <s:element name="Extension" type="wscb:ExtensionType" 
minOccurs="0" maxOccurs="1"/>
 <s:any processContents="lax" minOccurs="0" 
maxOccurs="unbounded" namespace="##other"/>
 </s:sequence>
 <s:anyAttribute/>
</s:complexType>
<s:complexType name="ExtensionType">
 <s:sequence>
 <s:any processContents="lax" minOccurs="1" 
maxOccurs="unbounded" namespace="##targetnamespace"/>
 </s:sequence>
 <s:anyAttribute/>
</s:complexType>

Example 3 -- A Callback type with extensibility

Because each extension in the targetnamespace is inside an Extension element, each subsequent target namespace extension will increase nesting by another layer. While this layer of nesting per extension is not desirable, it is what can be accomplished today when applying strict W3C XML Schema validation. It seems that having multiple nested elements is worthwhile if multiple compatible revisions can be made to a language. This technique allows validation of extensions in the targetnamespace, while retaining validation of the targetnamespace itself.

In general an extension can be defined by a new specification that makes a normative reference to the earlier specification and then defines the new element. No permission should be needed from the authors of the specification to make such an extension. In fact the major design point of XML namespaces is to allow decentralized extensions. The corollary is that permission is required for extensions in the same namespace. A namespace has an owner; non-owners changing the meaning of something can be harmful.

Understanding Extensions

Senders should ideally be able to extend existing XML documents with new elements without receivers having to change existing implementations. Extensibility is one step toward this goal, but achieving compatibility also requires a processing model for the extensions. The behavior of software when it encounters an extension should be clear. Thus, we introduce the next rule:

4. Provide Processing Model Rule: Languages SHOULD specify a processing model for dealing with extensions.

The simplest processing model that enables compatible changes is to ignore content that is not understood. This rule is

5. Must Ignore Rule: Document receivers MUST ignore any XML attributes or elements in a valid XML document that they do not recognize.

This rule does not require that the elements be physically removed, only ignored for processing purposes. There is a great deal of historic usage of the Must Ignore rule. HTML 1.1, 2.0, and 3.2 follow the Must Ignore rule; they specify that any unknown start tags or end tags are mapped to nothing during tokenization. HTTP 1.1 [6] specifies that a receiver should ignore any headers it doesn't understand: "Unrecognized header fields SHOULD be ignored by the recipient and MUST be forwarded by transparent proxies." The Must Ignore rule for XML was first introduced in 1997 by the WebDAV working group [19], standardized in the WebDAV specification RFC 2518 [5] section 14 and later separately published as the Flexible XML Processing Profile [2].

There are two broad types of vocabularies relating to dealing with extensions. The types are data oriented and presentation (or document) applications. For data oriented applications, such as web services, the rule is

6. Must Ignore All Rule: The Must Ignore rule applies to unrecognized elements and their descendants.

For example, if a message is received with unrecognized elements in a SOAP header block, they must be ignored unless marked as "mustUnderstand" (see Rule 10 below), but it is reasonable to expect that the unrecognized elements might be written to a log file.

Document oriented vocabularies may need a different rule as the application will typically want to present the content of an unknown element. The rule for document-oriented applications is

7. Must Ignore Container Rule: The Must Ignore rule applies only to unrecognized elements

This retains the element descendants, such as for display purposes.

Instead of ignoring unrecognized components, a language may provide a different model for handling extensions. One model is that a receiver will generate a fault if it finds a component it doesn't understand. An example might be a security specification where a receiver must understand any extension. This suffers from the significant drawback that it does not allow compatible changes to occur in the language; changes can't be ignored. Another model is a fallback model, where alternate elements are provided if the receiver does not understand the extension. XSLT 2.0 provides such a model.

Pages: 1, 2

Next Pagearrow