The Slippery Soap

October 17, 2001

Martin Gudgin and Timothy Ewald

In last month's column we put a stake in the ground, offering a definition of "web service". This month we're going to examine one of the underlying protocols upon which such services are being built, namely, SOAP. The original SOAP specification, version 0.9, was published in September 1999 and was revised twice before the publication of SOAP 1.1 in December 1999. SOAP 1.1 has become a de facto standard, with broad industry support from many vendors, large and small, providing client and server implementations. The specification defines a set of conventions for exchanging XML messages, including rules for encoding data structures, an extensibility mechanism, a binding to the HTTP protocol, and conventions for RPC style invocations. The W3C is working on turning SOAP into a W3C Recommendation through the work of the XML Protocol Working Group. The group is currently working on SOAP 1.2, using SOAP 1.1 as the starting point, and is making several changes and clarifications to the base specification.

Stepping back for a moment, it's worth asking why we need a protocol like SOAP at all. Or, to put it another way, what does SOAP bring to the table? Given that a web service involves exchanging information encoded in XML there is nothing to stop two parties from agreeing on a given XML vocabulary and structure, effectively defining their own protocol. However, this means that each pair of endpoints essentially defines an ad hoc protocol. So, given n endpoints the potential number of protocols is the sum of the numbers up to and including n-1. While implementing any single protocol may be a reasonably simple task, when n is of a significant size, say greater than 6, the implementation burden becomes quite significant. Having a standard protocol, rather than many ad hoc protocols, eases the implementation burden by bringing uniformity to certain aspects of communication. This ease of implementation leads, in turn, to processing facilities being built into other software, for example, server products, client products, toolkits, and operating systems. This frees the implementer of a web service (or clients of the service) to concentrate on the pieces specific to that service, rather than on the generic pieces that all web services require.

Message Recognition

SOAP defines the outer element for all SOAP messages that are exchanged between a web service and its client. Just as the stylesheet element in the XSLT namespace indicates to an XSLT processor that the XML is in fact a transform, the Envelope element in the SOAP namespace indicates to a SOAP processor that the XML is in fact a SOAP message. The processor can then look for the individual parts of the message; a mandatory Body element that contains the actual request and an optional Header element that contains extension elements. A skeleton SOAP message is shown below.

<soap:Envelope xmlns:soap='http://www.w3.org/2001/10/soap-envelope'>

  <soap:Header>



    <-- Headers go here -->



  </soap:Header>

  <soap:Body>



    <-- Request goes here -->



  </soap:Body>

</soap:Envelope>

Headers and Extensibility

The Header element in SOAP provides an extensibility mechanism. This element can contain any number of namespace qualified child elements. Each of these elements is some form of extension to the base SOAP protocol. Perhaps one element contains data associated with conversation or session management between a client and a server. Another element might contain authentication information or even information pertaining to an ongoing transaction. Yet another may contain locale information. Whatever their content or semantics each header element modifies the SOAP protocol in some way, providing extra context for the processing of the body of the message. For some extensions it will be important to the client that the server honors them; authentication is a good example. Others, such as locale information, may be less critical. Each header element may be annotated with a mustUnderstand attribute. If this attribute has a value of true or 1 then the header is considered to be mandatory. If the recipient of the message does not recognize or otherwise understand the header then it must generate a SOAP fault message. Headers without a mustUnderstand attribute (or with a mustUnderstand attribute with a value of false or 0) are to be considered optional. The recipient of such headers is not required to process them. An example SOAP message containing one mandatory and one optional extension is shown below.

<soap:Envelope xmlns:soap='http://www.w3.org/2001/10/soap-envelope'>

  <soap:Header>

    <h:Log xmlns:h='http://example.org/cvs/logging'>

      <trace>4</trace>

    </h:Log>

    <h:PServer xmlns:h='http://example.org/cvs/pserver'

               soap:mustUnderstand='true' >

      <username>anoncvs@example.org</username>

      <password>anoncvs</password>

    </h:PServer>

  </soap:Header>

  <soap:Body>

    <m:Checkout xmlns:m='http://example.org/cvs'>

      <source>/xml/soap/webservices/cvs</source>

      <revision>1.1</revision>

      <destination>/etc/usr/marting/source/xml</destination>

    </m:Checkout>

  </soap:Body>

</soap:Envelope>

The extensibility model provides a standard location for protocol extensions along with a standard mechanism for marking extensions as mandatory. Again, this eases implementation of application or infrastructure level code to process extensions.

Headers and the SOAP Processing Model

In a simple client-server exchange the processing model for messages that do not contain headers is straightforward. Typically the server recognizes the content of the Body as a request to perform some service. It performs the work and, optionally, sends back a response message. If the server does not recognize the content of the Body, or if some other error occurs during processing of the message, then it generates a SOAP fault message describing the error.

The presence of header elements complicates this processing model somewhat, and there has been a fair amount of debate in the SOAP community about several aspects of header processing. One such aspect is the meaning of the mustUnderstand attribute. What does it mean to "understand" a particular header? And, if a given processor understands the header ,is it required to process that header? This has led to the proposal of an additional mustProcess attribute. The effect of this attribute would be that the receiver would not only have to understand the header but process it too. Similarly, one could tighten the definition of the mustUnderstand attribute, which the SOAP 1.2 draft does, stating that mandatory headers must be processed according to the specification for that header or the SOAP processor must generate a fault message. In addition a SOAP 1.2 processor must ensure it understands all the mandatory headers in a given message, including the content of the Body, before actually processing any parts of the message. This cleans up the processing model somewhat, but makes life more difficult for streaming implementations that might like to process headers as they are encountered. Any attempt to process headers this way must be able to "rollback" the effects of header processing if a mandatory header is subsequently not understood by the SOAP processor.

Another area of debate is the order in which headers are processed. The SOAP spec does not define an order for the processing of headers or the body. It is up to the SOAP processor to determine the order. One obvious approach is to process headers in lexical order, but there may be cases where lexical order is inappropriate, perhaps because the sender of the message was unaware of interactions between certain headers in the message. One could define a header extension that modifies SOAP semantics to say that headers must be processed in a particular order. Marking such a header as mandatory would ensure that any SOAP processor receiving a message containing that header would know about the ordering requirement before it started processing the other headers in the message.

It is this ability to modify the semantics of the standard protocol which makes the header-based extensibility model expressive and open to abuse. Being able to define mandatory headers allows layering of rich functionality but also provides the means to subvert the base protocol. The SOAP specification allows headers to modify the semantics of their parent or peer elements. For instance, it's possible to define a mandatory header that turned the processing of other mandatory headers off. Consumers of web services need to be aware of the headers that a given endpoint expects and the semantics of those headers. Failure to exercise due diligence may result in clients accepting certain semantics they do not actually want.

At the moment there are few defined headers, none of which are standard. Over time various parties will define headers. Some will become standard, supported by many implementations; others will fall by the side. It is important that standard headers are well specified and do not subvert the workings of the underlying protocol. It's also worth noting that certain services, the Biztalk Server for example, use proprietary headers. While, strictly speaking, such services conform to the SOAP specification, they are not interoperable with clients (or other servers) which do not understand those non-standard headers. Developers planning to be clients of web services or the implementers of such services should think very carefully about which header extensions, if any, they are prepared to use.

References

• SOAP 1.2 Working Draft - Part One: Messaging Framework
• SOAP 1.2 Working Draft - Part Two: Adjuncts
• SOAP 1.1 Note