Processing SOAP Headers

July 17, 2002

Rich Salz

Last month we built a simple client for the Google API. In this month's column we'll look at how SOAP headers can be used to talk to an intermediate server that adds value to the basic search service. The value-add is actually pretty silly: we'll send the query, pick one of the results at random to return, and send it back as an HTML page in Pig Latin. Our goal, however, is to understand how to process SOAP headers, and why you'd want to do so. But first I want to thank Google for providing a wonderful Web API, which it is, module the concerns I addressed in my first column.

SOAP structures a message into two main parts: the headers and the body. I'll go out on a limb and say that almost all SOAP messages so far use the body. Very few put anything in the SOAP headers. I think the recent flurry of activity in SOAP security standards means that this will soon change, however, so it's worth understanding when and how to use SOAP header elements.

SOAP is more than just a sender-receiver protocol, although that, too, is certainly the dominant use today. SOAP supports the concept of a message passing from a recipient, possibly through one or more intermediaries, and ending up at its destination, more precisely known as the ultimate receiver.

Along the way, the intermediaries may perform processing on the message or its side-effects. For example, a message may pass through a transaction service, providing a client with guaranteed invocation in the presence of network failures; a security service may sit at an enterprise portal, providing authentication information; and so on.

An important aspect of these examples is that the basic operation is unchanged. While this isn't made explicit in the SOAP specifications, it's commonly accepted that intermediaries are intended to work primarily on the metadata of the SOAP message. SOAP headers are the ideal place for such data.

SOAP headers are also a good place to put optional information, and a good means to supporting evolving interfaces. For example, I use my ATM card to get money from my checking account. After several years, my bank upgraded to provide account linking, so that I can manage multiple accounts with one card. If I use my bank's ATM, I now have to specify if I want the withdrawal to be made from my primary, secondary, or tertiary account. If I use an ATM from another bank that isn't affiliated with my bank, I don't get asked that question. So the account identifier is clearly an optional parameter, with a reasonable default.

This is an ideal use-case for SOAP headers. We can define an element to specify an account:

<xsd:element name="AccountSubIdentifier" type="xsd:int"/>

We could then imagine a SOAP message which used that header:

<SOAP-ENV:Envelope>

  <SOAP-ENV:Header>

    <tns:AccountSubIdentifier>1</tns:AccountSubIdentifier>

  </SOAP-ENV:Header>

  <SOAP-ENV:Body>

    <tns:Action>withdrawal</tns:Action>

    <tns:Amount>100</tns:Amount>

    <tns:Fee>1.50</tns:Fee>

  </SOAP-ENV:Body>

</SOAP-ENV:Envelope>

Since a SOAP message may pass through several intermediaries and intermediaries are supposed to look at the headers, we need a way to specify which headers are intended for which intermediary. In SOAP this is done by using the "SOAP-ENV:actor" attribute. (It's a named "role" in SOAP 1.2.) The attribute value is a URI that somehow identifies the header's target. It can name a specific server -- e.g., the URL of my bank -- or it could name a generic service -- e.g., a URI that would be recognized by any "standard internet cache service" (if such existed). Like much of the SOAP framework, details are left to the application(s) involved.

SOAP defines two special actor (role) values, a "none" value means that the header is targeted not to any intermediaries, but rather to the ultimate destination. For example, a new ATM-like client could generate the following header, without having any idea of where the message is going:

<tns:AccountSubIdentifier 

SOAP-ENV:role="http://www.w3.org/2002/06/soap-envelope/role/none">

  1

</tns:AccountSubIdentifier>

The "next" value is probably less useful to developers today. It's a well-known URI that indicates that the header element is targeted to the next intermediary to receive the message. I believe it's less useful because multi-hop applications will probably be configured and built using host-specific URLs, rather than generic names like "the next point".

In the above example, I've said that the header is optional; the server will do something reasonable if omitted. Sometimes the meaning of a request can be changed by headers, such that it is not safe to ignore the header. Consider authentication information; if ignored, the server may treat the request similar to anonymous FTP, allowing limited "read-only" access to its resources. If the client authenticates himself, the server may provide additional access that modifies the resources it maintains.

That's actually a tricky example. Is authentication optional? It depends on the application. Sometimes, falling back to "unauthenticated" is okay, and other times it isn't. For those times when the client wants to make sure that the server understands its identity, the "mustUnderstand" attribute can be used. This is a boolean attribute, with 1 or true meaning that the attribute must be understood, and 0 or false meaning that it is okay to ignore it. For example, if I want to be sure that my deposit is going into my secondary account, I'd use the following:

<tns:AccountSubIdentifier

SOAP-ENV:role="http://www.w3.org/2002/06/soap-envelope/role/ultimateReceiver"

SOAP-ENV:mustUnderstand="1">

  2

</tns:AccountSubIdentifier>

The SOAP spec requires a SOAP processor to send a fault if it cannot understand a header that the message indicates that it must. Taken together, the actor (role) and mustUnderstand attributes provide a very powerful way to evolve interfaces over time. On the down side, they require careful use by the developer. As we've seen, they depend on the semantics of the operation and how it might have changed over time. This is quite different than code-generation wizards, which can at best only compare the syntax of two different interfaces.

With all that understood, let's look at our Pig Latin intermediary. For the sake of brevity, we'll skip the WSDL definition and describe it in prose. The server accepts Google "doGoogleSearch" requests, but returns the HTML translated. The server is identified by the URI "http://www.zolera.com/ZSI/xlate", and it understands a "Language" header in that namespace. The value of the header is a text string specifying the desired language. Currently only "pig latin" or "igpay atinlay" are supported; enhancements such as "Ubbi Dubbi" are left for future work!

Again, we'll use the Python ZSI SOAP toolkit. Because header processing can be so complex, the ZSI method names are deliberately verbose and evocative.

We start by parsing the message. We'll assume we're invoked from a CGI-like framework, and that the message body is available on our standard input:

from ZSI import *

import sys

ME = "http://www.zolera.com/ZSI/xlate"



# Note: bad style, not catching parse errors. :)

ps = ParsedSoap(sys.stdin)

According to the SOAP processing rules, we should first see if there are any mustUnderstand headers that we can't handle. The ZSI method returns the namespace and local name of all headers with a non-false mustUnderstand: header.

for h in ps.WhatMustIUnderstand():

    uri, lname = h

    if uri != ME or h != 'Language':

        f = FaultFromNotUnderstood(uri, lname, ME)

        f.AsSOAP(sys.stdout)

        sys.exit(1)

Note that ZSI makes it easy to send the proper type of SOAP Fault back to the client.

Next we look at all the header elements that are targeted to a specific actor. Since we're an intermediary, and we know that we're the last step before the ultimate destination, we'll send a fault if any actors other than us appear in the headers.

# See what named actors are in the headers.

# Written in a style to allow a server to handle

# multiple roles.

for a in ps.WhatActorsArePresent():

    if a.namespaceURI != ME:

        f = FaultFromActor(a.namespaceURI, ME);

        f.AsSOAP(sys.stdout)

        sys.exit(1)

Now that we've made sure all headers are intended for us and understood by us, we can process them. We already checked namespaces in the first step, so we can just look for the "Language" header and parse it as a string. We'll just hand-wave over the details of ZSI's parsing API:

what = "PigLatin"

for h in ps.GetMyHeaderElements():

    if h.localName == "Language":

        what = TC.String("Language").parse(h)

        what = what.lower()

    if what in ["piglatin", "pig latin", "igpay atinlay"]:

        what = "PigLatin"

    else:

        f = FaultFromActor(ME)

        f.AsSOAP(sys.stdout)

        sys.exit(1)

Note that even though the individual processing is fairly simple, the overall process is fairly complex and requires multiple passes over the header elements. In a streaming environment -- think SAX, not DOM -- that won't work. In fact, it's my bet that headers will spell the end of SAX-style SOAP processors. For example, a digital signature of a SOAP message naturally belongs in the header. In order to generate the signature, you need to generate a hash of the message content. How can you do that without buffering? If your SOAP messages use SOAP RPC encoding, you can append the signature as a non-root serialization element after your message. (We artfully ducked the whole issue of serialization roots last time, and we'll do so again.) But if you're using SOAP in "doc/literal" style to send XML documents, then you might not be able to arbitrary append data to your message. Should SOAP have defined a "SOAP-ENC:trailers" element, perhaps?