Building a Web Services Container in Python

January 20, 2004

Rich Salz

In the present run of columns, I'm using the web services framework provided by Python and the ZSI SOAP implementation to implement the XKMS registration service. Last month's column ended with a link to a skeleton server, but there was neither space nor time to explain it. This time we'll look at that server in some detail so that we can get an understanding of what features are provided by generic container servers (Apache Axis, J2EE servers, and the like).

Python provides a simple HTTP server framework in the BaseHTTPServer class. Create an instance of the class with TCP/IP binding information, and the name of your handler class. When a request comes in, the server will create an instance of your class, populate it with items from (and a reference to) the containing server object, and then call the appropriate method in the handler. You can subclass the server; a common reason to do this is to add additional debugging or other state information which the server can make available to the handler.

One of the nice features about the M2Crypto package is that it preserves this development model, with the SSLServer class replacing the standard Python server class. The following fragment lists the import statements we need, and shows how we subclass the server class:

import os, sys

from BaseHTTPServer import BaseHTTPRequestHandler

from ZSI import *

from M2Crypto import SSL

from M2Crypto.SSL.SSLServer import SSLServer



class HTTPS_Server(SSLServer):

    '''Subclass SSLServer; we add an optional "tracefile" parameter

    which will record all messages sent and received.'''

    def __init__(self, ME, handler, ssl_ctx, tracefile=None):

        SSLServer.__init__(self, ME, handler, ssl_ctx)

        self.tracefile = tracefile

Creating a server is easy. We first need to create an SSL context, which has the security (private key) and policy information (which protocol) that OpenSSL needs. In this case we want to use SSL 3 with the keypair we created last month. We don't care if the client uses an SSL certificate, so we're not going to do any verification or make sure that we know the client's CA. These choices translate into the following code to create a context, where home specifies the directory where the keypair files are found:

def init_ssl_context(home, debug=None):

    ctx = SSL.Context('sslv3')

    ctx.load_cert(certfile=home + '/openssl/ssl/cert.pem',

                  keyfile=home + '/openssl/ssl/plainkey.pem')

    ctx.set_verify(SSL.verify_none, 1)

    ctx.set_allow_unknown_ca(1)

    ctx.set_session_id_ctx('xkms_srv')

    if debug: ctx.set_info_callback()

    return ctx

Later on we'll define the XKMSRequestHandler class, but if we assume that the name is already in scope, then it is very simple to start our server. The os.environ.get() call is like the Unix shell's ${XKMSHOME-/opt/xkms} syntax. As with Perl, the environment is put into a dictionary, and the default value is used if the key XKMSHOME isn't found. The ('', 443) construct creates a Python tuple (immutable list) that the HTTP class interprets as "port 443 on all network interfaces". As is often typical of Python, the explanation is more complicated than the code itself:

sslctx = init_ssl_context(os.environ.get('XKMSHOME', '/opt/xkms'))

s = HTTPS_Server(('', 443), XKMSRequestHandler, sslctx)

s.serve_forever()

It doesn't do anything yet, but we've just implemented an HTTP/SSL server. In order to do something, we need to define the class that will handle requests. Our class is a subtype of a class provided by Python. The first thing we do is to define an identification string. The variable server_version will be output by the Python classes in the HTTP Server header. In keeping with common practice, we prefix our own information to the front:

class XKMSRequestHandler(BaseHTTPRequestHandler):

    server_version = 'ZSI/1.4-CVS XKMS/1.0 ' \

                   + BaseHTTPRequestHandler.server_version

This will result in HTTP output like this:

Server: ZSI/1.4 XKMS/1.0 BaseHTTP/0.2 Python/2.2.2

Our messages will be fairly small -- never more than a few tens of kilobytes -- so it's practical to buffer our responses in memory and use a Content-Length header. This method will send text identified as XML to the client:

def send_xml(self, text, code=200):

    self.send_response(code)

    self.send_header('Content-type', 'text/xml; charset="utf-8"')

    self.send_header('Content-Length', str(len(text)))

    self.end_headers()

    self.wfile.write(text)

    self.trace(text, 'SENT')

    self.wfile.flush()

ZSI has a Fault object to encapsulate SOAP faults. One method serializes the object into XML, as the following convenience methods shows. Note that we're using HTTP status code 500, which implies SOAP 1.1; if we were doing SOAP 1.2, we'd use HTTP status code 200.

def send_fault(self, f):

    self.send_xml(f.AsSOAP(), 500)

The trace method is an internal debugging utility. It shows that the request handler has a reference back to the server, self.server; and it also shows how we check to see if the server was created with tracing enabled:

def trace(self, text, what):

    '''Log a debug/trace message.'''

    F = self.server.tracefile

    if not F: return

    tstr = time.ctime(time.time()) 

    print >>F, '=' * 60, '\n',

        '%s %s %s %s:' % (what, self.client_address, self.path, tstr)

    print >>F, text

    print >>F, '=' * 60, '\n'

    F.flush()

To implement an HTTP method like GET or POST, we have to implement a method like do_GET or do_POST. Since all SOAP messages will come in as HTTP POST's, we can have a trivial implementation for GET that redirects the client to a more useful resource. A common convention for web services is to provide the relevant WSDL file in response to a particular GET. We'll do that later, but for now we'll use the "moved temporarily" HTTP status:

def do_GET(self):

    self.send_response(301)

    self.send_header('Content-type', 'text/html');

    self.send_header('Location', 'http://webservices.xml.com');

    self.end_headers()

    self.trace('', 'GET')

Now let's look at the implementation of HTTP POST. We get the Content-Length header and then read the message body. If something goes wrong, ZSI will raise a ParseException, which it can then convert into a SOAP fault. Python exceptions can also be converted, and if we provide the backtrace information (sys.exc_info), then ZSI will add that to the fault detail.

def do_POST(self):

    try:

        cl = int(self.headers['content-length'])

        IN = self.rfile.read(cl)

        self.trace(IN, 'RECEIVED')

        ps = ParsedSoap(IN)

    except ParseException, e:

        self.send_fault(FaultFromZSIException(e))

        return

    except Exception, e:

        self.send_fault(FaultFromException(e, 0, sys.exc_info()[2]))

        return

    if not check_xkms_method(ps): return

    if not check_xkms_headers(ps): return

    process_xkms(ps)

For example, posting garbage to the server will result in a reply like the following (SOAP envelope and namespace declarations omitted):

<SOAP-ENV:Fault>

  <faultcode>SOAP-ENV:Client</faultcode>

  <faultstring>Unparseable message</faultstring>

  <detail>

    <ZSI:ParseFaultDetail>

      <ZSI:string>Can't parse document (xml.parsers.expat.ExpatError): no

      element found: line 3, column 14</ZSI:string>

      <ZSI:trace>/ZSI:trace>

    </ZSI:ParseFaultDetail>

  </detail>

</SOAP-ENV:Fault>

At this point we have a fairly robust, SSL-enabled web services container. We could make it more generic by dispatching based on the URL or the namespace or element name of the data in the SOAP Body. Since Python can dynamically import new modules, we could have a configuration that maintains the dispatch mapping and load new code on the fly as needed. ZSI includes a "server dispatch" model that shows how to do some of that.

For our purposes, we'll just inline the XKMS implementation. Our first step is to see if the message is one we can implement.

def check_xkms_method(self, ps):

    root = ps.body_root

    if root.namespaceURI != 'http://www.w3.org/2002/03/xkms#':

        self.send_fault(

            Fault(Fault.Client,

                'Unknown namespace "%s"' % root.namespaceURI))

        return None

    if root.localName not in ['Register','Revoke']:

        self.send_fault(

            Fault(Fault.Client, 'Unknown operation "%s"' % root.localName))

        return None

    return 1

The next step is to look at the SOAP headers. We don't implement any headers, so make sure that none of the headers sent by the client are marked mustUnderstand. If we find any, we return a fault and stop processing.

def check_xkms_headers(self, ps):

    for (uri, localname) in ps.WhatMustIUnderstand():

        self.send_fault(FaultFromNotUnderstood(uri, lname, 'XKMS'))

        return None

    return 1

Next month we'll start to tear into the XML request message and do some real application-level work. Until then, you can download the server source and use it to start implementing your own web services.