WSDL Tales From The Trenches, Part 1

May 27, 2003

Johan Peeters

Recently I retrofitted WSDL to a set of existing web services. A customer had a server running and there was a client implementation. The client and server team had been working closely together and now the time had come for another client implementation by a development team on the other side of the globe. A clear specification of the services was needed, and that's what WSDL is for. So I set out to make explicit what was previously implicit. It turned out to be an instructive experience, reaffirming some good old software engineering practices and uncovering a new set of problems specific to web services, WSDL, and XML Schema.

There were clearly some design flaws at the outset which were hard to pinpoint. It is likely that these mistakes would not have been made if the designers had formally written down their service definitions in WSDL. So this is the core message of the series: write WSDL up front, do not generate it as an afterthought, as is often suggested by vendors.

In all the attention that has been lavished on web services, it is often difficult to distinguish between vision and reality. This series will be short on vision and long on reality. It will not provide an overview of WSDL, and it also assumes familiarity with W3C XML Schema. This first article in the series considers what sound software engineering practice and distributed computing experience offer to web service design. I review some of the important design decisions that a web service designer must make and offer some advice to guide the process. The rest of the series is about how to represent the design. In the second article I shine a light in some of the dark corners of the WSDL 1.1 specification, leaving out data type definitions, which are the subject of the third article. I look at WXS it from the perspective of someone who uses it to specify the data which will be sent across the web service interface.

Designing Web Services

I use the term web services specifically for a kind of technology which focuses on interoperability, which implies that it has been standardized: heterogeneous systems only work together by virtue of open standards. Are web services the solution to your problem? Web services do not earn you street credibility anymore, so you better have some other reason for using them. If you control both ends of the pipe, chances are that there are other, better suited technologies out there. We are only at the dawn of the era of open standards in distributed computing. The price for support of web services in this early adopter's market remains high. That price is paid by poor performance, high development cost, and degraded security.

The argument that web services are "firewall friendly" is misleading. Traditional firewalls protect corporate assets against intruders exploiting weaknesses in application software by only opening ports that run hardened services. Web services, on the other hand, expose application software via these ports. In other words, they subvert security by giving outsiders the access to applications that the network-level firewall was designed to deny. Contrary to being firewall-friendly, a new breed of firewall is needed. There are a number of new entrants in this market offering products to target the problem. The traditional firewall vendors are also starting to take note. But it is early days and the technology has yet to prove itself.

Even if you go with web services, remember that you do not need to use SOAP for everything. For example, I have seen forms being submitted as an argument to a SOAP-RPC request. I have seen forms being returned as part of a SOAP-RPC response. I fail to see where the added value of SOAP might be in these cases. Surely plain XML or HTML over HTTP would be simpler?

Do Not Generate WSDL

Is WSDL 1.1 fit for human consumption? The commonly offered advice is that WSDL should be generated, not written by hand. I take issue with this. It may be true that simple, demonstration services can be developed this way, but this approach falls down when applied to larger systems. Even a designer working on her own will soon lose an overview; it's even more difficult in the context of cooperative development by heterogeneous, geographically separated teams. Large distributed systems need designing; throwing them together and hoping they will work invites disaster.

When distributed components may be developed in a variety of programming languages, you need a language-neutral Interface Definition Language (IDL) to specify how services must be invoked. CORBA has one and so does DCOM. IDL is a contract between the service requester and provider, but it only captures syntax. Semantics are not covered; IDL leaves the question of what the service does unanswered.

WSDL is the IDL of web services. It defines how to invoke web services. It also specifies the responses that you may receive, both when the invocation succeeds and when it fails. A complete WSDL specification nails down the format of messages, the protocols used, and the addresses at which the services reside. Unfortunately, getting it said in clear, crisp WSDL is still no guarantee for the quality of the design. Like all IDLs, WSDL is strong on syntax and weak on semantics. Nonetheless, do not neglect this task as, at the end of the day, it is the semantics that matter; syntax merely serves to unlock them.

Designing the Interfaces

Before sitting down to write WSDL, make a clear agreement with the stakeholders about what the web service should do. Write use cases, unambiguously capturing how the web service interacts with its environment. An actor invokes a web service to achieve a goal. Write the "sunny day" scenarios that deliver the goal, but also include the "rainy day" scenarios that fail to deliver. What guarantees can the system extend when the goal succeeds? When it fails? Where does the responsibility of the client end and the server's responsibility begin?

The importance of getting the requirements right cannot be overemphasized. This is the time to think about the value proposition. What is it exactly? What data are needed from the client and what must be supplied by the server? Making a mistake here is expensive.

Here is a war story about an update web service taking a list of installed software packages and the amount of free persistent memory as parameters. The service should return a list of packages to upgrade. If there is sufficient free memory, the problem is easy. But what happens if there isn't? What upgrades should be selected? Should the server assume that the old versions will be removed by the client to make room for the new ones? These are hard questions leading to complex algorithms and buggy implementations. Of course, the system should never have been specified like this. The server should serve the list of packages, the dependencies between them, and their memory requirements. Figuring out which packages to install should be the client's job. Offloading that responsibility to the server does not add value for the client, but it does add cost to server development.

So run a cost-benefit analysis at an early stage. You can ignore technology details: consider the service to be a black box. But do not compromise on capturing the details of the interactions between the service and its environment.

Next consider how the use cases can be realized. Is there going to be a single interface (WSDL 1.1's portType) offering all functionality? Or are there several interfaces? Each interface may be offered at several endpoints, but should normally be indivisible. That is, an endpoint should offer all the functionality of an interface or none. Similarly, the interface should be semantically coherent. These are guidelines; there is nothing in the WSDL spec stopping you from slicing it differently. They simply help maintain sanity.

Are any supporting interfaces needed? How and when will they be invoked? What is the nature of the dependency? There are proposals for languages to address service choreographies, to allow services to hyperlink to other services, but standards are some way off. In other words, you are in uncharted territory.

Does your service return a result to the client? Must the client know whether the service completed successfully? Must the client call the service synchronously? Or asynchronously? Or will you provide a version for both invocation styles?

The Network is Not Transparent

The network is not transparent. There is an essential difference between invoking a local service and calling it remotely. Keep Peter Deutsh's eight fallacies of distributed computing pinned above your desk.

Failure semantics in a networked environment are different from the local case: when an invocation fails you do not always know whether the service executed or not.

WSDL 1.1 describes the following invocation styles: one-way, request-response, solicit-response, and notification. However, only one-way and request-response are supported by WSDL bindings. That's also pretty much the demarcation line between what is real and what is not. In other words, the usual case where the client expects a result back from the service cannot be realized asynchronously in a practical, production-grade, and standards-compliant way today. And that's a pity because that's the mechanism that is most resilient in the face of partial failure.

WSDL allows you to mix and match invocation styles and transport. Good judgment is needed here. A first requirement is that you understand exactly how the web service protocol stack you are proposing will behave, which is not always obvious. As an example, let us examine what happens when you invoke a web service asynchronously on top of a synchronous transport mechanism. Say you send a SOAP message one-way over HTTP: the client's business logic triggers the SOAP layer to invoke the remote service. This causes an HTTP request to be sent to the server. When the request is received by the server, it should immediately return an HTTP response without servicing the request. The server-side business logic should run after the response has been returned. The response should not contain a SOAP message. An HTTP 200 or 202 status code response does not mean that the service completed successfully. A failure code, on the other hand, should guarantee that the service invocation failed. The SOAP layer on the client side blocks until it receives the response or times out.

So one-way invocation does not mitigate network latency or network failure. At best it helps to avoid server-side business logic latency. I say "at best" because I would advise you to test this feature in your server's tool chain before you rely on it; it is not trivial for a vendor to implement correctly.

Using HTTP for transport the client may be sure that its request has been delivered. Note, however, that, if the client does not receive a response, it does not follow that the server has not received and correctly processed the request. For example, the network may fail as the response is being sent. In other words, implementing at least once invocation semantics is easy, not so for exactly once.

Web Services Aren't Distributed Objects

Objects are stateful. Try to avoid state as it ties up resources and compromises scalability. A loosely coupled architecture allows individual components to change without breaking the system. It's very likely that your web service will need to be extended in the future. How can you remain backward compatible? Old clients should not break with new versions.

While RPC trumps for ease of implementation, it does not sit well with loose coupling: it requires a fixed list of parameters to be passed with each invocation of the web service. This can be mitigated somewhat by allowing some of the parameters to be "nullable". I will not bore you with the details of the interoperability problems this may cause; a more fundamental limitation is that this technique allows parameters to be left out, not to be added. Document style invocations score better as parts of a document can be defined to be optional. They can also be designed for extensibility.

Define Narrow Interfaces

Examine carefully what you expose, both at the level of the data and the data encoding. Remember that you will need to continue to support the interface you export. The narrower the interface, the easier the implementation. The narrower the interface, the easier the maintenance. WSDL has been designed for flexibility. So it is easy to slip flexibility into web service specifications unwittingly and leave a lot of implementation options open. Resist this. Clients and servers are under an obligation to be able to handle all messages that are valid under the contract.

A typical beginner's error when implementing back-end systems is to open a new database connection to the database for each query. When I made this mistake, it turned out that opening database connections took more time than everything else combined. I know of two strategies for minimizing the performance hit: use a connection pool or use stored procedures.

In the web services biosphere, the cost of establishing a connection is also high. Again, the marginal cost of an invocation can be brought down by reusing an existing connection. On the other hand, also take into account that keeping a connection open ties up resources. However you look at it, the cost associated with web service invocations is much higher than a local function call and therefore they should be used sparingly.

At this point let us exercise the analogy with stored procedures. Instead of shipping out data and applying some logic to it in the client, only to find that more data is needed, stored procedures execute the logic within the database address space. The use of stored procedures in database applications is controversial for at least two reasons. First, the language to express stored procedures is not standardized and so their use leads to vendor lock-in. Second, the business logic should reside in a separate tier, not in the database.

Nonetheless, they address a real world problem and drawbacks need to be traded off against merits in each individual case. Similarly, in the web service area, you give the client control over logic at the expense of making many fine-grained calls. Coarse-grained services are more like stored procedures. They will not only have better performance, but also simplify the effort of implementation, deployment, and management of the system as a whole.

The rule of thumb is to design your web services to have narrow but coarse-grained interfaces. Narrowness is about exposing as little as possible, coarse-grainedness is about doing as much as possible with a single call.

Separate Business Logic From Policy

Do not address authentication and authorization in business logic. Transport level security may or may not satisfy your needs. The resources sections contains some references to articles that discuss this issue. If transport level security won't cut it, there are no standards yet. They are on their way, most notably from the OASIS Web Services Security Technical Committee. The mechanisms suggested do indeed promote a separation between business logic and policy: SOAP headers are pressed into service to provide security characteristics and the messages' payload is reserved for information pertinent to the business logic.

So much for the good news. The bad news is that the proposed standard only addresses web services with SOAP bindings. Another bit of bad news is that there is no standardized way to specify the Quality of Protection (QoP) of a service in a WSDL document. Microsoft, BEA, SAP, and IBM have put together a specification for Web Services Policy Attachments that addresses the issue. To my knowledge it has not been submitted to a standardization body.

Polluting the Design Space with Implementation Concerns

Ideally, when designing web services, you should not have to worry about implementation issues. The whole point of design is that you look at the system at a high level of abstraction, not allowing the implementation complexities to befuddle you. But you can make all the right design choices and the design will be useless if the tool chains that are going to be used do not support the constructs in your WSDL. I strongly recommend prototyping the design specification as a reality check. This edge is still raw.

Acknowledgments

Many thanks to Mark Portier and Caroline Greenman for their constructive comments.