Web Services Pitfalls
February 13, 2002
The latest hot ticket for vendors to sell and journalists to write about is web services. The appeal is natural: web services promise users and developers greater choice of components and services. This article examines perhaps the most futuristic of web services, those offered by a standalone service provider. In particular, it focuses on the infancy of the standards and technology in standalone web services.
What Are Web Services?
Web services is an umbrella term used to describe components and services that are addressable and available using web technology. The kinds of web services are typically user-oriented and browser-based, API-accessible, or system services functionality. A web service could be a browser-based e-mail program, an XML-based interface to an HR system, a SOAP service offered by a machine, a SOAP monitoring service, XML-based integration with an EAI or legacy system, and so on. The standards for the way components in a web service exchange data is crucial. Some of the infrastructure standards that are being created include SOAP, WSDL, UDDI, SAML, ebXML, as well as many vertical standards being created as well.
There are at least three common web service deployment types. In the first place, one can add a web service interface onto an existing product; examples of this approach include application servers, databases, messaging systems, enterprise resource planning tools, and so on. Generally software vendors are taking this approach. In the second place, customers deploy vendor products to solve current integration needs. The customer then uses the new web services functionality to integrate internally or with external partners more easily.
In the third place, an application service provider offers a web service interface, and its customers access and use the service using web service standards. An interesting difference between these deployment types is the different economic model they imply. Vendors gain make money from product sales; customers gain a return on their investment from increased efficiency and expanded customer revenue; and application service providers make money from recurring or rental revenue of the web service itself.
Web Services Essentials
Web services are not especially new. Application service providers (ASPs) have been providing end-user based web services for many years. While some have struggled, the majority of ASPs — particularly the focused ones — have continued to grow in customers and revenues over the past few years. In fact, many ASPs have provided APIs for parts of their service for a long time. Many provide APIs for provisioning, security, billing, and aspects of the business process. If you believe in the web services provider model, then you also have to believe in ASPs as there is little difference between them.
So where’s the rub? Many of the business and technical issues for service providers and consumers are incomplete. Consider one of the standard web service examples, a travel reservation service. The scenario usually goes something like this: an airline reservation service is currently available via a browser, then a web service API is added to allow applications to communicate directly. The ASP would provide the interface in a SOAP format, define the interface using WSDL, and register the service provider and service in a UDDI repository. A developer discovers the service and creates software that invokes the service.
Contracts and Billing
In reality it's not that simple, How many service providers offer a free service? Very few. The service provider will clearly want to earn revenue based upon service consumption. This typically involves a negotiated contract between the consumer and provider. In our example, the developer would have to know that a contract had already been reached with the service provider. Every time the service provider has a new consumer, and every time the consumer uses a different service provider, a new contract is required. If a service provider wants to reach one hundred different consumers, potentially one hundred contracts are needed.
The problem is harder if the consumer and producer want different pricing models based upon different characteristics. I'm skipping over the issue of how the consumer and producer actually exchange usage and billing information; suffice it to say that there are no widely deployed standards for usage and billing.
The service provider and consumer need a contract in place just as with any other non-technical customer-provider arrangement. As contract negotiation has many facets, this can be a difficult task to scale for either side.
The service's security model is crucial, of course. It is doubtful that a travel reservation service would allow any arbitrary user to book tickets. The ASP must know who is allowed to use which service, and it must know who is allowed to tell it who can use which service. One model is that the ASP will have a list of trusted users. From an API perspective, this becomes a little trickier than the web browser sign-on case. The service provider will specify how the authentication information will be encoded in the message. There are currently no standards widely deployed for encoding security credentials in a SOAP or B2B Message. Some are emerging, but they aren’t standardized yet.
How does the service know what are valid authentication credentials? A typical existing model is that the service provider manually enters a new username and password for each consumer when the contract is signed. It’s crucial that the credential be changed frequently.
Again, a key point is that the service provider and consumer need a mechanism in place to exchange security credentials, and typically a contract must be in place before the credential is exchanged.
The process of adding a user to a system, typically called provisioning, must also be addressed. In the travel reservation service example, user information must be entered into the reservation service: seat preferences, meal preferences, frequent flyer numbers, preferred alliance partners for flight selection, billing address, shipping address for the ticket, credit card information. For each user that can use the service, information needs to be entered. There can also be complications, where the price of the ticket is dependent upon the company that the user is part of. Often larger companies negotiate discounted rates based upon their size.
Thus the ASP must provide a mechanism for providing user information, and typically these would be in an automated API manner. The developer integrating the service must also incorporate logic to provision the service for the user. This logic would presumably be executed once the user is authorized to use the service.
The service provider and consumer need a mechanism to exchange provisioning information, and the service provider must know who it trusts as a source of that user information. It’s hard to imagine this without a contract in place.
DLL Hell Magnified?
Last, there's the issue of service version control. Service providers change their interfaces and workflows. Perhaps the travel reservation service provider decides to add a parameter to the fare quote request, for example: the specification of whether an electrical port is required for a particular trip. Usually the ASP will provide a new interface for the parameter, and it probably won’t keep the old interface available. It is very difficult for ASPs to maintain backwards compatibility, and they often don’t. Their databases, business logic, have all changed, so they don’t want to keep the old stuff running. A typical example of this is the way that organizations often don't offer two different versions of an ERP system at the same time, using the same underlying data.
What happens to our client when the interface changes? Generally, the client code breaks. The ASP's interface is expecting another parameter in the middle step, and the client software isn’t providing it. In the worst scenario, the ASP simply changed the interface and didn’t give any advance warning to the client. This is what typically happens in the web service provider market today. When the interface changes, the clients break. The consumer must assign a developer to discover the new parameter, perhaps changing interface and logic to collect and pass on the parameter. Depending upon the complexity, the consumer’s service may be down for an unknown amount of time, perhaps weeks or months.
This seems a lot like DLL hell, only worse. In DLL hell, a new application on your desktop installs a new DLL that breaks one of your existing applications. In web services DLL hell, a service upgrade breaks an arbitrarily large number of services that are on many different and separate machines, outside of a firewall. The consumer of the service can’t download an update, uninstall, or re-install.
There is no technical solution even mentioned in any of the presentations on web services. The problem of upgrading services and how that affects existing client software is not treated in any way.
The only solution today is through the use of contracts. Typical wording in the contract will state that the service provider must give 60 days notice of interface change, with test accounts available for testing available within 30 days.
But even then, each and every client must upgrade. The consumer simply has more warning. The key message though is that part of the contract between the service provider and the consumer is the treatment of versioning of interfaces.
What’s it all mean?
I’ve described some of the real-world issues of web service integration, particularly contract negotiation, billing, security, provisioning, version control. I’ve skipped other important issues like quality service, monitoring and support. But it turns out that any real and useful web service must provide some specification of these capabilities. It is unlikely that the service provider will simply provide these to the general world via a public registry. I expect that for quite some time, services will be offered via a private catalog. There will be a fairly extensive and human-centric process that occurs before data and information is entered in the catalog on behalf of consumers and service providers. The specifications for the various capabilities are also collected during this negotiation process. Consumers can then trust the validity of the catalog, and service providers can trust that consumers have been validated. Further consumers can go to one place to find all the information, rather than n different service providers. And Service Providers can update only one place, rather than m consumers.
Building an ecosystem of integrated web services is much more complex than the examples often suggest. There are many characteristics that must be accounted for, with contractual policies in place. The hypesters often don’t address these issues when describing web services, but they exist nonetheless.
The real summary is that web services will be used as an enabling technology to integrate applications together more quickly and easily. There are real-world integration problems that are being solved by web services standards compliant products and deployments. It will be quite some time, if ever, before there is widespread adoption of standalone web services that are usable without significant human intervention and contract negotiation.