XML Transactions for Web Services, Part 2

April 29, 2003

Faheem Khan

In the first part of this series, I introduced federated web service applications and transactional web services, including brief descriptions of of the WS-Coordination and WS-Transaction specifications. In this article I discuss and demonstrate the operation of atomic transactions in web services.

What is an Atomic Transaction?

The WS-Transaction specification defines a set of protocols to support atomic transactions. The term "atomic transactions" is not specific to web services, of course. It is a concept well known in database applications. In fact, "database transactions" is effectively a synonym for "atomic transactions". WS-Transaction defines the concept of an Atomic Transaction (AT) based on the proven concept of atomic database transactions.

Since web service ATs are conceptually similar to database transactions, it would be useful to consider them. Generically speaking, you use database transactions in the following way:

1. Create a new connection to the database to mark the start of a new transaction. This creates a new transactional context.

2. Add some SQL statements into the newly created transactional context. You can think of each SQL statement as a database activity (read/write operation). The statements do not execute immediately; rather, the changes produced at this stage are temporary and can be rolled back any time.

3. Next, you commit the temporary changes. When you ask the database management system to commit the activities of the transaction, the system will make a commit attempt.

4. If there were any problems with any of the activities in the transaction, you can roll back the entire set of activities. If you decide to roll back, the state of the database will revert to its original state before the start of the transaction. This means, in case of a roll back, the database will be in a state such that the transaction never happened at all.

5. If there were no problems with any of the activities in the transaction, your commit operation will succeed, thus making the temporary changes permanent.

So how does WS-Transaction define an AT? An AT has the following characteristics:

• It results either in the commitment of all activities or none of them. Activities involved in an AT are treated as an indivisible whole. The entire set of activities -- the atomic whole -- either succeeds or fails.

  The success of the atomic whole is normally referred to as a "Commit" operation, while the failure of any single activity results in a "Rollback" of the set of activities which constitutes the AT.

• We have seen that a database transaction considers the results of a transaction to be temporary until they are either committed or rolled back. A logical implication of this temporary storage of results is that the transaction are isolated from the rest of the system. If transactions are not isolated, other database operations and transactions may alter the database during the execution of one transaction, thus producing inconsistent or inaccurate results.

  Isolation in database transactions is a technical issue. But it implies two important side effects. When we try to keep a transaction isolated, we have to temporarily lock some of the database resources, which means these resources will not be available to other applications during the execution of the transaction. This leads to two side effects:

  • A transaction should take a little time as possible, freeing locked database resources as soon as possible. Thus, database transactions are short lived.
  • Only trusted users and applications should be allowed to access the transactional features of a database; being able to lock resources indefinitely is a denial of service attack which may be launched by malicious users.

Each of these points is applicable to all ATs. An AT should always be short lived and should operate within a trusted domain.

An Application Scenario

Let's consider the enterprise implementation of the PC assembler that I introduced in the first article. Recall that the bookOrder method implementation has to ensure the availability of components required to fulfill an order from the distributor.

The PC assembler's enterprise application implementation consists of various independent modules; those which will take part in the AT include the Sales, Inventory, PC Assembly Line Management, Accounts, and Database modules.

The bookOrder method is part of the sales module. When the sales module receives a bookOrder method invocation call, it performs the following activities, each of which is part of an AT:

1. The sales module asks the inventory module to check the availability of all the required components. If the required components are available, the inventory module will make them available to fulfill the current order. Depending upon business logic, the inventory module may ask the database module to mark those components as "booked" for the current order.

2. The sales module then asks the PC assembly line management module to check whether the PC assembly line has the manufacturing capacity to fulfill the order. If the capacity is available, the PC assembly management module will ask the database module to mark the manufacturing capacity as "booked" for the current order.

These two activities are part of an AT and both should occur simultaneously. In order to commit the transaction, the inventory should have the required components and there should be enough manufacturing capacity in the assembly line. If any one of the two requirements is not met, both the activities will be rolled back.

As evident from the two activities, the database module will have an active involvement in this AT. For the sake of simplicity, we have shown just one database module for the entire set of activities. In actual practice, there may be several databases in an enterprise scale application.

The database module has the task of managing persistent date storage and, thus, will be responsible for updating the outcome of the transaction in its data storage facility.

In addition to the two activities discussed above, the PC assembler's business logic has another requirement associated with the same AT:

3. The sales module will require the involvement of the accounts module in this transaction. The accounts module will be a silent listener. It will not directly participate in any activity in the transaction. But in order that it keep a record of the relevant accounting information it will be informed about the outcome of the AT.

Now that we have elaborated the application scenario, it's time to discuss the various WS-Transaction protocols that will be used to implement the AT.

Protocols for Atomic Transactions

Each participant has a role to play in the AT. The role of the participant depends upon its responsibilities. WS-Transaction defines five protocols for atomic transactions, each of which is actually a role that a participant will play while taking part in an AT.

Each of the five protocols defines a two-party communication mechanism, which means there will always be exactly two parties communicating according to any of the five protocols. One of the communicating parties will be a participant and the other will be a coordinator.

The coordinator provides two important services, Activation and Registration, to enable the AT. The participant and the coordinator form the two ends or two ports for each of the five protocols. So each protocol has a coordinator and a participant port.

Thus, all participants of an AT will talk to the coordinator in order to fulfill the business logic requirements of the AT. This is represented graphically in Figure 1, where all the participants of our sample AT scenario are shown interacting with a coordinator. Don't worry about the arrows and numbers in Figure 1. The last section of this article will explain Figure 1 in detail.

Figure 1: Exchange of Messages during the course of an AT

The five protocols for atomic transactions include the following.

Completion Protocol

A participant who wishes to initiate the commit or rollback of a transaction will use this protocol. Recall that in general an RDBMS application creates a new transactional context, throws SQL statements into the transaction, and then initiates a commit. Thus, the participant which initiates an AT will also initiate the commit or the rollback and will register for the completion protocol.

In our application scenario, the sales module initiates the transaction and will also ask the coordinator to try to commit. Therefore the sales module registers for the completion protocol.

At the end, when the transaction concludes with either a commit or a rollback, the coordinator will send a notification back to the participant who has registered for the completion protocol. The notification conveys the information about the outcome of the transaction.

CompletionWithAck Protocol

This protocol is the same as the completion protocol, except for one detail. The participant which registers for the CompletionWithAck protocol will need to send an acknowledgment back to the coordinator after receiving the outcome notification.

2PC Protocol

The "two phase commit" idea from transactions in distributed database applications. Imagine several database management applications participating in a single transaction. How can you design a transaction control system in such distributed environments?

An obvious strategy is to implement distributed transactions in two phases:

1. The first phase (the prepare phase) is akin to asking all database applications whether they have any problem with any of the activities we want them to perform during the transaction. In the prepare phase, the applications are not required to actually do anything. They are only required to report any problems they may encounter if they make the requested changes. If any of the participating applications reports a problem, the prepare phase will fail and the transaction will roll back.

2. If the prepare phase goes successfully, we are sure that none of the database applications have any problem with the changes required by the transaction. So we can go ahead with the actual commit operation and ask all the participants to perform the actual changes already conveyed to them before the start of the prepare phase.

This type of two-phase commit allows everyone to prepare before starting to perform a coordinated activity.

In our application scenario, we have included just one database module for the sake of simplicity. The database module will register for the 2PC protocol.

PhaseZero Protocol

The name "PhaseZero" suggests a phase that occurs before the start of the first of the two phases in the 2PC sequence. In our application scenario, the inventory module checks the availability of components with the database module and then asks the database module to mark the components as "booked". This requires the database module to know about the requirement of booking components for this order before the start of the 2PC sequence. Once the 2PC sequence has started, the inventory module will not have the chance of sending any more data to the database module.

Similarly, the assembly line management module also needs to tell the database module to book the assembly line capacity before the start of the 2PC sequence. The PhaseZero protocol provides an opportunity for applications to send data updates before the start of the 2PC sequence.

Therefore, both the inventory and the PC assembly line management modules will register for the PhaseZero protocol. You can think of PhaseZero protocol as an opportunity for middleware applications to flush any outstanding data to database applications before the start of the 2PC sequence.

OutcomeNotification protocol

This protocol is for applications that want to act as silent listeners in an AT. Such applications will have no active participation in the transaction, but they will be informed about its outcome.

In our application scenario, the accounts module will register for the OutcomeNotification protocol.

Exchange of Messages in Atomic Transactions

Look at Figure 1, which graphically depicts the exchange of messages that takes place in our AT scenario. Figure 1 shows all the actors that participate in the AT (the coordinator and the various modules of the PC assembler's enterprise). Messages are shown as numbered arrows. Each message of Figure 1 is explained below:

1. The sales module initiates a new AT by sending a CreateCoordinationContext SOAP message to the coordinator as shown in Listing 1. The CreateCoordinationContext message is sent inside the SOAP:Body element as a normal web service method invocation call.

The CreateCoordinationContext message contains three child elements, all of which belong to the WS-Coordination namespace. The WS-Transaction specification uses the WS-Coordination features for several jobs. The creation of a new coordination (or transactional) context is one such job.

The first child, the ActivationService element, contains the address of the activation server. The activation server is part of the coordinator (or you can say that activation is one of the services provided by the coordinator). The activation server is responsible for creating new transactional contexts.

The second child, the RequesterReference element, contains the address of the application that requested the creation of the new transaction context. In this case, the sales module is making this request, so the RequesterReference element refers to the sales module.

Notice the use of "http://schemas.xmlsoap.org/ws/2002/07/utility" namespace in these listings. It's just a helper namespace to wrap some low level information such as addresses. Both WS-Coordination and WS-Transaction often use this namespace.

The third child is the CoordinationType element, which contains the information about what type of coordination the new context will perform. The content of this element is a URI, "http://schemas.xmlsoap.org/ws/2002/08/wstx". This URI string is the identifier for AT. This means that the new context will coordinate an AT.

In the next part of this series, we'll see that the WS-Transaction specification also defines another type of transaction, a Business Activities (BA), which is very different from AT and is identified by a different URI.

2. The coordinator sends Listing 2 to the sales module in response to Listing 1. Listing 2 is a SOAP response which carries a CreateCoordinationContextResponse element in its body. The CreateCoordinationContextResponse element has two children.

The first child is the RequesterReference element, which is exactly the same as the one we mentioned in step 1 (address of the sales module).

The second child is the CoordinationContext element, which is perhaps the most important element in our discussion. This element is actually a wrapper for all the information that specifies the newly created coordination context. The CoordinationContext contains three child elements:

The first child is an Identifier element that carries a transaction identifier. Every transaction should have a unique identifier, which all participants will use in their activities relating to the particular transaction.

The second child is the CoordinationType element that is exactly the same as the one we explained in step 1 (the content of this element is a URI, which is an identifier for ATs).

The third child is the RegistrationService element that refers to an address. This address is the port where a registration server is listening. The requesting application (the sales module) and all other participants will use this address to register their respective roles in the transaction.

Activation and registration are two services provided by the coordinator. Participating applications use activation service to create a new transactional context and registration service to register their respective roles in the transaction.

The CoordinationContext element wraps three important bits of information: the transaction identifier, the type of transaction, and the address of the registration service.

3. On receipt of the SOAP response of Listing 2, the sales module needs to register for its role in the AT. So it sends a registration request to the coordinator. Listing 3 shows the registration request, which is a simple SOAP request carrying the Register method invocation call in its body. The Register request carries four bits of information, each as a child element.

The first child of the Register element is the RegistrationService element, which is the same as the one we came across in step 2. It carries the address of the port where the registration service of the coordinator can be reached.

The second child of the Register element is the RequesterReference element. Again this element is identical to the one we saw in steps 1 and 2.

The third child element is a ProtocolIdentifier element. It identifies the protocol (or the role) for which the participating application wants to register for. The content of the ProtocolIdentifier element is a URI (http://schemas.xmlsoap.org/ws/2002/08/wstx/Completion), which is the identifier for the completion protocol.

The fourth child is a ParticipantProtocolService element, which specifies the address of the port where the participating application has implemented its side of the protocol. Recall that all five protocols described in the last section are two-party communication protocols. One of the two parties is the coordinator and the other party is the participant. The ParticipantProtocolService element specifies the address of the second party.

4. The coordinator's registration service responds with a message that will mark the end of successful registration process. Listing 4 shows the RegisterResponse element wrapped inside the SOAP body. The RegisterResponse element has only two child elements.

The first child is the RequesterReference element, which is same as of steps 1, 2 and 3.

The second child is the CoordinatorProtocolService element, which specifies the address of the port at which the coordinator has implemented its side of the protocol. In step 3, the ParticipantProtocolService element specifies the participant side of the two-party communication protocol. In step 4, the CoordinatorProtocolService element specifies the coordinator side of the same protocol.

Some readers may object to Listings 3 and 4 for not containing enough information to uniquely identify the transaction instance that Listings 3 and 4 are meant to register for. This is an ambiguity that the current WS-Coordination and WS-Transaction specifications do not address. But the WS-Coordination specification does include an editor's note that suggests that this ambiguity will be addressed in a future update of the specification (refer to section 8.1, Appendix A of the WS-Coordination specification).

For now you can implement your own (application specific) methodology to identify the transaction instance. Perhaps the best way to do this is to include a transaction identifier within the ParticipantProtocolService element in Listing 3 and the CoordinatorProtocolService element in Listing 4. That would enhance the way port addresses are specified or referenced. In fact the editor's note mentioned suggests that the future update will enhance the way port addresses are referenced.

5. According to the business logic of our application, sales is not the only module that takes part in this AT. There are other intended participants of this AT. Now is time for the sales module to send some application specific messages to other intended participants. Listing 5 shows a generic message, which will be sent to the inventory, assembly line management, and account modules. Notice that Listing 5 wraps the application specific message in its body, while a CoordinationContext element (the same as the one we described in step 2) is wrapped inside the SOAP:Header element. The application specific body of the message will convey what the intended participant is supposed to do in the AT (e.g. that the inventory module is supposed to check the availability of components).

The SOAP:Header element is a part of the SOAP extensibility framework. This means that SOAP allows the development of SOAP-based protocols, which will use the SOAP:Header element to wrap protocol specific information. Many such protocols are emerging which use the SOAP:Header element to convey protocol specific information in application messages. WS-Coordination is one such protocol. Another is the Web Services Security (WSS) specification, which adds security information to the SOAP:Header element (refer to the Resources section for more information on WSS).

We have shown just one application specific message being sent by the sales module to other intended participants. In actual practice there may be any number of such messages. However, all messages should include the CoordinationContext element inside their SOAP:Header elements. This way the CoordinationContext element is used to propagate transaction information to other participants.

6. On receipt of Listing 5 the inventory module needs to register for the same transaction. So it sends a register message to the coordinator. Listing 6 shows that register message. Notice that the register message of Listing 6 is similar to the register message of Listing 3. The only differences are the address of the requesting application (this time the address is of the inventory module) and the protocol for which the inventory module wants to register (this time it is the PhaseZero protocol). Recall from our discussion in the last section that the inventory module will register for the PhaseZero protocol.

7. The coordinator will respond with the RegisterResponse message as shown in Listing 7. Listing 7 is similar to Listing 4, which is already explained in step 4.

8 and 9. These two steps are similar to 6 and 7. The PC assembler module also registers for the PhaseZero protocol. See the exchange of Listings 8 and 9.

10 and 11. The accounts module also registers for the same transaction. As already explained, the role of the accounts module is as a silent listener. So it registers for the OutcomeNotification protocol. See the exchange of Listings 10 and 11.

12. Now the inventory module sends an application specific message (Listing 12) to the database module. Notice that Listing 12 is similar to Listing 5 (the SOAP message that carries the coordination context in its header and an application specific message in its body).

13 and 14. The database module registers for the 2PC protocol in the same transaction. See the exchange of Listings 13 and 14.

15. Since all the relevant parties now know what they are supposed to do in this transactionm and they all have registered with the coordinator, it's time for the sales module to try to commit the AT. The sales module will send a commit request to the coordinator as shown in Listing 15. The Commit element in Listing 15 belongs to the WS-Transaction namespace (http://schemas.xmlsoap.org/ws/2002/08/wstx). Notice that there are only two child elements of the Commit element. The two child elements represent the two sides of the completion protocol that the sales module registered for in step 3. As the commit request is going from the sales module to the coordinator, so the source of the commit request is the sales module and the target of the request is the coordinator.

The first child of the Commit element is the TargetProtocolService element, which specifies the target of this commit request. The TargetProtocolService element contains the same address of the coordinator service that the coordinator exchanged with the sales module by sending the CoordinatorProtocolService element in Listing 4.

The second child is the SourceProtocolService element, which specifies the source of this request. The SourceProtocolService element contains the address of the port where the sales module has implemented its side of the completion protocol. Recall from step 3 that the sales module sent this address to the coordinator (the address was wrapped inside the ParticipantProtocolService element of Listing 3).

16. On receipt of the commit request of Listing 15, the coordinator initiates the commit sequence. The first step is the PhaseZero protocol. So the coordinator will check which of the participants have registered for the PhaseZero protocol. In our case, inventory and assembly line management modules have registered for the PhaseZero protocol. Therefore, the coordinator sends a PhaseZero message to the inventory module, as shown in Listing 16.

17. As already explained in the last section, the PhaseZero protocol provides an opportunity for middleware applications to flush outstanding data updates to resource managers. On receipt of Listings 16, the inventory module will flush any outstanding data to the database module in an application specific message as shown in Listing 17.

The SOAP:Header element in Listing 17 carries the CoordinationContext element of Listing 2, so that the database module knows which transactional context it should write to.

18. When the inventory module has successfully flushed the data to the database module, it will send a PhaseZeroCompleted message (Listing 18) to the coordinator, thus telling the coordinator that it has fulfilled its responsibility for the PhaseZero protocol.

19, 20 and 21. Similarly, the coordinator sends a PhaseZero message (Listing 19) to the assembly line management module. The assembly line management module flushes any outstanding data updates to the database module in an application specific message of Listing 20 and then sends a PhaseZeroComplete message (Listing 21) back to the coordinator.

22. Now the coordinator knows that PhaseZero is complete. So it will initiate the 2PC sequence. Only one participant (the database module) has registered for the 2PC protocol. So the coordinator will send a Prepare message (Listing 22) to the database module. Recall from our discussion in the last section that prepare is the first of the two phases in the 2PC sequence.

23. On receipt of the Prepare message of Listing 22, the database module will check if the changes required by the transaction will produce any problem in its back-end data storage. For example, if the assembly line is fully booked or the required components are not available in stock, the database module will not be able to commit the required transaction.

If everything is found to be in order, the inventory module will send a Prepared message (Listing 23) back to the coordinator.

24. Now the coordinator will issue a Commit message (Listing 24) to the database module (the second phase in the 2PC sequence).

25. The database module will make the actual changes in the database (book the components for this order and book the assembly line as well). Once the changes have been made, the database module will send a Committed message (Listing 25) back to the coordinator.

26. Now the coordinator sends a Committed message (Listing 26) back to the sales module, thus notifying that the transaction has been successfully completed.

27. The coordinator checks whether there is any participant which has registered for the OutcomeNotificaton protocol. The accounts module registered for the OutcomeNotification protocol. So the coordinator sends a Committed message (Listing 27) to the accounts module.

28. The accounts module responds with a Notified message (Listing 28), thus acknowledging receipt of the transaction outcome notification.

In the next part of this series, we will see what happens if things go wrong. Does AT rollback mean that we lose this business opportunity? There should be ways to compensate for rolled back transactions. For example, we might want to buy components from vendors if they are not available in stock. These questions lead us to Business Activities (BA), which I will examine in the next article, particularly how a BA crosses the boundaries of trusted domains.

Resources

• Check out the official WS-Coordination and WS-Transaction specifications.
• Business Transaction Protocol (BTP) by OASIS is another XML-based protocol that manages transactions. Check out the main BTP page at OASIS web site.
• This article discusses the use of database transactions in Java.
• This two-part series of articles (Part 1 and Part 2 ) gives a high level overview of WS-Coordination and WS-Transaction.
• We mentioned the Web Services Security (WSS) specification in this article. Check out the main page for the WSS specification at OASIS web site.