Web Services Security, Part 2
April 1, 2003
In my previous article I discussed the security requirements of web services in B2B integration applications. I also introduced some XML-based security standards from W3C and OASIS.
In this article, I will discuss three XML-based security standards -- XML Signatures, XML Encryption and Web Services Security -- which offer user authentication, message integrity and confidentiality features in SOAP communications. You can safely bet that these three standards fill the SOAP security hole I described previously. In what follows I explain how that hole is filled by demonstrating the creation, exchange, and processing of XML messages inside XML firewalls.
Basic Cryptographic Concepts
The discussion of message integrity, user authentication, and confidentiality employs some core concepts: keys, cryptography, signatures, and certificates. I will briefly discuss cryptographic basics. If you're in further details may refer to the Resources section, which contains a link to a freely downloadable handbook on applied cryptography.
A popular cryptographic technique is to use a pair of keys consisting of a public and a private key. First, you use a suitable cryptographic algorithm to generate your public-private key pair. Your public key will be open for use by anyone who wishes to securely communicate with you. You keep your private key confidential and do not give it to anybody. The public key is used to encrypt messages, while the matching private key is used to decrypt them.
In order to send you a confidential message, a person may ask for your public key. He encrypts the message using your public key and sends the encrypted message to you. You use your private key to decrypt the message. No one else will be able to decrypt the message, provided you have kept your private key confidential. This is known as asymmetric encryption. Public-private key pairs are also sometimes known as asymmetric keys.
There is another encryption method known as symmetric encryption. In symmetric encryption, you use the same key for encryption and decryption. In this case, the key has to be a shared secret between communication parties. The shared secret is referred to as a symmetric key. Symmetric encryption is computationally less expensive than compared to asymmetric encryption. Which is why asymmetric encryption is ordinarily only used to exchange the shared secret. Once both parties know the shared secret, they can use symmetric encryption.
Message digests are another concept used in secure communications over the Internet. Digest algorithms are like hashing functions: they consume (digest) data to calculate a hash value, called a message digest. The message digest depends upon the data as well as the digest algorithm. The digest value can be used to verify the integrity of a message; that is, to ensure that the data has not been altered while on its way from the sender to the receiver. The sender sends the message digest value with the message. On receipt of the message, the recipient repeats the digest calculation. If the message has been altered, the digest value will not match and the alteration will be detected.
But what if both the message and its digest value are altered? That kind of change may not be detectable at the recipient end. So a message digest algorithm alone is not enough to ensure message integrity. That's where we need digital signatures.
Keys are also used to produce and verify digital signatures. You can use a digest algorithm to calculate the digest value of your message and then use your private key to produce a digital signature over the digest value. The recipient of the message first checks the integrity of the hash value by repeating the digest calculation. The recepient then uses your public key to verify the signature. If the digest value has been altered, the signature will not verify at the recipient end. If both the digest value and signature verification steps succeed, you can conclude the following two things:
- The message has not been altered after digest calculation (message integrity); and
- the message is really coming from the owner of the public key (user authentication).
In its most basic form a digital certificate is a data structure that holds two bits of information:
- The identification (e.g. name, contact address, etc.) of the certificate owner (a person or an organization); and
- the public key of the certificate owner.
A certificate issuing authority issues certificates to people or organizations. The certificate includes the two essential bits of information, the owner's identity and public key. The certificate issuing authority will also sign the certificate using its own private key; anyone interested party can verify the integrity of the certificate by verifying the signature.
Message Integrity and User Authentication with XML Signatures
The XML Signature specification, XML Digital Signature, (XMLDS) has been jointly developed by W3C and IETF. It has been released as a recommendation by W3C. XML Signature defines the processing rules and syntax to wrap message integrity, message authentication, and user authentication data inside an XML format.
Recall from my previous article the interaction between a vacation tour operator and
partner hotels. Let's assume that the tour operator wants to invoke the
GetSpecialDiscountedBookingForPartners method of a partner hotel's web
service. This method provides online hotel booking service at special discounted rates.
These special discounted rates are only available for trusted business partners and
meant for the general public.
The tour operator includes message integrity and user authentication information within
GetSpecialDiscountedBookingForPartners SOAP method invocation. The hotel's
XML firewall, on receipt of the invocation, will need to look into the SOAP message
- The message has not been altered while on its way to the hotel's web service (message integrity); and
- the requester is really a trusted business partner (user authentication).
The XML firewall will only let the request pass onto the SOAP server if both these conditions are met. Figure 1 illustrates the process of user authentication in which the following sequence of events occurs:
- The tour operator sends a method invocation SOAP request to the hotel's web service. The request includes all the relevant security information (message integrity and user authentication).
- The hotel's web service is protected by an XML firewall, which receives all requests destined to the SOAP server. The XML firewall checks that the message as received is identical to the one that the requester intended to send.
- If the message integrity is found to be in order, the XML firewall reads the requester's identification information from the SOAP request and makes sure that the user is really a trusted business partner.
- If the requester is found to be a trusted business partner, the XML firewall allows the request pass onto the SOAP server.
Listing 1 is a
simple SOAP request that carries the
method call to the hotel's web service. The SOAP request of Listing 1 does not contain
any message integrity or user authentication data. Listing 1 is the starting
point to demonstrate XMLDS.
I'm using SOAP as an example XML format to demonstrate XMLDS, which isn't SOAP-specific. XMLDS can be used to insert signatures and message digests into any XML instance, SOAP or otherwise.
The following example will insert XMLDS tags inside the SOAP header. XMLDS is flexible and allows the insertion of XMLDS tags anywhere in an XML file. In fact there are three types of XML signatures: enveloping, enveloped and detached. When an XML signature wraps the data being signed, it is said to be an enveloping signature. If the XML data being signed wraps the signature (i.e. the XML signature becomes an element of the XML data being signed), it is said to be an enveloped signature. If the signature and the data being signed are kept separate -- the element being signed and the signature element are siblings -- it is said to be a detached signature. The XMLDS authoring example presented in this article uses detached signatures.
Four Steps to XML Digital Signature Authoring
The first step is to create a
Signature element. The
element will eventually wrap all the other XMLDS elements. Have a look at Listing 2, which has
exactly the same body as that of Listing 1. The only
difference between Listings 1 and 2 is that
Listing 2 contains the XMLDS namespace declaration
http://www.w3.org/2000/09/xmldsig#) and a SOAP header. The SOAP header wraps
Signature element in Listing 2 contains three
Listing 2 shows that the
Signature element is only a wrapper for other XMLDS tags. In steps 2, 3, and
4, we'll create the child nodes of the three
The second step is to create the child nodes of the
SignedInfo element. Listing 3 is the result of
SignedInfo child nodes into Listing 2. The complete
SignedInfo structure tells the details of the process that leads to an XML
signature. You can notice from Listing 3 that there are several children of the
SignedInfo element and
each of its children contains some bit of information as explained below.
CanonicalizationMethod is a required element that identifies the
canonicalization algorithm applied to the
SignedInfo element before producing
Canonicalization algorithms are important in XML signature applications because message digest algorithms treat XML data as octet streams. Two different octet streams can represent the same XML resource. For example, if you change the sequence of attributes occurring in an XML element, the resulting XML file will be a logically equivalent version of the original XML file. However the two logically equivalent XML files will contain two different octet streams and will produce different digest values.
Canonicalization algorithms are meant to produce identical octet streams for logically equivalent XML data. In order to make sure that logically equivalent XML documents produce the same digest value (and the same signature), we need to canonicalize our XML resources before digesting their octet streams.
CanonicalizationMethod element in Listing 3 has an
Algorithm, which has a URI string as value
http://www.w3.org/2001/10/xml-exc-c14n#). This URI string identifies
Exclusive XML Canonicalization, an algorithm by the W3C. The details of XML canonicalization
are beyond the scope of this article. Please refer to the resources section for a
articles that discusses XML canonicalization in detail.
At this stage, we have just created the
CanonicalizationMethod element. We
have not yet applied the canonicalization algorithm to anything. We will apply the
canonicalization algorithm to the
SignedInfo element after authoring all its
The next child of the
SignedInfo element in Listing 3 is a
SignatureMethod element, whose
Algorithm attribute identifies
the algorithm that will be used to produce the cryptographic signature.
The third child of the
SignedInfo element is a
There should be at least one
Reference element inside a
Reference element is used to hold various bits of information as
A reference to the data that is being signed. This is the job of the
URIattribute of the
Referenceelement. You may include the data to be signed within the XML document or you may keep it external. If your data and the signature reside within the same XML document, you will refer to it using a fragment identifier as a value of the
URIattribute of the
Referenceelement. This is what we have done in Listing 3. The value of the URI attribute points to the
GetSpecialDiscountedBookingForPartnerselement. If, on the other hand, your data is external to the XMLDS file, you will refer to it using a URI as the
URLattribute value of the
XMLDS allows you to perform some operations on your data before digesting and signing it. For example, you can canonicalize your data before signing it Or you may want to apply some XSL transformations on your data before digesting it. For instance, you may have some pricing data in a simple tabular form of model numbers and prices and you may want to transform the tabular form into a formal invoice before signing it. In this case, you may use an XSL transform as a template representing your invoice. This would mean that you intend to sign the complete formal invoice and not just the raw data included in XMLDS file.
Transformselement holds the information regarding what operations you performed on your data before signing it. Look at the
Transformselement in Listing 3, which contains one
Transformchild element. There can be any number of
Transformelement identifies a transformation algorithm. When you apply a transformation to your data before signing it, you will include a reference to what you did by adding a
Transformelement. This will tell the recipient application of your signed file to do the same transformation before attempting to verify the signature. In our case, we have applied just one operation, which is the canonicalization algorithm specified by the
Algorithmattribute of the
Transformelement in Listing 3.
If there is more than one
Transformelement, their order is important. Transformations are applied in the same order that they appear in a
Transformselement. All the transformations are performed before digesting the data. Hence, the output of the last
Transformelement is the input to the message digest algorithm.
What algorithm did you use to produce the digest value? The XMLDS specification suggests the use of SHA-1 digest algorithm. The
DigestMethodchild of the
Referenceelement holds this information in its
Algorithmattribute value (
The digest value itself. The
DigestValueelement in Listing 3 contains the actual digest value produced by digesting the canonicalized form of the
GetSpecialDiscountedBookingForPartnerselement. Note that binary data in raw form (such as the sequence of octets produced by message digest, signature, and encryption algorithms) cannot be wrapped inside XML markup as such; it may produce problems while XML parsing. Such data is base-64 encoded before wrapping inside XML markup. The result of base-64 encoding is that the encrypted data does not contain any byte that conflicts with XML processing rules.
SignedInfo and its child elements have been authored, you will
canonicalize the complete
SignedInfo element with the algorithm identified by
CanonicalizationMethod element. You will then produce the signature value
and wrap the signature value inside a
SignatureValue element as shown in Listing 4. While signing,
you will use the canonical form of the complete
SignedInfo element as data to
be signed. This includes all the child elements of the
Notice that the
SignedInfo structure contains a reference to the data being
URI attribute of the
Reference element), the digest
value, and the name of the signature method as well as other bits of information.
SignedInfo structure effectively means that you are signing the
digest value of your data along with a reference to the data itself.
Signature element in Listing 2 contains another
KeyInfo. The fourth step is to create its child elements. In Listing 5, the
KeyInfo element contains a
KeyName child element. The
KeyName element is an identifier for the key that will be used for signature
KeyName is just a placeholder for key identifiers. XMLDS does not
specify the mechanism which will relate the identifier with the actual key pair used
signing. It is up to XMLDS applications to design their own mechanism for key
identification. For example, the key identifier in Listing 5 (MyKeyIdentifier) may
a shared secret (a symmetric key) previously exchanged between the tour operator and
KeyInfo element is optional: you may or may not include a
KeyInfo element in a signature. The
KeyInfo element is optional
because a signature application may not want to include key information inside the
KeyInfo element may also be used in XML Encryption applications that
we will demonstrate in the next section.
These four steps are a very simple demonstration of XMLDS. Listing 5 is a complete SOAP message that carries message integrity and user authentication data in its header.
Now it's time to demonstrate the processing of the XMLDS-based header of Listing 5 at the hotel's web service end.
XML Digital Signature Validation
The validation procedure is simple and can be logically deduced from XMLDS authoring steps discussed earlier. It involves three main tasks.
First, canonicalize the
SignedInfo element. Recall that the
CanonicalizationMethod element specifies the canonicalization algorithm. Use
this canonical form of the
SignedInfo element for the rest of the validation
Second, check the integrity of the message by verifying the digest value contained
Reference element that we authored in step 2 above. For digest verification,
you need to know three things:
The data that needs to be digested. You dereference the
URIattribute of the
Referenceelement in order to get the data that needs to be digested.
Any transformations that may have been applied to the data before applying the digest algorithm. The
Transformselement contains this information. You will apply the same transformations to the data before digesting it.
The digest algorithm. This information is contained in the
Algorithmattribute value of the
DigestMethodelement. You will apply the message digest and verify that the digest value is the same as that contained in the
If the digest verification fails, the validation process fails and we're done.
If the digest value is found to be in order, the third task is to verify the signature.
the signature verification, you need the signer's key (the public key or the shared
You obtain the key information from the
KeyInfo element if it is present (or
your application may already know the keying information from some other means). Once
know the key to be used in signature validation, read the signature method used to
the signature. The
Algorithm attribute of the
element contains this information. Then use the canonical form of the
SignedInfo element and the key to confirm the signature value.
An XMLDS implementation can create SOAP headers to produce signed SOAP messages. The XML firewall sitting at the recipient's end will process the SOAP header to verify the signatures before forwarding the request to the SOAP server. This process is graphically illustrated in Figure 1. We can achieve the following two security objectives through this procedure:
- We can verify that the SOAP message that we received was really sent by the sender we think it came from.
- We can verify that the data we received has not been changed while on its way and is the same that the sender intended to send.
So now we are sure that the request for special discounted booking is really coming from a trusted partner hotel and that no one has altered the data on its way. But hackers can still see the data while traveling across the Internet. So let's see how the XML encryption specification solves this problem.
The XML Encryption specification satisfies confidentiality requirements in XML messages. XML encryption offers several features.
- You can encrypt a complete XML file.
- You can encrypt any single element of an XML file.
- You can encrypt only the contents of an XML element.
- You can encrypt non-XML data (e.g. a JPG image).
- You can encrypt an already encrypted element (i.e., "super-encryption").
Encrypting a Complete XML File
We'll start with encrypting a complete XML file. Have a look at Listing 6, where we have
shown an XML encrypted file. We have not shown the XML document that we encrypted
at Listing 6 because it
doesn't matter. Encrypting any XML file will produce the same XML structure, except
encrypted value wrapped inside the
CipherValue element in Listing 6.
EncryptedData element in Listing 6 holds the
encrypted data along with relevant information such as the algorithm used for encryption.
EncryptedData element contains the XML Encryption namespace declaration
http://www.w3.org/2001/04/xmlenc#) and has an attribute named
MimeType with value
text/xml. This attribute advises the
recipient of the XML encrypted file that we encrypted an XML file to produce the
The first child of the root
EncryptedData element is the
EncryptionMethod element. The
EncryptionMethod element has an
Algorithm, which specifies the algorithm we used for
encryption. The value that we have used for the Algorithm attribute is
http://www.w3.org/2001/04/xmlenc#3des-cbc, which specifies the triple DES
algorithm for encryption.
ds:KeyInfo element in Listing 6 is the same as
the one used in XMLDS. Note that the
ds:KeyInfo element has been borrowed form
the XMLDS namespace.
EncryptedData element has another child element named
CipherData, which in turn has a child element named
CipherValue element holds the encrypted content (the encrypted version of
the XML document that we wanted to encrypt). Therefore, encrypting the XML file produces
contents of the
Encrypting a Single Element
We have seen that the
EncryptedData structure holds the encrypted data along
with relevant information. While encrypting a single element of an XML file, we will
same concept. Look at Listing
7, in which we have we have encrypted the complete
GetSpecialDiscountedBookingForPartners element of Listing 1 by simply
replacing it with the
You can compare the
EncryptedData element of Listing 6 with the
EncryptedData element of Listing 7. You will notice
that there is one difference. Instead of the
MimeType attribute of Listing 6, we now have a
Type attribute in Listing 7. The value of
this attribute is
http:///www.w3.org/2001/04/xmlenc#Element, which means we
encrypted an XML element.
Whenever you are encrypting an element of an XML file, you will use the identifier
http:///www.w3.org/2001/04/xmlenc#Element as the
value. This tells the recipient of the XML encrypted file that the encrypted data
treated as an XML element in decrypted plain text form.
Encrypting the Contents of an Element
Look at Listing 8, in
which we have encrypted only the contents of the
GetSpecialDiscountedBookingForPartners element by replacing the contents with
EncryptedData structure. This is similar to what we did while encrypting an
element (Listing 7. There
is a difference; this time, the value of the
Type attribute of the
EncryptedData tag says
The value tells that the encrypted data should be treated as element content.
XML Encryption Processing
How will our XML firewall work with these encryption concepts? It will receive Listing 7 or 8 (SOAP messages with encrypted elements or content) and translate the contents to a decrypted form before forwarding the decrypted SOAP message request to the SOAP server.
The recipient of an XML encrypted file (e.g. the hotel's XML firewall in our case) will decrypt the XML encrypted file in the following sequence:
- Extract the encrypted content of the CypherValue element.
- Read the algorithm attribute value of the EncryptionMethod element.
- Read the Type attribute values of the EncryptedData element.
- Obtain the keying information form the ds:KeyInfo element.
- Use the information gathered in steps 1, 2, 3, and 4 to construct the plain text (decrypted) file.
An Introduction to Web Service Security
How will our XML firewall use XML signatures and encryption to protect SOAP servers? We have given many examples of using the two technologies individually, but the question of how to apply these two technologies in an XML firewall application to protect a SOAP server still needs to be addressed, especially since neither XMLDS nor XML Encryption are SOAP-specific. So why have we put all the signature related information in the SOAP header? Why not wrap it inside the SOAP body?
The Web Services Security (WSS) specification from OASIS defines the details of how to apply XML signature and XML encryption concepts in SOAP messaging. WSS relies on XMLDS and XML encryption for low level details and defines a higher-level syntax to wrap security information inside SOAP messages.
WSS describes a mechanism for securely exchanging SOAP messages. It provides the following three main security features:
- Message Integrity
- User Authentication
Have a look at Listing
9. It is a SOAP message that carries security information according to the WSS syntax.
Listing 9 is the same
GetSpecialDiscountedBookingForPartners SOAP request that we have seen many
times in this article. This time the request's header is carrying digital signature
information according to the WSS syntax.
Following are the simple points about Listing 9 that will help you understand WSS syntax:
SOAP:Envelopeelement in Listing 9 contains namespace declarations for SOAP, WSS, and XMLDS.
SOAP:Headerelement contains just one child element (
wsse:Security), which is the wrapper for all the security information in Listing 9. The
wsse:Securityelement in Listing 9 has two child elements, namely a
wsse:BinarySecurityTokenelement and a
wsse:BinarySecurityTokenelement contains a security token. A security token is like a security pass or an identity card that you are required to show if you want to enter a restricted access area. There are several types of electronic security tokens.
The most popular and widely used security token is a login-password pair, like the one you use while checking your e-mail.
A login-password pair is a human readable security token. There are some security tokens that are in binary form (and therefore not necessarily human readable). Such tokens are referred to as binary security tokens. For example an X509 certificate (a widely popular format for digital certificates developed by ITU-T) is a binary security token.
ValueTypeattribute of the
wsse:BinarySecurityTokenelement in Listing 9 tells what type of binary security token is wrapped inside this
BinarySecurityTokenelement. In Listing 9, the
wsse:X509v3as its value, which identifies X509 certificates.
EncodingTypeattribute of the
wsse:BinarySecurityTokenelement tells the encoding of the binary security token. As already explained, it is not possible to wrap binary data inside XML format as such. Therefore, we have to encode binary data (usually as a sequence of base-64 encoded values) before wrapping inside XML. The X509 certificate is wrapped inside the
wsse:BinarySecurityTokenelement as the element content.
ds:Signatureelement is the same as the one we discussed in the section on XML signatures. Note two important things:
Look at the
URIattribute of the
Referenceelement. Its value (
#myDiscountRequestBody) is a fragment identifier that points toward the SOAP:Body element. This means that the SOAP:Body element is the one that we have signed and wrapped the signature in XMLDS tags.
Secondly, also look at what the
ds:KeyInfoelement contains. It is a
wsse:SecurityTokenReferenceelement contains references to security tokens. In our case, it has a child element named
wsse:Reference, whose URI attribute points toward the
wsse:BinarySecurityTokenelement discussed in point 3 above. This means that the public key inside the X509 certificate (which the
wsse:BinarySecurityTokenelement wraps) will be used to verify the signature.
This is a very simple example to introduce signed WSS messages. The third and fourth parts of this series will explore further details of XML-based security, the different types of security tokens that we can use with WSS, and the use of XML encryption in WSS messages.
In the next article, we will discuss Security Assertions Markup Language (SAML), which provides a way for web service applications to share user authentication information. This sharing of authentication data is commonly referred to as single sign-on. SAML can be used as a security token in WSS applications. The next article will elaborate why, when, and how.