XML.com: How to Create a REST Protocol

How to Create a REST Protocol

December 01, 2004

Note: In this inaugural article of Joe Gregorio's new column, The Restful Web, he explains how to use the REST architectural style to create an application protocol with web-like properties. Future columns will offer additional REST-based applications and analyses. — Editor

If you follow web services, then you may have heard of REST. REST is an architectural style that can be used to guide the construction of web services. Recently, there have been attempts to create such services that have met with mixed success. This article outlines a series of steps you can follow in creating your protocol--guidance that will help you get all the benefits that REST has to offer, while avoiding common pitfalls.

What Is REST?

What is REST anyway? It is an architectural style. An architectural style is a named, coordinated set of architectural constraints.

A software architecture is defined by a configuration of architectural elements--components, connectors, and data--constrained in their relationships in order to achieve a desired set of architectural properties. [Roy Fielding]

Why Follow REST?

The whys of REST are covered on the REST Wiki. There are advantages to be had, from scalability, to simplifying implementation, to increasing the ability for your service to be reused. Jon Udell has two great articles, The Beauty of REST and Tangled in Threads: The power of the URL-line, that cover the benefits of a 'composable' system.

Look at it this way: if you're using the infrastructure of the web to move your data around, shouldn't you follow the best practices that will help your data move more smoothly through the system? It's better to go with the flow than to try to swim up stream.

How to Create a RESTful Interface

Now we get to the meat of the matter. Instead of covering this from an architectural view, I'll approach it from a recipe perspective, a series of steps you take and questions you can answer, that will hopefully guide you to creating a great REST interface.

To create a REST service, you need to answer the following questions, and you should answer them in this order:

What are the URIs?
What's the format?
What methods are supported at each URI?
What status codes could be returned?

In laying out these questions, I didn't use the proper naming conventions; instead, I used the common names for things that most developers are familiar with. As we answer each question, I'll also introduce the proper nomenclature.

Question 1: What Are the URIs?

Question 2: What's the Format?

The proper terminology here is "representation." You can't actually reach out and touch the resources behind the URIs you identified in Step 1. What you can do is exchange representations of those resources. When I do a GET on an "employee" resource, I need to get something back which conveys information about that employee. These days HTML, XML, images, sound, and movies are all potential formats that could be used. For now, we'll restrict ourselves to using XML: it's a pretty popular format, lots of tools are available to process it, and there is an expectation that when you say "web services," you are referring primarily to the exchange of XML documents. And, yeah, this is being published on XML.com!

For each of the resources you listed in Step 1, you need to decide what the representations are going to look like. If possible, reuse existing formats if they are applicable. This can increase the chances that your system can be composed with other systems.

In our example employee contact list above, we could have the following representations:

Employee Format

For the sake of exposition, I will make up a new XML format for this information. Exposition--"A statement or rhetorical discourse intended to give information about or an explanation of difficult material"--which just means that I get to cheat as long as you learn something in the process.

<employee xmlns='http://example.org/my-example-ns/'>
    <name>Full name goes here.</name>
    <title>Persons title goes here.</title>
    <phone-number>Phone number goes here.</phone-number>
</employee>

Employee List Format

Since each employee will have his or her own URI with all the details there, our list will only include that URI.

<employee-list xmlns='http://example.org/my-example-ns/'>
    <employee-ref href="URI of the first employee"/>
	 Full name of the first employee goes here.</employee>
    <employee-ref href="URI of employee #2"/>Full name</employee>
    .
    .
    <employee-ref href="URI of employee #N"/>Full name</employee>
</employee-list>

Note that we haven't mapped the representations to resources just yet. To do that, we need to consider the methods.

Question 3: What Methods Are Supported at Each URI?

In the proper nomenclature, how do the URIs we defined in Step 1 get dereferenced?

Agents may use a URI to access the referenced resource; this is called dereferencing the URI. Access may take many forms, including retrieving a representation of the resource (for instance, by using HTTP GET or HEAD), adding or modifying a representation of the resource (for instance, by using HTTP POST or PUT, which in some cases may change the actual state of the resource if the submitted representations are interpreted as instructions to that end), and deleting some or all representations of the resource (for instance, by using HTTP DELETE, which in some cases may result in the deletion of the resource itself). [Architecture of the World Wide Web, First Edition]

We'll restrict the discussion to accessing resources using one the four basic HTTP methods that can be applied to a URI: GET, POST, PUT, and DELETE. HEAD is really a GET without a response body, and there are also others defined by RFC 2616 like OPTIONS. In addition, specs like WebDAV introduce even more methods. That's nice, but you should be able to go very far with just the four basic methods, which align nicely with CRUD, an acronym from the database world which stands for Create, Retrieve, Update, and Delete.

HTTP Method	CRUD Action	Description
POST	CREATE	Create a new resource
GET	RETRIEVE	Retrieve a representation of a resource
PUT	UPDATE	Update a resource
DELETE	DELETE	Delete a resource

I hesitated to include this table. By presenting it, I wanted to point out the overlap in the four basic methods of HTTP. What I don't want to happen is that you start thinking of web resources as SQL tables. Don't do that.

Make sure your GETs are side-effect free. This is a biggie, the one where many services get it wrong. GETs must be both safe and idempotent. In turn, anything which does not have side effects should use GET.

So if you want to create a new resource, use POST. Want to retrieve a representation of a resource? Use GET. To update a current resource, use PUT. Finally, to delete a resource, use DELETE.

Once you have decided on the URIs (Step 1), picked representations (Step 2), and decided on methods (Step 3), you need to match them all up. At the very least, you'll need to pick a representation for the response of every GET and a representation to place in the request for PUT and POST. Optionally, you may want to consider the representation, if any is returned from a POST.

Let's go back to our employee contact list, and now we can match up the resources, the representations, and the methods.

Resource	Method	Representation
Employee	GET	Employee Format
Employee	PUT	Employee Format
Employee	DELETE	N/A
All Employees	GET	Employee List Format
All Employees	POST	Employee Format

Question 4: What Status Codes Could Be Returned?

Not only do you need to know what type of representation is returned, you also need to enumerate the typical HTTP status codes that could be returned. In a perfect world, this step wouldn't be necessary since a good implementation would handle every status code correctly. In practice, you should list all the status codes you expect to return. This will provide good guidance for implementers on the conditions they should be testing for.

We'll update our table for our employee contact list to include the expected status codes

Resource	Method	Representation	Status Codes
Employee	GET	Employee Format	200, 301, 410
Employee	PUT	Employee Format	200, 301, 400, 410
Employee	DELETE	N/A	200, 204
All Employees	GET	Employee List Format	200, 301
All Employees	POST	Employee Format	201, 400

Warning Signs

Even following these steps, you could still make some mistakes. Paul Prescod has created a list of Common REST Mistakes that is worth reviewing as you work on your protocol.

Review

Let's review. To build a good REST service you need to answer the following questions:

What are the URIs?
What's the format?
What methods are supported at each URI?
What status codes could be returned?

And that's all there is to it. You don't believe me? Good, because it's not true. There's lots more to discuss, like compression, etags, caching, extensibility, idioms, and implementations. See you next month.

Share your experience in our forums.
(* You must be a member of XML.com to use this feature.)
Comment on this Article

Titles Only

Newest First

great
2007-07-28 14:29:53 profesjonalna [Reply]

This is really great article I think I will write a post with link to this brilliant idea!

Tom Black
http://www.article.webdirectorybid.org

DELETE and PUT
2005-05-11 09:27:42 dbacher987 [Reply]

The issue with using delete and put is that some user agents (notably Safari and some versions of Opera) don't support them from scripting, which means your service becomes inaccessible to users of those agents.

You need to provide an alternate mechanism (using POST) of performing delete and put operations. Note: by alternate, I mean implement both, and have the post link invoke delete itself.

You can yell about standards until you're blue in the face, etc. but if end users can't run your program, the standard doesn't matter.

Thanks Joe
2004-12-03 06:52:46 dpawson1 [Reply]

Nice clear explanation.
Thanks Joe.

Mark may be right, but if we can get the basics through this series, the finer points can be absorbed later.

Look forward to the next article.

DaveP

PUT versus POST
2004-12-03 04:08:37 jjn1056@yahoo.com [Reply]

I know from reading some of the REST forums that there is an ungoing discussion about PUT versus POST. First of all, a lot web browsers don't support PUT and many web servers block PUT as well, so a lot of people allow POST to do double duty. If you read the HTTP recommendation, it says that PUT is to be used to insert a given file at a given location, so in theory that could be used both for inserting new files and updating old ones. On the other hand POST is used to send a file and query parameters to a given URI, so that is more about modifying or processing that it is about replacing a given file.

Anyone out there in the REST community can help me wrap my head around this? It's a little hard for me to just accept PUT=INSERT, POST=UPDATE.
- PUT versus POST
  2004-12-03 05:01:54 jgregorio [Reply]
  
  The Atom Wiki contains a rather comprehensive list of all clients and servers and their support for PUT and DELETE.
  
  http://www.intertwingly.net/wiki/pie/PutDeleteSupport

PUT is not an update
2004-12-02 21:01:02 John Schlesinger [Reply]

Although it is tempting to think of PUT as an update, this is very misleading. It is actually more like an insert. POST is more like an update. So I think these are the wrong way around in the article.

As Roy Fielding says in http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html

" Methods can also have the property of "idempotence" in that (aside from error or expiration issues) the side-effects of N > 0 identical requests is the same as for a single request. The methods GET, HEAD, PUT and DELETE share this property. Also, the methods OPTIONS and TRACE SHOULD NOT have side effects, and so are inherently idempotent."

Update is never idempotent, whereas insert it.

John Schlesinger
- PUT is not an update
  2005-05-04 07:06:11 Lars Huttar [Reply]
  
  ""[idempotent means] the side-effects of N > 0 identical requests is the same as for a single request"
  ...
  Update is never idempotent, whereas insert i[s]."
  
  Somebody help me with this... It would seem to me the other way around: Update should be idempotent, not Insert. If you update, say, an employee #127's name to "Smith" multiple times, it's the same as doing it once. But if you Insert a new employee named Smith multiple times, you get multiple new employees, whereas if you do it once, you get one new employee. Thus, intuitively at least, Update is idempotent and Insert is not.
  What am I missing?
  
  Anyway, I appreciate the knowledgeable debate on PUT vs. POST. It is a confusing issue.
  
  Thanks,
  Lars
- PUT is not an update
  2004-12-03 04:58:54 jgregorio [Reply]
  
  To quote from RFC 2616:
  
  On POST[1]: " Extending a database through an append operation."
  
  On PUT[2]: " The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server."
  
  [1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5
  [2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6
  - PUT is not an update
    2004-12-06 08:18:12 jdodds [Reply]
    
    HTTP was originally created for managing collections of documents.
    
    PUT is intended to be used for adding new documents. A PUT only becomes an 'update' when the specified document already exists. PUT can effectively be both CREATE and UPDATE.
    
    POST is really much more general than either a CRUD CREATE or a PUT. (Joe, I think you did yourself a disservice with the quote that you pulled from section 9.5 of the HTTP spec.)
    
    There is, however, a more subtle point that isn't explicit in the article. From section 9.6:
    
    "The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request -- the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource."
    
    While I tend to disagree with the CRUD table in the article, I don't have the same issue with the final table which shows the protocol and reflects that a POST as CREATE is performed against the 'All Employees' collection and the PUT as UPDATE is performed against a specific 'employee'.
    
    I do have one question: What does it mean to POST to an employee?
    - PUT is not an update
      2004-12-06 08:40:12 jgregorio [Reply]
      
      "I do have one question: What does it mean to POST to an employee?"
      
      The best you can say is that POST isn't defined for an employee by this article. Then again neither is OPTIONS, or LOCK, or any of the other methods you could apply. It does mean that a later spec or article could come along and define what it means to POST to an employee.

Comments
2004-12-02 07:40:22 Mark Baker [Reply]

Nice Joe, but I disagree that #s 3 and 4 are part of the protocol; they just be work that needs to be done for the implementation of the service. Doing either of those in the protocol risks creating profiles of HTTP where clients only, for example, understand certain response code rather than those of HTTP itself. Consider a service which only ever returns 201; a client may be tempted never to check for 200, and would therefore barf if the service happened to use it in the future. That's why it's important to understand the fallback rules, i.e. that 2xx means "Success" even if you don't recognize the "xx" part.
- Comments
  2004-12-03 04:52:51 jgregorio [Reply]
  
  Mark,
  That is an excellent point. I added step 4 not to say that you are *only* going to support those status codes, but to highlight the fact that when putting together a web service you need to think about the HTTP status codes up front. In the future I'll always try to remember to mention fallback handling.