Listen Print Discuss
Show Me the Code

Show Me the Code

by Joe Gregorio
March 02, 2005

In my inaugural article, I outlined the four basic steps you needed to follow when creating a RESTful web service. Now let's take those basic steps and follow them through a worked example. To stay on familiar ground we'll create something that you may find familiar: a web bookmark service. It's hip, all the cool kids have one, and by the end of this article, so will we — or at least we'll have the specification for one. The actual implementation will be fleshed out over future articles.

The idea of a bookmark service is that it allows you to store your web bookmarks in a centralized place. Newer services add a social aspect by allowing you to categorize your links by tags and then view all the links that people have marked with the same tag.

To design such a service we need to answer our four questions.

  1. What are the resources?
  2. What are the representations?
  3. What methods are supported at each resource?
  4. What status codes could be returned?

Resources

What are the resources for a bookmarking system? Each bookmark we add should be a resource. In addition, aggregations of bookmarks should also be resources. Here are some example resources:

  • A single bookmark
  • All the bookmarks I've ever created
  • All the bookmarks I've ever created for some keyword
  • My 20 most recent bookmarks
  • The 20 most recent bookmarks for a specific keyword, across all users
  • The 20 most recent bookmarks, across all users
  • A list of all the keywords used by a user
  • The 20 most popular keywords used by everyone

There are many more aggregate resources we may want to build, but these examples should be enough to get us going.

What we want to do here is scan the list repeatedly looking for similarities. You'll note that the first resource in the list is a single bookmark. Most of the rest are aggregations of bookmarks. Only the keyword resources aren't covered so far, so we will make them their own kind of resource. In total we have three kinds of resources:

Bookmark Resource
A single bookmark and associated information, such as when the bookmark was created.
Bookmark Collection Resource
A collection of individual bookmarks grouped together.
Keyword List Resource
A list of keywords.

Since we now know the types of resources we will be dealing with, let's define the URI space that those resources will be living in.

It sounds so simple, doesn't it? "Let's define the URI space." How about, "Let's go fall off a log." Well, it actually isn't that difficult.

We'll need one or two guiding principles to help us. One principle we'll follow is using the built-in hierarchy building that the URI "path" component allows. That is, we'll use the path portion of the URIs in our system to reflect the hierarchy of our resources. Conversely, if we have resources that are non-hierarchical we'll use either query parameters or Matrix URIs.

In our system, each user's bookmarks belong to that user, so for the path component of our URIs, the user should be higher in the hierarchy (that is, to the left of the path) than the user's bookmarks. If bookmarks are categorized by tags, then the bookmarks would appear below (that is, to the right of) the tag in the path. In the following table the URIs listed are just relative URIs. We will flesh out the rest of the URI later.

URI Type of Resource Description
[user]/bookmark/[id]/ Bookmark A single bookmark for "user".
[user]/bookmarks/ Bookmark Collection The 20 most recent bookmarks for "user".
[user]/bookmarks/all/ Bookmark Collection All the bookmarks for "user".
[user]/bookmarks/tags/[tag] Bookmark Collection The 20 most recent bookmarks for "user" that were filed in the category "tag".
[user]/bookmarks/date/[Y]/[M]/ Bookmark Collection All the bookmarks for "user" that were created in a certain year [Y] or month [M].
[user]/config/ Keyword List A list of all the "tags" a user has ever used.

The user "all" is a special user. The aggregate of all the users in the system is represented by the user "all". That is, all/bookmarks/ is a Bookmark Collection of the 20 most recent bookmarks added to the system across all users.

Representations

Now for the second question, "What are the representations?"

Here we don't have to get too creative, as an XML vocabulary for bookmarks already exists. XBEL is the XML Bookmark Exchange Language. It was designed by the Python XML Special Interest Group. Using an existing popular format will allow greater opportunities for reuse and remixing of our service once it is up and running.

Let's look at XBEL and see how it fits in with our service. First, we'll review some regular XBEL documents and then discuss what we need to add or remove to accommodate our service. Here is an example XBEL document containing a single bookmark.

<?xml version="1.0"?>
<!DOCTYPE xbel PUBLIC 
       "+//IDN python.org//DTD XML Bookmark Exchange
        Language 1.0//EN//XML" 
       "http://www.python.org/topics/xml/dtds/xbel-1.0.dtd">
<xbel version="1.0">
   <bookmark href=
"http://www.xml.com/pub/a/2005/02/09/xml-http-request.html">
      <title>Very Dynamic Web Interfaces</title>
      <desc>Using XMLHttpRequest to build dynamic web
            interfaces.
      </desc>
   </bookmark>
</xbel>

Example: XBEL Document for a single bookmark

Note that most of the elements in an XBEL document are optional. Here is the simplest XBEL document containing just a single bookmark.

<xbel version="1.0">
   <bookmark href=
"http://www.xml.com/pub/a/2005/02/09/xml-http-request.html"
/>
</xbel>

Example: Shortest XBEL Document for a single bookmark

To simplify, I'll drop the XML declaration and doctype on future examples. Here is a more complicated example with nested folders of bookmarks:

<xbel version="1.0">
  <folder>
    <title>XML.com</title>
    <folder>
      <title>Web Services</title>
      <bookmark href=
 "http://webservices.xml.com/pub/a/ws/2005/01/12/salz.html">
        <title>Freeze the Core</title>
        <desc>Rich Salz explains why the 
          web services core should be frozen.
        </desc>
      </bookmark>
      <bookmark href=
           "http://www.xml.com/pub/a/2005/01/12/saml2.html">
        <title>SAML 2</title>
      </bookmark>
    </folder>
    <bookmark href=
          "http://www.xml.com/pub/a/2005/02/09/xforms.html">
      <title>Top 10 XForms Engines</title>
    </bookmark>
  </folder>
</xbel>

Example: XBEL Document for a bookmark collection

It appears that XBEL has most of the features we need for our bookmarking service. We can use an XBEL document with a single bookmark entry as the representation of a Bookmark Resource. We can also use an XBEL document with multiple bookmarks as the representation for a Bookmark Collection.

There are only two things missing from XBEL. The first is a place for the URI of the Bookmark Resource for each bookmark in a Bookmark Collection. Each <bookmark/> element has an href attribute that's the bookmark URI. In addition to that information, we also want to store the URI of the Bookmark Resource. Luckily XBEL has a spot for additional metadata for each bookmark. You can add <metadata/> elements to the <info/> element. The XBEL specification requires that the <metadata/> element have an "owner" attribute that is unique, preferably a URI, that points to a description of the metadata added. Using the extensibility of the metadata element we can add a place for the URI of the Bookmark Resource for each bookmark.

<xbel version="1.0">
  <bookmark href=
"http://www.xml.com/pub/a/2005/02/09/xml-http-request.html">
    <title>Very Dynamic Web Interfaces</title>
    <info>
      <metadata owner=
              "http://example.com/documentation/xbel/edit" >
        http://example.com/jcgregorio/bookmark/12
      </metadata>
    </info>
    <title>Very Dynamic Web Interfaces</title>
    <desc>Using XMLHttpRequest to build dynamic web
          interfaces.
    </desc>    
  </bookmark>
</xbel>

Example: Extended XBEL Document for a single bookmark

In this example, we extended an XBEL Document for a single bookmark. An XBEL Document for a collection of bookmarks can be extended in the same way, by adding <metadata/> elements to all of the <bookmark/> elements.

The URI we chose for the "owner" attribute is just a stand-in; we wouldn't really deploy this service by pointing to the example.com domain. Before we go public with the service we'll need to secure a place for our documentation and update that URI to point there.

The second thing missing from XBEL is a place to hold category information. Again the "metadata" element is appropriate for this information. Here is an example of an XBEL document being extended to include the category tag. We allow multiple tags separated by whitespace as element content for such a "metadata" element.

<xbel version="1.0">
  <bookmark href=
"http://www.xml.com/pub/a/2005/02/09/xml-http-request.html">
    <title>Very Dynamic Web Interfaces</title>
    <info>
      <metadata owner=
              "http://example.com/documentation/xbel/tags" >
        XML JavaScript XMLHttpRequest
      </metadata>
    </info>
    <title>Very Dynamic Web Interfaces</title>
    <desc>Using XMLHttpRequest to build dynamic web
          interfaces.
    </desc>    
  </bookmark>
</xbel>

Example: Extended XBEL Document for a single bookmark

The only thing we don't have is a representation for keywords. We only need something simple:

<tags>
    <tag>dogs</tag>
    <tag>cats</tag>
    <tag>pets</tag>
</tags>

Example: Keyword List Document

Methods

On to the third question, "What methods are supported at each resource?"

Now we need to decide which methods are supported by which resources and which representations they will use. The Bookmark Resource represents a single bookmark, and we're using an XBEL document with just a single bookmark in it to represent such a resource. To retrieve the representation of a Bookmark Resource, we GET it. To update a bookmark, we will PUT the information to the Bookmark Resource. To remove a bookmark, we DELETE it.

The operations on Bookmark Collection Resources include GET to retrieve a list of all the bookmarks in that collection, in the form of an XBEL document. And we'll POST an XBEL document with just a single bookmark to the Bookmark Collection Resource in order to add another bookmark to that collection.

That gives us the following types of resources and the methods they support:

Resource Method Representation Description
Bookmark
  GET Extended XBEL Document for a single bookmark Get a bookmark.
  PUT Extended XBEL Document for a single bookmark Update a bookmark.
  DELETE n/a Delete a bookmark.
Bookmark Collection
  GET Extended XBEL Document for a bookmark collection Get a collection of bookmarks.
  POST Extended XBEL Document for a single bookmark Add a bookmark to a collection.
Keyword Lists
  GET Keyword List Document Get a list of keywords.

Sanity Check

Have we maintained idempotency for our GETs? Yes, GET on a Bookmark Resource, or a Bookmark Collection Resource, or a Keyword List Resource doesn't change either the Bookmark, the Bookmark Collection, or the Keyword List Resource.

Another check is to ensure that all of the methods that aren't GETs are non-idempotent. PUTing a new representation and DELETEing a Bookmark can definitely change the state of the server. Creating a new Bookmark with POST also changes the state, so none of them could be done by GET. We're good so far.

Status Codes

The only status code we will be explicitly looking for is 201 Created when a new bookmark is created by POSTing to the Bookmark Collection Resource. Besides that any well-written client should have good default handling for 2xx status codes for success, 3xx for redirection and 4xx codes for errors.

Example Interactions

Let's look at some examples of what we would expect to see "on the wire" for our protocol.

We will for now assume that our service is hosted at http://example.com. To get the latest bookmarks for the user "jcgregorio" we do a GET on http://example.com/jcgregorio/bookmarks/recent/. What is returned will be an Extended XBEL document with the last 20 bookmarks.

GET /jcgregorio/bookmarks/recent HTTP/1.1
Authorization: Basic ZnJpZW5kczpvbmt1
User-Agent: curl/7.10.3 (win32) libcurl/7.10.3
Host: example.com
Accept: application/xbel+xml

And we get the following response:

HTTP/1.1 200 OK
Date: Sun, 13 Feb 2005 04:12:33 GMT
Server: Apache/2.0.46 (Red Hat)
Last-Modified: Sat, 12 Feb 2005 04:02:59 GMT
ETag: "444338-263-d71fc2c0"
Content-Length: 611
Content-Type: application/xbel+xml

<xbel version="1.0">
  <bookmark href=
 "http://webservices.xml.com/pub/a/ws/2005/01/12/salz.html">
    <title>Freeze the Core</title>
     <metadata owner =
              "http://example.com/documentation/xbel/edit" >
       http://example.com/jcgregorio/bookmark/1
     </metadata>
    <desc>Rich Salz explains why the 
      web services core should be frozen.
    </desc>
  </bookmark>
  <bookmark href=
           "http://www.xml.com/pub/a/2005/01/12/saml2.html">
    <title>SAML 2</title>
    <metadata owner=
              "http://example.com/documentation/xbel/edit" >
      http://example.com/jcgregorio/bookmark/2
    </metadata>
    <desc>Federated Identity</desc>
  </bookmark>
</xbel>

Note that each bookmark has an extra "metadata" element that points to a Bookmark Resource for that bookmark.

To create a new bookmark, we POST an XBEL document with a single bookmark to the appropriate Bookmark Collection.

POST /jcgregorio/bookmarks/ HTTP/1.1
Authorization: Basic ZnJpsd5kczpvbmx5
Content-Type: application/xbel+xml
Host: example.com

<xbel version="1.0">
  <bookmark href=
          "http://www.xml.com/pub/a/2005/02/09/xforms.html">
    <title>Top 10 XForms Engines</title>
    <metadata owner=
              "http://example.com/documentation/xbel/tags" >
      XML XForms
    </metadata>
    <desc>Review of available implementations.</desc>
  </bookmark><xbel version="1.0">
</xbel>

The server could then respond with:

HTTP/1.1 201 Created
Date: Sat, 12 Feb 2005 09:12:13 GMT
Server: Apache/2.0.46 (Red Hat)
Content-Length: 611
Content-Type: application/xbel+xml
Location: http://example.com/jcgregorio/bookmark/3

Note that the server returned the status code of 201 Created and a Location: header with the URI of the newly created Bookmark Resource.

What We Haven't Specified

Let's spend a minute covering the facets of our protocol that we haven't specified.

We haven't specified an authentication mechanism for our protocol. That's because HTTP provides support for discovering the allowed authentication mechanisms via the WWW-Authenticate: header. This is a generic, and extensible, mechanism provided by HTTP to enumerate allowable authentications, and there's no advantage in trying to either route around it or hard code a specific authentication mechanism into our protocol.

We haven't specified a compression mechanism for our protocol. XML is fairly verbose, and we could probably gain some speed by specifying that the files be transferred compressed. We haven't done that because, again, HTTP supports a generic and extensible mechanism for specifying when compression is allowed and what are the allowed types of compression.

Next Time

At the beginning of the Examples section I said “assume that our service is hosted at http://example.com.” That meant that all of the URIs for our system simply started with "http://example.com”. It also assumed that we would publish the structure of our URIs into the bookmarking service. That's some pretty serious hand waving. What I've glossed over is the whole idea of service discovery, which I'll cover in a future article.

Comment on this articleShare your experience in our forums.
(* You must be a member of XML.com to use this feature.)
Comment on this Article

Titles Only Titles Only Newest First
  • Change to Search Options
    2007-09-21 16:21:25 JNeumann [Reply]

    The article includes URIs that contain the actual criteria on which a search is based. For instance: [user]/bookmarks/date/[Y]/[M]/. What happens if the search options change? What if you want to add the ability to search within a date range? Do you make a new URI? [user]/bookmarks/date_range/[Y_from]/[M_from]/[D_from]/[Y_to]/[M_to]/[D_to]/.
    The URI with just /date then doesn't make much sense. Should it actually be /for_year_and_month? Also, what happens if we change our /date_range to only use years and months? I'm not seeing much flexibility here. Nor are the calls really self-documenting.


  • Question on resource URI
    2007-01-04 12:32:15 PauloMerson [Reply]

    Great article! Quick question:
    In the example, you use
    "http://example.com/jcgregorio/bookmarks/recent/"
    but in the URI table you use
    "[user]/bookmarks"
    for the 20 most recent bookmarks.
    Should it be
    "[user]/bookmarks/recent" instead?
    Thanks!
    Paulo Merson

  • Accessing multiple ressources using GET
    2006-02-06 13:50:00 mawi100 [Reply]

    In your article you describe different methods for accessing a certain subset of bookmarks. While these subsets in your example are clearly defined ('20 most recent') I am wondering how that would work in the following situation: I want to retrieve information from about 50 different sensors (eg. air quality sensors distributed in a city) where these sensors can be randomly selected from a list or by certain criteria. Would it make sense to add each single sensor to the end of the URL (such as '/bookmarks/id1/id15/id16/...')? Or would it make more sense to POST a XML file that contains the searched sensor ids? I guess the latter approach would not comply with REST, while the first approach implies a hierachy which does not exist.
    What would be the best RESTful approach?
    Thanks a lot.

    • Accessing multiple ressources using GET
      2007-01-19 12:19:00 aronr [Reply]

      This may help: "users should not derive metadata from the URI itself". (From Hao He, "Implementing REST Web Services: Best Practices and Guidelines", http://www.xml.com/pub/a/2004/08/11/rest.html)


      That implies that the number or codes in the URIs used by your RESTful service to identify unique, individual sensor readings should not correspond to a physical numbering or coding scheme used by those readings. That's because that scheme could well change later on, or the service might need to support multiple numbering or coding schemes for such data.

      • Accessing multiple ressources using GET
        2007-01-19 12:13:48 aronr [Reply]

        The above should have read: "sensors or sensor readings" rather than just "sensors'.

        • Accessing multiple ressources using GET
          2007-01-19 16:47:43 aronr [Reply]

          One more followup (sign): According to Roy T. Fielding, it *can* be useful to have meaningful resource hierarchies, so the suggestion to not use specific codes for sensors in URIs is not meant to generally discourage that practice. According to his quote, cited on the RestWiki:


          "REST does not require that a URI be opaque. The only place where the word opaque occurs in my dissertation is where I complain about the opaqueness of cookies. In fact, RESTful applications are, at all times, encouraged to use human-meaningful, hierarchical identifiers in order to maximize the serendipitous use of the information beyond what is anticipated by the original application."


          http://rest.blueoxen.net/cgi-bin/wiki.pl?RestAndUriOpacity#nid1SK


          Specifically regarding providing sensor data, there is a RESTful service for earthquake engineering data whose API is published at:


          http://it.nees.org/library/data/neescentral-web-services-api.php


          Perhaps that might provide some guidance or at least inspiration for coming up with a URI hierarchy and conventions in your own application ...

  • Minor correction
    2005-03-04 22:20:10 Mike Dierken [Reply]

    Great article! Can't wait for the book.


    Minor point - "Another check is to ensure that all of the methods that aren't GETs are non-idempotent." This is incorrect, as PUT and DELETE are always idempotent (repeating the request results in the same end state). The non-safe methods (the ones that modify data) should be done with something other than GET.

  • Why PUT vs. POST?
    2005-03-04 07:30:41 velebak [Reply]

    When updating bookmarks, you specify using PUT for a single but POST for a collection. What benefit does PUT bring, and wouldn't a POST work just as well?


    Thanks,
    Keith Veleba

    • Why PUT vs. POST?
      2005-03-04 22:23:28 Mike Dierken [Reply]

      A PUT request is idempotent - it can be repeated and the client can be sure of the results. A POST has no such guarantee - it could append some text, and doing that twice results in something different than doing it once.


      The benefit of repeatable requests is that if the network goes down, or a reply doesn't reach the client, your software can simply send the request again - the system as a whole becomes tolerant of failures.


      In the example, the collection could be replace with a PUT, but that's a design choice of the developer.

      • Why PUT vs. POST?
        2005-03-05 07:34:05 velebak [Reply]

        That makes sense, I guess it's all in how your application code handles PUT and POST operations. I was thinking too far down in the code again. Thanks for the clarification!


        Keith