XML.com: XML From the Inside Out
oreilly.comSafari Bookshelf.Conferences.

advertisement

The Atom API

The Atom API

October 15, 2003

Atom is an up-and-coming format for editing, syndicating, and archiving weblogs and other episodic web sites. The final details are still being hashed out, but that's never stopped me before, having written several articles about XHTML 2. To understand the problems that Atom is designed to solve, we should look briefly at what came before it.

The LiveJournal API

In the beginning, there was LiveJournal. LiveJournal had an API, the LiveJournal Client/Server API. It worked over HTTP, and it looked like this:

POST /interface/flat HTTP/1.1
Host: www.livejournal.com
Content-type: application/x-www-form-urlencoded

mode=login&user=test&password=test
HTTP/1.1 200 OK
Content-Type: text/plain

name
Mr. Test Account
success
OK
message
Hello Test Account!

A function call is an HTTP POST to a specific URL, which is the same for all function calls. The function name is given as a parameter in a list of form-encoded key-value pairs. The result is also a list of key-value pairs which are separated by carriage returns instead of ampersands.

Things to notice right off the bat:

  • Escaping issues with input. You may have a library that handles this, otherwise you'll need to escape everything manually. The documentation for the LiveJournal API steps you through all the gotchas.
  • Unmarshalling the output. You'll need to write a parser to pick out key CR value CR key CR value CR, etc., before you even get around to interpreting the results.
  • Escaping issues with the output. Carriage returns are special -- they're delimiters -- so carriage returns within the return values need to be escaped. Also, there are at least four ways to write a carriage return. The LiveJournal documentation suggests, but doesn't mandate, "\r\n".
  • Passwords are sent in the clear, as plain text.

The Blogger API

Next in our abbreviated history is the Blogger API. The Blogger API was created by Evan Williams of Pyra Labs and was quickly adopted by virtually everyone. It defined a series of functions, such as blogger.newPost, which took as arguments application_key (application-specific, each developer signed up to receive one), blog_id (defined within the Blogger system), username, password, entry_text, and a boolean flag publish which controlled whether to publish the new post immediately or leave it in draft mode.

The Blogger API was based on XML-RPC, so a call to newPost(APP_KEY, BLOG_ID, USERNAME, PASSWORD, ENTRY_TEXT, PUBLISH) would send this over the wire:

POST /api/RPC2 HTTP/1.1
Host: plant.blogger.com
Content-Type: text/xml

<?xml version='1.0'?>
<methodCall>
  <methodName>blogger.newPost</methodName>
  <params>
    <param>
      <value>
        <string>APP_KEY</string>
      </value>
    </param>
    <param>
      <value>
        <string>BLOG_ID</string>
      </value>
    </param>
    <param>
      <value>
        <string>USERNAME</string>
      </value>
    </param>
    <param>
      <value>
        <string>PASSWORD</string>
      </value>
    </param>
    <param>
      <value>
        <string>ENTRY_TEXT</string>
      </value>
    </param>
    <param>
      <value>
        <boolean>PUBLISH</boolean>
      </value>
    </param>
  <params>
</methodCall>

Things to note here:

  • It's XML. While escaping is still an issue, at least it's a well-understood issue.
  • It's XML-RPC. There are XML-RPC libraries for very many programming languages, so you'll never have to see or think about the raw wire format. Until you need to debug it, of course. Do you handle all 4 serializations of boolean values? Did you know <string> is optional and is omitted by some XML-RPC servers? And so forth.
  • Argument order matters: parameters are not named, so order must matter. XML-RPC has no capability to serialize optional arguments. What we have is a function with six required parameters in a specific order. What we do not have is any way to extend this function, say to add another argument, except by defining a new function with a different name and seven arguments.
  • Despite being widely adopted, this API is really quite Blogger-specific. Blogger did not at the time have the capability to associate titles or any other sort of metadata with entries; hence, the Blogger API has no capability for these.
  • Passwords are sent in the clear, as plain text.

The MetaWeblog API

In direct response to the perceived limitations of the Blogger API, especially the lack of extensibility, since many people wanted titles, UserLand created the MetaWeblog API. It solved some of the problems but at the cost of added complexity. It was also based on XML-RPC, but it replaced the single entry_text string argument with a struct which could hold multiple pieces of information.

As the MetaWeblog API spec puts it,

The MetaWeblog API uses an XML-RPC struct to represent a weblog post. Rather than invent a new vocabulary for the metadata of a weblog post, we use the vocabulary for an item in RSS 2.0. So you can refer to a post's title, link and description; or its author, comments, enclosure, guid, etc. using the already-familiar names given to those elements in RSS 2.0.

In other words, if you want to post a new entry that would be represented like this in RSS,

<item>
  <title>My Weblog Entry</title>
  <description>This is my first post to my weblog.</description>
  <pubDate>Mon, 13 Oct 2003 13:29:54 GMT</pubDate>
  <author>Mark Pilgrim (f8dy@example.com)</author>
  <category>Unfiled</category>

you would create that entry with a struct, something like this:

>>> import xmlrpclib
>>> server = xmlrpclib.ServerProxy('http://www.example.com/RPC2')
>>> server.metaWeblog.newPost(BLOG_ID, USERNAME, PASSWORD,
  {'title': 'My Weblog Entry',
   'description': 'This is my first post to my weblog.',
   'dateCreated': '2003-10-13T13:29:54',
   'author': 'Mark Pilgrim (f8dy@example.com)',
   'category': 'Unfiled'},
   xmlrpclib.True)

What goes over the wire after this call is insanely complicated, far too much to include inline here. Looking through the wire format, and the higher-level source code, suggests out a number of problems with the MetaWeblog API:

  • Despite the spec's claim that the vocabulary "comes from RSS 2.0", it doesn't really. For example, creation dates in RSS are stored in <pubDate>, but in the API the creation date goes in <dateCreated>.
  • The date formats don't match. Dates in RSS 2.0 are in RFC-822 format, but XML-RPC only supports ISO-8601 formatted dates.
  • Amazingly, XML-RPC has no concept of time zones. Convention dictates that all dates are UTC, but this is implementation-dependent.
  • Some elements in RSS (source, enclosure, and category) can have attributes. These are also special-cased. For enclosure, the MetaWeblog API tells us to "pass a struct with sub-elements whose names match the names of the attributes according to the RSS 2.0 spec, url, length and type." For source, "pass a struct with sub-elements, url and name."
  • Since we're using a struct to hold all the metadata for the entry, we can't have more than one value per property. The most common case of this is creating an entry in multiple categories. The MetaWeblog API dodges this issue by defining a separate categories element within the struct which is an array of strings. Other cases -- a post with multiple authors, for example -- are simply impossible.
  • RSS categories can also have both attributes and a value -- the value being the name of the category, and a domain attribute that specifies the domain in which the category name resides. To serialize this, the MetaWeblog API tells us: "If an element has both attributes and a value, make the element a struct, include the attributes as sub-elements, and create a sub-element for the value with the name _value. Note that this means that no element can be passed through the API that has an attribute whose name is _value."
  • Multiple categories with domains are impossible. The special-case categories element does not handle serializing attributes for each category; it is always simply a list of strings.
  • Passwords are sent in the clear, as plain text.

But wait, there's more. RSS 2.0 is extensible through namespaces, so in theory the MetaWeblog API is extensible too. It says:

RSS 2.0 allows for the use of namespaces. If you wish to transmit an element that is part of a namespace, include a sub-struct in the struct passed to newPost and editPost whose name is the URL that specifies the namespace. The sub-element(s) of the struct are the value(s) from the namespace that you wish to transmit.

Thankfully, nobody actually does this. Movable Type extends the MetaWeblog API by simply defining a bunch of new elements in the struct called mt_allow_comments, mt_allow_pings, and so forth.

In other words, what we have here is an RPC-based API that starts with an XML-centric data model (RSS 2.0), shoves it into a struct, defines separate special cases for everything that isn't simply a name-value pair, ignores everything that isn't handled by the special cases, reinvents the concept of XML namespaces, and then serializes it all in a verbose XML format that looks like this. So we've reinvented XML, over RPC, over XML. Badly.

And passwords are still sent in the clear.

The Atom API

The Atom API was designed because the MetaWeblog API proved that RPC-based APIs were simply the wrong solution for this problem. The Blogger API was about as complicated as you could reasonably get before things went completely off the rails. "Shove everything into a struct" was an idea that sounded like it might solve some problems; but, as you can see, it caused more problems than it solved.

In direct response to the mess that is the MetaWeblog API, the Atom API was designed with several guiding principles in mind:

  • Well-defined data model -- with schemas and everything!
  • Doc-literal style web services, not RPC
  • Take full advantage of XML and namespaces
  • Take full advantage of HTTP
  • Secure, so no passwords in the clear

Pages: 1, 2

Next Pagearrow