Sign In/My Account | View Cart  
advertisement


Listen Print Discuss
Atom Authentication

Atom Authentication

by Mark Pilgrim
December 17, 2003

I wish I didn't need to write this article. My life would be much simpler if Atom could just use existing HTTP authentication, as-is. But it can't; I'm going to tell you why and then I'm going to tell you what we're doing instead.

Let's back up. Atom, in case you missed it, is a new standard that uses XML over HTTP to publish and syndicate web-based content. It is initially targeted at weblogs, and most of the early adopters so far have been weblog vendors and users. It consists of the Atom API, which I discussed last month, and the Atom syndication format, which I will discuss next month. This month I want to talk about authentication.

As with all design decisions, it helps to list the problems you are trying to solve, and the audience you are trying to solve them for, in the form of a character sketch. Let's talk about Bob. Bob has a weblog. Bob hosts his weblog on a low-end web hosting service, one which hosts several hundred weblogs on a single machine (a single IP address). Bob can FTP files to his web directory, and he can run CGI scripts in Perl and Python. But Bob has no remote shell access on his server (that costs extra), no .htaccess rights to change his Apache configuration (his hosting provider set AllowOverride None), no PHP scripts, and no database. He can serve static HTML files and run CGI scripts, and that's it.

This is not a contrived scenario. Many web hosting providers offer exactly this environment, and two popular weblog publishing systems, Movable Type and Blosxom, run as CGI scripts in this environment. There are thousands of real people like Bob.

Bob also travels a lot, going to various O'Reilly conferences. He would like to be able to post to his Atom-enabled weblog from an Atom-enabled client, without anyone else at the conference (who might be listening in on the wireless network) being able to steal Bob's password or take over his weblog.

As we saw last month, all previous weblog publishing APIs send passwords over the wire in clear text. Clearly none of these APIs will work for Bob. A number of solutions were proposed during the development of Atom, but none of them help Bob.

  1. Use HTTP basic authentication. This does not technically send passwords over the wire in clear text, but it encodes them in a way that is easily reversible. So this doesn't actually help Bob since it's not an improvement over clear text.
  2. MD5-hash the password and only send the hash. This would solve the password sniffing problem, since you couldn't reverse engineer the hash to recover the original password. But it doesn't help Bob because it's susceptible to replay attacks. Someone at Bob's conference could sniff Bob's transaction and recover the password hash, then reuse it to post to Bob's weblog themselves. Even though the attacker never learns Bob's actual password, they can still learn enough to successfully pretend to be Bob and take over his weblog.
  3. Use HTTP basic authentication over SSL. This would solve the password sniffing problem, but it doesn't help Bob because he can't use SSL. Due to the way SSL handshaking works, each SSL certificate requires its own IP address, which Bob doesn't have (remember, his weblog is one among hundreds which all share a single IP address). Also, configuring the web server to listen on port 443 requires root access, which Bob doesn't have.
  4. Use HTTP digest authentication. This would also solve the password sniffing problem, and it would solve the replay problem, but most web hosting providers don't turn on digest authentication (it requires an Apache module that is not on by default). Even if Bob's ISP had mod_digest_auth enabled, it wouldn't help Bob, because he has no .htaccess rights to configure his passwords; and, because of the way Apache works, CGIs can't implement digest authentication on their own. (Scripts handled by an Apache module, such as mod_php or mod_perl, can implement HTTP digest authentication. But external CGI processes can't because Apache does not pass the necessary headers along to the CGI script. But that still doesn't help Bob because his hosting provider doesn't offer PHP; and, even if they did, his weblog software doesn't run on PHP anyway.)

It looks like Bob is screwed.

WSSE Username Token

A little-known fact about RFC 2617 is that HTTP authentication is extensible. The RFC defines and Apache has modules for Basic and Digest authentication, but developers are free to define different algorithms for use within the HTTP authentication framework, and servers are free to insist that clients support those algorithms if they want access to the server's resources.

After much haggling, the algorithm we chose was WSSE Username Token (PDF). WSSE is a family of open security specifications for web services, specifically SOAP web services. However, the Username Token algorithm is not SOAP-specific; it can be easily adapted to work within the HTTP authentication framework, and it solves all of Bob's problems.

The algorithm itself works like this:

  1. Start with 2 pieces of information: username and password.
  2. Create a nonce, which is a cryptographically random string. This is harder than it sounds; if an attacker can guess your next nonce, they can still attempt a replay attack. Most cryptography libraries have routines to generate decent nonces, the specifics of doing which are beyond the scope of this article.
  3. Create a "creation timestamp" of the current time, in W3DTF format.

  4. Create a password digest:

    PasswordDigest = Base64 \ (SHA1 (Nonce + CreationTimestamp + Password))

An example will help make this clearer.

  1. Let's say Bob's username is "bob", and his password is "taadtaadpstcsm".
  2. Bob creates a nonce, "d36e316282959a9ed4c89851497a717f".
  3. Bob created this nonce at "2003-12-15T14:43:07Z", so that's the creation timestamp.
  4. Bob's password digest is Base64(SHA1 ("d36e316282959a9ed4c89851497a717f" + "2003-12-15T14:43:07Z" + "taadtaadpstcsm")), which is "quR/EWLAV4xLf9Zqyw4pDmfV90Y=". Most languages have built-in libraries to create SHA-1 hashes and to encode strings in Base64 format.

Now let's see how this algorithm fits into the HTTP authentication framework.

Extending HTTP authentication

Bob's weblog is at http://bob.example.com/, and his Atom endpoint is at http://bob.example.com/atom.cgi. Bob's Atom-enabled client tries to post to his weblog, by sending an HTTP POST request with his Atom entry:

POST /atom.cgi HTTP/1.1
Host: bob.example.com
Content-Type: application/atom+xml

<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://purl.org/atom/ns#">
  <title>My Entry Title</title> 
  <created>2003-12-15T14:43:07Z</created> 
  <content type="application/xhtml+xml" xml:lang="en"> 
    <div xmlns="http://www.w3.org/1999/xhtml">
      <p>Hello, <em>weblog</em> world!</p>
      <p>This is my third post <strong>ever</strong>!</p>
    </div>
  </content>  
</entry>

But this request didn't include any authentication information. The server responds with an HTTP 401 Unauthorized:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: WSSE realm="foo", profile="UsernameToken"

The profile is constant and should always be "UsernameToken". The realm is determined by the server and can be anything.

Bob repeats his request, this time with his authentication credentials: username, password digest, nonce, and creation date.

POST /atom.cgi HTTP/1.1
Host: bob.example.com
Content-Type: application/atom+xml
Authorization: WSSE profile="UsernameToken"
X-WSSE: UsernameToken Username="bob", PasswordDigest="quR/EWLAV4xLf9Zqyw4pDmfV9OY=", Nonce="d36e316282959a9ed4c89851497a717f", Created="2003-12-15T14:43:07Z"

<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://purl.org/atom/ns#">
  <title>My Entry Title</title>
  <created>2003-12-15T14:43:07Z</created>
  <content type="application/xhtml+xml" xml:lang="en">
    <div xmlns="http://www.w3.org/1999/xhtml">
      <p>Hello, <em>weblog</em> world!</p>
      <p>This is my third post <strong>ever</strong>!</p>
    </div>
  </content>
</entry>

Bob's Atom-enabled weblogging software looks in the X-WSSE: header for the actual authentication credentials, and recreates the steps Bob took in order to verify that Bob knows his password. If Bob got his password wrong, the server simply responds with an HTTP 401 Unauthorized with the WWW-Authenticate: header, same as before; or, optionally, with some explanatory text in the body of the message to tell the client what's going on. If Bob got his password right, the server accepts the request, posts the new entry, responds with an HTTP 201 Created, and gives the location of the newly created entry.

I want to briefly mention three design decisions and the problems they solve. Remember when I said that CGI scripts couldn't implement HTTP authentication because Apache didn't pass along the appropriate headers? The header I was talking about is the Authorization: header. In this case, Apache will notice the Authorization: header and notice that the authentication algorithm is "WSSE". Apache doesn't have a module to handle this, so it will strip the Authorization: header and pass the rest of the headers (including X-WSSE:) on to the CGI script.

Second, if Bob has previously successfully authenticated with this same nonce, the server may recognize that and reject the request (with a 401). The server may keep track of nonces for a limited amount of time (usually a matter of minutes) and reject duplicates. If a client tries to reuse a nonce after that time, the server can simply reject it based on the creation timestamp (since the password hash is made from both the nonce and the timestamp). In turn, Bob's Atom-enabled client software generates a new nonce and creation timestamp with each request. This will protect against replay attacks.

Third, this leaves the door open for future versions of Atom supporting other profiles of WSSE, such as Kerberos. But let's jump off that bridge when we come to it.

Optimizing HTTP Authentication

OK, so that's how it looks when it works, and that's how it looks when it doesn't work. But as you can see, this is an inefficient process. Bob sent his entire entry to the server, then the server rejected it, then Bob sent his entire entry to the server again -- this time with his WSSE-formatted credentials. But there's nothing about HTTP authentication or the WSSE Username Token algorithm that requires this extra round trip. If Bob's Atom-enabled client software knows ahead of time that Bob's Atom-enabled server is going to ask for WSSE Username Token authentication, it can simply calculate the credentials and send them with the initial request. In the best case, the server sees the credentials, verifies them, processes the request, and returns the appropriate success code -- all without ever generating a 401. In the worst case, the server doesn't actually support WSSE Username Token, so it simply responds with a 401 and a WWW-Authenticate: header that details what algorithms it does support.

Summary

More Dive Into XML Columns

Identifying Atom

XML on the Web Has Failed

The Atom Link Model

Normalizing Syndicated Feed Content

The Atom API

Let's make sure we've found a solution that works for Bob. Extending HTTP authentication with WSSE Username Token:

  • Doesn't require root access to install Apache modules.
  • Doesn't require .htaccess rights to set up passwords. (The CGI application can manage passwords itself.)
  • Doesn't require Bob to have his own web server or even his own IP address. It works with virtual name-based hosting.
  • Works with pure-CGI applications. CGI scripts can read the X-WSSE: header to get the credentials, and they can generate a 401 error and a WWW-Authenticate: header if authentication fails.
  • Doesn't send passwords in the clear. Bob can blog with confidence at O'Reilly conferences.
  • Protects against replay attacks.

So that's what Atom authentication looks like and that's why. Because it's the simplest thing that works for Bob.

Disclaimer: the Atom API has not been finalized. While this authentication scheme has been deployed by several vendors, it may still change slightly before the Atom API goes 1.0. Feel free to implement it now, but be prepared to reimplement it later.

Comment on this articleShare your views in our forum.
(* You must be a member of XML.com to use this feature.)
Comment on this Article

Titles Only Titles Only Newest First
  • Works great for RESTful services
    2009-08-28 07:54:54 cherevik [Reply]

    The same approach works great for web services with a REST API that must be accessed from desktop or iphone apps. See an example here http://cherevik.com/?p=57. One would think that OAuth would do the trick, but it does not work in a non-browser environment. A WSSE-based token scheme is better approach.

  • Locksmith Los Angeles 1-818-386-1022
    2008-09-24 18:39:04 0 [Reply]

    AAA Locksmith in Los Angeles 1- 818-386-1022
    Los Angeles Locksmith provides fast, reliable, professional 24 Hour Emergency Locksmith services at extremely competitive rates.
    We Install New Locks Unlock Autos, Homes doors locks, and Businesseslocks installation
    Get reimbursed for lockouts if you have roadside assistance
    We Rekey locks for Homes and Businesses doors locks


  • Out of date?
    2007-04-02 05:22:18 Singer-Scientist [Reply]

    Is this discussion relevant now that the latest versions of the Atom Publishing Protocol (e.g. http://bitworking.org/projects/atom/draft-ietf-atompub-protocol-14.html) reference TLS as the HTTP security layer?

  • No secret sauce
    2007-03-06 09:30:54 cryptoanarchist [Reply]

    Your discussion of the need for Atom authentication is good and valid. However, I disagree that the scheme you propose at the end of the article is necessarily secure or even adds a significant amount of security over cleartext passwords.


    The problem is the dictionary attack of possible passwords is very easy. A number of studies on the weakness of user chosen passwords have concluded that users are just very, very bad at choosing strong passwords.


    As a result, the number of possible input passwords in a dictionary attack is very low, 20-25 bits of max entropy. Since nonce and timestamp are known values, dictionary attacking the unknown value requires trivial CPU time.


    At best, you are achieving additional layers of obfuscation and requiring the attacker to have access to an SHA-1 implementation.


    However, don't let that distract from the very good discussion on the need for better Atom Authentication. Better solutions in that area that folks like Bob can deploy are problems in need of solutions.

  • server keeping passwords in clear
    2006-04-04 16:03:03 joaquinmiller [Reply]

    I'm with Jon Prettyman. It is not best practice for a server to keep passwords in clear.


    But i see no reason why Bob could not know the algorithim (and any necessary data) that the server uses to encrypt the passwords, and encrypt his before sending it...


    O!


    Hmmm...


    Any comments?


    =joaquin aka http://mylid.net/joaquin

  • about x-wsse?
    2005-08-02 20:52:48 ÓÞ¹«×ÓËï [Reply]

    I hava a problem and want to ask you .Can you help me ?
    The problem is :
    where does the user's information save while using x-wsse for authentication ?
    for example ,user's name,password. is it Cookie,session or any other ?


  • Unclear on #3
    2004-01-09 11:29:08 Rich Salz [Reply]

    For #3, are you limiting yourself to the case where Bob has his own domain name that points to the service, or something else? One workaround is to do the redirects at the HTTP level, not the DNS (cname) level. I'm not arguing the overall decision, just trying to make sure I understand the requirements.


  • Setting up the server auth.
    2004-01-06 04:18:44 simon kittle [Reply]

    I know I must be being incredibly stupid, but say I'm Bob. How do I setup the server to require WSSE authentication in the first place ?


    You state: "But this request didn't include any authentication information. The server responds with an HTTP 401 Unauthorized:"


    How does Bob setup the server to request any authentication at all, rather than just serving files? Surely that requires an .htaccess as well?



    And on this point, would it not be possible to use a similar scheme to add authentication at the CGI level, without any server involvement?


    The credentials are passed to a CGI which is simply served in the standard way. If the password was wrong the CGI would print a message and exit, refusing to do anything. Obviously it doesn't protect all the other files in the directory, and it's open to attack via bugs in the CGI, but it may be useful in some circumstances.


    • Setting up the server auth.
      2004-01-06 21:13:21 Mark Pilgrim [Reply]

      There is a good example of this in XML::Atom, Ben Trott's Perl implementation of the Atom API. It includes a skeleton server that handles the API (including authentication), with hooks to build the rest of your application on top. It can run as a CGI in a non-.htaccess environment. Check it out.


      http://search.cpan.org/~btrott/XML-Atom-0.05/

      • Setting up the server auth.
        2004-01-08 12:46:57 simon kittle [Reply]

        Thanks for that, it's interesting.


        But it does mean the authentication happens at the CGI level, right?


        I think I was confused because you said "the server will return 401" and I guess I assumed the web server but presumably you were meaning the CGI based Atom server right?


        I just wanted to get this clear because obviously that would mean no other files in that specific directory are protected.





        • Setting up the server auth.
          2007-04-10 11:21:36 Robert_Hayden [Reply]

          I don't think this tries to protect any files on the web server; I think this authentication only keeps the CGI script from doing things that someone other than Bob ask it to do.


          I believe the CGI script actually sends the 401 header.
          When the Atom client responds, the meat of its response is in the X-WSSE header which Apache passes to the CGI script.

  • This sucks.
    2004-01-03 22:05:41 Mark Nottingham [Reply]

    I find it unbelievably unfortunate that the tools available are so miserable that we have to resort to this kind of hackery.


    Not surprising, mind you, just unfortunate.


    I understand and appreciate the security issues driving the Apache folks to do this, but it only reinforces my conviction that Web server configuration and application deployment are holding back the Web's potential, big time.


    Let's fix it.

  • Reversible passwords
    2003-12-29 17:39:13 Jon Prettyman [Reply]

    I think this method requires the server to be able to produce a clear text version of the password. That's not how basic auth implementations typically work. You might want to factor that into your design.

  • Sniffing vs Spoofing
    2003-12-22 16:22:27 Alex Jacobson [Reply]

    If you are concerned about local cinference participants sniffing Bob's password, you might also be concerned that they intercept his packets and just modify the contents of his posts.


    The attack looks something like this: Advertise a wireless network with the same name as the conference's wireless net. Get Bob to obtain DHCP from your fake net, but route it to the Internet. Now you can modify the content of Bob's posts.


    The value of content-md5 or content-sha should be part of the computed passworddigest.

  • Patent status?
    2003-12-19 12:00:28 Micah Dubinko [Reply]

    What is the patent status of SHA1?

    • Patent status?
      2003-12-21 22:37:47 Mark Pilgrim [Reply]

      SHA-1 is not patent-encumbered, according to this letter from NIST dated June 24, 1998: http://grouper.ieee.org/groups/1363/P1363/letters/NIST.txt


      .NET, Java, Perl, Python, and PHP all come with SHA-1 implementations, and there are multiple public domain SHA-1 implementations as well.

  • Clarification
    2003-12-19 03:47:59 Danny Ayers [Reply]

    No final agreement has yet been reached on what Atom authentication will look like. Authentication was elided from the latest version of the AtomAPI spec (09) based on the belief that such mechanisms belong in their own RFC, and there is a lot to suggest Atom authentication shouldn't be tied to any single mechanism. Having said that, in cases like Bob's where techniques such as HTTP digest authentication aren't available, the method described here would certainly be a good choice.

  • Base64 encoding of SHA1 digest
    2003-12-18 07:01:13 Piotr Lakomy [Reply]

    The base64 encoding of the SHA1(Password + CreationTimestamp + Nonce) should be "quR/EWLAV4xLf9Zqyw4pDmfV9OY".


    Looks like example in the article applies base64 to to the hex encoding of SHA1 value:
    base64("aae47f1162c0578c4b7fd66acb0e290e67d5f4e6") = "YWFlNDdmMTE2MmMwNTc4YzRiN2ZkNjZhY2IwZTI5MGU2N2Q1ZjRlNg=="
    (Note two filler "=" charactes at the end).


    Here is perl code to verify:


    use Digest::SHA1 qw(sha1 sha1_hex sha1_base64);;
    use MIME::Base64 qw(encode_base64);


    $password = "taadtaadpstcsm";
    $nonce = "d36e316282959a9ed4c89851497a717f";
    $timestamp = "2003-12-15T14:43:07Z";
    $text = $nonce . $timestamp . $password;


    print "text = $text\n";
    print "SHA1 hex = " . sha1_hex($text) . "\n";
    print "SHA1 b64 = " . sha1_base64($text) . "\n";
    print "SHA1 b64(hex) = " . encode_base64(sha1_hex($text));


    Very good article, I like it.





    • Base64 encoding of SHA1 digest
      2006-01-11 07:57:52 JY123 [Reply]

      Hi,
      I am working on a project that needs "Base64 encoding of SHA1 digest". Your article is very helpful. I have downloaded sha1 code from http://www.veign.com/vrc_codeview.asp?type=app&id=76, and wrote a Base64 code. Both are tested successfully using known examples.


      But I failed to reproduce your results when I calculate Base64(sha1(Nonce+TimeStamp+Password).
      The input string I used is:


      d36e316282959a9ed4c89851497a717f2003-12-15T14:43:07Ztaadtaadpstcsm


      But the result I got is:
      sha1=BADE76A5F7B0AFDE6D96BDEE566AAAACECEA7B75
      base64=QkFERTc2QTVGN0IwQUZERTZEOTZCREVFNTY2QUFBQUNFQ0VBN0I3NQ==


      I could not reproduce your result. What did I miss? Please let me know.


      Thanks
      Jack Yan