Menu

Tracking Packages with RSS

March 16, 2005

Yakov Shafranovich

With the increasing popularity of RSS and Atom, syndication is beginning to be used for many more innovative purposes than simply distributing website updates. In this article I want to show how to simplify such mundane tasks as tracking packages by converting tracking data into an RSS 2.0 feed.

Why RSS?

The first obvious question that comes to mind is why bother with RSS? After all, major U.S. package carriers, including the old-fashioned United States Postal Service, already provide free tracking systems on their websites. To answer this question, one must ask why we use RSS all together. For most folks, including myself, it is a burden to check our list of favorite websites every morning. For some, the list can easily number in the hundreds. Now imagine if you were buying things from eBay, or doing your holiday shopping at Amazon: you might be expecting multiple or urgent packages, and I can bet that most of us will be tracking them pretty often, just like those favorite websites of ours. But if RSS works for website updates, why not serve up tracking information packages in an ready-to-eat RSS feed as well?

Previous Efforts

The idea of using RSS for tracking packages is not mine. Back in July of 2004, Ben Hammersley wrote a screen scraping script for parsing HTML data produced by FedEx's online tracking system into RSS. Unfortunately, screen scraping is often unreliable, since carriers can change their websites at any time. This led to the creation of a UPS tracking service as a .net dll by Jason Young in December of 2004 that utilized UPS's XML API instead of screen scraping. Because I wanted something that would run on Linux without Project Mono, I chose to write my own alternative and with that the track2rss project was born. Current support is provided for FedEx, UPS and USPS, with DHL USA support coming soon. In this article I will be concentrating on the UPS tracking API.

Overview of the UPS XML API

UPS provides two types of tracking APIs for developers collectively called UPS Online Tools: HTML-based and XML-based. The XML API is well documented on UPS's website, (registration required) and only uses XML 1.0, as opposed to SOAP and WSDL. Most other carriers have very similar APIs (although FedEx uses XML schema). The API process consists of an input XML packet sent in the body of an HTTP POST transaction over SSL to the URL provided by UPS. The response packet will either contain an error or tracking information, with an occasional HTTP 500 error for those times when the server is really dead (see an example of an error and a normal output packet).

Astute readers will note that the input packet is actually two XML packets combined together. That is due to the fact that UPS requires authentication information to be passed in a separate XML packet prepended to the actual request packet within the same HTTP request. This authentication data consists of the username and password used to log in to UPS's E-Tools website, and a special "XML Access Key" which must be obtained from the same site.

Note that UPS recommends that the user name and password used for production be different from the regular username and password in order to enhance security. While I understand the inherent need for authentication information, nevertheless I would have preferred that authentication information not include such information or be passed around in plain text even over SSL. The amount of damage that can be done with this information warrants additional precautions be taken by carriers, and with alternatives such as checksums and digests it shouldn't have taken much work to secure these APIs. Additionally, the use of such sensitive authentication information precludes more innovative uses of this API such as accessing it via Javascript. However, since the current API requires plain text user names and passwords, developers should try to securely store the authentication information in their applications.

Project Overview

After obtaining the relevant API information from UPS and registering to get the XML Access Key, I now faced the task of choosing the right tools for translating the package tracking information into RSS. Seeing that UPS's XML API and RSS were just different dialects of XML, I decided to use XSLT, a W3C-standardized language for transforming XML, instead of writing a regular program. One of the main reasons why I chose to go with XSLT is because the XML processing APIs in most popular languages are still too cumbersome for casual use. With a conventional programming language I would need to parse the XML and then transform it into code, while with XSLT I could leave the parsing task to the XSLT processor and do the actual transformation in just a few lines. Additionally, since XSLT and XML are portable across many platforms, I could port my project to any platform. Also, putting XML transformation instructions into an external stylesheet allows for creation of other XSLT stylesheets in the future to generate different types of XML, such as Atom, and easily add support for other carriers without changing the main program.

Of course, ending up with a bunch of XSLT stylesheets doesn't do us any good. We still need a way actually execute them. Any other language with XSLT support would suffice for this purpose including Python, Java, .NET languages, etc., as well as any frameworks that support XSLT templates such as Struts or AxKit. However, I decided to write a short wrapper program in Perl because it is portable across many platforms and is available by default on many hosting providers (of course an XSLT processor such as libXSLT is still required). It would be trivial to rewrite the same wrapper in any other language, something that I might do in the future for this project.

Playing with XSLT

In the end, the track2rss project consisted of two parts: a set of XSLT stylesheets to process the XML and a 200 line wrapper script written in Perl. I started off generating the correct XML input packet. The resulting stylesheet simply takes an empty XML file and several parameters as passed into the XSLT processor, and puts them into the right places in the XML (see the stylesheet and sample input packet). Of course it is not necessary to use XSLT or the XML APIs to do this; I could have simply recorded the whole packet inside the wrapper script and used some sort of a replace function to plug in the parameters. But since I am using the XSLT for output anyway, I decided to use it for input as well as follows :

<?xml version="1.0"?>

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:output method = "xml" indent="yes"/>

   

<xsl:param name="service_key"/>

<xsl:param name="service_username"/>

<xsl:param name="service_password"/>

<xsl:param name="tracking_number"/>

<xsl:param name="version"/>

   

<xsl:template match="/">	

	<AccessRequest xml:lang="en-US">

	<AccessLicenseNumber><xsl:value-of select="$service_key"/>

	</AccessLicenseNumber>

	<UserId><xsl:value-of select="$service_username"/></UserId>

	<Password><xsl:value-of select="$service_password"/></Password>

	</AccessRequest>

   

	<xsl:text disable-output-escaping="yes">&lt;?xml version='1.0'?&gt;</xsl:text>

	<TrackRequest xml:lang='en-US'>

	<Request>

	<TransactionReference>

	<CustomerContext><xsl:value-of select="$version"/></CustomerContext>

	<XpciVersion>1.0001</XpciVersion>

	</TransactionReference>

	<RequestAction>Track</RequestAction>

	<RequestOption>activity</RequestOption>

	</Request>

	<TrackingNumber><xsl:value-of select="$tracking_number"/></TrackingNumber>

	</TrackRequest>

</xsl:template>

   

<xsl:template match="text()">

</xsl:template>

   

</xsl:stylesheet>

Unlike the input stylesheet, the output stylesheet operates directly on the XML packets received from the carrier. However, I also wanted to include several other things in the feed produced by this project:

  • An error message if the transaction failed.
  • A link to the HTML version of the carrier's tracking site.
  • A mandatory disclaimer.
  • The version and name of the program that generated the feed.
  • The ability to set a custom CSS stylesheet to format the output RSS feed in a browser.

I started the output stylesheet by passing two parameters indicating a URL to the CSS stylesheet and the version of the program:


<?xml version="1.0"?>

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">



<xsl:param name="url_stylesheet"/>

<xsl:param name="version"/>

The first step in the actual template is to check for the CSS parameter ($url_stylesheet) and set it if present. Being that the CSS stylesheet is an XML processing instruction ("<?xml-stylesheet?>"), it must be placed in the beginning of the output XML file (note that the use of the xsl:text tags to escape it):

<xsl:template match="/">

<xsl:if test="$url_stylesheet">

	<xsl:text disable-output-escaping="yes">&lt;?xml-stylesheet href="</xsl:text>

		<xsl:value-of select="$url_stylesheet"/>

	<xsl:text disable-output-escaping="yes">" type="text/css"?&gt;</xsl:text>

</xsl:if>

At this point I need to start creating the actual RSS feed. The basic RSS 2.0 feed consists of a single root rss element containing within it a single channel element. Within the channel element there are some metadata elements containing information about the feed and multiple item elements. I put some text and the tracking number into the title element, along with a link to UPS's tracking website which goes into the link element :

<rss version="2.0">

<channel>

   <title>UPS Tracking Information for 

     <xsl:value-of select="TrackResponse/Shipment/Package/TrackingNumber"/></title>

   <link>http://wwwapps.ups.com/WebTracking/processInputRequest?

        sort_by=status&amp;tracknums_displayed=1

   &amp;TypeOfInquiryNumber=T&amp;loc=en_US&amp;InquiryNumber1=

   <xsl:value-of select="TrackResponse/Shipment/Package/TrackingNumber"/>

        &amp;track.x=0&amp;track.y=0</link>

To comply with UPS's license for the API, I include a disclaimer in the description element along with the tracking number (note the use of HTML br tags for formatting the output in RSS readers):

<description>This RSS feed tracks UPS package #

   <xsl:value-of select="TrackResponse/Shipment/Package/TrackingNumber"/>

   &lt;br/&gt;

   DISCLAIMER GOES HERE

</description>

Here I indicate the language of the feed and the version of the program, using the $version parameter :

<language>en-us</language>

<generator><xsl:value-of select="$version"/></generator>

The next step is to check the /TrackResponse/Response/ResponseStatusCode element for errors. If there are no errors, then we can call the rest of the stylesheet. Otherwise, an item element will be created with the error message instead. As the last step, we close off all the tags :

<xsl:choose>

   <xsl:when test="/TrackResponse/Response/ResponseStatusCode = '1'">

   	<xsl:apply-templates select="TrackResponse/Shipment/Package/Activity"/>

   </xsl:when>

   <xsl:otherwise>

   <item>

   	<title>TRACKING REQUEST FAILED</title>

   	<description>Failed to retrieve data from UPS, error message:

		<xsl:value-of select="TrackResponse/Response/Error/ErrorDescription"/>

	</description>

   </item>

   </xsl:otherwise>

</xsl:choose>

</channel>

</rss>

</xsl:template>

Now that the main channel element has been taken care of, the next step is to process the actual tracking information. For this we have to match all of the individual Activity elements and transform them into the corresponding RSS item elements in a separate template. Since UPS uses special codes for activity types, we need to use xsl:choose to transform them into human-readable descriptions and put them inside the title element :

<xsl:template match="Activity">

<item>

<title>

<xsl:choose>

   <xsl:when test="Status/StatusType/Code = 'I'">IN TRANSIT</xsl:when>

   <xsl:when test="Status/StatusType/Code = 'D'">DELIVERED</xsl:when>

   <xsl:when test="Status/StatusType/Code = 'X'">EXCEPTION</xsl:when>

   <xsl:when test="Status/StatusType/Code = 'P'">PICKUP</xsl:when>

   <xsl:when test="Status/StatusType/Code = 'M'">MANIFEST PICKUP</xsl:when>

   <xsl:otherwise>UNKNOWN</xsl:otherwise>

</xsl:choose>

</title>

For the description element, we include the date/time, location and status of the tracking event. In order to transform the UPS's date/time formats, some processing is required with the XSLT substring function. Additionally, since the ActivityLocation/Address/City element is optional, an xsl:if statement is needed (note the use of br tags for formatting) :

<description>

   Date/Time :

   <xsl:value-of select="substring(Date, 5, 2)"/>/

   <xsl:value-of select="substring(Date, 7, 2)"/>/

   <xsl:value-of select="substring(Date, 1, 4)"/>

   &#160;#160;   

   <xsl:value-of select="substring(Time, 1, 2)"/>:

   <xsl:value-of select="substring(Time, 3, 2)"/>:

   <xsl:value-of select="substring(Time, 5, 2)"/>

   &lt;br/&gt;

   

   Location :

   <xsl:if test="ActivityLocation/Address/City">

   	<xsl:value-of select="ActivityLocation/Address/City"/>,&#160;#160;

   </xsl:if>

   <xsl:value-of select="ActivityLocation/Address/StateProvinceCode"/>&#160;#160;

   <xsl:value-of select="ActivityLocation/Address/CountryCode"/>

   &lt;br/&gt;

   Status: <xsl:value-of select="Status/StatusType/Description"/>

</description>

We end up with a link element, which is identical to the link element we used in the channel element above, and follow by closing the tags. Even though the RSS 2.0 specification does not mandate the link element, some readers like FireFox will not parse the feed without it (note that we navigate back up to get the tracking number) :

<link>http://wwwapps.ups.com/WebTracking/processInputRequest?

     sort_by=status&amp;tracknums_displayed=1

&amp;TypeOfInquiryNumber=T&amp;loc=en_US&amp;InquiryNumber1=

<xsl:value-of select="../../../../TrackResponse/Shipment/Package/TrackingNumber"/>

     &amp;track.x=0&amp;track.y=0

</link>

</item>

</xsl:template>

The complete output stylesheet can be found here along with the resulting RSS feed.

Writing the Wrapper Script

Once we are done with the XSLT part of the project, its time to move to the wrapper script that actually runs it. It consists of four parts:

  1. The first part contains the necessary settings for each carrier such as authentication information, URLs, etc. Due to security and portability issues, it would be better to store them in a secure fashion in a separate configuration file, but to keep things simple I chose not to do that here.
  2. The second part checks that all the input parameters to the script are not empty, and parses them from the HTTP request.
  3. The third part generates the request packet by invoking the XSLT processor with the input template, and sends the request to UPS.
  4. The fourth part processes the response and transforms the resulting XML into RSS via another call to the XSLT processor.

Leaving aside the mechanics of setting variables and parsing HTTP requests, I want to concentrate on the interaction between the wrapper and the XSLT templates. In this example, I picked the XML:libXSLT Perl module which itself interfaces to the GNOME's libXSLT library (the other XSLT processor for Perl, XML::XSLT did not support enough XSLT features to process my stylesheets correctly). To use the library, I needed to add the following to the Perl script:

use XML::LibXSLT;

Getting past all the initial code, I initialize the XSLT library and create an empty XML object which will be used to generate the input XML packet:

my $parser = XML::LibXML->new();

my $xslt = XML::LibXSLT->new();

my $source = $parser->parse_string('<?xml version="1.0"?><xml/>');

After that, we parse the input XSLT stylesheet from a file and invoke the XSLT processor. The parameters which are passed to the processor are defined in the beginning of the script and contain the authentication information for UPS, and the tracking number of the package (which is extracted from the input request when the script is called) :

my $style_doc = $parser->parse_file($input_xsl);

my $stylesheet = $xslt->parse_stylesheet($style_doc);

my $results = $stylesheet->transform($source,

	XML::LibXSLT::xpath_to_string(service_key => $service_key),

   	XML::LibXSLT::xpath_to_string(service_username => $service_username),

	XML::LibXSLT::xpath_to_string(service_password => $service_password),

	XML::LibXSLT::xpath_to_string(tracking_number => $tracking_number)

	);

Once the request packet is generated via XSLT, I send it on its merry way by using the famous libwww-perl library:


my $req;

$ua = LWP::UserAgent->new;



$req = HTTP::Request->new(POST => $service_url_track);

$req->content_type('application/x-www-form-urlencoded');

$req->add_content($stylesheet->output_string($results));



my $res = $ua->request($req);

To process the response, I follow a similar routine, except that I use the response returned from UPS instead of the empty XML:

my $source = $parser->parse_string($response->content);

my $style_doc = $parser->parse_file($output_xsl);

my $stylesheet = $xslt->parse_stylesheet($style_doc);

my $results = $stylesheet->transform($source,

	XML::LibXSLT::xpath_to_string(version => $version)

	XML::LibXSLT::xpath_to_string(url_stylesheet => $url_stylesheet)

   );

Once processed, we simply print the output to the user and exit:


print "Content-Type: application/xml\n\n";

print $stylesheet->output_string($results);

exit;

The full script can be downloaded here. Figures 1 and 2 below show how the resulting feed appears in the Bloglines web-based RSS reader and Liferea desktop RSS reader for Linux/GNOME.

Bloglines Screenshot
Figure 1. BlogLines Screenshot

Liferea Screenshot
Figure 2. Liferea Screenshot

Conclusion

Using XSLT for generating RSS feeds for UPS package tracking turned out to be a much simpler task than writing a straightforward program. Having XSLT do the heavy lifting of dealing with XML freed me to concentrate on the mundane programming tasks of working with HTTP requests and responses, illustrating how XSLT can reduce the complexity of XML processing. In the future, I plan to integrate additional carriers into the track2rss project, as well as additional output formats and languages.