Tracking Packages with RSS
March 16, 2005
With the increasing popularity of RSS and Atom, syndication is beginning to be used for many more innovative purposes than simply distributing website updates. In this article I want to show how to simplify such mundane tasks as tracking packages by converting tracking data into an RSS 2.0 feed.
The first obvious question that comes to mind is why bother with RSS? After all, major U.S. package carriers, including the old-fashioned United States Postal Service, already provide free tracking systems on their websites. To answer this question, one must ask why we use RSS all together. For most folks, including myself, it is a burden to check our list of favorite websites every morning. For some, the list can easily number in the hundreds. Now imagine if you were buying things from eBay, or doing your holiday shopping at Amazon: you might be expecting multiple or urgent packages, and I can bet that most of us will be tracking them pretty often, just like those favorite websites of ours. But if RSS works for website updates, why not serve up tracking information packages in an ready-to-eat RSS feed as well?
The idea of using RSS for tracking packages is not mine. Back in July of 2004, Ben Hammersley wrote a screen
scraping script for parsing HTML data produced by FedEx's online tracking system into RSS.
Unfortunately, screen scraping is often unreliable, since carriers can change their
at any time. This led to the creation of a UPS tracking
service as a
.net dll by Jason
Young in December of 2004 that utilized UPS's XML API instead of screen
scraping. Because I wanted something that would run on Linux without Project Mono, I chose to write my own alternative and
with that the track2rss project was born. Current
support is provided for FedEx, UPS and USPS, with DHL USA support coming soon. In this article I will be concentrating on the
UPS tracking API.
Overview of the UPS XML API
UPS provides two types of tracking APIs for developers collectively called UPS Online Tools: HTML-based and XML-based. The XML API is well documented on UPS's website, (registration required) and only uses XML 1.0, as opposed to SOAP and WSDL. Most other carriers have very similar APIs (although FedEx uses XML schema). The API process consists of an input XML packet sent in the body of an HTTP POST transaction over SSL to the URL provided by UPS. The response packet will either contain an error or tracking information, with an occasional HTTP 500 error for those times when the server is really dead (see an example of an error and a normal output packet).
Astute readers will note that the input packet is actually two XML packets combined together. That is due to the fact that UPS requires authentication information to be passed in a separate XML packet prepended to the actual request packet within the same HTTP request. This authentication data consists of the username and password used to log in to UPS's E-Tools website, and a special "XML Access Key" which must be obtained from the same site.
After obtaining the relevant API information from UPS and registering to get the XML Access Key, I now faced the task of choosing the right tools for translating the package tracking information into RSS. Seeing that UPS's XML API and RSS were just different dialects of XML, I decided to use XSLT, a W3C-standardized language for transforming XML, instead of writing a regular program. One of the main reasons why I chose to go with XSLT is because the XML processing APIs in most popular languages are still too cumbersome for casual use. With a conventional programming language I would need to parse the XML and then transform it into code, while with XSLT I could leave the parsing task to the XSLT processor and do the actual transformation in just a few lines. Additionally, since XSLT and XML are portable across many platforms, I could port my project to any platform. Also, putting XML transformation instructions into an external stylesheet allows for creation of other XSLT stylesheets in the future to generate different types of XML, such as Atom, and easily add support for other carriers without changing the main program.
Of course, ending up with a bunch of XSLT stylesheets doesn't do us any good. We still need a way actually execute them. Any other language with XSLT support would suffice for this purpose including Python, Java, .NET languages, etc., as well as any frameworks that support XSLT templates such as Struts or AxKit. However, I decided to write a short wrapper program in Perl because it is portable across many platforms and is available by default on many hosting providers (of course an XSLT processor such as libXSLT is still required). It would be trivial to rewrite the same wrapper in any other language, something that I might do in the future for this project.
Playing with XSLT
In the end, the track2rss project consisted of two parts: a set of XSLT stylesheets to process the XML and a 200 line wrapper script written in Perl. I started off generating the correct XML input packet. The resulting stylesheet simply takes an empty XML file and several parameters as passed into the XSLT processor, and puts them into the right places in the XML (see the stylesheet and sample input packet). Of course it is not necessary to use XSLT or the XML APIs to do this; I could have simply recorded the whole packet inside the wrapper script and used some sort of a replace function to plug in the parameters. But since I am using the XSLT for output anyway, I decided to use it for input as well as follows :
<?xml version="1.0"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output method = "xml" indent="yes"/> <xsl:param name="service_key"/> <xsl:param name="service_username"/> <xsl:param name="service_password"/> <xsl:param name="tracking_number"/> <xsl:param name="version"/> <xsl:template match="/"> <AccessRequest xml:lang="en-US"> <AccessLicenseNumber><xsl:value-of select="$service_key"/> </AccessLicenseNumber> <UserId><xsl:value-of select="$service_username"/></UserId> <Password><xsl:value-of select="$service_password"/></Password> </AccessRequest> <xsl:text disable-output-escaping="yes"><?xml version='1.0'?></xsl:text> <TrackRequest xml:lang='en-US'> <Request> <TransactionReference> <CustomerContext><xsl:value-of select="$version"/></CustomerContext> <XpciVersion>1.0001</XpciVersion> </TransactionReference> <RequestAction>Track</RequestAction> <RequestOption>activity</RequestOption> </Request> <TrackingNumber><xsl:value-of select="$tracking_number"/></TrackingNumber> </TrackRequest> </xsl:template> <xsl:template match="text()"> </xsl:template> </xsl:stylesheet>
Unlike the input stylesheet, the output stylesheet operates directly on the XML packets received from the carrier. However, I also wanted to include several other things in the feed produced by this project:
- An error message if the transaction failed.
- A link to the HTML version of the carrier's tracking site.
- A mandatory disclaimer.
- The version and name of the program that generated the feed.
- The ability to set a custom CSS stylesheet to format the output RSS feed in a browser.
I started the output stylesheet by passing two parameters indicating a URL to the CSS stylesheet and the version of the program:
<?xml version="1.0"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:param name="url_stylesheet"/> <xsl:param name="version"/>
The first step in the actual template is to check for the CSS parameter
$url_stylesheet) and set it if present. Being that the CSS stylesheet is an
XML processing instruction ("
<?xml-stylesheet?>"), it must be placed in
the beginning of the output XML file (note that the use of the
xsl:text tags to
<xsl:template match="/"> <xsl:if test="$url_stylesheet"> <xsl:text disable-output-escaping="yes"><?xml-stylesheet href="</xsl:text> <xsl:value-of select="$url_stylesheet"/> <xsl:text disable-output-escaping="yes">" type="text/css"?></xsl:text> </xsl:if>
At this point I need to start creating the actual RSS feed. The basic RSS 2.0 feed
of a single root
rss element containing within it a single
element. Within the
channel element there are some metadata elements containing
information about the feed and multiple
item elements. I put some text and the
tracking number into the
title element, along with a link to UPS's tracking
website which goes into the
link element :
<rss version="2.0"> <channel> <title>UPS Tracking Information for <xsl:value-of select="TrackResponse/Shipment/Package/TrackingNumber"/></title> <link>http://wwwapps.ups.com/WebTracking/processInputRequest? sort_by=status&tracknums_displayed=1 &TypeOfInquiryNumber=T&loc=en_US&InquiryNumber1= <xsl:value-of select="TrackResponse/Shipment/Package/TrackingNumber"/> &track.x=0&track.y=0</link>
To comply with UPS's license for the API, I include a disclaimer in the
description element along with the tracking number (note the use of HTML
br tags for formatting the output in RSS readers):
<description>This RSS feed tracks UPS package # <xsl:value-of select="TrackResponse/Shipment/Package/TrackingNumber"/> <br/> DISCLAIMER GOES HERE </description>
Here I indicate the language of the feed and the version of the program, using the
$version parameter :
<language>en-us</language> <generator><xsl:value-of select="$version"/></generator>
The next step is to check the
element for errors. If there are no errors, then we can call the rest of the stylesheet.
item element will be created with the error message instead. As
the last step, we close off all the tags :
<xsl:choose> <xsl:when test="/TrackResponse/Response/ResponseStatusCode = '1'"> <xsl:apply-templates select="TrackResponse/Shipment/Package/Activity"/> </xsl:when> <xsl:otherwise> <item> <title>TRACKING REQUEST FAILED</title> <description>Failed to retrieve data from UPS, error message: <xsl:value-of select="TrackResponse/Response/Error/ErrorDescription"/> </description> </item> </xsl:otherwise> </xsl:choose> </channel> </rss> </xsl:template>
Now that the main
channel element has been taken care of, the next step is to
process the actual tracking information. For this we have to match all of the individual
Activity elements and transform them into the corresponding RSS
item elements in a separate template. Since UPS uses special codes for
activity types, we need to use
xsl:choose to transform them into human-readable
descriptions and put them inside the
title element :
<xsl:template match="Activity"> <item> <title> <xsl:choose> <xsl:when test="Status/StatusType/Code = 'I'">IN TRANSIT</xsl:when> <xsl:when test="Status/StatusType/Code = 'D'">DELIVERED</xsl:when> <xsl:when test="Status/StatusType/Code = 'X'">EXCEPTION</xsl:when> <xsl:when test="Status/StatusType/Code = 'P'">PICKUP</xsl:when> <xsl:when test="Status/StatusType/Code = 'M'">MANIFEST PICKUP</xsl:when> <xsl:otherwise>UNKNOWN</xsl:otherwise> </xsl:choose> </title>
description element, we include the date/time, location and status of
the tracking event. In order to transform the UPS's date/time formats, some processing
required with the XSLT
substring function. Additionally, since the
ActivityLocation/Address/City element is optional, an
statement is needed (note the use of
br tags for formatting) :
<description> Date/Time : <xsl:value-of select="substring(Date, 5, 2)"/>/ <xsl:value-of select="substring(Date, 7, 2)"/>/ <xsl:value-of select="substring(Date, 1, 4)"/>  #160; <xsl:value-of select="substring(Time, 1, 2)"/>: <xsl:value-of select="substring(Time, 3, 2)"/>: <xsl:value-of select="substring(Time, 5, 2)"/> <br/> Location : <xsl:if test="ActivityLocation/Address/City"> <xsl:value-of select="ActivityLocation/Address/City"/>, #160; </xsl:if> <xsl:value-of select="ActivityLocation/Address/StateProvinceCode"/> #160; <xsl:value-of select="ActivityLocation/Address/CountryCode"/> <br/> Status: <xsl:value-of select="Status/StatusType/Description"/> </description>
We end up with a
link element, which is identical to the
element we used in the
channel element above, and follow by closing the tags.
Even though the RSS 2.0 specification does not mandate the
link element, some
readers like FireFox will not parse the feed without it (note that we navigate back up to get the
tracking number) :
<link>http://wwwapps.ups.com/WebTracking/processInputRequest? sort_by=status&tracknums_displayed=1 &TypeOfInquiryNumber=T&loc=en_US&InquiryNumber1= <xsl:value-of select="../../../../TrackResponse/Shipment/Package/TrackingNumber"/> &track.x=0&track.y=0 </link> </item> </xsl:template>
Writing the Wrapper Script
Once we are done with the XSLT part of the project, its time to move to the wrapper script that actually runs it. It consists of four parts:
- The first part contains the necessary settings for each carrier such as authentication information, URLs, etc. Due to security and portability issues, it would be better to store them in a secure fashion in a separate configuration file, but to keep things simple I chose not to do that here.
- The second part checks that all the input parameters to the script are not empty, and parses them from the HTTP request.
- The third part generates the request packet by invoking the XSLT processor with the input template, and sends the request to UPS.
- The fourth part processes the response and transforms the resulting XML into RSS via another call to the XSLT processor.
Leaving aside the mechanics of setting variables and parsing HTTP requests, I want to concentrate on the interaction between the wrapper and the XSLT templates. In this example, I picked the XML:libXSLT Perl module which itself interfaces to the GNOME's libXSLT library (the other XSLT processor for Perl, XML::XSLT did not support enough XSLT features to process my stylesheets correctly). To use the library, I needed to add the following to the Perl script:
Getting past all the initial code, I initialize the XSLT library and create an empty XML object which will be used to generate the input XML packet:
my $parser = XML::LibXML->new(); my $xslt = XML::LibXSLT->new(); my $source = $parser->parse_string('<?xml version="1.0"?><xml/>');
After that, we parse the input XSLT stylesheet from a file and invoke the XSLT processor. The parameters which are passed to the processor are defined in the beginning of the script and contain the authentication information for UPS, and the tracking number of the package (which is extracted from the input request when the script is called) :
my $style_doc = $parser->parse_file($input_xsl); my $stylesheet = $xslt->parse_stylesheet($style_doc); my $results = $stylesheet->transform($source, XML::LibXSLT::xpath_to_string(service_key => $service_key), XML::LibXSLT::xpath_to_string(service_username => $service_username), XML::LibXSLT::xpath_to_string(service_password => $service_password), XML::LibXSLT::xpath_to_string(tracking_number => $tracking_number) );
Once the request packet is generated via XSLT, I send it on its merry way by using the famous libwww-perl library:
my $req; $ua = LWP::UserAgent->new; $req = HTTP::Request->new(POST => $service_url_track); $req->content_type('application/x-www-form-urlencoded'); $req->add_content($stylesheet->output_string($results)); my $res = $ua->request($req);
To process the response, I follow a similar routine, except that I use the response returned from UPS instead of the empty XML:
my $source = $parser->parse_string($response->content); my $style_doc = $parser->parse_file($output_xsl); my $stylesheet = $xslt->parse_stylesheet($style_doc); my $results = $stylesheet->transform($source, XML::LibXSLT::xpath_to_string(version => $version) XML::LibXSLT::xpath_to_string(url_stylesheet => $url_stylesheet) );
Once processed, we simply print the output to the user and exit:
print "Content-Type: application/xml\n\n"; print $stylesheet->output_string($results); exit;
Figure 1. BlogLines Screenshot
Figure 2. Liferea Screenshot
Using XSLT for generating RSS feeds for UPS package tracking turned out to be a much simpler task than writing a straightforward program. Having XSLT do the heavy lifting of dealing with XML freed me to concentrate on the mundane programming tasks of working with HTTP requests and responses, illustrating how XSLT can reduce the complexity of XML processing. In the future, I plan to integrate additional carriers into the track2rss project, as well as additional output formats and languages.