Menu

An SVG Case Study: Integrated, Dynamic Avalanche Forecasting

April 23, 2003

Chris Cochella and Tyler Cruickshank

Introduction

It's common to wander the Web, cobbling bits of information together to understand some concept or make a decision. Weather information related to a specific activity is a good example. Trying to figure out the backcountry snow conditions at 10000ft in southern Colorado is a difficult problem. At some point an inner voice says, "Why can't someone simply collect this information in one place and display it in a simple, useful, unadvertised, and uncluttered manner". There are even desktop tools like Watson that try to help with this effort by mining or harvesting generally useful information like movie listings.

Problem

We are avid backcountry skiers (backcountry snow conditions are not considered generally useful). A wise backcountry skier is always aware of the specific local and regional weather conditions in the mountains that contribute to avalanche danger. For winter backcountry enthusiasts like us, the problem is that all of the weather data available (i.e., National Oceanic and Atmospheric Association's National Weather Service) from remote mountain stations and ski areas is scattered throughout the Web -- in various formats, of varying frequency, contained in difficult to read text files, and differing in measured parameters. Cobbling this information together at 6AM prior to skiing is not our idea of fun. Thus, our goal is to collect all of this data in one place and then graphically display related parameters in a Web information appliance. We call this appliance the Avalanche Meteorology Toolkit (AMT).

To view the AMT in action click here or you can view the AMT SVG interface as screen capture in Figure 1. You will need the Adobe SVG plugin to view the demonstration. The results of this project were also presented at the 2002 International Snow Science Workshop. A copy of the presentation abstract can be downloaded here as PDF.

Avalanche Meteorology Toolkit, Figure 1
Figure 1: Avalanche Meteorology Toolkit

The process of bringing all this information together is messy. But it can be done by using Perl to do data clean up, handing off to a MySQL database for storage, and then to Scalable Vector Graphics (SVG) for a lightweight graphics display, which can be viewed by Adobe's SVG Plugin. The article Data-Driven SVG Apps by Pramod Jain is useful for understanding the multi-tier approach (data storage, data processing, display). For more information on why SVG is so useful, read SVG On the Rise by Dean Jackson.

Our Solution

This project requires five steps, utilizes two Perl scripts, one SVG information appliance interface, and one SVG file for each weather station. The steps can be broken into two separate phases: information harvesting (steps 1-2) and information display (steps 3-5).

Harvesting

Step 1: Data Collection

The first Perl script retrieves, parses, and stores the desired data from external data sources. This script uses two handy perl modules, LWP::Simple and DBI::MySql. Both modules are easily installed using Active State Perl's Perl Package Manager (ppm) or CPAN.

The second script extracts data from the database and creates our customized weather graphics using SVG.

LWP::Simple is a simple web client library for Perl. The getStore() method retrieves a weather station file for Alta, Utah, from the Web and then stores the file in a local directory:

$url
= 'http://www.wrh.noaa.gov/Saltlake/avalanche/RPT/COLLINS.RPT.html';
$myFile = getStore($url);

Now that we have the file stored locally we can use Perl's regular expressions to extract data. In this example we extract the relevant temperature, date, and time values. The following HTML is a sample of the retrieved file:

		<HTML>
		<HEAD><TITLE>
		</TITLE></HEAD>
		<BODY BACKGROUND="/Saltlake/background2.jpg">
		<PRE>

		3-16-2003> Collins Study Plot > 9,664'
		Month Air 12 Hr Total 1 Hour since
		Day Hour Temp Snow Depth precip 0400
		
		-------------------------------------------------

		3 16 2000 23.0 0 92 0 .27
		3 16 1900 24.0 0 92 0 .27
		3 16 1800 26.0 0 92 0 .27
		3 16 1700 30.0 0 92 0 .27
		3 16 1600 32.0 0 92 0 .27
		</PRE>
		</HTML>

After opening the file we have to locate the starting point of the actual data and begin to parse the file line by line:

@data= split(/\s+/);
push(@temperature,$data[4]);
$date_time = "$data[0]-$data[1]-$data[2]";
push(@dateTime,$date_time);

The method split() parses the line of text on whitespace (\s+), and we place the individual pieces of data into the temperature array. In the AMT, for each chart we retrieve up to three different files, parse up to seven different parameters, and store the data in individual arrays.

All of our temperature data now resides in the array @temperature. The DBI::MySql module (database interface to MySQL) can now be used to move our temperature, date, and time values from our array into the database. The first step is to check whether or not the temperature values from each date-time record are already in the database. This is a particularly important step if you are running your script at regular intervals and, as a result, run the risk of duplicating records.

Step 2: Storage

We won't go into all the details of using DBI::Mysql here, so you may want to scan through the Short Guide to DBI by Mark-Jason Dominus for further explanation of DBI and MySQL. In our code we compare the harvested data with that already in the database, storing new data in the database. The data harvesting runs automatically every hour. Now that we have our data we need to display it in a clear, lightweight manner.

Display

Step 3: Retrieval

We now begin the more interesting part of working with the data. First, we need to use the Perl DBI once more to yank out the data that we are interested in graphing. We use a second Perl script that retrieves data and processes it to create SVG graphics.

First, we will address the SQL code. Our script extracts seven days' worth of data, and stores it in the array @data.

   $dbh = DBI->connect(
       "DBI:mysql:database=weather:host=localhost",
       "myUser","myPass")|| die "Cannot connect: $DBI::errstr";

   $select = "SELECT * FROM weather.alta WHERE TO_DAYS(NOW()) 
        -TO_DAYS(date_time) <= 7 ORDER BY date_time DESC";

   $sth =$dbh->prepare($select) || 
        die "Couldn't prepare statement: " . $dbh->errstr; 

   $sth->execute() || 
        die "Could not execute statement: " . $sth->errstr;

   while((@weatherInfo) = $sth->fetchrow_array()){ 
        $data[$ctr] =[@snow_info];
        $ctr++; 
   }

Step 4: Chart Generation

Once the information for a given weather station is retrieved we need to process it and output it as SVG. A raster image of a complete chart is shown in Figure 2 below. Each station is output as a separate file so that our Toolkit can retrieve specific weather station information on demand without sending data for all stations. This also makes it easy to add weather stations in the future.

Figure 2: Weather Station Chart
Avalanche Meteorology Toolkit, Figure 2

We begin constructing this file by creating the standard SVG file header: 

<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 20010904//EN" 
    "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">

We then create a coordinate system for the graph and output the beginning SVG so that the graph scales 100% of the view box provided by graph's container. In this case the graph's container will be specified by the Toolkit interface.

<svgwidth="100%" height="100%">

SVG provides a tag to uniquely identify a group of visual elements. Here we use SVG's group ("g") tag to give the elements that define our graphic a name ("id"). The name that defines our graphic will be used by our JavaScript event handler. The next thing we want to point out is SVG's handy transform attribute. First, scale provides a way to scale or resize your object or entire group of graphic objects. The translate attribute lets us move the graphic to a particular location within the coordinate system. Why are we doing this? In AMT we have a rectangular view port within which we display the various graphs of the weather stations. The individual weather station SVG graphics have their own size and coordinate system; we must transform the size and positioning of the graphic so that once it is added to the AMT, it fits inside the rectangular view port. This means that you can use just about any SVG graphic anywhere you want.

<g id="altaGraph" width="6000" height="4500" 
  transform="scale(.104,.0973) translate(2625,1230)">

Then we draw the various graphic primitives such as the outer chart box:

<rect x="0" y="0" width="6000" height="4500"
  fill="#dcdcdc" stroke="black" stroke-width="20"/>

We place a trash can image for disposing or removing a particular chart with the onclick="snowTkMetRemove()" function (this is the really exciting part which we will discuss in step 5 below):

<image xlink:href="./images/trash.gif" x="5700" y="50"
width="300" height="300" style="display:inline"onclick="snowTkMetRemove(evt)"/>

Once the base layer of graphical components are constructed, and we have prepared the data, it is a simple matter of outputing the data like the Temperature line segments (blue line graph):

<polyline fill="none" stroke="red" stroke-width="7"
  points="400,1533.33 431.13,1533.33"/>

There are many other lines, rectangles, and text values which are placed on the chart but are not displayed here. For more information on SVG details refer to SVG Essentials by J. David Eisenberg.

We have automated the chart generation process to follow the data harvesting. There is one additional step for chart creation: compression. The Adobe SVG plug in permits zip compression of the source files. This is useful in our case because the SVG text files are approximately 200 Kilobytes. That's not huge, but when you are trying to get out the door, time is of the essence. By zipping these files with gzip we can gain an order of magnitude reduction in file size. For example, one chart is 199955 Kilobytes before compression and 18247 Kilobytes after. The relative lightweightedness is one of the hidden benefits of SVG over other vector graphics.

Now that the data has been harvested and the charts have been generated, we need to integrate all the charts into a single viewer.

Step 5: Where it all comes together

Each weather station is a separate SVG file and presentation. However, for our purposes we need to integrate each station into a regional Toolkit so that the user can jump from one station to another and see a consistent output of relevant information. In a sense, the Toolkit is a visual information appliance used for displaying external weather data.

For the toolkit interface, you can see that we employ some of the same SVG display options like creating a box, adding an image and a transparent overlay to create a list of "Meteorological Graphs" shown in Figure 3 and in the code sample below:

Avalanche Meteorology Toolkit, Figure 3
Figure 3: List of Graphs 
<g id="smallWindowTemplate">
<desc> The background Template </desc>

<image xlink:href="./images/avalanche1.jpg" x="35"
  y="120" width="220" height="280"style="display:inline"/>

<rect x="45" y="130" width="200"height="260" 
  fill="yellow" opacity=".6"stroke="none" stroke-width="5"/>

<rect x="35" y="120" width="220"height="280" 
  fill="none" stroke-opacity=".4"stroke="red" 
  stroke-width="3"/>

<text style='font-size:12pt; font-weight:bold; 
  font-family:sans-serif;fill:black; text-anchor:center;' 
  x="60"y="150">Meteorological Graphs</text>

<text style='font-size:12pt; font-weight:bold; 
  font-family:sans-serif;fill:black; text-anchor:center;'
  x="70"y="180">Utah:</text>

</g>

We then add each weather graph choice as a radio button:

<g id="messageWindow">

<text style='font-size:10pt; font-family:sans-serif; 
  fill:black;text-anchor:left;' 
  x="95" y="200">Alta</text>

<circle id="wx1a" cx="85" cy="195" r="5" 
  fill="blue" stroke="black" stroke-width="2" 
  onclick="snowTkMetALTA(evt)" />

</g>

Aside from the basic visual elements, we have included an ECMAScript onclick event. This is where we take advantage of Adobe's SVG plugin server-connection capabilities to load external data sources. In this case, our external data is the station-specific SVG graphs we produced in steps 3 through 4. This functionality is very nice because once we have developed the AMT appliance we no longer have to touch it, even though the data is constantly changing. We simply refer to that external data source, which also means we download only what is requested. Figure 4 shows a specific weather station graph within the complete Toolkit interface.

Avalanche Meteorology Toolkit, Figure 4
Figure 4: The Completed Toolkit

This process of loading and unloading external data sources requires three methods: one to retrieve the external SVG file with the getURL method, another to act as a callback method that parses, returns, and inserts the XML into the main SVG document tree, and a final method to destroy or delete the inserted XML when the trash icon is selected (see the snowTkMetRemove method in the above code listing that adds the trash can image). These 3 methods are listed below along with the beginning of the SVG document for the Toolkit interface.

<?xmlversion="1.0"?>

<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 20010904//EN" 
  "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">

<svg width="900" height="900" viewBox="0 0900 900" 
  preserveAspectRatio="xMinYMin" >

<!-- ECMAScript --> 

<script type='text/ecmascript'><![CDATA[

// Method calling the getURL()

function snowTkMetALTA (evt)
{
  var node = evt.target.ownerDocument.getElementById("altaGraph"); 
  node=evt.getTarget;
  var url='./altaGraph.svgz';
  getURL(url,snowTkMetCallbackALTA);
}

function snowTkMetCallbackALTA (document)
{
  fragment=parseXML(document.content,document);
  document.getElementById("messageWindow").appendChild(fragment);
}

function snowTkMetRemoveAlta (evt) 
{
  var node = evt.target.ownerDocument.getElementById("altaGraph");
  node.parentNode.removeChild(node);
}

</script>

There are a couple of items to take note of in the ECMAScript. First, notice that we provide the directory path to our weather station graphic. The method getURL() retrieves it and then appends it on to our AMT in the snowTkMetCallbackALTA function. Second, the getElementById() method is given two different arguments: "messageWindow" and "altaGraph". The "messageWindow" argument is the group id of the location in our AMT where we want to append our weather graph. Now, the group "messageWindow" will have a new element that is actually a group of elements that defines our weather graph. "altaGraph" is the group id of the weather station SVG graphic that we want to add.

This is where giving names to graphic elements is important. The "altaGraph" group contains scale and translate information that allows the graphic to fit inside the view port once it is appended to "messageWindow". Not only is it important for placing or inserting the weather graph, it is also important for specifying what portion of the document to remove. (For more information, see Antoine Quint's article SVG Tips and Tricks: Adobe's SVG Viewer at XML.com.) When we are done with each chart, we simply remove it from the document tree and add a new one-very clean.

Summary

The primary benefits of our approach are the integration of disparate information sources into a single, lightweight display of relevant real-time information. Perl, or course, is great for text processing. SVG along with Adobe's server-connection capability has proven to be a very flexible and lightweight means of information display. We are planning the integration of additional dynamic information like avalanche pictures posted by the public, integration of the Utah Avalanche Center forecast advisory, and a regional map of recent avalanches. By combining all this information in one place, backcountry travelers and avalanche forecasters have been able to quickly assess backcountry conditions and make appropriate, informed decisions. There is no end to the possibilities for creating dynamic, lightweight displays of real-time data: other scientific monitoring devices, traffic monitoring, manufacturing processes, financial data, and so on.

Resources