XML.com: XML From the Inside Out
oreilly.comSafari Bookshelf.Conferences.

advertisement

An SVG Case Study: Integrated, Dynamic Avalanche Forecasting

April 23, 2003

Introduction

It's common to wander the Web, cobbling bits of information together to understand some concept or make a decision. Weather information related to a specific activity is a good example. Trying to figure out the backcountry snow conditions at 10000ft in southern Colorado is a difficult problem. At some point an inner voice says, "Why can't someone simply collect this information in one place and display it in a simple, useful, unadvertised, and uncluttered manner". There are even desktop tools like Watson that try to help with this effort by mining or harvesting generally useful information like movie listings.

Problem

We are avid backcountry skiers (backcountry snow conditions are not considered generally useful). A wise backcountry skier is always aware of the specific local and regional weather conditions in the mountains that contribute to avalanche danger. For winter backcountry enthusiasts like us, the problem is that all of the weather data available (i.e., National Oceanic and Atmospheric Association's National Weather Service) from remote mountain stations and ski areas is scattered throughout the Web -- in various formats, of varying frequency, contained in difficult to read text files, and differing in measured parameters. Cobbling this information together at 6AM prior to skiing is not our idea of fun. Thus, our goal is to collect all of this data in one place and then graphically display related parameters in a Web information appliance. We call this appliance the Avalanche Meteorology Toolkit (AMT).

To view the AMT in action click here or you can view the AMT SVG interface as screen capture in Figure 1. You will need the Adobe SVG plugin to view the demonstration. The results of this project were also presented at the 2002 International Snow Science Workshop. A copy of the presentation abstract can be downloaded here as PDF.

Avalanche Meteorology Toolkit, Figure 1
Figure 1: Avalanche Meteorology Toolkit

The process of bringing all this information together is messy. But it can be done by using Perl to do data clean up, handing off to a MySQL database for storage, and then to Scalable Vector Graphics (SVG) for a lightweight graphics display, which can be viewed by Adobe's SVG Plugin. The article Data-Driven SVG Apps by Pramod Jain is useful for understanding the multi-tier approach (data storage, data processing, display). For more information on why SVG is so useful, read SVG On the Rise by Dean Jackson.

Our Solution

This project requires five steps, utilizes two Perl scripts, one SVG information appliance interface, and one SVG file for each weather station. The steps can be broken into two separate phases: information harvesting (steps 1-2) and information display (steps 3-5).

Harvesting

Step 1: Data Collection

The first Perl script retrieves, parses, and stores the desired data from external data sources. This script uses two handy perl modules, LWP::Simple and DBI::MySql. Both modules are easily installed using Active State Perl's Perl Package Manager (ppm) or CPAN.

The second script extracts data from the database and creates our customized weather graphics using SVG.

LWP::Simple is a simple web client library for Perl. The getStore() method retrieves a weather station file for Alta, Utah, from the Web and then stores the file in a local directory:

$url
= 'http://www.wrh.noaa.gov/Saltlake/avalanche/RPT/COLLINS.RPT.html';
$myFile = getStore($url);

Now that we have the file stored locally we can use Perl's regular expressions to extract data. In this example we extract the relevant temperature, date, and time values. The following HTML is a sample of the retrieved file:


		<HTML>
		<HEAD><TITLE>
		</TITLE></HEAD>
		<BODY BACKGROUND="/Saltlake/background2.jpg">
		<PRE>

		3-16-2003> Collins Study Plot > 9,664'
		Month Air 12 Hr Total 1 Hour since
		Day Hour Temp Snow Depth precip 0400
		
		-------------------------------------------------

		3 16 2000 23.0 0 92 0 .27
		3 16 1900 24.0 0 92 0 .27
		3 16 1800 26.0 0 92 0 .27
		3 16 1700 30.0 0 92 0 .27
		3 16 1600 32.0 0 92 0 .27
		</PRE>
		</HTML>

After opening the file we have to locate the starting point of the actual data and begin to parse the file line by line:

@data= split(/\s+/);
push(@temperature,$data[4]);
$date_time = "$data[0]-$data[1]-$data[2]";
push(@dateTime,$date_time);

The method split() parses the line of text on whitespace (\s+), and we place the individual pieces of data into the temperature array. In the AMT, for each chart we retrieve up to three different files, parse up to seven different parameters, and store the data in individual arrays.

All of our temperature data now resides in the array @temperature. The DBI::MySql module (database interface to MySQL) can now be used to move our temperature, date, and time values from our array into the database. The first step is to check whether or not the temperature values from each date-time record are already in the database. This is a particularly important step if you are running your script at regular intervals and, as a result, run the risk of duplicating records.

Step 2: Storage

We won't go into all the details of using DBI::Mysql here, so you may want to scan through the Short Guide to DBI by Mark-Jason Dominus for further explanation of DBI and MySQL. In our code we compare the harvested data with that already in the database, storing new data in the database. The data harvesting runs automatically every hour. Now that we have our data we need to display it in a clear, lightweight manner.

Display

Step 3: Retrieval

We now begin the more interesting part of working with the data. First, we need to use the Perl DBI once more to yank out the data that we are interested in graphing. We use a second Perl script that retrieves data and processes it to create SVG graphics.

First, we will address the SQL code. Our script extracts seven days' worth of data, and stores it in the array @data.

   $dbh = DBI->connect(
       "DBI:mysql:database=weather:host=localhost",
       "myUser","myPass")|| die "Cannot connect: $DBI::errstr";

   $select = "SELECT * FROM weather.alta WHERE TO_DAYS(NOW()) 
        -TO_DAYS(date_time) <= 7 ORDER BY date_time DESC";

   $sth =$dbh->prepare($select) || 
        die "Couldn't prepare statement: " . $dbh->errstr; 

   $sth->execute() || 
        die "Could not execute statement: " . $sth->errstr;

   while((@weatherInfo) = $sth->fetchrow_array()){ 
        $data[$ctr] =[@snow_info];
        $ctr++; 
   }

Step 4: Chart Generation

Once the information for a given weather station is retrieved we need to process it and output it as SVG. A raster image of a complete chart is shown in Figure 2 below. Each station is output as a separate file so that our Toolkit can retrieve specific weather station information on demand without sending data for all stations. This also makes it easy to add weather stations in the future.

Figure 2: Weather Station Chart
Avalanche Meteorology Toolkit, Figure 2

We begin constructing this file by creating the standard SVG file header:

<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 20010904//EN" 
    "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">

We then create a coordinate system for the graph and output the beginning SVG so that the graph scales 100% of the view box provided by graph's container. In this case the graph's container will be specified by the Toolkit interface.

<svgwidth="100%" height="100%">

Pages: 1, 2

Next Pagearrow