Music and Metadata
by Chris Mitchell | Pages: 1, 2

What About Our Poor Suffering Hero?

The software tools available from the SIMILE project are good, but how can they help our poor Shawn? Well, unbeknownst to him, a plucky music lover has downloaded these two programs and put them to good use. A long-term fan of U.K.-based independent music and a regular visitor to sites that list a huge number of U.K. music events, this plucky developer, also living in London, has generated an RDF document that lists the coming week's music events using Solvent and Piggy Bank. This RDF document lists all the information available on drownedinsound.com, including the addresses, various artist information, or the artists played by the DJ(s). It then uses the geolocation service to get information detailing the longitude and latitude of the event, which could be used to find its location on a mapping service such as Google Maps. This information is consolidated into an RDF document that is published on a website. Although we have until now only been talking about RDF and seeing it through Piggy Bank, RDF can also be viewed directly, as it is an XML-derived markup language; Example 1 illustrates an RDF description of the music events listing page and the screen scraper. As you can see, this lists the genres of music being played at the event.

<?xml:namespace ns = "http://www.w3.org/RDF/RDF/" prefix ="RDF" ?>
<?xml:namespace ns = "http://example.org /music/" prefix = "EM" ?>

<RDF:RDF>
  <RDF:Description RDF:HREF = "http://uri-of-event -1">
    <EM:Genre>Rock</DC:Genre >
    <EM:Artists>Rage against the machine</ Artists >
    <EM:Artists>Beastie Boys</ Artists >
   </RDF:Description>
</RDF:RDF>

Example 1. RDF description of a music event

Shawn, interested in restaurants in London, has browsed OpenGuides – a network of free, community-maintained information that includes a guide to good food in London. Our hero has looked at some great food and bookmarked the RDF from the web pages describing various restaurants. This was done simply by clicking on a button in Firefox that was added when he installed Piggy Bank. As easy as adding a favorite to his shortcut menu, the information about the restaurant and its web address is stored because an RDF data document was referenced in the header portion of the web page by OpenGuides. This information has been added to the information in Piggy Bank already obtained from the music lover's list of London music events. Our hero can now use his Semantic Web browser, Piggy Bank – which has consumed both feeds – to search this information. The relationships between the geocoordinate labels used in the documents have also been defined by a third person, and, using Piggy Bank's inbuilt Google Maps functionality, Shawn can have them plotted on a map showing the restaurants and events for that week. How far is this from the truth? Well, all of it – apart from the inferred connection of the geolocation labels – is absolutely doable in the tools described. The relationship mapping between the longitude and latitude labels is easily doable, just not with the graphical interface to the current version of Piggy Bank; the map visualization can be plotted once the longitude and latitude are labeled under a specific tag name. This is actually achieved with the listed screen scraper, which can be used with Piggy Bank to plot this week's music events as shown in Figure 4. Defining the relationship of other labels to this label would require additional relationships to be defined, as well as the inference step, which really makes this all so much more interesting. It should be stated that this was not Piggy Bank's primary goal. Inference, as previously stated, is a simple list of rules: if this label is present, then you can assume this. In a RDF inference system, the simplest one in the Semantic Web, this consists of just 12 rules easily implemented in a programming language.

Figure 4. Google Maps used in Piggy Bank to plot this week's London music events

In fact, there is a much more functional inference engine in Jena, which Piggy Bank is built around. All the tools are present, and with the current versions of these tools, a complete framework is in place. However, these tools are now waiting for those plucky developers that were so crucial in our example.

Turning Up to a Hip-Hop Club Night, Thinking It's Going to be Dance

Following from our current example, what would happen if we knew other artists who had appeared on albums with Rage Against The Machine? This might give us a reasonable understanding that their music is similar. Many artists may not be simply related in this way, but this is pretty close to the playlist models used by Pandora [5] and Music Strands [6] or other collaborative filtering applications such as the ones used by Amazon for books. The relationship of artists through albums is easily expressed in RDF and obtainable from a site called MusicBrainz. How music genres are related to one another (music genre taxonomies) can explored in many web pages such as Wikipedia, Amazon, or MP3.com. With all this information, many new applications become possible – simply by being able to reuse the information that is currently on the Web in new ways on the Semantic Web. Again, please keep this simple: web pages and information labels are being augmented with artists, genres, and events based on what can be implicitly assumed from people's structured labels of web pages and information. A C# example of using music genre taxonomy with the Semantic Web to query music and try to improve search performance can be found here [7]. Given all this information, knowing how the genres are related does not seem so trivial a point as in our bargain bin example. Having the ability to consolidate all this information about music artists, genres, and their relationships is great, but this is the same for documents, images, or video files. It would be better to get some information related to the recorded music files themselves.

At this stage, our valiant Semantic Web trusty is walking out his front door listening to the album his unsuspecting date mentioned, thinking he is all set. In his pocket, he has the address for a great, reasonably priced Italian restaurant and a club with good reviews that plays dance music. However, the club he found had listed "Rage Against the Machine" for their hip-hop roots, not for Zack de la Rocha's collaboration with dance artists.

There is a way of describing recorded music that labels music using RDF [8]. A common format for doing this is known as MPEG-7, although the principles are the same for a number of multimedia markup languages. This has the functionality of describing music using subjective labels that rely on frequency information in the music. These descriptions can be used to describe how similar-sounding two bits of recorded music are, or even what musical genre they are likely to be from [9]. If his Semantic Web browser had been equipped with this ability, Shawn would not have sat in a reasonably priced Italian restaurant trying to convince his lovely date that he greatly enjoys the type of music she's into, and casually telling her that there is a club that plays dance music nearby if she would like to go.

MPEG-7 has a number of feature descriptors that can be used to subjectively label various attributes of music such as timbre or beat. This creates a more musically tied labeling system. Examples of MPEG-7 documents can be found here [10], and they describe, among other things, the time-frequency properties of a piece of music. Given the example sound track, the Semantic Web music browser could have pointed out that Shawn was actually looking for hip-hop events, not dance as he specified, and suggested some good events in the area. These same features have also been used by people to construct music similarity (MusicIP [11]), and music fingerprinting (Shazam [12], MusicBrainz), allowing two music files to be compared or uniquely identified. If this information is fed into an inference system, like the Semantic Web, it leads to some very useful labels being added to all kinds of music information. In our case, this would mean that his date would have had a much better time that night than she did politely listening to hip-hop. The music could have been good, but the musician and DJs had rushed together a set after spending far too many hours looking for a break for their new song the record label needs next month.

Musician Problems

We have been concentrating on the needs of someone trying to find music events. However, musicians can also benefit from the Semantic Web, especially if it is equipped with MPEG-7 style descriptors. In our example, the musicians are rushed because they had problems finding a break. Normally, this process consists of trawling vast audio sample libraries, both on and offline. However, every time a break is found that is close to the one being sorted, it is invariably the right type of rhythm but the wrong type of sound, or the right type of sound and the wrong rhythm or groove. This, then, requires starting your search all over again. With the feature descriptors of MPEG-7, it is possible to state that you are looking for something that sounds similar to one break but has a different groove type or rhythm, narrowing your search in each step. If online sample stores stored their music with an MPEG-7 document and the inferences that are possible with the Semantic Web when samples are labeled with common information (such as the music style the sample is best used for and the creator), then our rushed musicians would have put on a much better show for our friends.

Does He Get That Third Date?

How far off are we from this really useful community-driven tool for music? Well, I'm sitting here now seeing all the building blocks and tools, wanting to implement a Semantic Web music browser – hopefully like so many of you are. However, I do not see any evidence out there of anyone doing this, so if anyone wants to give me a hand, that would be just great. If you like the ideas of the Semantic Web, have programming skills, and love music, why not join me in a open source project and get music into the Semantic Web – if only to help out our hapless friend on his third date.

References

1. SIMILE is focused on developing robust, open source tools based on Semantic Web technologies that improve access, management, and reuse among digital assets.
2. Solvent is a Firefox extension that helps you write screen scrapers for Piggy Bank.
3. Piggy Bank is a Firefox extension that turns your browser into a mashup platform by allowing you to extract data from different websites and mix them together.
4. Piggy-Bank, Installing screen scrapers
5. Pandora
6. Music Strands
7. C# Semantic Web and music genre taxonomies
8. RDF MPEG-7
9. Music Similarity application using MPEG-7
10. MPEG-7 examples
11. MusicIP
12. Shazam

Resources

1. Screen scraper and geolocation server-side script

---

---

2. Los Angeles Locksmith 323-678-2704 Los Angeles Locksmith
   2010-06-16 12:32:39 carpetcare
4. Amen brother
   2006-12-19 00:48:24 jbrettjones
6. Not legal RDF/XML or Namespaced XML
   2006-11-30 09:47:05 dajobe
8. PICSFROMSPACE
   2006-11-23 00:46:12 walweb