Applied Network Theory
February 11, 2003
Network theory enjoys tremendous vogue at the moment. Books by Duncan Watts, Malcolm Gladwell, Albert-László Barabási, and Mark Buchanan have created such intense buzz that Watts recently complained to The New York Times about the time he must spend deflating the hype. Why all the excitement? The research suggests a kind of grand unified theory: networks made out of anything (molecules, nerve cells, electrical grids, transportation systems, web links, human beings) obey the same laws of growth and arrive at similar structures. In every kind of network, a few nodes differentiate. They attract more links, become hubs or routers, and radically shorten the distance between arbitrary endpoints.
In social networks, of course, the nodes, hubs, and routers are people, a detail that theorists abstract away from in order to clarify principles of network growth and structure. It is surprising, and no doubt significant, to find that the hub molecules of cellular metabolism, and the social connectors identified by Malcom Gladwell, can be understood to play analogous structural roles in their respective networks. But this is only an analogy. In the brave new world of social network analysis, where the effectiveness of someone with a job description like "liaison between engineering and marketing" can be quantified in terms of message flow and connectedness, we'll need to be really careful about pushing the analogy too far. The network is the computer, but when some of the nodes are human minds, link density and connection strength can only tell part of the story.
Malcolm Gladwell's experiment in surname recognition showed that social connectors aren't just a little bit above average; they're way above. The same holds true for the people Gladwell calls mavens, whose minds are repositories of specialized knowledge. I've known a few of them. Fifteen years ago I worked on one of the first commercial applications of CD-ROM technology. The product (whose descendants still exist today) was a database of business information about U.S. public corporations. It combined 10K reports, financial ratings, news stories, and executive biographies to create a compelling tool for business analysts. The seamless integration of those disparate sources was remarkable not only by 1988 standards, but also by today's standards. That was true because a single mind created and maintained the metadata maps that correlated the data sources and made the product more than the sum of its parts. That mind belonged to Bill Bassett who, everyone agreed, was uniquely wired to perform this information-mapping task. It was necessary, but not sufficient, to situate Bill at the confluence of the data sources. The magic happened because Bill had a rare talent for information mapping, and because he enjoyed exercising that talent.
At Yahoo, that ecological niche is occupied by Jeffrey Friedl. Those of us who have experienced the whirlwind of a Friedl lecture on regular expressions know he is a force of nature. (If you haven't already, check out his excellent book on the subject, Mastering Regular Expressions.) Operating at the hubs of data-flow networks, people like Bassett and Friedl play crucial roles that are not yet widely acknowledged or understood. In the realm of software development, we have learned over the years that the best programmers are not just a little better than average--they are outrageously, orders-of-magnitude above average. We give these people titles such as "chief architect," and we understand that they are a keystone species in the software ecosystem. We don't yet accord similar status to highly accomplished social networkers and information mappers, but I think that's about to change.
In blogspace we can see these new keystone species evolve. A weblog is simultaneously the hub of a social network and a router of information flows. It erodes the distinction between "who you know" and "what you know" in ways that we have only begun to fathom. In this environment, connectors and mavens can really stretch their wings and can exercise their talents to the fullest degree. The effects will be felt everywhere, but perhaps most powerfully (in the near term) in the realm of software development. An analysis by my friend Andy Singleton suggests why. Andy's latest venture, Assembla, helps its clients find and apply resources (notably offshore--read: non-U.S.--software developers and open source software componentry) that can drive down the cost and accelerate the pace of development. In an essay called Trends Driving IT Deflation Andy writes:
"I believe that in today's market it is more important to understand where software and software components come from than it is to be able to build software. Research, standard selection, negotiation, and creative alteration can deliver software very quickly." [Assembla.com]
Elsewhere he notes that the best model for this method of organizing work is the film industry, where the art of getting things done with dynamically assembled teams reaches its apotheosis. Others have made the same point, including Jeremy Rifkin:
"The Hollywood culture industries have had a long experience with network-based approaches to organization and, for that reason, are fast becoming the prototype for the reorganization of the rest of the capitalist system along network lines." [The Age of Access]
Rifkin's remarkable and underappreciated book includes this quote from a 1995 Inc. Magazine story by Joel Kotkin and David Friedman entitled Why Every Business Will Be Like Show Business:
"Hollywood [has mutated] from an industry of classic, huge, vertically integrated corporations into the world's best example of a network economy.... Eventually, every knowledge-intensive industry will end up in the same flattened, atomized state. Hollywood has just gotten there first." [Inc. Magazine, via Jeremy Rifkin]
The network is not only the computer. It is also the operating system and the software development environment. Coders will thrive in this environment, but increasingly, so will social connectors and information mappers. Network theory tells us that some of these hubs will outperform others. It doesn't explain why. Perhaps there are general laws that produce favored hubs in any kind of network. But in a knowledge network the hubs are people imbued with a talent, and driven by a passion, for connecting people, information, and components. Software development doesn't yet recognize that professional role, but I predict that it will.