Less Is More In E-Business: The XML/edi Group
November 10, 1999
This week, David Webber from the XML/EDI Group, in an article co-authored with Alan Kotok from the Data Interchange Standards Association (DISA), presents his perspective on the integration of XML with EDI. The authors argue strongly that simplicity is the key to the widespread adoption of XML e-business solutions. The article urges the W3C to look to the needs of mainstream business as well as publishers, and not to overcomplicate XML standards.
XML Can Provide the "Eureka Event" for E-Business
At the Graphic Communications Association's (GCA's) executive conference last year, Gerry Galewski, a philosopher on information technologies, gave a provocative explanation on why it often takes years to truly appreciate the full potential of new technology:
"... when a breakthrough in technology is achieved, it takes us a while as a culture to figure out what we really have. New developments are culturally assimilated often based on what has come before. We can't help but place the new developments within an historical context.
• XML/EDI Group
• Data Interchange Standards Association
• The World Wide Web Consortium (W3C)
• Graphic Communications Association
• Interactive Financial Exchange
• Open Travel Alliance
"Here's an example: In 1844 Samuel Morse invented the ability to transmit information coded into electromagnetic pulses. He sent the first message of dot dash dot dot dash from Baltimore to Washington DC, and therefore people called this telegraphy.
"That first message Morse sent was 'What hath God wrought.' Telegraphy became ingrained into the cultural consciousness. It was easy to understand and deploy.
"Fifty years later, Marconi made a technological breakthrough. He broadcast electromagnetic waves through the air. But what did he send? The ability to modulate a signal was well understood. But Marconi sent dot dash dot dot dash. That is what was ingrained into the cultural consciousness of the time. So people called this wonderful new tool, simply "Wireless Telegraphy." Within their frame of reference, they didn't know what they really had. It took another twenty years for Lee Deforest to apply practical knowledge that had been around for decades. Deforest had the Eureka event, and gave us radio.
"Now let's look at how we do business in the 1990s. In the 30s and 40s and 50s and 60s professional managers defined the common business processes that we use to this day. Then computers and networks were developed. And we set out to take advantage of this new technology and automate our processes, and naturally we did that based on a cultural context. Therefore we called this new capability 'Electronic Document Interchange.'
"But the underlying document model driving the process stayed the same. We called it 'Paperless ordering,' or 'Paperless invoicing,' yet the fundamental process flow stayed the same. Even though it enabled entirely different business methods such as 'just in time inventory,' we still had not reached that next fundamental level of understanding. This is now changing. Eureka events have taken place.
"The existence of the Internet has created the ability to re-invent the way that we fundamentally do business to make us all more interconnected, closer in time and space, with less manual work, our processes more timely, and our operations more and more streamlined."
We have only now begun to see how XML delivered via the Internet can change the way businesses interact with each other. This is a trend we can expect to accelerate. XML may have a history as a powerful tool for content publishing, but business data exchanges are now becoming XML's "killer app."
The question is, how can businesses realize the full power of XML? Using Galewski's radio analogy, how do we move from wireless telegraphy to HDTV?
The XML/EDI Group: Grass-roots Organization Championing XML with EDI
Enabling businesses to realize the benefits of XML in data exchanges was the motivation for the founding of the XML/EDI Group in July 1997. The grass-roots group is formed around an Internet mailing list, now with around 1,400 participants spread throughout the world. The group's focus is on fusing the benefits of traditional EDI with XML, thereby making this new technology more accessible to the vast majority of companies who previously found business-to-business electronic transactions too expensive and cumbersome.
With chapters in North America and Europe, the XML/EDI Group has proposed guidelines for business data exchanges using XML, written a white paper on global XML repositories, and now seeks refinements and simplifications in XML itself to make it a better business engine. The Graphic Communications Association Research Institute provides management support for the group, while the group remains independent, receiving no funding from vendors of XML systems, networks, or services.
In this article we explore critical issues in XML and e-business, and present the XML/EDI Group's XML for E-Business Initiative.
A Simpler XML Standard Can Make the World of E-Business Happen
Establishing the XML-based mechanisms to support e-business interactions is the first critical step. It is with that end in mind that the XML/EDI Group has launched an initiative to determine XML syntax for machine-to-machine exchanges, and develop a set of recommendations on XML for e-business.
The W3C is currently reviewing a complex family of draft documents from XML technology working groups. Many of these working groups have as their goal to add greater functionality to the base XML language for document generation and processing.
While these new functions will no doubt have value in the publishing world, XML's e-business needs are more immediate and offer greater financial rewards than the simplification of syntax mechanisms.
Opportunities for XML are limited by the mechanics of the syntax and the methods and techniques they either enable or mandate. Traditional EDI technology was hamstrung in the past by arcane syntax that only experts could fathom. Much of the success of HTML, on the other hand, has come from the broad accessibility of that technology. Even lay people with little computer knowledge can create Web content. To gain and ensure broad use of XML, general users must get results as easily and consistently as they do with HTML.
The XML for E-Business Initiative will extract from the many XML syntax extensions only the relevant and critical components needed for e-business. This initiative will achieve two goals:
- Empower software developers.
- Ensure that XML parser implementations for e-business remain as simple and compact as possible (not overburdened with feature-creep).
This does not preclude developing a more complex XML syntax needed for publishing as a superset later once the urgent needs of e-business are met.
Understanding the Scope and Getting Organized for E-Business
Business data exchange using XML is moving rapidly toward becoming the de facto next generation technology. It is only a little over two years since XML 1.0 became a W3C recommendation, and already many industry groups are adopting XML-based vocabularies. Equally importantly, a broad range of software vendors are providing solid XML authoring tools and server delivery solutions.
While these developments represent a fine start, the real test of the benefits of business data exchange with XML will come from expanding its reach into the tens of thousands of smaller businesses who previously had not had the resources or tools to effectively perform business transactions with their trading partners.
In this context, we look at the needs of supporting an extended network of trading partners without being swamped by the sheer scale and number of interface combinations. Later in the article we recommend ways that the developers of XML can address these needs and provide mechanisms to support tens of thousands of interacting partners within an industry.
First, we need to understand the extent of current business-to-business information exchanges and the forecasts for the expansion in this area via the Internet. While retail consumer Web sites get plenty of media—and investor—attention, the real expanding Internet market comes from data exchanges between businesses.
IDC predicts the value of business-to-business commerce over the Internet will grow from under $100 billion in 1999, to about $500 billion in 2002, and to $1.3 trillion by 2003—truly radio eclipsing telegraphy! IDC expects retail consumer commerce over the Internet to grow to about $100 billion (about where business exchanges stand today) in this same time frame.
To achieve a thirteen-fold expansion in only four years means the number of businesses exchanging data over the Internet will expand dramatically. That expansion will be lead by smaller businesses who are driving the wholesale move to the Internet as a means of conducting business.
Having identified the present and future scale of the world of e-business, we need to define its characteristics.
- Reliability that builds trust
- Security that builds confidence
- Manageability that ensures performance
Trading partners want to exchange information both inside and outside their industries with ease. Customers, both business and consumer, want to assume these services work automatically and are as simple to use as a telephone or fax machine is today. For example, fax machines work equally well for domestic or overseas transmissions: exchanging business data outside one's normal circle of trading partners should not require an entirely different protocol from that used for everyday transactions.
Creating XML/EDI specifications that encourage this kind of transparent interoperability will require a focused approach based on an understanding of global business needs—not just North American—as well as a new kind of collaborative relationship among companies creating XML specifications. So far, American companies appear fond of creating business consortia designed to create competitive advantage in their own local markets. But as we have learned in the past five years, the Web, and the commerce it has spawned, truly are world-wide.
The Rise of Business Consortia
Business consortia represent companies in their own industries, as well as those across complementary industries and boundaries. The trend over the past year and a half has been for such consortia (often led by traditional brick-and-mortar trade associations) to take advantage of Web-based collaborative tools, Web sites, Internet listservers, and e-mail to replace the extended comment and review periods of traditional standards bodies.
Despite the emergence of consortia, there is still a role for traditional industry standards bodies, particularly in defining the underlying interoperability technologies. However, they need to take greater advantage of Internet-based collaborative tools to break through the limitations imposed by traditional paper-based requirements for public notice and comment.
Business consortia define scenarios, messages, content models and data elements needed for doing business electronically. Where previous EDI standards exist, they can build on this valuable experience. They can also extend the reach beyond current EDI trading partners, as well as fill in the gaps that predefined EDI transactions have left out. Examples of these collaborative business consortia are RosettaNet in the computer technology industry, the Open Travel Alliance (OTA), and the Interactive Financial Exchange (IFX). OTA and IFX currently work with the Data Interchange Standards Association, the organization that manages the North American standard (X12) for EDI.
Achieving Integration and Interoperability: Moving Beyond EDI
EDI helped pave the way for this looming breakthrough of true e-business. Well before the Web and even the Internet as we know it existed, companies exchanged data using standardized transactions tailored for each industry. Many companies in the automotive, grocery, and electronics industries, among others, achieved significant savings and improved business processes using EDI. In the early 1980s, the American National Standards Institute (ANSI), seeing the benefits of standards in electronic business, established an accredited standards committee for EDI and gave it the designation X12. The X12 standard lead the way in defining and promoting fixed EDI transaction sets in North America. Similarly, the UN/EDIFACT standard is used elsewhere in the world. Since 1987, the Data Interchange Standards Association (DISA) has managed and published the X12 standard.
Many of the current XML initiatives are still wedded to these EDI ideas of fixed transactions, now set down in XML syntax instead. While in some cases businesses need to exchange predefined collections of data that resemble their hard copy or EDI ancestors, companies will need to exchange data more broadly, more frequently, and in different ways than before. For example, if business partners want to tighten the supply chain relationship, have joint planning and forecasting, and more frequent replenishment of inventories in a way that reduces overall costs (see http://www.cpfr.org/), then they need different kinds of data exchanges. Materials, transportation, and warehousing vendors will need a real-time, or close to real-time, view of inventories to get the right materials to the right place at the right time. In these scenarios, traditional EDI-style approaches will not provide enough flexibility, scalability, or maintainability.
One lesson that XML enthusiasts can learn from EDI is the importance of standards to encourage companies to invest in technology. Without standards, each company (not only each industry) will fashion its own way of exchanging data; the result is unmanageable gridlock.
XML brings closer the promise of data integration that EDI enthusiasts often talked about but rarely achieved. EDI transactions constitute a flow of information from sender to receiver. In contrast, the new needs of Internet-driven business relations call for integrated end-to-end information processing. This will allow trading partners to build the logic for integrating e-business data flows into their systems. For example, XML banking vocabularies such as IFX provide for electronic bill presentment integrated with payment and inter-account transfers. Earlier X12-EDI transactions offered electronic invoices, credit-debit adjustments, and payment orders, but did not tie them together with the payment process.
Talking the integration talk is one thing; walking the walk is quite another. To achieve this kind of integration and interoperability requires a fusion of five sets of ideas and technologies involving XML/EDI:
XML: to take advantage of the Internet for transport and delivery, coupled with using metadata for identifying business objects
EDI: for the defining of, and linkage to, business transactions and processes
XML repositories: for storing the glossaries that define the information content used in specific business processes
Templates: for defining and implementing the processing rules and information interactions
Software Agents: to provide control, facilitation, and scalability
XML.com recently discussed the symbiotic potential between XML and EDI; see XML and EDI, Lessons Learned and Baggage to Leave Behind.
In the next section, we take some of these ideas further.
The XML/edi Group's XML for E-Business Initiative
Industries and individual companies need to establish standard mechanisms for making XML work for business data exchange. As we have seen with EDI, standards encourage businesses to invest in data exchange technology and help draw in vendors to develop packaged software that supports those standards. The XML/edi Group, among others, have been focusing on key enabling technologies for delivering on such functional XML/EDI standards.
An overview of the first draft of the XML for E-Business recommendations is presented in Table 1, reflecting a series of functions based on business needs. In the rest of this section, we expand on certain key elements of these recommendations: XML repositories and Bizcodes. We then look at the importance of maintainability and scalability in XML/EDI applications.
Table 1: XML/edi Group's XML for E-Business Recommendations
|Enable XML repositories and XML glossaries with Bizcodes to create a basis for simple automated mapping between XML vocabularies, dialects or schemas, and DTD and EDI (code and element) re-use.|
|Utilization of the XLink and XPointer specifications for implementing Bizcodes, which
link elements to their definitions in repositories; see the XML/edi Group's Repository White
|Ability to validate application interchange elements via DTDs and ATTLISTs, without needing complex programming constructs and or additional parser helper components.|
|Basic datatyping with five primitive types as per the 24 September
XML-Schema working draft (byte, date, integer, sequence, SQL and Java), plus a Picture
Mask definition based only on the following picture codes "#$XNULBPV.,/-" as a
direct means to express e-business field format rules within a DTD or
|Open ability to exchange authenticated and secure transactions across all XML transaction servers.|
|Digital signature, and authentication certificates as detailed in the
Extensible Forms Description Language, XML For All, and BizTalk methods; looking at
these methods, a single implementation may be derived for common use.
|Ability to exchange and route transactions between XML transaction servers. This will allow Information and Content Exchange, Biztalk, Java Message Service, Interactive Financial Exchange, RosettaNet and EDI messaging servers to talk to each other with one consistent envelope based on XML. It will also ensure a firm minimum for the legally required tracking information associated with messages that vendor products should handle.|
|Common XML syntax for interchange envelopes, with common persistent
tracking and registration authority recommendations for open transaction interchanges.
Use of standard business methods, such as Dun and Bradstreet and UNSPSC codes, for
company identification plus a mechanism for ISPs to set up their own business
interchange registration companies using a consistent format.
|Open the syntax for creating SQL statements that will work against a variety of back-end XML-based database products. This step will greatly simplify implementation, maintenance, and training. It will also ensure broad and rapid adoption, and allow robot and/or 4GL software to consistently generate valid access statements.|
|XML extensions to SQL via the CONTAINS verb as implemented by Oracle.
This feature provides a robust and immediate means for SQL-based access querying to
content stored in a database column or object class.
XML Repositories Store the E-Business Valuables
Once XML vocabularies have been developed for the data elements exchanged in business processes, there is a need to have them available for use. This is the function performed by a repository. Data definitions are stored in the repository for retrieval and use by e-business applications. In particular, a network of relationships between data definitions is built up, with the re-use of element definitions from other industries, and the establishment of data elements between which there is an effective equivalence.
We propose that to document these relationships, repositories rely on semantically neutral references called "Bizcodes". Just like the familiar UPC/EAN bar codes for products and items that provide simple semantically neutral numbers for manufacturers, distributors, and retailers to use for inventory control, Bizcodes provide neutral identifiers for relating data elements among electronic transactions and across industries.
For example, a party that engages a company to acquire goods or services could be called a customer, client, guest, buyer, renter, visitor, or user depending on the industry. XML vocabularies for these industries would identify these entities using the terms most familiar to those industries: <CustomerNr>, or <ClientNr> or <BuyerID>, and so on. Each industry may use its own vocabulary (with different terms and tag names) but they all essentially reference the same thing.
Traditional EDI has addressed this consideration with code and element dictionaries, but has not provided the roadmap for the cross-referencing. Instead, individual EDI tool vendors have provided their own industry specific vocabularies mapped to the standards.
In XML parlance, this integration of the terms, and the data model along with the business use semantics, is known as a schema. So a vocabulary combined with data structures and business rules can be expressed in XML syntax using schema definition rules. In the XML 1.0 specification the schema modeling tools are known as the Document Type Definition (DTD) syntax. The W3C's draft XML-Schema specification will extend the modeling capability far beyond what is currently possible with DTDs.
In the XML repository for each of our example industries, a glossary would equate a specific identifier to the equivalent Bizcode. An industry could create a label for the Customer/Client/Buyer identifier example and call it "BE000123". A company from another line of business that needs to interact with companies using that industry's XML vocabulary could precisely relate its customer or buyer or client identifiers to BE000123.
As a result, trading partners in a vertical industry can use the business terms and rules with which they are familiar, yet still relate to allied businesses. In the financial world, stock brokerage houses can relate to personal banking integrators and to financial regulators, where each group uses related information that overlaps across the business domains. Financial industry associations through their XML syntax working groups are already looking into these ways of interacting.
Another important use of Bizcodes is to locate, re-use and transform information (often called mapping) both within business transactions and more broadly across the web itself. An example is product catalogues. A vendor may have a catalogue displayed on the web using its own vocabulary, such as <ProdCode>, <Cat>, <InStockQty>, <Delivery>, <Product>, <ListPrice>, <WebDisc> and <TechSpecs>. From their published Bizcode references (these can be conveniently documented in an XML DTD), software can then automatically relate these references to a search process that locates customers for specific products.
In a global economy, Bizcodes can also provide an effective method of translating between languages. A customer in Japan, for example, could see search results displayed in their own language; replacing the original English terms. Notice in this context that Bizcodes can work more robustly and directly than traditional vocabulary labelling techniques, or mapping the vocabulary itself, where any variance in the spelling or naming will cause significant failures to occur.
Technically, Bizcodes are implemented using fixed attributes in DTDs. More information on repositories and Bizcodes is available in the XML/edi group's XML Repositories white paper. Note that Bizcodes are here introduced as the "ATTLIST mechanism".
Data Exchanges Must Be Scalable and Maintainable
At the beginning of this article we saw how IDC are predicting a major upsurge in business use of XML and the Internet. To successfully expand traditional EDI style systems to a global audience of tens of thousands of trading partners means developing and deploying new, innovative, and very robust software processes. Simple arithmetic is at work here. If your current EDI department processes 50,000 transactions daily, working with 200 trading partners, will that support group be able handle ten times that number of partners? Would you, or could you, hire more people to accommodate that volume?
Clearly the new breed of XML/edi systems must be flexible, and able to handle a large expansion of exchanges without excessive human interaction. This is why there is a need to understand how we can integrate software agent technologies into XML/edi systems.
Up to now, the picture seems discouraging. Despite the grim lessons of Y2K compliance that have cost businesses billions of dollars, many software development teams are still happily creating hard-coded systems with ready-made limits on their ability to handle new and different business data.
We are not talking about highly advanced capabilities here, but simple measures like avoiding creating an information system that will only handle 2 address lines in an address, or no more than 10 items per order, and so on. What happens when an order with 11 items, or 3 address lines comes in?
XML provides the means to express metadata without specifying hard-coded constraints. This means that developers building new XML implementations should avoid hard-coded constraints in the XML content to prevent such problems in process models.
Creating XML software systems that are significantly flexible, fault tolerant and scalable is a design and engineering challenge. Software agents can help this process. The most immediate need is for tools that can track XML transaction content and alert support staff to potential failure conditions in the information, particularly relating to versioning. Traditional EDI implementors have long understood the need to manage versioning between trading partners. In an XML-driven world, such changes are exacerbated by the ease that trading partners can change and revise the details in their transaction content and structures.
XML for E-Business: A Growing Concern
In addition to the work being pursued by the XML/edi Group, other groups are recognizing similar needs and requirements. Among initiatives in this are CommerceNet's eCo Framework and the ebXML initiative launched by the United Nations' CEFACT and OASIS.
The inaugural ebXML initiative meeting takes place this November. The stated objectives of the ebXML initiative closely mirror those envisioned in the XML for E-Business Initiative. What may differentiate the two is in that the former has a higher level, top-down approach, while in the XML/edi Group's work there is a detail-oriented bottom-up approach. As such, both approaches are complementary.
Further Thoughts and Review
The W3C and other bodies must recognize the needs of doing business with XML when crafting further refinements to the XML standard. They must ensure that the emerging complexity found in extended portions of new working drafts on XML technologies does not derail the process of providing simple, consistent, light weight, easy-to-learn and broadly maintainable e-business systems.
Michael Swaine, editor-at-large for Dr. Dobb's Journal, in his November 1999 DDJ column "Programming Paradigms", puts all this very succinctly while discussing the Oxygen Project at MIT (introduced on MIT's site in this transcript). Under the heading of Attaining Invisibility, Swaine writes: "The goal of MIT's Oxygen Project is to push computer technology and information technology beyond what passes for ease-of-use today, making it not only accessible to everyone, but richly exploitable by everyone".
Swaine then identifies the key technologies that will deliver on this promise. "The five technologies of speech (and other perceptual capabilities), knowledge access, automation, collaboration and customization are the only new kids on the block. Out of the thousands of things that we can imagine doing in the new world of information, these five are the foundations on which any new activities that help us do more by doing less will be built".
Doing more by doing less—this mantra is central to the vision for XML/edi. Realizing e-business is an early step towards a world of transparent computing envisioned by the MIT project.
Much work in e-business remains to be done: XML repositories coupled with Bizcodes by themselves cannot capture all the aspects of automating a data integration process. When an application system transforms and re-uses information across domains, both structural and business semantic changes occur, and software may also re-combine data to produce new data.
However, our objective here has been to introduce the core concepts behind the XML/edi Group Initiative on XML for E-Business, and show how these aspects are critical for implementing broad-based global electronic commerce via the Internet.