EDI, Warts and All
August 4, 1999
The Data Interchange Standards Association (DISA), the not-for-profit group that oversees the development of EDI standards in the United States, defines EDI as:
the computer-to-computer exchange of business data in standard formats. In EDI, information is organized according to a specified format set by both parties, allowing a "hands off" computer transaction that requires no human intervention or rekeying on either end. The information contained in an EDI transaction set is, for the most part, the same as on a conventionally printed document.
This definition helps explain why EDI has had success, at least with Fortune 1000 companies, but also has the seeds of many of the problems it faces . First, consider a little history of how and why EDI developed.
|"In EDI, information is organized according to a specified format set by both parties, allowing a "hands off" computer transaction that requires no human intervention or rekeying on either end."|
A dedicated group of visionaries led by the late Ed Guilbert of Washington, DC saw the need to reduce the growing burden of paper documents needed to ship goods from one place to another. Guilbert came out of the U.S. Air Force logistics command and played a key role in organizing the Berlin Airlift in 1948. He formed the Transportation Data Coordinating Committee (TDCC) in the early 1970s to develop voluntary standards for electronic formats to replace the growing pile of hard copy documents needed for shippers, transportation companies, customs authorities, warehouses, and receiving companies.
TDCC standards defined file formats and data elements for freight shipments, and continued until the early 1990s, when ANSI's Accredited Standards Committee X12 incorporated them into the overall North American EDI standards. The X12 standards identify a series of basic electronic documents called transaction sets. These transaction sets cover common business documents such as purchase orders, invoices, and manifests but also specialized single-industry documents such as those for government or health care use. EDI for health care has grown significantly in recent years; newer versions of the X12 standard read almost like medical textbooks.
The X12 transaction sets break down into data segments, consisting of collections of related data elements. The date/time data segment for example consists of data elements for:
- Date/time qualifier, a code indicating the type of date or time, such as date of the transaction or time of delivery
- Date, in CCYYMMDD format
- Time, in 24 hour format
- Time code or zones
- A date/time period format qualifier, indicating the date and time format -- various combinations of century, year, month, date, hours, minutes, and seconds
- Date/time period, an expression of one or a range of dates and times
The X12 standard is modeled on a relational database and, to its credit, makes many of the data segments and elements interchangeable. Dates, for example, appear in every transaction set, and use the same date/time segment shown above. Where a segment needs a simple date or time, it borrows those specific data elements without all of the qualifiers in the date/time segment.
|"EDI has enabled many companies to realize substantial savings on processing hard copy documents."|
For all the criticism heaped on EDI, the work of thousands of companies would look very different and much uglier if it did not exist. EDI has enabled many companies to realize substantial savings on processing hard copy documents, but that is only the beginning. Business practices like vendor-managed inventories, where suppliers keep close track of customer inventories and replenish as needed, could never happen without electronic product activity reports coming from the customer. Evaluated receipts settlements, a practice that literally removes invoices and the accounting they entail from the business process, can happen only if companies exchange purchasing, shipping, and receiving data electronically. As a result, many enterprises using EDI have streamlined their business processes and improved productivity markedly.
EDI has generated other very real and tangible results. Because organizations exchange data electronically, they can track individual units of production, raw materials for example, where before the volume of data would swamp human capacities. Graphic Communications Association (GCA, my employer) established a standard EDI transaction for transmitting paper performance data from printing plants to paper mills, where the mills can plug the data directly into their statistical quality programs. The result is a higher quality product and less downtime by the customer.
|"Rigid transaction set formats are one of the problems of EDI, but th at is also one of its strengths."|
Organizations exchanging EDI transactions can set up their systems for hands-off processing, as the DISA definition notes, because the standards provide for a predictable structure to the data stream. Goldfarb and Prescod, in The XML Handbook (Prentice-Hall, 1998, p. 103) point out rigid transaction set formats as one of the problems of EDI, but in fact, that is one of its strengths. Rigidity can also mean stability, and if a company plans to quickly process thousands of detailed transactions or line items within these transactions, they must have predictability in the incoming data stream.
While EDI deserves a lot of the credit for these improvements in business processes, it also needs to recognize its shortcomings. EDI standards are large, complex, difficult to implement, and often have high price tags attached for software, networks, and consultants. A look at the date/time segment above provides a glimpse of its complexity. The X12 standard addresses the needs of nearly all industries and businesses, and therefore needs to cover nearly all contingencies. As a result, defining something as seemingly simple as date and time requires a definitional exercise befitting a Talmudic scholar, a process repeated for almost all data that organizations want to send or receive. It can take months or even years for an industry to define an implementation of an X12 transaction set.
|"Defining something as seemingly simple as date and time requires a definitional exercise befitting a Talmudic scholar."|
People often talk of EDI standards in the plural, for good reason. While North America may have its X12 standard, the rest of the world uses the UN/EDIFACT standard for EDI. UN/EDIFACT resembles X12 in many ways but still has many differences that require companies doing business internationally to carry at least two sets of electronic formats for each transaction. X12 has implemented some of the features of the UN/EDIFACT system, but they are still different standards.
To make matters worse, each industry defines its implementation guidelines for the X12 standard differently. In many respects, one cannot avoid this situation since each industry has its own set of business rules and practices. But unless the respective industry groups developing the transactions make a special effort to agree on common definitions, interoperability depends as much on sweet good fortune as anything else. Companies serving many industries, such as warehousing or logistics services, need to carry each industry's EDI implementations in their systems. At the same time, companies wanting to move their management services to EDI will find different implementations for banking, insurance, telecommunications, and health care for example.
|"In addition to different EDI standards around the world and in different industries, the X12 standards change every year."|
In addition to different EDI standards around the world and in different industries, the X12 standards change every year. The most basic change is the addition of new transaction sets. In 1992, the X12 standard had 105 transaction sets in version 3020 of the standard. By 1998, the number had ballooned to 297 transaction sets in version 4010. Besides the new transaction sets, the technical working groups that maintain the standard will change the data segments, elements, and codes as well. For example, GCA's EDI Committee recently upgraded its transaction sets for paper inventory control. In doing this task, it found that the code for short tons (2000 pounds, as opposed to metric tons or 1000 kilograms), the most common measure of roll paper volume in North America, had changed in the X12 standard. Every company using EDI for paper purchases and shipments in the industry had to change their code tables as a result.
Using EDI day-to-day gets pricey. Translator software that takes data from legacy systems and formats them in the X12 syntax and back again, needs to change with the growing and ever changing X12 standard. Therefore it often has a high initial price tag and maintenance costs. Enterprise management software (e.g., SAP, Peoplesoft) often has EDI modules that make implementation a little easier, but they still need to keep up with changes in the EDI standards.
Until recently, organizations sent just about all EDI transactions over value-added networks that also had high price tags both for startup and ongoing operations, often charging per thousand characters transmitted. The Internet pricing model of flat monthly rates has forced most of the networks to change their pricing structures.