Envision: Text-Mining for Gold

Supply chain professionals face tidal waves of data. A new technology called text mining can help them surf those waves and find new value for their companies.

[From iSource Business, October 2001] Corporations and supply chains are awash in exponentially increasing amounts of information. Two statistics put this point in stark relief: The research firm IDC has reported that unstructured data will continue to double every two months at large companies, and Forrester Research is predicting that the amount of text-based, unstructured data will rise to 800 terabytes by 2004. To put this volume of text in perspective, it equates to 10 copies of Tolstoy's War and Peace for every person in the United States, or one e-mail per second for the next 300 years.

Supply chain professionals will not be immune to this trend. For example, the National Association of Purchasing Management (NAPM) has reported that, with outsourcing becoming a fact of life, the typical Fortune 1000 corporation has seen a 20 percent increase in the number of its suppliers over the past five years  this, despite efforts to reduce supply bases. Moreover, each time a corporation engages a new client, the company may require additional suppliers or may have to change current supplier relationships. That equates to tens of thousands of supplier files and contracts that supply chain professionals need to manage.

To date, the process of managing all the documents inherent to the supply chain has been largely manual, which is to say time-consuming and labor intensive. But a technology called text mining holds the promise of turning increasingly unmanageable supplier files into an asset for savvy supply chain professionals.

The concept behind text mining is fairly simple: software searches through electronic text (like contracts) and identifies relevant content that allows a document to be categorized and filed automatically. Later, the documents can be searched, sorted and accessed instantly. Where, at one time, contract administrators needed to perform deep-dive analyses into supplier contracts and manually pull relevant content from the contract (such as discount schedules, indemnification terms, rebate language or cancellation clauses), text mining will now obtain, organize and plot the same results on a spreadsheet at the rate of hundreds of contracts within minutes.

The technology underlying text mining is remarkably sophisticated. Text-mining software combines linguistic techniques with defined business rules to perform syntactic and semantic analysis on documents, extracting information based on context and meaning. For instance, the software can distinguish between the use of the word acquisition to refer to a business relationship (Company A is in acquisition discussions with Company B) or to a product (Company A manufactures data acquisition systems). On the other hand, text mining will understand that some words essentially have the same meaning within the context of a given paragraph, such as airplane, aircraft, plane and jet.

Seeing Into the Future

For this reason, the information obtained through text mining can be more relevant  and therefore more valuable  than simple keyword searches. The possibilities for this technology are endless. Text mining cuts through volumes of irrelevant information and focuses on what is deemed important. This type of tool can put better information more quickly into the hands of supply chain professionals, empowering them to make better-informed decisions and to focus on adding value to their companies rather than on performing clerical tasks.

In the future, supply chain professionals won't need to wade through supplier contracts, nor will they have to worry about managing the deals they negotiated. Text mining technology will manage the business relationship with the supplier by tracking the business content and communicating the supplier's performance to the contract negotiator automatically. For example, if you commit to spend $15 million a year to achieve certain discount levels, text mining software will establish trigger points to advise you periodically by e-mail as to how well you're performing. This would be especially important when a contract establishes penalty charges for missing your commitment. On the other hand, text mining software could be used as part of a supplier review process, allowing you to pull data on supplier commitments and performance from contracts, plot the results and determine which suppliers you would employ in future sourcing engagements.

This type of automation will become critical for successful supply chain management as business relationships become more complex and companies look to the supply chain to deliver greater value through the consolidation and organization of business knowledge. Although text mining is a relatively new concept, its potential for delivering productivity gains will make it a powerful tool for managing your supply chain.