Connecting the Data Dots – Integration’s Important Role in the Freight Technology Revolution

(Originally published in Freight Revolution, 2018 Edition by Freight­Waves and Über Freight)

Let’s take a trip back in time. As we travel a century or two into the past, consider for a moment some of the natural resources and rare earth elements that form the basis of modern technology: Lithium. Beryllium. Cobalt. Titanium. All unarguably integral to semi­con­duc­tors, smartphones, aircraft and alternative energy. And all nearly inac­ces­sible (and unprocess­able) until now. They’ve always been there, of course, but these raw materials laid in wait until the technology was available to act upon and transform them into finished products.

The freight trans­porta­tion industry is on the precipice of broaching this same frontier with data. Data is everywhere, and data has always been everywhere. Freight data is ubiquitous, it is useful, and acting upon this data has the potential to streamline operations, increase efficiency and spark new growth within freight trans­porta­tion like nothing else before it – save for the internal combustion engine.


The industry’s ability to access, interpret and effectively utilize all of this data, however, is still in its infancy. The problem with data being everywhere is that it’s, well — everywhere, and it exists in all shapes, sizes and schemas. In a perfect world, the freight industry would confer to collaborate and construct a single unified schema to unite the disparate data streams generated from classic EDI, ELDs, modern APIs, the IoT and the imminent transition of data integrity to the blockchain-based ledger.

It’s estimated that the move to a universal data standard and commu­ni­ca­tion methodology will occur right around when pigs begin to take flight. In the meantime, the expanding role of middleware and integration technology providers will become the key to connecting the disparate streams of freight data being unleashed upon the industry and exchanged 24/7/365 between shippers, carriers and their trading partners. 

Why will middleware providers still be needed? Isn’t it the goal in and of itself to eliminate as many potential layers as possible between partners? 

Data integration and translation between various appli­ca­tions, partners, protocols and formats requires a specific domain expertise, and devoting in-house technology resources to tackle project-specific tasks can be a risky venture during a tran­si­tionary period such as the data and technology revolution the freight trans­porta­tion industry is currently undergoing. Standards may shift (or fade), various partners can present a wide variety of tech­no­log­ical require­ments or mandates, and an ever-tightening market for the human resources required for imple­menting data integration technology can result in a less-than-fruitful hiring endeavor. The ability to implement and the agility to switch providers, appli­ca­tions or approaches to technology to accommodate partners will be crucial to navigating this tran­si­tional phase of the data revolution. 

The ability to implement and the agility to switch providers, appli­ca­tions or approaches to technology to accommodate partners will be crucial to navigating this tran­si­tional phase of the data revolution.

Freight Revolution 2018 Edition

We can explore two potential use cases in the freight trans­porta­tion industry that exemplify the benefits of working alongside a middleware or integration provider in order to leap the technology gap in a more agile manner, while also minimizing the exposure of investing in new technology while standards and best practices are in flux.

The first use case details a freight trans­porta­tion brokerage fully invested in the EDI (Electronic Data Interchange) ecosystem in order to integrate data with carriers, shippers and freight appli­ca­tions around the globe. They explore moving to a fully API-based system to exploit the real-time capa­bil­i­ties inherent in application-to-application commu­ni­ca­tion but quickly learn that while the technology exists, only a select few partners are capable or willing to utilize it, there is a learning curve to imple­men­ta­tion, and they are lacking the internal knowledge necessary to build out the project. In this scenario, a middleware or integration provider bridges both the knowledge and technology gaps for the brokerage and empowers them not to replace, but to extend their existing capa­bil­i­ties by providing a flexible solution that can integrate all flavors of data formats, document types, and commu­ni­ca­tion protocols while scaling with their needs in an agile manner.

The second use case entails the imminent rise of blockchain technology and the competing interests certain to be at play during the frontier days of the distributed ledger.

In a perfect world, freight shipment data, documents and trans­ac­tions would feed into and reside on THE immutable, centralized public blockchain. In reality, private blockchains have gained a foothold as individual shippers, corpo­ra­tions and part­ner­ships prototype similar yet separate solutions to maintaining supply chain data integrity. While the current use of blockchain-based ledgers to record trans­ac­tions is far from mandatory, the diverging development presents the interesting scenario in that customers and trading partners of these firms may not only be required to have the ability to place data onto a private blockchain, they may have to allow for the capability to integrate with multiple ledgers mandated by their various partners.

Once again, the willingness and ability to engage a middleware or integration provider capable of bridging this technology or skills gap will enable the hypo­thet­ical firm to take the leap head-on into the fascinating new world of freight data and application integration without the potential risk of placing their eggs entirely into one basket. An extensible third party will not only be able to manage the various rela­tion­ships, the integration provider’s breadth of knowledge, experience and technology lends a degree of agility simply not found within the in-house imple­men­ta­tion process.

Regardless of whether you decide to partner with a third party or blaze your own trail in order to take advantage of new tools, this is an extremely exciting time to be on the front lines of the freight trans­porta­tion data and technology revolution. The readiness to finally take advantage of millions of miles of data is palpable, and the ability (and willingness) to finally shake off the industry’s stodgy, technology-resistant reputation in favor of cutting edge method­olo­gies and protocols like the blockchain and APIs is clearly within reach.

Author

Phil Johnson

Director of Marketing, Kleinschmidt