the Padi platform: open-source tools for open data
We are at the cusp of a “perfect storm” of smart systems innovations. Multiple parallel technology developments, including data modeling and machine learning, are accelerating and enabling more complex and adaptive systems such as digital twins. Along with the value these innovations bring, however, grows the complexity of connecting and integrating machines, equipment and data in a meaningful context. A new open-source software tool, Connection Profile, enables simple, durable and context-sensitive integration between complex systems without any wasteful custom development. It is a great leap forward to the original vision of the internet as a single, seamlessly integrated cloud.
THE REALITIES OF “CLOUD COMPUTING”
Back in the 1990s, when the web (as opposed to the internet) was just getting going, people used to represent the internet with a little cartoon of a cloud. Countless presentations to venture capitalists featured little pictures of desktop computers, modems, hard drives, etc., all connected by wiggly lines to the cloud-cartoon. When technologists needed investment, they would just draw a diagram where everything was connected to a cloud. Here’s a classic example:
Despite the naiveté of these pictures, they portrayed an important truth: There was one cloud, not many. Microsoft Cloud did not exist back then, nor did Apple Cloud, Google Cloud, Amazon Cloud, Salesforce Cloud, or ABC Widgets Cloud. There was simply “the cloud”—the one and only shared cloud of information, the common property of humanity. Nobody owned, or could ever own, “the cloud” of the 1990s dot-com diagrams.
Further, that singular cloud embodied a huge idea about connectivity: When you connected to the one-and-only cloud, you were connected to everything else that was connected to the cloud. Like all peer-to-peer (P2P) architectures, the original design of the internet embodied a radically decentralized view of computing. In the internet depicted in that early cloud, your personal computer wouldn’t have “information in it.” Rather, your personal computer would be “in the information.”
In a mere twenty years that idea has been completely inverted. Even though cloud providers would argue that URLs and Linked Data (LD) exist to link everything, the private clouds of today have no inherent ability to interoperate significantly with other clouds, private or public. These private clouds are all based upon proprietary designs built to compete, not cooperate. Cloud-computing is in fact the same old corporate client-server computing we had before, just dressed up in cloud-clothing.
ENTER THE PADI PLATFORM
The latest integration platform to cross our radar is called Padi, which is the brainchild of IoT and automation pioneer Anto Budiardjo. Padi means “rice plants” in Indonesian, and the rice-bowl in the company’s logo represents the thousands of “grains” of device information that are of interest at any moment, compared to the billions of grains that are irrelevant to a given user.
The purpose of integration platforms is to let system developers work more collaboratively by simplifying the bewildering number of systems and devices they have to deal with on a daily basis. Budiardjo realized some years ago that most integration platforms, though they used the internet to some degree, were not genuinely internet-centric. They kept solving the “integration problem” over and over again but often with too much customization.
Existing technology has proven cumbersome and costly to apply with many conflicting protocols and incomplete integration tools. Padi, by contrast, integrates data flows in a reusable way with generalized, extensible concepts similar to those that underlie the internet itself and use an innovative open-source concept called Connection Profiles to allow the connection and integration of any internet endpoint to any other.
THE PROMISE OF MACHINE DATA and DIGITAL TWINS
Before delving into the new thinking that makes this story possible, let’s talk about why it’s necessary at all. Machine data can offer extraordinary business advantages to both the companies that manufacture and support machines as well as the users of machines. The ability to detect patterns from aggregating data is the “holy grail” of smart systems. New machine data and learning technologies enables not only data patterns but a much higher order of intelligence to emerge from large collections of ordinary sensor and machine data.
The fact that a rapidly expanding range of devices have the capability to automatically transmit information about status, performance and usage and can interact with systems and people anywhere in real time points to the increasing complexity of adaptive machines and systems. The emergence of digital twins is a very good example of this.
A TURBINE AND ITS DIGITAL TWIN
What is a digital twin? The Digital Twin Consortium (DTC) defines a digital twin as “a virtual representation of real-world machines and processes, synchronized at a specified frequency and fidelity.” Digital twins of complex machines or processes actually consists of many models of component parts and sub-systems, each one pinned to measurement data collected by sensors. Digital twins can include current actual operating information, as well as analytics capabilities and a recommendation engine to enable real-time operations. Whether driven by historical or live data-feeds, they provide users with visibility into how the machine or entity is performing at any given moment, as well as what to anticipate in the future.
Digital twins are combining networking, software, data and analytics innovations in new and unprecedented ways to optimize complex machines and systems.
But are they?
CONNECTIVITY and INTEGRATION CHALLENGES CREATES DIGITAL ORPHANS
Whatever we call these opportunities—the Internet of Things, digital twins or machine learning—they all presuppose technologies, standards and tools that make connecting equipment and integrating machine data, cheap, easy and seamless.
How will the physical and digital integration aspects of these new opportunities actually take place? How will virtually any device, right down to a lowly light bulb, become a peer that connects at will to the global data network?
Obviously, billions of devices of wildly varying types cannot each receive individual attention and configuration, or conform to elaborate a priori specifications. If it literally takes a network engineer to screw in a smart light bulb, digital systems and twins are never going to work.
Many schemes and so-called “standards” for device connectivity and device data integration already exist. But of course, all those “solutions” add up to one big problem. We don’t want many standards; we want one.
For too many years now, systems developers and users have faced the rapid proliferation of many random endpoints trying to connect to other endpoints in the absence of any meaningful context. The PADI platform’s magic lies in how it achieves its connections among diverse equipment and systems. Its designers set as their goal the Shangri-la of integration—connecting anything to anything, no matter what or where a given “thing” was. This goal came out of Budiardjo’s long work in the device integration field, where he spent many years designing specialized drivers and one-off middleware for project-specific issues. As anyone who has done this type of work knows, the solutions to these difficult challenges are not immediately reusable in other contexts, and the engineering also tends to be brittle and break under the slightest stress—for example, a single URL changing.
To be successful, integration platforms need to go to a much deeper, more abstracted level. The mechanism that its creators came up with—Connection Profiles—are not defined for specific projects, but rather for specific use cases—for example, to connect a certain type of client to a certain type of server for a specific purpose. This is done in a way that allows that Connection Profile to be used for any instance where a client and server of the same specifications are being connected.
Connection Profiles itemize the characteristics of a “client” (for example, a device) or a “server” (for example, a data repository). (Note, however, that “client” and “server” are terms used largely for convenience. Any entity utilizing Connection Profiles could function as a client and a server simultaneously.) Connection Profiles have unique names, just like internet domains, and there is a registrar of Connection Profiles to ensure that they retain the necessary clarity.
Finally, integration platforms need to maintain a connection instance over the course of its lifetime. Anyone with real-world experience integrating internet-connected devices will immediately recognize the importance of this statement. Things in any system change over time—URLs get modified, software gets updated, something physical gets broken or replaced, and so on. With Connection Profiles, the connection becomes dynamic. The server tells the platform that X or Y has changed, the platform tells all clients, and within moments the connection is up and running again—with minimal downtime and no human intervention. You can swap pieces of equipment or modify the software, and the connections will continue to work.
In the world before Connection Profiles, you’d decide on a project-by-project basis that this needs to talk to that, and you’d begin a very involved and expensive development cycle involving engineers in a room writing specifications, deciding what protocols or standards to use, and then getting custom code written. Then you’d have to install it, with people from both sides of the equation present, and pray that for the life-cycle of that equipment (often decades), the integration would continue to function.
Many consultants make a lot of money doing this type of integration work, and Padi—with its open-source and extensible Connection Profiles—is a disruptive technology in their world. Instead of specifying a project-specific specification, you create a Connection Profile instance that says, in essence, “I want to connect a train-ticketing system with an energy management system.” Any necessary development becomes part of a product-development process, not a project-development process, and all development is reusable and can be made available again and again. At installation time, the systems are connected with minimal if any configuration, such configurations being project-specific information.
This essay is supported by our white paper on Padi’s open-source Connection Profiles.
Fill out the form below to download the paper for free.