Quantum computers, operating in a realm where classical physics does not apply, will intensify these developments, enabling us to look at reality as if through a magical microscope, untangling the seeming chaos of traffic patterns or the heretofore indecipherable pulse of global markets.

As we move into this confluence of worlds with the union of spatial, cognitive, and distributed computing, how systems will be designed and architected needs to radically change. However, we are not that certain about how many people really understand this. Most people assume that “the technologists are taking care of it.”  They take it on faith that the best possible designs for the future of digital infrastructure, computing and networking will emerge from large incumbent players.

But those are big, unfounded assumptions.  In fact, most entrenched players are showing little appetite for radical departures from current practices. Yet current practice will not serve the needs of a genuinely connected world.

Quantum computers, operating in a realm where classical physics does not apply, will intensify these developments, enabling us to look at reality as if through a magical microscope, untangling the seeming chaos of traffic patterns or the heretofore indecipherable pulse of global markets.

As we move into this confluence of worlds with the union of spatial, cognitive, and distributed computing, how systems will be designed and architected needs to radically change. However, we are not that certain about how many people really understand this. Most people assume that “the technologists are taking care of it.”  They take it on faith that the best possible designs for the future of digital infrastructure, computing and networking will emerge from large incumbent players.

But those are big, unfounded assumptions.  In fact, most entrenched players are showing little appetite for radical departures from current practices. Yet current practice will not serve the needs of a genuinely connected world.

COMPLEX ADAPTIVE SYSTEMS REQUIRE ORCHESTRATION

Before delving further into the new thinking that makes this story possible, let’s talk about why it’s necessary at all.

People understand digital systems are interwoven into nearly every aspect of life and increasingly expect more from the technology they consume. And yet digital architects and developers struggle to build new innovations on top of legacy technologies that simply cannot support the required performance, scale and flexibility necessary for modern digital services. To meet the demand for new digital experiences and deliver technology solutions that improve people’s lives, we need to better align the foundational elements of our digital infrastructure. This represents a true shift in how we collectively think about the relationships between and among the core technologies that enable and inform our digital world.

Our evolving “thesis” goes something like this: As the complexity of these systems increases, the number and diversity of stakeholders expands, and the volume and nature of their interactions grows. The systems technology and architecture will, by necessity, need to become more aligned and, by necessity, become more tightly coupled. Better alignment and orchestration of core technologies will, in turn, inform radically new modes of collaboration, including increased sharing of system data and services. but only if these innovations are better orchestrated and become mutually supportive without inhibiting one or the other.

Some basic design principles must be put in place to guide the growth of this vast, distributed system that must remain organized as it evolves according to a logic all its own. It demands that we think about architecture in a completely new way, one that addresses the loosely coupled and misaligned nature of today’s hardware, software and network technologies.

These combinatorial innovations are what we call “complex adaptive systems” where multiple technologies converge and reinforce one another. In other words, a system of systems where the relationships and arrangement of independently operated systems allows for collaborative interactions among them, giving rise to emerging capabilities and values while maintaining each of the individual system’s own capabilities and values.

Whatever we call these opportunities—smart complex systems, adaptive or autonomous systems, or system of systems—they all presuppose technologies, standards, and tools that make integrating data, hardware, software, systems and people cheap, easy, and seamless. How will the physical and digital integration aspects of these new opportunities actually take place? How will virtually any device – right down to a lowly light bulb – become a peer that connects at will to the global data network?

Obviously, billions of devices of wildly varying types cannot each receive individual attention and configuration, or conform to elaborate a priori specifications. If it literally takes a network engineer to screw in a smart light bulb, digital systems are never going to work.

EVOLUTION OF HARDWARE and SOFTWARE TO ENABLE AI, ML & NEW USER EXPERIENCES

In a world driven by data analytics and diverse workloads, the need for compute performance has spurred an onslaught of competing processor designs and architectures to help accelerate digital services.

The days of ‘one size fits all’ processors and hardware are gone. Now we see more and more specialized hardware designs to address very application-specific compute requirements. CPUs are being combined with GPUs along with FPGA solutions, ASICs and custom designs to accelerate compute performance for applications and use cases such as artificial intelligence (AI) and machine learning (ML) workloads and new multi-modal user experiences including augmented reality (AR) and virtual reality (VR).

The past decade has seen an explosion in processor parallelism with the move to multicores. The next major architectural trend is the move from symmetric multiprocessors to heterogeneous systems, with multiple different processors and memories. These new silicon architectures enable the acceleration of data-parallel applications such as deep learning, image and video processing, and financial applications.

The challenges with modernizing digital infrastructure don’t end with hardware innovations. Each new wave of hardware is inevitably tied to and integral with cloud computing and networking performance, all of which requires new software. The exponential pace of these parallel innovations is making software development untenable.

The impacts of these technological advances are twofold: On the one hand, new software technologies are enabling many new business and consumer-focused applications including AI, robotics and Internet of Things (IoT) solutions, as well as augmented reality and virtual reality. On the other hand, they have greatly increased the complexity of software development up and down the technology stack.

Applications are now distributed and dynamic in nature, needing to work across different cloud configurations, diverse locations and devices. Software development needs a completely new approach – one where open-source software development and libraries are modular and interchangeable and where applications can be decomposed into microservices and components that can be assembled and then re-used. New development frameworks will need to simplify application development and shorten time to market by better organizing developer tools and run-time support for multi-cloud, edge, IoT and application-specific hardware devices.

Enterprises have diverse users, functions and entities, all with an overabundance of data flows and interactions. Today, many organizations are struggling to turn the operational data generated by their people, machines and fleets into tangible business value. Data is often trapped in machines, equipment and incompatible systems, or stored locally on workstations and drives. Extracting value from diverse data types and disparate data sources requires special skills that are in short supply — cloud server provisioning, data science, multiple programming languages and more.

What’s required are orchestration tools that creatively combine multiple innovations in cloud infrastructure management, workflow automation and data application development to reduce complexity and better leverage architects, developers, integrators, subject matter experts and business users to rapidly and collaboratively build diverse distributed data applications.

Today, relatively few companies understand the necessity of re-thinking how their systems and applications will be developed, consumed and managed. Even the latest platforms, cloud-native solutions, edge, and IoT device development platforms have shortcomings. They tackle only bits and pieces of the problem, but do not address the overarching complexities of software development, tooling, testing, security and monitoring.

In the age of smart distributed systems, simply “bolting together” diverse edge devices ranging from smartphones to IMECS and integrating them onto 5G radio-area networks (RAN) and then trying to integrate these devices with distributed application technologies enabled by micro-container services or with data intensive AI or IoT applications doesn’t add up to a coherent architecture. Similarly, integrating a diverse hodgepodge of technologies will not render a seamless user experience or enable the metaverse environment we are anxiously awaiting.

DIGITAL ARCHITECTURE COLLABORATION

We are at the dawn of a new age of information and infrastructure technology in which distributed networked systems combine with pervasive to take the notion that the network is the computer to new heights. These systems will require new hardware designs, better software development tools and more dynamic networks that overcome infrastructural limitations.

Today, multiple parallel technology developments appear to be increasingly reinforcing and accelerating one another. But are these innovations actually aligned and will the building blocks of digital infrastructure really play well together?

Perhaps the most important impact resulting from the shift to digital systems has to do with how these technologies are developed and the architectural relationships between and among the differing technologies. Networking, computing, applications, device management and identity have all, for the most part, evolved in relatively autonomous development paths. We strongly believe this will need to change.

The two biggest challenges with developing digital infrastructure are: 1) aligning players and developments with an agreed upon architecture; and, 2) synchronizing developments across the ecosystem of core technology developers. This type of coordination and collaboration requires a subtle dance of alignment, timing and delivery – the ability to see where, and how fast, capabilities are progressing and thus align development work and also manage complexity.

All of the traditional categories of players that are theoretically driving digital infrastructure innovations (IT systems, telecom systems, automation and control systems, software development and more) have historically operated within well-established technical development protocols and business models that reflected the distinct competencies each group has developed over the last 20+ years. In short, they are trapped in their own “sunk cost” economics and blinded to any alternative future beyond the linear development path they have been on.

As the number and diversity of stakeholders expands (users, architects, developers, supporters, etc.), and the volume and nature of their interactions grows, the discrete technologies that comprise digital infrastructure will need to become more and more tightly coupled. Each core technology must be viewed in close proximity to all of the other core technologies and, by necessity, need to be mutually supportive without inhibiting the other core technologies.  However, trying to coordinate and align the respective roles of each core technology often creates contention.

We are coming to see the continuously evolving relationship between these core technologies as fertile ground for innovation.  They need to be interwoven and mutually supportive to leverage their combined potential. However, trying to coordinate and leverage the respective roles of technology architecture often creates contention.

As is often the case, all of this adds up to the need for a new architecture for distributed systems that is a profoundly different architecture than what’s in common use today and new types of relationships that foster cooperation.

The shift to truly intelligent distributed digital systems requires a new generation of architectural capabilities that are designed and built from the ground up. As we enter the middle of the third decade of the 21st century, many of our biggest challenges with developing digital infrastructure still originate directly from our inability to creatively collaborate to solve diverse technical issues. ◆

Stream our latest Future Perfect Tech podcast