UI / UX and AR / VR
Digital computing has radically transformed human affairs, but that transformation has taken place entirely on the computer’s terms. All the marvels of computing have occurred in rigidly regulated, closed systems, with IT floating blissfully above the profound messiness of reality.
What comes next will be a true paradigm shift.
XR: EXTENDED REALITY
COMPUTING’S NEXT INCARNATION WILL BE A PARADIGM SHIFT
If you want the benefits of computing, either sit down at your computer or stare endlessly into your smart phone. Those are your choices.
To this day, computing and smart phones are like a baby in diapers—cared for by people, coddled by people, tolerated by people and, rather astonishingly, people only expect it to continue to get cheaper, faster, easier to lift, and perhaps more entertaining. But they don’t seem to expect much more.
Technology moves fast but it doesn’t move instantaneously. Most teenagers in 2021 have never held a “floppy” in their hands, and yet floppy disks were essential to personal computer users for at least two decades. Back then, you loaded your file into the computer off a floppy or you didn’t work on the computer. A lot of things have gone obsolete to get us to where we are. Today, USB “thumb” drives are seeing the same gradual sunset of obsolescence that floppies underwent a generation ago.
Some consumers resent—or even rebel against—the changes to (or disappearance of!) the ports on their machines, not to mention the things they used to put into them. Even so, most people have transitioned fairly smoothly from one version of their computer to the next.
That’s because computer peripherals can change (or become abstracted into “the cloud”) without disrupting the core computing “platform” at the center of your world—namely, a QWERTY keyboard with a trackpad and a screen. The “WIMP” user-interface (windows, icons, menus, pointing device) and its attendant hardware has been modified over the years, but it’s all still very much with us—recognizable and usable but deeply confining.
That situation is about to change. Although it was unthinkable just ten years ago, today one can read the news, answer emails and texts, post to social platforms, and participate in a video meeting as easily on your phone as on your laptop—and in many cases far easier. Consider what that says about innovation and the way that the combinatorial power of technologies can build on each other. In a sense, it’s the whole story.
THE NEXT COMPUTING REALITY: SPATIAL COMPUTING
PERVASIVE COMPUTING BECOMES SPATIAL COMPUTING
We’ve been prophesying pervasive or ubiquitous computing for more than thirty years, and in that time our understanding of the phenomenon has evolved into spatial computing. In the last decade, for example, the success of various unicorn startups—for vehicle and residence sharing, home task services, on-demand food delivery, etc.—has depended upon the digitization of spatial information about the real world.
Although we’ve come a long way from the era of floppy disks, the “mobile revolution” that allows us to join Zoom meetings, edit docs and navigate from phones is still limiting. We are still doing things on the computer’s terms. Indeed, we are even more tethered to our devices than we had been before.
Fifteen years ago, it was space-age to have the web in our hands, but staring at our handheld rectangles across physical space takes our attention off the world we naturally live in.
The next leap is going to cross a much larger chasm between computers and their users—not by making us like them, but by making them closer to us. How? In the short term, this will happen by replacing their screens with head-mounted devices (HMDs) that block out ordinary reality and present an entirely different world, or that overlay onto our world real-time data-driven graphics. These visual interfaces won’t require typing on—or even touching—a keyboard. Instead, their interface will involve hand and head gestures, haptic (skin level) feedback, and voice-recognition.
VR & AR HMDs Are Driving XR Market Adoption
source: Harbor Research
In addition to specialized applications for healthcare, real estate, retail, education, engineering, etc., HMDs will incorporate all the functions of smartphones for voice communication, texting, email, video browsing, and social platforms. Once these HMDs become lightweight and affordable enough, they will be standard issue for every consumer in developed countries. And depending upon the speed of innovation, it is even possible that undeveloped segments of the world may skip the handheld smartphone entirely in favor of the head-mounted one.
In the longer term, the use cases may merge into a metaverse, a world of digital twin interactions with data driving an invisible economy.
A clip from our webinar, ”Extending Reality: The Future of AR, VR & MR“
THE PROMISE OF MACHINE DATA AND DIGITAL TWINS
Machine data can offer extraordinary business advantages to both the companies that manufacture and support machines as well as the users of machines. The ability to detect patterns from aggregating data is the “holy grail” of smart systems. New machine data and learning technologies enables not only data patterns but a much higher order of intelligence to emerge from large collections of ordinary sensor and machine data.
The fact that a rapidly expanding range of devices have the capability to automatically transmit information about status, performance and usage and can interact with systems and people anywhere in real time points to the increasing complexity of adaptive machines and systems. The emergence of digital twins is a very good example of this.
A turbine and its digital twin
source: Harbor Research
What is a digital twin? The Digital Twin Consortium (DTC) defines a digital twin as “a virtual representation of real-world machines and processes, synchronized at a specified frequency and fidelity.” Digital twins of complex machines or processes actually consist of many models of component parts and sub-systems, each one pinned to measurement data collected by sensors.
Digital twins can include current actual operating information, as well as analytics capabilities and a recommendation engine to enable real-time operations. Whether driven by historical or live data-feeds, they provide users with visibility into how the machine or entity is performing at any given moment, as well as what to anticipate in the future.
Digital twins are combining networking, software, data and analytics innovations in new and unprecedented ways to optimize complex machines and systems.
XR COMES IN SEVERAL FORMS
The way we interact with data and digital twins is already shifting to HMDs and more human-centric interfaces. Extended Reality (XR) is the blanket term for this new realm of immersive simulations, which is typically divided into three categories:
- Virtual Reality (VR) immerses the user in an imaginary or replicated world (or simulates the real world) in a way that completely blocks out the real physical environment.
- Augmented Reality (AR) layers or superimposes virtual images, video and sounds on the real world, which remains perceivable. This is most commonly seen in smartphone apps that show the real world in the phone’s camera with digital assets placed upon it. The “heads-up” displays of digital eyeglasses and windshields are other early examples of AR in practice.
- Mixed Reality (MR) is a hypothetical combination of the two, which could involve experiences like a “virtual wall” superimposed on reality that a user could actually “bump into” or bounce a virtual ball against.
Breaking down the XR universe
source: Harbor Research
XR in general broadens our visual command of the space we’re working and playing in. We are no longer conscious of, or limited by, the edges of finite screens. Further, the gestural control (via hand and head movements) used with HMDs represents a new, far from completely evolved, but very powerful method of navigating a computational environment.
That said, there are technical issues that remain to be resolved. XR visualizations are computationally intensive and require high-end hardware to produce lifelike real-time visualizations. Improvements in size, weight, battery life, and cellular technology must all be addressed for HMDs to become a widely adopted mobile computing platform. But this has always been true of digital innovation, and these are not sufficient reasons for any firm to lag behind. As we’ve seen in previous computing incarnations, hardware will quickly catch up to software requirements.
For the foreseeable future, HMDs will co-exist with present-day computing incarnations (desktops, laptops, tablets and smartphones). In the long-term, however, as HMDs become lighter, more powerful, and less obtrusive, we foresee their use by all people in every walk of life.
THE MARKET FOR XR
AR IS A GROWING ENTERPRISE MARKET
XR has the potential for massive market disruption. We estimate that VR and AR together could become an $80B+ market by 2025. Our projections for that year include:
XR market growth by 2025 in billions
source: Harbor Research
XR WILL SOON BE PERVASIVE ACROSS INDUSTRIES AND USE CASES
Although we focus on VR and AR as the major drivers of adoption of this new computing incarnation, we believe that as processor power increases and application development becomes more and more sophisticated, AR will be the more fruitful realm for business because it allows the power of computing to be embedded in normal reality.
It is still true that high adoption costs, uncertain value, and awkward user experience design are hindering adoption at the present time. However, successful XR pilots by industry leaders like Walmart, Ford, and Verizon have demonstrated the value of XR in enterprise, and this will be a major provocation for the market in these technologies to explode in the next five years.
Healthcare, retail, and manufacturing will experience the highest AR/VR adoption rate during that time, followed closely by construction and professional services. The top horizontal use cases we expect to see adopted by these verticals are training and education, and virtual workspaces and remote services.
XR market map by use case and industry
source: Harbor Research
We think the challenges with integrating these systems into our everyday lives will require a far greater understanding of the user’s needs in a particular context-of-use. Smart Systems and the IoT push this even further.
Computing power and networking is embedded in more and more of the objects and environments around us. Hence, the social and physical contexts in which connected devices and services can be used is even more complex and varied. This makes the need to rethink computing models essential to enabling effective or compelling user experiences.
As XR test-pilots continue to make headlines, work settings will also continue to shift in the wake of the pandemic, and hardware prices will continue to fall. Across hardware, software, and services, the XR market is quickly reaching maturity for enterprise use. Personal XR use will soon follow, driven by gaming and now “work from home” requirements.
For players looking to enter the XR market, the time to start developing your solution is now. The new interfaces of Extended Reality will soon allow computers to do what they always promised: Disappear. ◆