Spinning the web of mutualistic technologies

The combination of AI and XR

AI and XR have an intrinsically beneficial relationship. The articles, The many ways AI can empower XR and The irresistible marriage of AI and XR describe many examples how AI can enable powerful XR applications.

Leslie Shannon, head of trend and innovation scouting at Nokia, and Gábor Sörös, senior scientist at Nokia Bell Labs, recently discussed the interaction of these technologies in On the road to spatial computing – where digital meets physical. Shannon notes that spatial computing – a concept related to extended reality – “makes the actual world clickable”, and how the combination of spatial computing and artificial intelligence leads to intelligence augmentation.

Intelligence augmentation allows computing applications to understand the human and physical world, thereby “unifying the digital and the physical world”.

Moreover, if users of these technologies record their interaction with the real world – as many smart glasses already do – the world can become searchable via generative AI. For instance, if someone were to put car keys on a shelf, he or she could later ask the system for their location to find and retrieve the keys. Other applications could include listing known participants at a meeting or personal event.

Sörös highlights that by recognising objects and materials in a room, the system can render the room into a digital twin that then can find use for a wide range of applications. He believes that reliving memories could become a potential first consumer killer app for spatial multimedia. Shannon summarises the relationship between AI and XR, quoting AR Insider author Mike Boland: “XR can be the face of AI, while AI is the brains of XR.”

The interaction of AI, XR and robotics

Similarly, virtual, XR-enhanced environments will enable advanced robotics. Siemens, for instance, maintains that AI-enabled and virtual training gets robots to work faster – and will empower a more flexible era of automation. Environments that are simulated in an industrial metaverse can serve as a training ground for robots to acquire advanced skills much more quickly than traditional methods allow for.

“Immersive environments in the industrial metaverse that use simulation, digital twins and AI to mimic real-world conditions in detail” offer an effective training ground in which industrial factories and operations can be recreated. Virtual robots can learn to navigate the premises, take over new tasks and acquire safety-relevant skills. Alex Greenberg, director of robotics 4.0 simulation at Siemens Digital Industries Software, expands on the benefits.

“You can also simulate how humans move and behave,” he says. “You can program their task behaviour but also randomise behaviour to see where potential issues lie. Virtual reality can establish collaborative environments where suppliers and manufacturers can work together and run iterations in simulations. You can reduce risks on the shop floor.”

The outlook for future robotics is promising. Péter Fankhauser, CEO and co-founder of Anybotics, a provider of autonomous robots, argues: “I expect our robots to become so intelligent that they can independently generate their own missions based on the knowledge accumulated from digital twins.”

Amit Goel, director of product management for autonomous machines at Nvidia, also speaks to the value of digital twins for industrial applications. “You can simulate sensors, you can simulate robots, you can simulate people, you can simulate machines, you can train your AI agents – all of that in the simulated world,” he says.

The mutual benefits of sensors and robotics

Advanced robotics in conjunction with digital twins to train them in virtual environments and to ensure increasingly effective operation drives an entire submarket for sensors. Sensors can enhance robotics’ capabilities to become even more useful, flexible and safe. Sensors can also leverage robots as mobile platforms to quickly and cost-effectively collect data that can feed into digital twins, potentially creating positive feedback loops between digital twins and robotic applications.

Global Market Insights expects the robot sensor market to grow from $2.3bn in 2023 to $5bn by 2032, thereby doubling in the span of a decade. Smaller, increasingly efficient and more cost-effective sensors are opening the door for increased adoption in industrial and manufacturing sectors, but also for agricultural, defence, healthcare and logistics applications.

Benefits in industrial uses include detection of objects, avoiding collisions, and establishing increasingly precise operations – these benefits will over time establish opportunities in other sectors as robotic applications become more common outside of industrial sectors.

The increasing adoption of collaborative robots, “co-bots”, is another driver for robot sensors. In addition, the growing – more speculative – markets of autonomous vehicles and humanoids is adding demand for such sectors.

The report also highlights the fusion of AI with robotics as a key market trend, because “AI algorithms enable sensors to process large volumes of data, recognise patterns and make autonomous decisions”.

Sensors not only support robotic applications, robots can also serve as sensor platforms to harvest data quickly and efficiently from areas that can be difficult to reach or prohibitively expensive to equip with fixed sensors. Carmaker BMW, for example, is using Boston Dynamics’ Spot – a four-legged, doglike, mobile robot – that way.

Operators at the Hams Hall plant in the UK created a digital twin of the facility. The twin showed that inspection data was missing from areas in the factory. To avoid the cost and time requirements to install fixed sensors at the indicated areas, engineers decided to use a robot as a sensor platform to gather the needed data.

Similarly, scientists at the Department of Energy’s Oak Ridge National Laboratory (ORNL) employ autonomous robots for field monitoring, and the collection of samples from plantations and biological ecosystems. The data is then used for smart field management.

Udaya Kalluri, project lead and scientist in ORNL’s Biosciences Division, explains: “One of our goals with this project is to make connections between the laboratory and the field seamless – to order, for instance, automated sampling at a specific coordinate where we want to learn more.”

Understanding the intersection of mutualistic technologies

While many combinations of sensors, AI, XR and robotics develop as a result of current corporate and research needs, other benefits of these technology combinations will only become clear in the coming years, similar to the way the internet unfolded its genuinely transformative power over decades.

Scientists and developers are investigating the possibilities the combination of these technologies enable. Researchers at the Department of Engineering Sciences at the University of Agder in Norway, for example, are “exploring the synergies between collaborative robotics, digital twins, augmentation and industry 5.0 for smart manufacturing”.

The researchers see XR as part of augmentation, and AI in this context can create more intelligent processes in manufacturing. “There is a distinct lack of exploration into the diverse application scenarios that arise from the synergistic interaction of cobotics, DTs [digital twins], augmentation and AI,” the researchers note. “This gap hampers the establishment of a comprehensive understanding of the practical implications and potential use cases of these technologies working in tandem.”

A publication by the World Economic Forum and Capgemini is attempting to develop a better understanding of how various technologies can create symbiotic relationships for commercial applications. The Technology convergence report looks at the impact of 23 technology pairings across eight crucial technologies – among them AI, robotics and spatial intelligence, including digital twins and some types of sensors. Not surprisingly, the authors identify AI as the connective tissue among these technologies to drive convergence.

One of the report’s key insights will likely become a main theme in the coming years. “Technological convergence is dissolving traditional industry boundaries and enabling organisations to redefine their role in the value chain,” it says. 

Martin Schwirn is the author of “Small data, big disruptions: How to spot signals of change and manage uncertainty’ (ISBN 9781632651921). He is also senior advisor for strategic foresight at Business Finland, helping startups and incumbents to find their position in tomorrow’s marketplace.