From knowing to doing: embodied AI gets to work

Rules versus reasoning

If the machines now entering factories, warehouses and even offices look superficially familiar, what’s less obvious is the underlying software that lets them perceive, reason and act.

“Classic industrial automation and first-generation video analytics follow explicit, narrow logic: a fixed tripwire, a motion threshold, a robot arm replaying the same programmed path,” says Babak Behzad, director of engineering for AI, search and analytics at Verkada, a physical AI technology supplier. “That works well for repeatable, structured tasks, but it’s brittle the moment the environment varies.”

Modern systems increasingly use vision-language models, trained on large volumes of real and synthetic data, to generalise across situations they were never explicitly programmed for: a forklift approaching a pedestrian from an unusual angle, or a door propped open for a legitimate but unlogged reason. “The system is reasoning about context, not just matching a pattern,” he says.

The real dividing line, in Behzad’s view, isn’t whether a system has a camera or a sensor – plenty of legacy automation does – but whether its behaviour is the product of learned, generalisable perception and reasoning, or of an explicit rule its developers wrote in advance.

By delegating physically demanding and repetitive tasks to our robotic fleet, we significantly reduce worker strain and safety risks, allowing our human workforce to focus on high-value, creative roles

Acting in the physical world also changes the computing problem. Unlike conventional AI workloads, physical AI systems must operate within bounded time constraints, says Fong, and in applications such as robotics, autonomous systems and healthcare devices, average performance is often less important than consistent performance.

AMD’s answer is determinism – predictable timing behaviour across the entire system. “Tail latency, jitter and deadline misses can have a significant impact on safety, reliability and overall system performance,” says Fong.

That is steering suppliers towards heterogeneous architectures: in AMD’s case, CPUs for deterministic control and orchestration, graphics processing units (GPUs) for perception and computer vision, neural processing units for power-efficient inference, and adaptive compute and field-programmable gate arrays for sensor processing and real-time acceleration. And because such systems can stay in the field for decades, Fong adds, buyers also need open software ecosystems, long-term support and compatibility with existing IT and operational technology.

Verkada’s approach is to split inference by task, rather than defaulting everything to edge or cloud. Anything latency-critical – a deterrence response, or a safety trip that has to fire in under a second – runs on the device itself, on purpose-built silicon such as Ambarella’s CV7x chips inside its cameras.

That also buys resilience, with devices continuing to make local decisions when connectivity drops in warehouses or on remote perimeters. Heavier reasoning, such as correlating an event against hours of history, cross-referencing multiple cameras or running a larger vision-language model, goes to the cloud.

Fong sees the industry converging on the same pattern, predicting that physical AI architectures will “increasingly operate as distributed nervous systems, where intelligence is distributed across sensors, edge processors, controllers and cloud infrastructure”.

Behzad offers buyers a test for cutting through supplier claims: “Enterprises should be wary of any physical AI vendor whose architecture assumes constant, high-bandwidth cloud connectivity. That is not a realistic assumption for a warehouse, a remote substation or a large campus perimeter.”

HMGICS’s experience shows how much groundwork is involved. The facility runs on a 5G-enabled campus network with mobile edge computing, deployed with Singtel, providing the low-latency connectivity needed for fleets of mobile robots to be managed, rerouted and kept safely apart – all integrated with the plant's enterprise resource planning, manufacturing execution and warehouse management systems, layered with AI, robotics control and real-time analytics.

The data gap

For all the compute, the scarcer resource is often data. Transferring what is learned in simulation to the messiness of reality remains a bottleneck in physical AI, but Verkada starts from an unusual position: its device fleet supplies a continuously refreshed stream of aggregated, anonymised performance signal showing where its models underperform and on what kinds of scene.

Its partnership with Nvidia – announced in early July, with the chipmaker also taking a stake in the company – uses Nvidia’s Cosmos world foundation models and synthetic data pipelines less to simulate reality from scratch than to fill gaps in real-world data, generating physically consistent examples of events too rare, dangerous or sensitive to capture at scale, such as a forklift near-miss, a weapon in frame or a fall in a hospital room. Verkada says mean average precision, the standard accuracy score for this kind of search, has risen by 68% since the work began.

The problem that remains, says Behzad, “is less about volume and more about distribution”, because real-world environments differ by region and industry in ways generic training data does not capture. “High-visibility safety apparel standards in Australia differ from the US, which showed up as real accuracy gaps in the field until we gathered region-specific data,” he says.

Enterprises should be wary of any physical AI vendor whose architecture assumes constant, high-bandwidth cloud connectivity. That is not a realistic assumption for a warehouse, a remote substation or a large campus perimeter

Su agrees that data heads the list of barriers to enterprise adoption, citing the lack of quality data in specific tasks, such as airport luggage transfer, device assembly and facility cleaning.

Integration is the second, with most Chinese vendors focusing on manufacturing and shipping robots but less on helping enterprises put them to work, Su says. Cost is the third, because human labour in some markets remains more cost-efficient than deploying humanoid robots at scale. The fourth is the absence of general-purpose capability: even robots running the most advanced embodied AI are still only good at specific tasks.

According to Su, real-world production deployments are mostly found in media and entertainment, research and development, education and customer services, while extended pilots are mainly in warehousing, manufacturing and airports.

HMGICS, for its part, did not treat its smart factory as a one-off pilot. Instead, it adopted “a holistic digital transformation framework that allows us to build, validate and scale in one continuous process”, refining technologies inside a live manufacturing environment before deploying them across Hyundai Motor Group’s global production network.

Getting to production brings its own discipline. Machine learning operations and model update pipelines for physical AI in the field should be treated as fleet management rather than app deployment, says Behzad.

That means centralised visibility of every device and the model version it is running; automated compression and validation of new models before they ship, since edge silicon has hard limits on model size; staged, reversible rollouts rather than big-bang updates; and confidence thresholds that can be tuned per site or region, because a model that performs well in one environment can under- or over-fire in another.

The piece organisations underweight, he adds, is the loop back from the field. When frontline teams flag a false positive or an edge case, that signal needs a clear path back into the next training cycle – not just a support ticket that gets closed.

“To be precise about what that loop actually moves: it’s performance signal, not customer footage or data. We’re tracking things like which categories of events get misclassified and where, then using that aggregated signal – plus synthetic data where we need more examples of something rare – to improve the model,” says Behzad, who suggests buyers ask suppliers exactly how that feedback loop works, and what data moves through it before they ask about accuracy benchmarks.

Trust, safety and the humans

A compromised robot is a physical threat, not just a digital one. HMGICS says cyber security “is embedded across our manufacturing environment”, with a team continuously monitoring evolving risks, and safety measures designed into its automation and robotics to protect both people and equipment.

The wider industry is still working out the guard rails. Liability and insurance are “somewhat a barrier”, says Su, but the market has not really reached that stage yet: “The industry is still trying to resolve functional safety concerns through hardware-software security and safety standards, but all these efforts are still in their early days.”

Regulation will bite everywhere, he adds, particularly on data: “Vendors need to adhere to strict restrictions on data hosting, which severely limit how data can be handled. They may need to work with the local supply chain to host data and AI within customer borders.”

For Behzad, the hardest unsolved problems are as much about trust as technology. The first is calibrating autonomy – deciding which actions a system should take on its own and which need a human in the loop, and making that threshold legible and auditable rather than a black box. “Get this wrong in a physical setting and the cost of a mistake is much higher and more tangible than a mistake in the digital world,” he says.

The second is memory. Most physical AI reacts to what’s in front of it in the moment, but the systems that actually earn trust need to build up knowledge of a specific site over time (this loading dock normally sees a delivery truck at 6am, this hallway is a break room after 10pm), the same way an experienced human operator does. “This is a genuinely hard, still-emerging area of the field,” Behzad adds.

Then there’s also the challenge of orchestrating cameras, robots, drones and sensors from different suppliers, and – in Behzad’s view – the winners in this space will be the companies that already have the sensor footprint and the real-world deployment experience, not just the best model.

The human operators, meanwhile, are not going anywhere. At HMGICS, the workforce is being continuously upskilled – in AI literacy, data awareness and cyber security fundamentals, as well as deeper technical expertise – as roles shift from performing repetitive tasks to managing automated systems.

“By delegating physically demanding and repetitive tasks to our robotic fleet, we significantly reduce worker strain and safety risks, allowing our human workforce to focus on high-value, creative roles,” the company says.

However the hype cycle plays out, Fong is in little doubt about the potential of physical and embodied AI: “The next decade of AI will be defined not only by systems that ‘know’, but by systems that ‘do’.” On the factory floor in western Singapore, the doing is already well under way.