Qualcomm and Meta collaborate to enable on-device AI applications using Llama 2

The extent of the metaverse looks unlimited, with a potential capability of running on any connected device, and in a bid to provide users with private, more reliable and personalised extended reality (XR) experiences, Qualcomm Technologies and Meta have embarked on a project to optimise the execution of Meta’s Llama 2 large language models directly on-device – without relying on the sole use of cloud services.

Meta and Qualcomm Technologies have an existing business relationship in trying to create the next generation of premium device experiences. The companies’ current collaboration to support the Llama ecosystem spans research and product engineering efforts.

The companies believe the ability to run generative artificial intelligence (AI) models like Llama 2 on devices such as smartphones, PCs, virtual reality (VR) and augmented reality (AR) headsets and vehicles allows developers to save on cloud costs, and to provide users with private, more reliable and personalised experiences.

As it was announcing the new working relationship, Qualcomm claimed its experience in on-device AI “uniquely” positioned it to support the Llama ecosystem, having what it said was an “unmatched” footprint at the edge – with billions of smartphones, vehicles, XR headsets and glasses, PCs and internet of things (IoT) devices powered by its AI hardware and software solutions.

As a result, Qualcomm Technologies plans to make available on-device AI implementations based on Llama 2, Meta’s second generation of its large language model, to enable the creation of new AI applications. It said this would allow customers, partners and developers to build use cases, such as intelligent virtual assistants, productivity applications, content creation tools, entertainment and more. These new on-device AI experiences, powered by Qualcomm’s Snapdragon mobile platform, are designed to work in areas with no connectivity and in airplane mode.

“We applaud Meta’s approach to open and responsible AI and are committed to driving innovation and reducing barriers to entry for developers of any size by bringing generative AI on-device,” said Durga Malladi, senior vice-president and general manager of technology, planning and edge solutions businesses at Qualcomm Technologies. “To effectively scale generative AI into the mainstream, AI will need to run on both the cloud and devices at the edge, such as smartphones, laptops, vehicles and IoT devices.”

Qualcomm Technologies is scheduled to make available Llama 2-based AI implementation on devices powered by Snapdragon from 2024 onwards. Developers can start optimising applications for on-device AI immediately using the Qualcomm AI Stack, a set of tools designed to process AI more efficiently on Snapdragon, making on-device AI possible even in small, thin and light devices.

The Meta announcement comes just after Qualcomm Technologies announced new features for its Snapdragon Spaces XR Developer Platform to hasten a cross-device, extended reality (XR) ecosystem and create a head-worn future that it said would be “truly open”. It added that it had helped pioneer smartphone augmented reality over 13 years ago, and that with Snapdragon Spaces, it was reimagining what smartphone-powered AR can be when combined with AR glasses.