This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm.
On-device artificial intelligence (AI) is critical to making your everyday AI experiences fast and security-rich. That’s why it’s such a win that Qualcomm Technologies and Meta have worked together to support the Llama 3.2 large language models (LLMs) family for use on devices that contain Snapdragon platforms.
Meta unveiled Llama 3.2 at its Meta Connect event on Wednesday. It’s the next iteration of their large language model family, bringing multilingual, multimodal capabilities, long context length and more.
This collaboration marks the latest development in making generative AI accessible on edge devices such as smartphones and AI PCs, enabling a host of benefits including improved responsiveness, heightened privacy and personalized user experiences as compared to cloud-based AI.
More broadly, it moves the bar on what we can expect from devices and follows Qualcomm Technologies’ mission to bring more AI to the devices themselves. Our Snapdragon X Series Compute platforms power a new wave of Windows Copilot+ PCs and the Snapdragon 8 Gen 3 for Galaxy is helping to enable many of Galaxy Z Fold6 and Z Flip6’s AI features.
Optimizing Llama 3.2 models, including the new lightweight 1B and 3B models, as well as the multi-modal 11B parameter model, for devices powered by Snapdragon 8 Gen 3 and Snapdragon X Series can significantly enhance user interactions.
Tools and resources for developers
To facilitate the adoption and optimization of Llama 3.2 on Snapdragon and Qualcomm platforms, developers will have access to extensive resources and tools. They can start experimenting with Llama 3.2 today through various implementations:
- Ollama is an open-source project that serves as a powerful and user-friendly platform for running LLMs on Snapdragon X Series platforms. It provides the capabilities for developers to easily run cutting edge LLM models on-device and provides developers the right tools to create their customizable AI experience. Download and use Ollama with Llama 3.2 1B and 3B directly on laptops equipped with Snapdragon X Elite.
- ExecuTorch and Llama.cpp enable you to run models directly on laptops equipped with Snapdragon X Elite and smartphones with Snapdragon 8 Gen 3.
Furthermore, the Qualcomm AI Hub is a centralized platform for developers to access tools and resourcesfor accelerating on-device AI development and deployment , including Llama 2 and 3, available for download today, and will soon include Llama 3.2.
Vinesh Sukumar
Senior Director, Product Management, Qualcomm Technologies
Rodrigo Caruso Neves do Amaral
Staff Marketing Communications Specialist, Qualcomm Technologies
