This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm.
With over 100 pre-optimized AI and generative AI models, the Qualcomm AI Hub is a developer’s gateway to superior on-device AI performance
Generative AI has been evolving to run on device, in addition to the cloud. It is transforming how we interact with our devices and making the user experience more personalized, reliable and private. These capabilities were available in smartphones released earlier this year, and soon we’ll see them in the next generation of PCs. But devices are only one part of the story.
Making AI models run on device with adequate performance can be complex and challenging, particularly when it comes to software compatibility and hardware optimization. The Qualcomm AI Hub is a developer-first online platform that simplifies the process and streamlines on-device AI development for Snapdragon and Qualcomm platforms. Developers can unlock the full potential of on-device AI and drive the next wave of innovation in the AI landscape.
The Qualcomm AI Hub aims to simplify the developer workflow, from conceptualization to optimization, testing and deployment. Developers can select a specific device or chipset, and the Qualcomm AI Hub will automatically convert models from Pytorch or ONNX for deployment on TensorFlow Lite, ONNX RT and Qualcomm AI Engine direct. This workflow, including testing on locally hosted cloud devices, takes less than five minutes and requires only a few lines of code. AI models optimized through the Qualcomm AI Hub can achieve up to four times faster inferencing performance, leading to superior user experiences.
This online testing capability quickly provides valuable insights into model performance and resource usage, enabling developers to iterate and optimize more effectively. This translates into accelerated development cycles and faster time-to-market.
To provide further support to developers in deploying their AI applications, we have introduced two major capabilities:
- Bring your own model (BYOM): The Qualcomm AI Hub now allows developers to bring their own AI models to the platform. This flexibility enables developers to work with their preferred frameworks and models, while the Qualcomm AI Hub takes care of compiling and optimizing them for Snapdragon and Qualcomm platforms. BYOM opens a world of possibilities, allowing developers to build AI applications tailored to their unique requirements and preferences.
- Expansion to Snapdragon X Compute Platforms: Building on the success of the Qualcomm AI Hub models for mobile and IoT, we are now extending support for PCs powered by the Snapdragon X Elite and Snapdragon X Plus. This expansion means that developers can leverage the optimization and testing capabilities of Qualcomm AI Hub for the new generation of AI-enabled PCs, from leading global PC brands, including Acer, ASUS, Dell, HP, Lenovo, Microsoft Surface and Samsung — further enhancing performance and efficiency of new and exciting AI applications.
Plug-and-play success with AI Models
The Qualcomm AI Hub includes a library of over 100 pre-optimized AI and generative AI models — such as Llama v2 7B, ControlNet, Stable Diffusion, Baichuan 7B and Inception v-3 — that are plug and play. These models were optimized to take advantage of Snapdragon and Qualcomm platforms and their heterogeneous computing capabilities across the neural processing unit (NPU), central processing unit (CPU) and graphics processing unit (GPU). We are already seeing companies such as GoPro, Cephable, Algoriddim, FullTrack and many others take their apps to the next level by utilizing the Qualcomm AI Hub.
The Qualcomm AI Hub revolutionizes the AI development process by simplifying workflows, optimizing models and providing seamless testing capabilities. With the ability to bring their own models and the expansion to Snapdragon X Series, developers have the tools they need to build AI applications that deliver exceptional performance, efficiency and user experiences. It is how on-device AI will scale across devices and become truly ubiquitous.
Rodrigo Caruso Neves do Amaral
Staff Marketing Communications Specialist, Qualcomm Technologies, Inc.
