Highlights:
-
Qualcomm AI On-Prem Appliance Solution is designed for generative AI inference and computer vision workloads on dedicated on-premises hardware – allowing sensitive customer data, fine-tuned models, and inference loads to remain on premises.
-
Qualcomm AI Inference Suite provides ready-to-use AI applications and agents, tools and libraries for operationalizing AI from computer vision to generative AI – spanning across on-premises solutions and cloud deployments.
-
Aetina, Honeywell, and IBM are amongst the first wave of industry leaders supporting the AI On-Prem Appliance and AI Inference Suite.
Jan 6, 2025 – LAS VEGAS – At CES 2025, Qualcomm Technologies, Inc., today announced Qualcomm® AI On-Prem Appliance Solution, an on-premises desktop or wall-mounted hardware solution, and Qualcomm® AI Inference Suite, a set of software and services for AI inferencing spanning from near-edge to cloud. The combination of these new offerings allows for small and medium businesses, enterprises and industrial organizations to run custom and off-the-shelf AI applications on their premises, including generative workloads. Running AI inference on premises can deliver significant savings in operational costs and overall total cost of ownership (TCO), compared to the cost of renting third-party AI infrastructure.
Using the AI On-Prem Appliance Solution in concert with the AI Inference Suite, customers can now use generative AI leveraging their proprietary data, fine-tuned models, and technology infrastructure to automate human and machine processes and applications in virtually any end environment, such as retail stores, quick service restaurants, shopping outlets, dealerships, hospitals, factories and shop floors – where the workflow is well established, repeatable and ready for automation.
“Our new AI On-Prem Appliance Solution and AI Inference Suite changes the TCO economics of AI deployment by enabling processing of generative AI workloads from cloud-only to a local, on-premises deployment. For a wide variety of AI automation use cases such as in-store assistants, worker coaching, site-specific information, safety compliance, and sales and service enablement at store, dealerships or factory-floors, our AI On-Prem Appliance Solution significantly reduces AI operational costs for enterprise and industrial needs,” said Nakul Duggal, Group General Manager for Automotive, Industrial IoT and Cloud Computing, Qualcomm Technologies, Inc. “Enterprises can now accelerate deployment of generative AI applications leveraging their own models, with privacy, personalization and customization while remaining in full control, with confidence that their data will not leave their premises.”
The Qualcomm AI On-Prem Appliance Solution is powered by the Qualcomm® Cloud AI roadmap of accelerators. It combines the accessibility and performance of a datacenter inference server with the power efficiency, weight and form factor, data privacy, personalization and control of an on-premises AI solution. The AI On-Prem Appliance Solution supports wide ranging capabilities:
- Scalability from a standalone desktop product to a wall-mounted appliance cluster which provides a host of local AI services like voice agents in-a-box, offload of small language models (SLMs), large language models (LLMs) and large multimodal models (LMMs), and retrieval augmented generation (RAG) functions for intelligent indexed search and summarization, agentic AI, AI workflow automation, image generation, code generation, computer vision, and camera processing.
- Support for a wide range of generative AI, natural language processing, and computer vision models, both open-source and proprietary, to enable workflow automation for many enterprise applications such as intelligent multi-lingual search, custom AI assistants and agents, code generation, automated drafting and note-taking, and more.
- Camera AI with image, video, and streaming processing for computer vision applications focused on security, safety and site monitoring.
The Qualcomm AI Inference Suite for On-Prem enables enterprise customers and third-party developers with a comprehensive set of tools and libraries for developing or porting generative AI applications to the AI On-Prem Appliance Solution. It features a rich set of application programming interfaces (APIs) including user management and administration, chat, image generation, audio and video generative AI capabilities, OpenAI API compatibility, and RAG. The suite supports integration with popular generative AI models, frameworks, and deployment using Kubernetes or bare containers.
Industry leaders embrace the Qualcomm AI On-Prem Solution and AI Inference Suite
Honeywell is collaborating with Qualcomm Technologies on the design, evaluation and/or deployment of AI workflow automation use cases using the AI On-Prem Solution and AI Inference Suite.
Aetina is among the first OEMs to provide on-premises equipment for enterprise deployments based on the AI On-Prem Appliance Solution. These compact, wall-powered on-premises AI appliances utilize a mix of SLMs for natural language processing, along with fine-tuned LLMs and LMMs (up to 70B parameters) to run enterprise AI agents supporting real-time responses and AI workflow automation functions such as intelligent indexed search and content creation – all enabled by best-in-class power consumption and superior cost-of-ownership architecture.
IBM is collaborating to bring its watsonx data and AI platform and Granite family of AI models for deployment across On-Prem AI appliances, in addition to cloud, to support a range of enterprise and industrial use cases in automotive, manufacturing, retail, and telecommunications.
Interested enterprise users and developers can assess the possibilities of the Qualcomm AI On-Prem Appliance Solution and capabilities of AI Inference Suite can find more information in our Edge AI Appliance and Box page.
About Qualcomm
Qualcomm relentlessly innovates to deliver intelligent computing everywhere, helping the world tackle some of its most important challenges. Our proven solutions drive transformation across major industries, and our Snapdragon® branded platforms power extraordinary consumer experiences. Building on our nearly 40-year leadership in setting industry standards and creating era-defining technology breakthroughs, we deliver leading edge AI, high-performance, low-power computing, and unrivaled connectivity. Together with our ecosystem partners, we enable next-generation digital transformation to enrich lives, improve businesses, and advance societies. At Qualcomm, we are engineering human progress.
