This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm.
The software will help facilitate your personal preferences, apps and gen AI models to work together and provide revolutionary on-device AI capabilities
In the rapidly evolving landscape of generative artificial intelligence (AI) and how we interact with our devices, the ability to offer proactive personalized experiences is essential. And Qualcomm Technologies’ introduction of the Qualcomm AI Orchestrator is the key to taking those experiences to the next level.
Qualcomm AI Orchestrator facilitates between your personal preferences, local context and the capabilities of the apps and services on your device, bringing in an agentic AI experience. It represents a revolution in on-device AI and the kind of personalized AI approach your devices can deliver.
This groundbreaking technology harmonizes all AI functionalities to provide a rich and meaningful interaction experience. By keeping information on the device as a personal knowledge graph, the experience will be more responsive.
Here’s a closer look at what makes the Qualcomm AI Orchestrator stand out.
AI that understands your intent
The Qualcomm AI Orchestrator leverages your context (e.g., your favorite cuisine, workout routines, your frequent contacts and messages, your previous preference history) stored on-device as a personal knowledge graph, to better understand your intent.
By understanding your personal preferences fused with local context, the Qualcomm AI Orchestrator can integrate this knowledge into your generative AI assistant so you can get a personalized response and action that is meaningful to you.
Improving the on-device AI experience is important because of several key advantages:
- you get improved responsiveness,
- heightened privacy and
- personalized user experiences.
Additionally, the Qualcomm AI Orchestrator supports various modalities of interacting with the device including text, vision and speech, with multimodal models.
Multimodal support enables interactions with the AI device that are natural and personal. The orchestrator can effectively understand your intent whether you’re asking a question, generating images or interacting through visual and camera inputs.
Orchestration: The agent in action
To conduct this AI symphony, the orchestrator understands your applications’ capabilities, the actions they can take and their associated APIs. It also supports retrieval-augmented generation (RAG) — a generative AI technology that can use locally available data (e.g., documents or images stored on the device) to make the AI agent an expert on the provided data. With your personal context, the ability to navigate through various generative AI models, and knowing your app’s capabilities and RAG, the orchestrator agent is now ready to take your questions and requests, then plan the interactions between all these elements. The result is an incredible experience that’s tailored to your needs and preferences.
Streamline your day
Imagine a scenario where you start your day with a bunch of notifications on your phone. You don’t have time to read them until your lunch break, so instead of reading all the notifications yourself, your generative AI assistant automatically creates a notification summary and can pick out important ones.
One that you wouldn’t want to miss is a text from your mom asking you to meet for dinner at 6:30 pm. You respond: “Plan dinner for me with my mom at her favorite restaurant” and your AI assistant springs into action:
- It checks both your calendars to confirm availability and books the appointment.
- Knowing your mom’s preferences, the assistant suggests her favorite restaurant and reserves a table using your favorite reservations app, then blocks the time on your calendar.
When you see everything is set, you seamlessly go back to work on your laptop powered by Snapdragon technology and as you engage your tasks, you scan the reservation on your calendar view.
Responding to unexpected situations
As it gets later in the day, you receive a notification that it’s time to leave work to get to dinner with your mom, with your maps app automatically opened and the best route already loaded. It’s time to close your laptop and hit the road!
Once you enter your car with Snapdragon, that route has already transferred to your favorite navigation app and off you go. Not too long after, a text arrives that your mom will be 15 minutes late. Your AI assistant promptly responds, “I’ve notified the restaurant that you will be 15 minutes late and prepaid for parking.”
As you have 15 minutes to spare, the assistant:
- automatically suggests a new route with a stop at the store to grab flowers, then
- has the store place your mom’s favorite roses in your shopping cart so it’s ready to pick up when you arrive — ensuring a thoughtful touch for the evening.
This integration of notification summarization, understanding your request in full context, calendar management, location-based services and personalized recommendations creates a hassle-free and efficient experience, allowing you to focus on quality time with your loved ones.
This example is only one among numerous ways in which the orchestrator can lighten your tasks. The Qualcomm AI Orchestrator can act as your private expert, a knowledgeable assistant and planner, all while understanding your personal context and preferences.
Qualcomm AI Orchestrator: A new addition to the Qualcomm AI Stack
The Qualcomm AI Orchestrator will be an integral part of the Qualcomm AI Stack. It sits between your apps and the AI frameworks and runtimes, providing the orchestration needed for all the functionalities mentioned above.
As we move forward, the Qualcomm AI Orchestrator is poised to expand its capabilities, including seamless integration of device-to-device and device-to-car configurations. The orchestrator will leverage the best aspects of each device and generate the most compelling experience for the user. This future-ready design enables the orchestrator to continually evolve, helping to meet the ever-changing demands of AI technology.
The Qualcomm AI Orchestrator is more than just a tool; it’s a revolution in on-device AI integration. By harmonizing AI functionalities and providing a rich, personalized experience, it sets a new benchmark for AI-based interactions with devices. It will soon be in many devices powered by Qualcomm and Snapdragon technology. Tune into our annual Snapdragon Summit for more information as we continue to push the boundaries of what’s possible with AI.
Durga Malladi
SVP and GM, Technology Planning and Edge Solutions, Qualcomm Technologies, Inc.
