This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm.
From live translations to personalized content management — the new era of mobile intelligence
It’s 2024, and generative artificial intelligence (AI) is finally in people’s hands. Literally. This year’s early slate of flagship smartphone releases is a showcase of AI integration into the handsets themselves. Meaning generative AI technology’s real-world use cases are becoming increasingly clear and available to consumers.
Generative AI technologies like large language models (LLMs) or diffusion models, the powerhouses behind today’s conversational AI assistants and image generators, have long been associated with the need for massive computational resources. However, advances in mobile hardware and technologies are reducing the need for cloud processing to do AI-powered tasks, making it possible for a growing list of smartphones to handle these complex tasks natively.
This evolution marks a crucial turning point in the integration of AI into our daily lives. As more devices — especially smartphones — become capable of running AI features natively, we are entering an era where AI will be a fundamental part of our everyday experience. This transition opens up a world of possibilities for more intuitive, personalized and efficient interactions with our devices, revolutionizing the way we engage with technology.
On-device AI capabilities on the ASUS Zenfone 11 Ultra.
Generative AI’s deep integration into 2024’s latest flagship smartphones
Generative AI’s direct integration into device software has allowed for supercharged AI smartphone features in a variety of categories like communication enhancement, generative image editing tools, personalization and accessibility. On-device generative AI is being used to create more intuitive, user-centric features and automate tasks that were previously time-consuming or impossible for mobile devices. And this is only the beginning.
This shift towards AI-driven functionalities is clear in the latest flagship smartphone releases from major manufacturers like Samsung, ASUS, Xiaomi, Oppo, Vivo and Honor, which are incorporating generative AI to improve various aspects of the user experience. These smartphone makers have a shared approach in that they have adopted the Snapdragon 8 Gen 3 Mobile Platform to work as the brains of the phone and efficiently run generative AI applications locally on device.
The Snapdragon 8 Gen 3, which features an integrated neural processing unit (NPU) built specifically to enable generative AI, also utilizes heterogeneous computing for AI workloads. The Snapdragon 8 Gen 3 can take advantage of a diverse combination of processors beyond the NPU, including the central processing unit (CPU), graphics processing unit (GPU) and always-on sensor processor, to increase performance and optimize energy efficiency.
By intelligently utilizing the right processor alongside the NPU, this heterogeneous computing approach optimizes application performance, thermal efficiency and battery life. As a result, it enables new and improved generative AI experiences on device.
I’m excited to share several real-world use-cases that have been shaken up and reimagined by having generative AI run directly on the smartphone.
Generative AI-powered interaction tools and refinement
In 2024, smartphone manufacturers are integrating generative AI capabilities directly into the keyboard software, offering a major upgrade to the fundamental way users interact with their devices. Samsung is leveraging AI models to offer features such as tone suggestions and intelligent rewriting. Samsung’s Galaxy AI, for example, allows users to rephrase sentences in different styles and moods, while also providing spelling and grammar corrections.
Live translation and transcription
The Samsung Galaxy S24 and ASUS Zenfone 11 Ultra also harness the power of on-device AI to break down language barriers with their advanced live translation features. The S24’s Live Translate and Zenfone 11 Ultra’s AI Call Translator enable real-time translation of phone calls into the user’s preferred language, facilitating seamless cross-cultural communication.
Additionally, the Zenfone’s AI Transcript automatically converts voice calls into written text, allowing users to easily review, search and share important conversation details. Similarly, the S24’s Live Translate provides real-time transcription of translated calls, displaying the text on screen for convenient reference. By harnessing advanced on-device AI, these smartphones streamline note-taking, improve information retention and enable quick access to key discussion points, making them invaluable tools for professionals and multitaskers in an increasingly fast-paced world.
Generative AI-powered image content generation and editing
The Xiaomi 14 and Xiaomi 14 Ultra introduce an innovative “AI Portrait” feature that allows users to train the smartphone on their own face using photos from their gallery. Users can then generate realistic AI selfies in almost any setting by simply entering a text prompt. The AI model generates four images in 30 to 40 seconds based on the prompt, with options for higher resolution versions.
The Oppo Find X7 Ultra includes an impressive on-device AI feature called the AIGC Eraser, which allows users to remove unwanted objects or people from their photos without requiring an internet connection. By downloading the offline pack, the AIGC Eraser can function entirely on the device itself, providing a convenient and privacy-focused solution for image editing.
The Zenfone 11 AI wallpaper generator instantly crafts a personalized image that reflects a user’s individuality by allowing them to input a prompt and select their preferred style and tone — plus, it runs completely on device.
AI-powered personalization features through LLMs
In 2024, several flagship smartphones are pushing the boundaries of on-device AI with the integration of powerful LLMs that enable advanced natural language processing and generation capabilities. The Samsung Galaxy S24 series hosts LLMs locally to enable a range of new features, including the ability to generate human-like text based on prompts, and even assist with writing emails or documents. The model’s performance is optimized for the S24’s hardware, ensuring fast and efficient processing while maintaining user privacy by keeping data on the device.
Meanwhile, the Oppo Find X7 Ultra boasts AndesGPT. AndesGPT focuses on delivering contextually aware, multi-turn conversations to provide a more natural and engaging user experience. The 7B version of this model enables the Find X7 Ultra to serve as a virtual assistant, offering personalized recommendations, answering questions and even helping with tasks like scheduling or translation — all without sending data to the cloud. A larger AndesGPT that runs in the cloud can also assist the on-device AI LLM for a hybrid AI solution.
The Honor 6 Pro also joins the on-device LLM trend with Yoyo, a model developed by Honor’s in-house AI team. Yoyo is designed to learn and adapt to user preferences over time, providing increasingly tailored responses and suggestions. It powers features like smart reply suggestions in messaging apps, context-aware search results and even a creative writing assistant that can help users craft stories or poetry.
Advanced generative AI-powered interface tools
A new generative AI-powered feature from Honor is its Magic Portal which allows users to drag and drop elements like text or images between apps, aiming to streamline interactions. The AI predicts user intent and suggests relevant apps for sharing content. At launch, Magic Portal is expected to support around 100 of the most frequently used apps globally, with potential for future expansion.
But these features are only the start of what AI-powered devices can do.
One transformative application of this technology is the proverbial AI assistant, designed to understand user goals and manage tasks through simple voice or text commands, and streamline daily activities such as planning trips, shopping and editing photos.
This approach simplifies the user experience by eliminating the need to navigate through multiple apps.
On-device AI processing unlocking new horizons
On-device AI processing is a big deal for a number of reasons that we’ve covered in previous OnQ blogs. This shift from cloud-based to on-device processing is a game-changer, offering enhanced privacy, improved performance and deeper personalization. By keeping data local, on-device generative AI significantly reduces the risk of data breaches and unauthorized access, addressing the growing concerns over data privacy in our increasingly digital world.
Moreover, the move to on-device processing translates to faster response times and smoother interactions for users. Without the need to send data back and forth to the cloud, latency is drastically reduced, leading to a more immediate and reliable performance.1
With generative AI interfaces, there has been a fundamental shift to the way we interact with technology, offering considerable time savings and transforming mundane tasks into engaging experiences.
This move towards intuitive, generative AI-driven interfaces could democratize access to technology, making it more inclusive for individuals with varying levels of tech-savviness or disabilities. Furthermore, the development of such generative AI-powered devices, capable of processing AI on-device for enhanced speed and efficiency, underscores a commitment to privacy, security and user-centric innovation. As this concept moves from prototype to reality, it challenges the current app-centric model and paves the way for a more accessible, simplified digital life.
Smartphone generative AI features are a window to tomorrow
The integration of generative AI into smartphones is a huge step forward in our interaction with these devices. The incorporation of AI features directly into mobile devices signifies a move toward greater convenience, personalization and accessibility.
This development suggests that smartphones are transitioning to become not just tools, but extensions of the user, capable of understanding and responding to individual needs in a more nuanced manner. Looking forward, the potential for AI in smartphones as well as other devices, such as PCs, XR, cars and other edge devices, appears boundless.
Future advancements may introduce even more sophisticated AI capabilities, such as augmented reality interfaces, AI-driven health monitoring and eventually self-directing AI-powered agents which will be able to complete complex tasks, directly accessible through our mobile devices. As AI technology progresses, it is expected to further enhance the capabilities of smartphones, making them more intelligent, responsive and integral to our daily lives.
But all that to say, consumers can get a sense of what’s possible tomorrow today when they purchase the latest round of premium smartphones powered by Snapdragon available right now.
Jerry Chang
Senior Manager, Marketing, Qualcomm Technologies
Reference
- Qualcomm. (May 2023). The future of AI is hybrid Part I: Unlocking the generative AI future with on-device and hybrid AI. Retrieved on March 26, 2024 from: https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Whitepaper-The-future-of-AI-is-hybrid-Part-1-Unlocking-the-generative-AI-future-with-on-device-and-hybrid-AI.pdf
