This blog post was originally published at Qualcomm’s website. It is reprinted here with the permission of Qualcomm.
Explore the most pivotal advancements that shaped the landscape of generative AI
From Alan Turing’s pioneering work to the cutting-edge transformers of the present, the field of generative artificial intelligence (AI) has witnessed remarkable breakthroughs — and today we invite you to delve into a timeline of generative AI.
We’ve included everything from early attempts at machine learning to the transformative power of large generative models, so you can discover how these breakthroughs have revolutionized natural language processing, image generation and more to push the boundaries of what AI can achieve today and into the future of AI.
What is generative AI?
To explore the history of breakthroughs in generative AI, it’s helpful to understand what generative AI even means. Generative AI models generate high-quality images, text, audio, synthetic data and other types of content. These models often learn to create this new content based on the patterns and relationships in datasets of existing content. Very big models trained on very large amounts of data are called foundational models, most of which are large language models (LLMs) trained on natural language and predicting the next word.
Why is generative AI important?
In the past few years, generative AI has rapidly evolved, transforming how machines interact with and understand humans. It’s an enormous step forward in the history of AI — rather than just classifying or analyzing existing data, generative AI can create entirely new content. Companies can also now build their own customized models on top of foundational models, with tremendous potential for quickly adapting to numerous downstream tasks without the need for task-specific training from scratch.
The most compelling benefit of generative AI is arguably its efficiency. For all kinds of users, generative AI will enable groundbreaking productivity and creativity. Enterprises can automate specific tasks and shift their energy, time and resources to more strategic goals. This can reduce labor costs, optimize operations and unlock insights into a company’s business processes. For professionals and content creators, generative AI tools can help them efficiently create new ideas, plan and schedule content, edit, research and more.
Alan Turing’s accomplishments included helping invent a device that could crack secret codes sent by German armed forces that were sent via “Enigma” machines (pictured here).
Generative AI timeline: 1940s to 1960s
Though it has garnered tremendous attention in recent years, the roots of generative AI go back to the birth of AI in the middle of the 20th Century.
The Turing test
In 1947, mathematician Alan Turing first mentioned “intelligent machinery” in a paper exploring whether a machine could detect rational behavior. In a 1950 paper, he introduced the concept of the Turing Test, in which a human evaluator would judge text-based conversations between a human and a machine designed to generate human-like responses. If the evaluator could not reliably tell the machine from the human, the machine would pass the test.
ELIZA
One of the first functioning examples of generative AI was the ELIZA chatbot, created by British scientist Joseph Weizenbaum in 1961. The first talking computer program that simulated the work of a psychotherapist, ELIZA could communicate with a human by engaging in text-based conversations with rudimentary responses.
Generative AI timeline: 1980s to 2010s
Advancements in machine learning algorithms fueled the evolution of generative AI, enabling machines to learn from data and improve their performance over time.
RNNs and LSTM networks
The introduction of Recurrent Neural Networks (RNNs) in the late 1980s and Long Short-Term Memory (LSTM) networks in 1997 enhanced the ability of AI systems to process sequential data. LSTM’s ability to understand order dependence was crucial for solving intricate problems, including speech recognition and machine translation.
Breakthroughs in generative models
The creation of Generative Adversarial Networks (GANs) in 2014 was a fundamental breakthrough in generative AI. A GAN is an unsupervised machine learning (ML) algorithm that engages two competing neural networks. One network is a model generating content, and the other is discriminative, attempting to determine the authenticity of the content. After many attempts, the generator will eventually produce high-resolution images that the discriminator cannot distinguish from the real images.
Other methods developed around the same time, such as variational autoencoders (VAEs), diffusion models and flow-based models, and continued to evolve the image generation process.
Transformer architecture and introduction of GPT models
Similar to RNNs, transformer models, introduced in 2017, process sequences such as natural language text. Transformer models capture context by understanding the interplay of words in a sentence. Unlike earlier ML models that process sequences step by step, transformers process all parts at the same time, making them far more efficient and powerful.
The transformer architecture led to the creation of LLMs such as GPT (Generative Pre-trained Transformer), first built by OpenAI in 2018. GPTs are neural networks using a deep learning architecture to generate text, engage in conversations with users and complete numerous language tasks. Users can apply a GPT to automate and refine tasks like coding, content writing, researching complex topics and text translation. Its real value is its ability to do so at great speed and scale.
ChatGPT enables machines to engage with users in natural conversations, answering questions and generating content.
Generative AI timeline: 2020s
ChatGPT
ChatGPT, launched by OpenAI in November of 2022, reached a staggering one million users in five days.1 Initially powered by OpenAI’s GPT-3.5, ChatGPT enables machines to engage in coherent and context-aware conversations. Users can also ask ChatGPT to generate text and other content of a desired style, length, format and level of detail.
Llama from Meta
Meta’s Llama (Large Language Model Meta AI) is a collection of state-of-the-art foundation language models and was a turning point in open-source AI development. Though its foundational models are smaller than those of GPT-3 and others, it achieves comparable results with much less computational power. In 2023 at Snapdragon Summit, we achieved the world’s fastest Llama 2-7B on a phone, demonstrating chat with an AI assistant that runs completely on the device.
PaLM and Gemini from Google
Google’s PaLM (Pathways Language Model) was first announced in April 2022 and remained private until March 2023, when Google launched an API for PaLM. PaLM was another breakthrough in natural language processing (NLP), scaling up to an astonishing 540 billion parameters.
Google ‘s latest model, Gemini represents a significant milestone due to its state-of-the-art performance and next-generation capabilities. It’s optimized for various sizes and can seamlessly understand and combine different modalities, including text, code, audio, image and video. Gemini comes in three different sizes: Ultra, Pro and Nano.
Built with responsibility and safety at its core, Gemini undergoes comprehensive safety evaluations and incorporates safeguards to address potential risks.
BLOOM
Launched in July of 2022, BLOOM (Big Science Language Open-science Open-access Multilingual) is the brainchild of BigScience, a collective of more than 1,000 volunteer researchers worldwide. BLOOM is a multilingual model that generates coherent text in 13 programming languages and 46 different languages. BLOOM at 176 billion parameters is a very large open-access AI model, available for anyone to use, free of cost, making innovation possible for small businesses, individuals and nonprofits.
Generative AI text-to-image models
DALL-E, Midjourney and Stable Diffusion are advanced generative AI models that create and manipulate visual content based on textual input. DALL-E, by OpenAI, and Midjourney are proprietary models that create images with a high level of detail and realism. Stable Diffusion also generates high-quality images but is open source. In February of 2023, we showed the world’s first on-device demonstration of Stable Diffusion on an Android phone.
As AI continues to advance and evolve, many industries are adopting this technology to improve society.
The impact and future of generative AI
Generative AI shows potential for wide use in diverse industries. Educators and healthcare professionals could use it to develop learning plans or patient rehabilitation training. Graphic and fashion designers could generate new ideas for visual assets, logos, styles and patterns. Personalized digital assistants can develop individual diet and exercise plans, make travel reservations and pay bills. Developers can accelerate coding. Users will more easily engage in chat-like conversations with their devices. Generative AI also shows near-limitless potential for scientific research and analysis.
The potential benefits of generative AI for on-device applications alone can further improve the user experience through enhanced data privacy and security, reduced latency, increased performance and contextual personalization while lowering the required costs and energy consumption of cloud-based AI.
Generative AI will continue to play a crucial role in shaping the future of technology by pushing the boundaries of what machines can achieve. New advancements in generative AI may spur laptop replacements and a general move from the cloud to on-device processing. Personal AI assistants will make smartphones even more indispensable. Creatives and marketers will see an increase in productivity levels, time-to-market and efficiency. Consumers will increasingly demand that their devices work together across open ecosystems, and extended reality experiences will redefine our world.
Expanding horizons
Generative AI has enjoyed an accelerating journey of innovation and importance, from its roots in AI’s earliest days in the 1950s to today’s near-constant release of innovations and demonstrations leveraging its many capabilities. The technology has had a profound impact on creativity, efficiency and innovation across a wide range of industries. As generative AI continues to evolve, its applications are likely to expand, bringing about further advancements and reshaping how we interact with and leverage AI.
Considering its rate of innovation, staying informed about the latest developments in generative AI is crucial for understanding the potential applications and impact on various industries. The journey of generative AI reflects not only technological progress but also the limitless possibilities that lie ahead in the realm of AI.
Pat Lawlor
Director, Technical Marketing, Qualcomm Technologies, Inc.
Jerry Chang
Senior Manager, Marketing, Qualcomm Technologies
Sources
- Devon, Cheyenne. (Nov. 30, 2023). On ChatGPT’s one-year anniversary, it has more than 1.7 billion users—here’s what it may do next. CNBC. Retrieved on January 12, 2024 from: https://www.cnbc.com/2023/11/30/chatgpts-one-year-anniversary-how-the-viral-ai-chatbot-has-changed.html
