Over the past five years, AI models have become much more complex and capable, tailored to perform specific tasks across industries and provide better efficiency, accuracy and automation. However, the cost of training in these systems has exploded. According to data presented by AltIndex.com, AI model training costs have skyrocketed by more than 4,300% since 2020.
44x Higher Cost in Just Four Years
The release of OpenAI’s ChatGPT-4 in March last year kickstarted the global AI hype. Google followed suit with its advanced AI model, Gemini, nine months later. Although both systems brought improvements, new capabilities, and better applications, they have been much more expensive to train than previous AI models.
The Artificial Intelligence Index Report released by Stanford University and research firm Epoch AI, which analyzed cloud computing rental prices, the model’s training duration, the hardware’s utilization rate, and the value of the training hardware, showed how much precisely. The difference is shocking.
Statistics show the cost of training Gemini, a large language model that can be inputted with text, voice commands, and images, stood between $30 million and $191 million, excluding staff salaries which can make up from 29% to 49% of the final price. Most of the development cost, between 47% and 67%, was for the hardware, while 2%–6% went to energy consumption. Compared to Gemini, GPT-4 had a much lower technical creation cost of between $41 million and $78, which still rose to over $100 million when including staff costs.
Looking back, the cost of earlier AI models was much lower. For instance, it cost just $930 to train Transformer, a groundbreaking neural network architecture introduced in 2017. Google’s BERT-Large, released a year later, cost $3,288. Meta’s RoBERTa Large, introduced in 2019, had a training cost of over $160,000, which snowballed to over $4,3 million with OpenAI’s GPT3 a year later. However, that is still 44 times less compared to the latest AI training models released last year.
The Cost of Largest Training Runs Could Hit Over One Billion Dollars by 2027
Even with 44 times lower AI model training costs, keeping up with cutting-edge AI development became a privilege for only the most funded organizations. Although companies are already exploring ways to reduce them, like developing smaller models for specific tasks or using synthetic data for training, the snowballed AI model training costs remain a huge challenge for sustainable AI development.
Unfortunately, with the surging need for substantial computational resources, more complex model architectures, and large-scale data requirements, they will only continue rising in the future. According to the Artificial Intelligence Index Report, the largest training runs will cost more than a billion dollars by 2027 if the trend continues.
