The Training Costs of AI Models Over Time

What We’re Showing

The rising cost of training AI models over time, based on data from the 2024 Artificial Intelligence Index Report released by Stanford University.

How Training Cost is Determined

Stanford University collaborated with research firm Epoch AI to estimate AI model training costs, which were based on cloud compute rental prices.

Key factors that were analyzed include the model's training duration, the hardware’s utilization rate, and the value of the training hardware.

Ballooning Training Costs

The cost of training AI models has increased dramatically these past few years. For instance, it cost just $930 to train Transformer, a groundbreaking neural network architecture introduced in 2017—yet it cost over $78 million to train GPT-4 last year.

Google’s AI model, Gemini Ultra, costs even more, at a staggering $191 million. As of early 2024, the model outperformed GPT-4 on several metrics, most notably across the Massive Multitask Language Understanding (MMLU) benchmark.

This benchmark serves as a crucial yardstick for gauging the capabilities of large language models, evaluating its knowledge and problem solving proficiency across 57 subject areas.