Gen AI training costs soar yet risks are poorly measured, says Stanford AI report

Key Points:

  • The number of significant new AI models from industry is surging compared to academia and government.
  • The soaring costs of training large language models like OpenAI’s GPT-4 and poor measurement of risks in AI.
  • Challenges in standardization of responsible AI reporting and the proliferation of benchmark tests for assessing AI safety.


In the domain of artificial intelligence (AI), the landscape has seen an evident shift towards industry—marked by a surge in the emergence of significant new AI models from commercial entities compared to academia and government sectors. Stanford University’s Institute for Human-Centered Artificial Intelligence (HAI) has illuminated key concerns in its latest report, “The AI Index 2024 Annual Report.” Notably, the report underscores the exponential rise in the costs associated with training state-of-the-art AI models, exemplified by OpenAI’s GPT-4 and Google’s Gemini Ultra, which incurred soaring expenses totaling $78 million and $191 million, respectively.


However, alongside the escalating costs lies a troubling dearth of standardized measures to evaluate the risks posed by these large AI models. The report highlights a fragmented landscape of responsible AI benchmarks, reflecting a significant challenge in systematically assessing and comparing potential risks and limitations of top AI models. As the AI industry, particularly Gen AI, gains momentum, underpinned by growing commercial interests and real-world deployments, the need for cohesive standards in responsible AI reporting becomes increasingly imperative.


Moreover, the report sheds light on the rapid expansion in the industrial market for AI, with a notable emphasis on generative AI investment surging in 2023 and a substantial increase in the release of machine learning models by industry players. This shift towards industrial dominance is exemplified by a spike in Fortune 500 companies incorporating AI in their operations. Nevertheless, as industrial AI continues to flourish, concerns persist regarding the burgeoning training costs of AI models and the evolving challenges in ensuring their safety, transparency, and data privacy.


Unveiling crucial insights on the spiraling training costs, the report, in collaboration with research institute Epoch AI, highlights a staggering escalation in expenses aligned with the growing computational demands of large AI models. Illustratively, the report traces the evolution of training costs from the original Transformer model in 2017 to the recent OpenAI’s GPT-4 and Google’s Gemini Ultra, thereby spotlighting the profound financial investments indispensable for developing cutting-edge AI capabilities.


Amidst these concerns, the imperative for streamlined and consistent evaluation frameworks for responsible AI emerges as a vital imperative to navigate the complex terrain of AI ethics and accountability. The report underscores the necessity for aligning on standardized benchmarks to enhance responsible AI reporting and facilitate comprehensive assessments of AI models across diverse domains.


While AI continues to catalyze enhanced productivity and efficiency across various sectors, with notable improvements observed in project completion rates, quality, and speed bolstered by AI-empowered tools, such as Microsoft Copilot and GitHub’s Copilot, there are also nuanced impacts observed across different labor cohorts. From augmenting the performance of less-skilled consultants to enhancing the handling capacity of call-center agents, AI’s transformative influence extends to diverse professional realms.


However, the multifaceted nature of AI’s impact unfolds complexities, as evidenced by contrasting outcomes in distinct professional settings. The phenomenon of complacency induced by reliance on AI tools among recruitment professionals, as elucidated in a Harvard study, underlines the nuanced challenges intertwined with AI adoption and underscores the pivotal role of vigilance in leveraging AI to augment human capabilities while mitigating potential pitfalls.



Prompt Engineering Guides



©2024 The Horizon