News Hub

AI has surpassed human performance on a number of benchmarks, finds Stanford report

Written by Thu 18 Apr 2024

The latest edition of the Stanford University AI Index Report found that not only have artificial intelligence (AI) models outperformed humans on several benchmarks, but low-skilled workers could see the biggest boost from AI.

Despite the advances of AI in some areas such as image classification and visual reasoning, it is still unable to match humans on more complex tasks that require mathematical abilities and visual common sense reasoning.

“The data is in: AI makes workers more productive and leads to higher quality work. In 2023, several studies assessed AI’s impact on labour, suggesting that AI enables workers to complete tasks more quickly and to improve the quality of their output,” said the report.

With the use of AI, workers were able to complete work quicker and improve the quality of their output. Challenges remain around reaching appropriate standards in responsible AI reporting, with leading firms in the space, like OpenAI, Google, and Anthropic, using a range of different benchmarks.

While there may be no question that AI is developing fast, the cost to train these advanced models is extremely high and only increasing. The report finds that OpenAI’s GPT-4 used an estimated £62 million ($78 million) worth of compute to train, with Google’s Gemini Ultra costing £122 million ($191 million) in computing power spend.

“Frontier models get way more expensive … According to AI Index estimates, the training costs of state-of-the-art AI models have reached unprecedented levels,” said the report.

Frontier AI research is predominately being carried out by industry firms, with 51 of the leading machine learning models coming from business, 15 coming solely from academia and 21 resulting in industry-academia collaborations, over the past year.

As AI grows, so too are regulations that govern its usage. In the United States alone, 25 regulations were introduced last year around AI use, an increase from just one in 2016.

Last month, the European Union’s Parliament approved the EU Artificial Intelligence (AI) Act to ensure the safety and compliance of the technology whilst enhancing innovation. The regulation was endorsed by Members of the European Parliament (MEPs) with 523 votes in favour, 46 against, and 49 abstentions.

Join Tech Show London

12-13 March 2025, ExCeL London

Be a part of the latest tech conversations and discover pioneering innovations.

You won’t want to miss one of the most exciting technology events of the year.

Written by Thu 18 Apr 2024

Send us a correction Send us a news tip