Question

When will AI achieve competency on multi-choice questions across diverse fields of expertise (MMLU)?

26 comments26

Total Forecasters86

Community Prediction2025-06-27

Make a Prediction

My Prediction

community

lower 25%

median

upper 75%

>2039-12-31

...

2024-09-25 16:28

2025-06-27 19:53

2026-08-26 16:20

0.9%

Authors:

isinlor

Opened:

Closes:

Scheduled resolution:

AI Demonstrations

Computing and Math

Artificial Intelligence

Machine learning

Artificial intelligence

Computer science

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027-05-01

Comments

? comments

Authors:

isinlor

Opened:

Closes:

Scheduled resolution:

AI Demonstrations

Computing and Math

Artificial Intelligence

Machine learning

Artificial intelligence

Computer science

The Long Road to Genuine AI Mastery

Time•Sep 12, 2024

OpenAI releases o1, its first model with ‘reasoning’ abilities

Verge•Sep 13, 2024

The Evolution of LLMs Through Real-Time Learning

Psychology Today•Sep 13, 2024

Learn more about Metaculus NewsMatch

On January 1, 2025, which frontier AI lab will have a publicly available model with the highest score on the MMLU benchmark?

What will be state-of-the-art accuracy on the Massive Multitask dataset on the following dates?

When will an AI be able to solve over 80% of competition-level coding problems on its first try?

2027-05-01