METR (reported by MIT Technology Review) · Time horizon benchmark

METR time horizon plot update

METR has updated its time horizon benchmark, which measures how long a task a model can complete relative to human completion time. The new suite adds longer tasks and more open-ended ones, such as designing AI evaluations and scoring philosophical arguments. Models fall short of human performance on these judgment-heavy tasks, though newer models such as Opus 5.5 score "borderline above the bar" on evaluation design, up from "clearly below the bar" a year earlier. Many completion times are estimates rather than measured human results.

Open the source ↗

Similar Research & Safety releases

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.