On Forecasting Lags in AI Risk Evaluation

Abstract

AI risk evaluations are key to building safety cases for new AI systems. These evaluations help us track how quickly new AI systems are gaining competencies and propensities to engage in harmful behavior. However, it is not currently clear how well we are doing in covering the surface of potential risks that AI systems could present as they continue to improve. This paper proposes a modelling framework for predicting lags in AI risk evaluations, illustrating our approach using data from METR's evaluations of the ability for AI to complete long tasks. We find in our simulations that a business-as-usual approach to tracking AI risks is vulnerable to substantial detection lags when facing budget constraints. We advise greater adoption of the logistic estimator proposed by METR where appropriate, as it mitigates against such lags.