Predictability in Autonomous Driving Systems

Abstract

We study predictability in autonomous driving (AD) through capability-oriented evaluation, using assessors - lightweight meta-models that forecast a primary model’s per-instance performance. We extend this notion from language to vision-based AD, aiming to anticipate failures rather than merely explain them after the fact. Our current focus is traffic-sign detection: given a fixed detector, we train an assessor to predict frame-level quality from a compact feature vector that fuses scene context, environmental proxies and model-intrinsic signals. Enabling “predict-before-you-detect” risk flagging. In future work, we aim to (i) move from frame- to object-level assessment, (ii) integrate assessors with end-to-end vision-language AD systems, and (iii) study safety-relevant scaling laws across model size, data and compute. We will further incorporate real-world disengagement reports and adopt demand–ability profiling to relate instance difficulty (e.g., occlusion, weather) to system capability, toward transparent, independent reliability estimates for deployment.