Explore-Then-Commit with Dynamic Expertise Amplification

Abstract

In resource-constrained real-world deployments, greedy exploration strategies that perpetually expand their budgets in pursuit of maximal reward become fundamentally impractical, demanding a principled framework that achieves high performance while strictly honoring computational and financial limits. Optimal allocation of limited resources between exploring new directions and exploiting proven strategies is a defining challenge in scientific research, education, and AI for Social Good. We introduce IDEA (Intelligent Dynamic Expertise Amplification via Explore-Then-Commit), a principled framework that recasts research strategy optimization as an expertise-aware contextual bandit problem. Theoretically, we prove that IDEA achieves expected regret E[R(T)] = O( √ KT log T) − Ω(ρ(T−τ )∥E∞∥), where the subtractive expertise term drives effective regret strictly negative for large T, a fundamental guarantee absent from state-of-the-art neural bandit methods, whose regret bounds are purely additive upper bounds of Oe( √ deT) with no compounding benefit. Computationally, IDEA requires only O(T(d 2+Kd)) operations, achieving a provable Θ(m2L/(Kd)) reduction in both time and memory over leading neural contextual bandit baselines. Empirically, in 1,300 controlled research trajectory simulations, IDEA delivers a cumulative reward of 15.5% higher than all state of-the-art bandit methods with strong statistical significance (p<0.05). On the real-world ASSISTments dataset (200,000 student interactions, 31,997 students, 252 skills, T=15,000 rounds), IDEA achieves a +21.7% higher mean reward over the best competing method (p<0.0001), while running 2.2× faster. These results establish IDEA as a theoretically grounded and practically superior framework for resource constrained decision-making in AI for Social Good.