Planner Museum: Evaluating Classical Planners Over Time

Abstract

The International Planning Competition (IPC) has been running since 1998, providing a standardized platform on which different planners can compete on equal footing. Over the years, the competition has showcased diverse approaches to planning and has thus become a record of the paradigm shifts the field has undergone. In this paper, we introduce the Planner Museum, a project that collects distinguished planning systems from past competitions and enables them to run on modern computers. By evaluating these historical planners alongside modern ones on challenging benchmark problems from multiple IPCs, we quantify the rate of improvement of state-of-the-art planners over the past 28 years and show that progress has been steady.