Beyond Message Passing: Modern GNN Architectures for Online Planner Selection

Abstract

As planning is computationally hard, the performance of automated planners varies greatly across planning tasks. Thus, the ability to predict planner performance on a given task is of great importance. While various learning methods have been applied in cost-optimal planning, Graph Neural Networks (GNNs) were found to perform well. However, existing work only explores a limited range of homogeneous GNN architectures and focuses primarily on the model perspective. We address these limitations by approaching the problem from both data and model perspectives. From the data perspective, we analyze the planners' performance data in terms of Shapley values to assess the potential contribution of individual planners to a portfolio. Our insights enable us to effectively reduce the portfolio from 17 to 6 planners, improving practicality and performance. From the model perspective, our work extends previous investigations of homogeneous graphs by modeling planning tasks as heterogeneous graphs and applying the heterogeneous Relational Graph Convolutional Network (RGCN) and Relational Graph Attention Network (RGAT) models. To analyze the problem in more depth, we thoroughly investigate the impact of GNN model, graph representation, node features, and prediction task. Going further, we propose a hybrid approach in which graph representations obtained by GNNs are used as input to a classical machine learning model (XGBoost), resulting in both a more resource-efficient and accurate approach. Our best model (RGCN+XGBoost) achieves 91.7% accuracy, a substantial improvement over previous methods with 87%, while requiring fewer computational resources. Overall, we demonstrate the effectiveness of heterogeneous GNN-based online planner selection methods, opening up new exciting avenues for future research.