Cultural Algorithm Guided Policy Gradient with Parameter Exploration

Abstract

This study explores the integration of cultural algorithms (CA) with the Policy Gradients with Parameter-Based Exploration (PGPE) algorithm for the task of MNIST hand-written digit classification within the EvoJAX framework. The PGPE algorithm is enhanced by incorporating a belief space, consisting on Domain, Situational, and History knowledge sources (KS), to guide the search process and improve convergence speed. The PGPE algorithm, implemented within the EvoJAX framework, can efficiently find an optimal parameter-space policy for the MNIST task. However, increasing the complexity of the task and policy space, such as the CheXpert dataset and DenseNet, requires a more sophisticated approach to efficiently navigate the search space. We introduce CA-PGPE, a novel approach that integrates CA with PGPE to guide the search process and improve convergence speed. Future work will focus on incorporating exploratory knowledge sources and evaluate the enhanced CA-PGPE algorithm on more complex datasets and model architectures, such as CIFAR-10 and CheXpert with DenseNet.