Learning Distributed Scheduling via LLM-Augmented Reinforcement Learning

Abstract

The distributed flexible job-shop scheduling problem (DFJSP) aims to coordinate job execution across distributed factories to achieve production goals. Existing reinforcement learning (RL)-based scheduling algorithms have made processes in learning adaptive scheduling polices, but rely on shallow networks and simple handcrafted rewards. These designs limit global state reasoning and accurate credit assignment under sparse rewards, thereby hindering the balanced workload distribution and efficient policy learning. To address these limitations, we propose a Large Language Model (LLM)-augmented RL algorithm tailored for DFJSP by leveraging the contextual reasoning and prior knowledge of LLM. Specifically, we propose an LLM-driven factory assignment mechanism that encodes global factory states and job features into structured queries, enabling context-aware and effective coordination among factories. Furthermore, we design an LLM-informed reward model that encodes scheduling-aware semantics into multi-dimensional proxy rewards for precise credit assignment during training. Theoretical analysis establishes bounds on the reward approximation error and demonstrates that the designed factory assignment can effectively reduce global workload variance. Moreover, extensive experiments on two benchmarks (i.e., Hurink and Brandimarte) and simulation-based DFJSP instances of varying scales demonstrate that our algorithm outperforms state-of-the-art RL algorithms, achieving the average makespan improvement ranging from 0.61% up to 25.78%.