Planning for Operational Control Systems with Predictable Exogenous Events


  • Ronen Brafman Ben-Gurion University of the Negev
  • Carmel Domshlak Technion - Israel Institute of Technology
  • Yagil Engel IBM Research
  • Zohar Feldman IBM Research


Various operational control systems (OCS) are naturally modeled as Markov Decision Processes. OCS often enjoy access to predictions of future events that have substantial impact on their operations. For example, reliable forecasts of extreme weather conditions are widely available, and such events can affect typical request patterns for customer response management systems, the flight and service time of airplanes, or the supply and demand patterns for electricity. The space of exogenous events impacting OCS can be very large, prohibiting their modeling within the MDP; moreover, for many of these exogenous events there is no useful predictive, probabilistic model. Realtime predictions, however, possibly with a short lead-time, are often available. In this work we motivate a model which combines offline MDP infinite horizon planning with realtime adjustments given specific predictions of future exogenous events, and suggest a framework in which such predictions are captured and trigger real-time planning problems. We propose a number of variants of existing MDP solution algorithms, adapted to this context, and evaluate them empirically.




How to Cite

Brafman, R., Domshlak, C., Engel, Y., & Feldman, Z. (2011). Planning for Operational Control Systems with Predictable Exogenous Events. Proceedings of the AAAI Conference on Artificial Intelligence, 25(1), 940-945. Retrieved from



Reasoning about Plans, Processes and Actions