Multi-Objective MDPs with Conditional Lexicographic Reward Preferences


  • Kyle Wray University of Massachusetts, Amherst
  • Shlomo Zilberstein University of Massachusetts, Amherst
  • Abdel-Illah Mouaddib University of Caen



multi-objective, momdp, lmdp, mdp, lexicographic preferences


Sequential decision problems that involve multiple objectives are prevalent. Consider for example a driver of a semi-autonomous car who may want to optimize competing objectives such as travel time and the effort associated with manual driving. We introduce a rich model called Lexicographic MDP (LMDP) and a corresponding planning algorithm called LVI that generalize previous work by allowing for conditional lexicographic preferences with slack. We analyze the convergence characteristics of LVI and establish its game theoretic properties. The performance of LVI in practice is tested within a realistic benchmark problem in the domain of semi-autonomous driving. Finally, we demonstrate how GPU-based optimization can improve the scalability of LVI and other value iteration algorithms for MDPs.




How to Cite

Wray, K., Zilberstein, S., & Mouaddib, A.-I. (2015). Multi-Objective MDPs with Conditional Lexicographic Reward Preferences. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1).