Automatic Discovery and Transfer of Task Hierarchies in Reinforcement Learning


  • Neville Mehta Oregon State University
  • Soumya Ray Case Western Reserve University
  • Prasad Tadepalli Oregon State University
  • Thomas Dietterich Oregon State University



Sequential decision tasks present many opportunities for the study of transfer learning. A principal one among them is the existence of multiple domains that share the same underlying causal structure for actions. We describe an approach that exploits this shared causal structure to discover a hierarchical task structure in a source domain, which in turn speeds up learning of task execution knowledge in a new target domain. Our approach is theoretically justified and compares favorably to manually designed task hierarchies in learning efficiency in the target domain. We demonstrate that causally motivated task hierarchies transfer more robustly than other kinds of detailed knowledge that depend on the idiosyncrasies of the source domain and are hence less transferable.

Author Biographies

Neville Mehta, Oregon State University

PhD Candidate, Department of Computer Science

Soumya Ray, Case Western Reserve University

Assistant Professor, Department of Electrical Engineering and Computer Science

Prasad Tadepalli, Oregon State University

Professor, Computer Science Department

Thomas Dietterich, Oregon State University

Professor and Director of Intelligent Systems Research




How to Cite

Mehta, N., Ray, S., Tadepalli, P., & Dietterich, T. (2011). Automatic Discovery and Transfer of Task Hierarchies in Reinforcement Learning. AI Magazine, 32(1), 35.