Title:
Automatic Task Decomposition and State Abstraction from Demonstration
Automatic Task Decomposition and State Abstraction from Demonstration
Author(s)
Cobo, Luis C.
Isbell, Charles L.
Thomaz, Andrea L.
Isbell, Charles L.
Thomaz, Andrea L.
Advisor(s)
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
Both Learning from Demonstration (LfD) and Reinforcement Learning (RL) are popular approaches for building decision-making agents. LfD applies supervised learning to a set of human demonstrations to infer and imitate the human policy, while RL uses only a reward signal and exploration to find an optimal policy. For complex tasks both of these techniques may be ineffective. LfD may require many more demonstrations than it is feasible to obtain, and RL can take an inadmissible amount of time to converge. We present Automatic Decomposition and Abstraction from demonstration (ADA), an algorithm that uses mutual information measures over a set of human demonstrations to decompose a sequential decision process into several sub- tasks, finding state abstractions for each one of these sub- tasks. ADA then projects the human demonstrations into the abstracted state space to build a policy. This policy can later be improved using RL algorithms to surpass the performance of the human teacher. We find empirically that ADA can find satisficing policies for problems that are too complex to be solved with traditional LfD and RL algorithms. In particular, we show that we can use mutual information across state features to leverage human demonstrations to reduce the effects of the curse of dimensionality by finding subtasks and abstractions in sequential decision processes.
Sponsor
Date Issued
2012-06
Extent
Resource Type
Text
Resource Subtype
Proceedings