Person:
Isbell,
Charles L.
Publication Search Results
Batch versus Interactive Learning by Demonstration
Horizon-based Value Iteration
Person:
Isbell,
Charles L.
1 - 3 of 3
Item
Agents that operate in human environments will
need to be able to learn new skills from everyday people.
Learning from demonstration (LfD) is a popular paradigm for
this. Drawing from our interest in Socially Guided Machine
Learning, we explore the impact of interactivity on learning
from demonstration. We present findings from a study with
human subjects showing people who are able to interact with
the learning agent provide better demonstrations (in part) by
adapting based on learner performance which results in improved
learning performance. We also find that interactivity increases a
sense of engagement and may encourage players to participate
longer. Our exploration of interactivity sheds light on how best
to obtain demonstrations for LfD applications.
Item
We present a horizon-based value iteration algorithm called Reverse
Value Iteration (RVI). Empirical results on a variety of domains,
both synthetic and real, show RVI often yields speedups of
several orders of magnitude. RVI does this by ordering backups by
horizons, with preference given to closer horizons, thereby avoiding
many unnecessary and incorrect backups. We also compare
to related work, including prioritized and partitioned value iteration
approaches, and show that our technique performs favorably.
The techniques presented in RVI are complementary and can be
used in conjunction with previous techniques. We prove that RVI
converges and often has better (but never worse) complexity than
standard value iteration. To the authors’ knowledge, this is the first
comprehensive theoretical and empirical treatment of such an approach
to value iteration.