(Georgia Institute of Technology, 2007)
Cantino, Andrew S.; Roberts, David L.; Isbell, Charles L.
In this paper, we address the problem of building a system of autonomous tour
guides for a complex environment, such as a museum with many visitors. Visitors
may have varying preferences for types of art or may wish to visit different areas
across multiple visits. Often, these goals conflict. For example, many visitors may
wish to see the museum's most popular work, but that could cause congestion,
ruining the experience. Thus, our task is to build a set of agents that can satisfy
their visitors' goals while simultaneously providing quality experiences for all.
We use targeted trajectory distribution MDPs (TTD-MDPs), a technology developed
to guide players in an interactive entertainment setting. The solution to a
TTD-MDP is a probabilistic policy that results in a specific distribution of trajectories
through a state space. We motivate TTD-MDPs for the museum tour problem,
then describe the development of a number of models of museum visitors.
Additionally, we propose a museum model and simulate tours using personalized
TTD-MDP tour guides for each kind of visitor. We explain how the use of probabilistic
policies reduces the congestion experienced by visitors while preserving
their ability to pursue and realize goals.
(Georgia Institute of Technology, 2007)
Zang, Peng; Irani, Arya; Isbell, Charles L.
We present a horizon-based value iteration algorithm called Reverse
Value Iteration (RVI). Empirical results on a variety of domains,
both synthetic and real, show RVI often yields speedups of
several orders of magnitude. RVI does this by ordering backups by
horizons, with preference given to closer horizons, thereby avoiding
many unnecessary and incorrect backups. We also compare
to related work, including prioritized and partitioned value iteration
approaches, and show that our technique performs favorably.
The techniques presented in RVI are complementary and can be
used in conjunction with previous techniques. We prove that RVI
converges and often has better (but never worse) complexity than
standard value iteration. To the authors’ knowledge, this is the first
comprehensive theoretical and empirical treatment of such an approach
to value iteration.