Organizational Unit:

Mobile Robot Laboratory

Permanent Link

https://hdl.handle.net/1853/70779

Parent Organization

Organizational Unit

College of Computing

ArchiveSpace Name Record

https://finding-aids.library.gatech.edu/agents/corporate_entities/1110

Full item page

Publication Search Results

Now showing 1 - 9 of 9

Learning of Parameter-Adaptive Reactive Controllers for Robotic Navigation

(Georgia Institute of Technology, 1997) Ramesh, Ashwin ; Santamaria, Juan Carlos

Reactive controllers are widely used in mobile robots because they are able to achieve successful performance in real-time. However, the configuration of a reactive controller depends highly on the operating conditions of the robot and the environment; thus, a reactive controller configured for one class of environments may not perform adequately in another. This paper presents a formulation of parameter-adaptive reactive controllers. Parameter-adaptive reactive controllers inherit all the advantages of traditional reactive controllers, but in addition they are able to adjust themselves to the current operating conditions of the robot and the environment in order to improve task performance. Additionally, the paper describes a multistrategy learning algorithm that combines ideas from case-based reasoning and reinforcement learning to construct a mapping between the operating conditions of the mobile robot and the appropriate controller configuration; this mapping is in turn used to adapt the controller configuration dynamically. The algorithm is implemented and evaluated in a robotic navigation system that controls a Denning MRV-III mobile robot.
A New Heuristic Approach for Dual Control

(Georgia Institute of Technology, 1997) Ram, Ashwin ; Santamaria, Juan Carlos

Autonomous agents engaged in a continuous interaction with an incompletely known environment face the problem of dual control [Fel’dbaum 1965]. Simply stated, actions are necessary not only for studying the environment, but also for making progress on the task. In other words, actions must bear a “dual” character: They must be investigators to some degree, but also directors to some degree. Because the number of variables involved in the solution of the dual control problem increases with the number of decision stages, the exact solution of the dual control problem is computationally intractable except for a few special cases. This paper provides an overview of dual control theory and proposes a heuristic approach towards obtaining a near-optimal dual control method that can be implemented. The proposed algorithm selects control actions taking into account the information contained in past observations as well as the possible information that future observations may reveal. In short, the algorithm anticipates the fact that future learning is possible and selects the control actions accordingly. The algorithm uses memory-based methods to associate long-term benefit estimates to belief states and actions, and selects the actions to execute next according to such estimates. The algorithm uses the outcome of every experience to progressively refine the long-term benefit estimates so that it can make better, improved decisions as it progresses. The algorithm is tested on a classical simulation problem.
Experiments With Reinforcement Learning in Problems With Continuous State and Action Spaces

(Georgia Institute of Technology, 1996) Ram, Ashwin ; Santamaria, Juan Carlos ; Sutton, Richard S.

A key element in the solution of reinforcement learning problems is the value function. The purpose of this function is to measure the long-term utility or value of any given state and it is important because an agent can use it to decide what to do next. A common problem in reinforcement learning when applied to systems having continuous states and action spaces is that the value function must operate with a domain consisting of real-valued variables, which means that it should be able to represent the value of infinitely many state and action pairs. For this reason, function approximators are used to represent the value function when a close-form solution of the optimal policy is not available. In this paper, we extend a previously proposed reinforcement learning algorithm so that it can be used with function approximators that generalize the value of individual experiences across both, state and action spaces. In particular, we discuss the benefits of using sparse coarse-coded function approximators to represent value functions and describe in detail three implementations: CMAC, instance-based, and case-based. Additionally, we discuss how function approximators having different degrees of resolution in different regions of the state and action spaces may influence the performance and learning efficiency of the agent. We propose a simple and modular technique that can be used to implement function approximators with non-uniform degrees of resolution so that it can represent the value function with higher accuracy in important regions of the state and action spaces. We performed extensive experiments in the double integrator and pendulum swing up systems to demonstrate the proposed ideas.
Structured Light Systems for Dent Recognition: Lessons Learned

(Georgia Institute of Technology, 1995) Arkin, Ronald C. ; Santamaria, Juan Carlos

This paper describes the results from a feasibility analysis performed on two different structured light system designs and the image processing algorithms they require for dent detection and localization. The impact of each structured light system is analyzed in terms of their mechanical realization and the complexity of the image processing algorithms required for robust dent detection. The two design alternatives considered consist of projecting vertical or horizontal laser stripes on the drum surface. The first alternative produces straight lines in the image plane and requires scanning the drum surface horizontally, whereas the second alternative produces conic curves on the camera plane and requires scanning the drum surface vertically. That is, the first alternative favors image processing against mechanical realization while the second alternative favors mechanical realization against image processing. The results from simulated and real structured light systems are presented and their major advantages and disadvantages for dent detection are presented. The paper concludes with the lessons learned from experiments with real and simulated structured light system prototypes.
Io, Ganymede and Callisto - a Multiagent Robot Trash-Collecting Team

(Georgia Institute of Technology, 1995) Balch, Tucker ; Boone, Gary Noel ; Collins, Tom ; Forbes, Harold ; MacKenzie, Douglas Christopher ; Santamaria, Juan Carlos

Georgia Tech won the Office Cleanup Event at the 1994 AAAI Mobile Robot Competition with a multi-robot cooperating team. This paper describes the design and implementation of these reactive trash-collecting robots, including details of multiagent cooperation, color vision for the detection of perceptual object classes, temporal sequencing of behaviors for task completion, and a language for specifying motor schema-based robot behaviors.
Model-Based Echolocation of Environmental Objects

(Georgia Institute of Technology, 1995) Arkin, Ronald C. ; Santamaria, Juan Carlos

This paper presents an algorithm that can recognize and localize objects given a model of their contours using only ultrasonic range data. The algorithm exploits a physical model of the ultrasonic beam and combines several readings to extract outline object segments from the environment. It then detects patterns of outline segments that correspond to predefined models of object contours, performing both object recognition and localization. The algorithm is robust since it can account for noise and inaccurate readings as well as efficient since it uses a relaxation technique that can incorporate new data incrementally without recalculating from scratch.
A Multistrategy Case-Based and Reinforcement Learning Approach to Self-Improving Reactive Control Systems for Autonomous Robotic Navigation

(Georgia Institute of Technology, 1993) Ram, Ashwin ; Santamaria, Juan Carlos

This paper presents a self-improving reactive control system for autonomous robotic navigation. The navigation module uses a schema-based reactive control system to perform the navigation task. The learning module combines case-based reasoning and reinforcement learning to continuously tune the navigation system through experience. The case-based reasoning component perceives and characterizes the system’s environment, retrieves an appropriate case, and uses the recommendations of the case to tune the parameters of the reactive control system. The reinforcement learning component refines the content of the cases based on the current experience. Together, the learning components perform on-line adaptation, resulting in improved performance as the reactive control system tunes itself to the environment, as well as on-line learning, resulting in an improved library of cases that capture environmental regularities necessary to perform on-line adaptation. The system is extensively evaluated through simulation studies using several performance metrics and system configurations.
Knowledge Compilation and Speedup Learning in Continuous Task Domains

(Georgia Institute of Technology, 1993) Ram, Ashwin ; Santamaria, Juan Carlos

Many techniques for speedup learning and knowledge compilation focus on the learning and optimization of macro-operators or control rules in task domains that can be characterized using a problem-space search paradigm. However, such a characterization does not fit well the class of task domains in which the problem solver is required to perform in a continuous manner. For example, in many robotic domains, the problem solver is required to monitor real-valued perceptual inputs and vary its motor control parameters in a continuous, on-line manner to successfully accomplish its task. In such domains, discrete symbolic states and operators are difficult to define. To improve its performance in continuous problem domains, a problem solver must learn, modify, and use “continuous operators” that continuously map input sensory information to appropriate control outputs. Additionally, the problem solver must learn the contexts in which those continuous operators are applicable. We propose a learning method that can compile sensorimotor experiences into continuous operators, which can then be used to improve performance of the problem solver. The method speeds up the task performance as well as results in improvements in the quality of the resulting solutions. The method is implemented in a robotic navigation system, which is evaluated through extensive experimentation.
Multistrategy Learning in Reactive Control Systems for Autonomous Robotic Navigation

(Georgia Institute of Technology, 1993) Ram, Ashwin ; Santamaria, Juan Carlos

This paper presents a self-improving reactive control system for autonomous robotic navigation. The navigation module uses a schema-based reactive control system to perform the navigation task. The learning module combines case-based reasoning and reinforcement learning to continuously tune the navigation system through experience. The case-based reasoning component perceives and characterizes the system's environment, retrieves an appropriate case, and uses the recommendations of the case to tune the parameters of the reactive control system. The reinforcement learning component refines the content of the cases based on the current experience. Together, the learning components perform on-line adaptation, resulting in improved performance as the reactive control system tunes itself to the environment, as well as on-line case learning, resulting in an improved library of cases that capture environmental regularities necessary to perform on-line adaptation. The system is extensively evaluated through simulation studies using several performance metrics and system configurations.