Conflict-Aware Risk-Averse and Safe Reinforcement Learning: A Meta-Cognitive Learning Framework

Modares, Hamidreza

Title:

Conflict-Aware Risk-Averse and Safe Reinforcement Learning: A Meta-Cognitive Learning Framework

dc.contributor.author	Modares, Hamidreza	en_US
dc.contributor.corporatename	Georgia Institute of Technology. Institute for Robotics and Intelligent Machines	en_US
dc.contributor.corporatename	Michigan State University. Department of Mechanical Engineering	en_US
dc.date.accessioned	2021-04-23T17:55:05Z
dc.date.available	2021-04-23T17:55:05Z
dc.date.issued	2021-04-14
dc.description	Presented online April 14, 2021 at 12:15 p.m.	en_US
dc.description	Hamidreza Modares is an Assistant Professor in the Department of Mechanical Engineering at Michigan State University. Prior to joining Michigan State University, he was an Assistant Professor in the Department of Electrical Engineering, Missouri University of Science and Technology. His current research interests include control and security of cyber–physical systems, machine learning in control, distributed control of multi-agent systems, and robotics. He is an Associate Editor of IEEE Transactions on Neural Networks and Learning Systems.	en_US
dc.description	Runtime: 60:34 minutes	en_US
dc.description.abstract	While the success of reinforcement learning (RL) in computer games has shown impressive engineering feat, unlike the computer games, safety-critical settings such as unmanned vehicles must thrash around in the real world, which makes the entire enterprise unpredictable. Standard RL practice generally implants pre-specified performance metrics or objectives into the RL agent to encode the designers’ intention and preferences in achieving different and sometimes conflicting goals (e.g., cost efficiency, safety, speed of response, accuracy, etc.). Optimizing pre-specified performance metrics, however, cannot provide safety and performance guarantees across a vast variety of circumstances that the system might encounter in non-stationary and hostile environments. In this talk, I will discuss novel metacognitive RL algorithms to learn not only a control policy that optimizes accumulated reward values, but also what reward functions to optimize in the first place to formally assure safety with a good enough performance. I will present safe RL algorithms that adapt the focus of attention of RL algorithm to its variety of performance and safety objectives to resolve conflict and thus assure the feasibility of the reward function in a new circumstance. Moreover, model-free RL algorithms will be presented to solve the risk-averse optimal control (RAOC) problem to optimize the expected utility of outcomes while reducing the variance of cost under aleatory uncertainties (i.e., randomness). This is because, performance-critical systems must not only optimize the expected performance, but also reduce its variance to avoid performance fluctuation during RL’s course of operation. To solve the RAOC problem, I will present the three variants of RL algorithms and analyze their advantages and preferences for different situations/systems: 1) a one-shot static convex program based RL, 2) an iterative value iteration algorithm that solves a linear programming optimization at each iteration, and 3) an iterative policy iteration algorithm that solves a convex optimization at each iteration and guarantees the stability of the consecutive control policies.	en_US
dc.format.extent	60:34 minutes
dc.identifier.uri	http://hdl.handle.net/1853/64458	en_US
dc.language.iso	en_US	en_US
dc.publisher	Georgia Institute of Technology	en_US
dc.relation.ispartofseries	IRIM Seminar Series
dc.subject	Safe control	en_US
dc.subject	Reinforcement learning	en_US
dc.title	Conflict-Aware Risk-Averse and Safe Reinforcement Learning: A Meta-Cognitive Learning Framework	en_US
dc.type	Moving Image
dc.type.genre	Lecture
dspace.entity.type	Publication
local.contributor.corporatename	Institute for Robotics and Intelligent Machines (IRIM)
local.relation.ispartofseries	IRIM Seminar Series
relation.isOrgUnitOfPublication	66259949-abfd-45c2-9dcc-5a6f2c013bcf
relation.isSeriesOfPublication	9bcc24f0-cb07-4df8-9acb-94b7b80c1e46