Title:
Feynman-Kac Numerical Techniques for Stochastic Optimal Control
Feynman-Kac Numerical Techniques for Stochastic Optimal Control
| dc.contributor.advisor | Tsiotras, Panagiotis | |
| dc.contributor.author | Hawkins, Kelsey Pal | |
| dc.contributor.committeeMember | Berenson, Dmitry | |
| dc.contributor.committeeMember | Coogan, Samuel | |
| dc.contributor.committeeMember | Theodorou, Evangelos | |
| dc.contributor.committeeMember | Vamvoudakis, Kyriakos | |
| dc.contributor.department | Interactive Computing | |
| dc.date.accessioned | 2022-01-14T16:07:08Z | |
| dc.date.available | 2022-01-14T16:07:08Z | |
| dc.date.created | 2021-12 | |
| dc.date.issued | 2021-08-25 | |
| dc.date.submitted | December 2021 | |
| dc.date.updated | 2022-01-14T16:07:09Z | |
| dc.description.abstract | Three significant advancements are proposed for improving numerical methods in the solution of forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control (SOC) problems. First, we propose a novel characterization of FBSDE estimators as either on-policy or off-policy, highlighting the intuition for these techniques that the distribution over which value functions are approximated should, to some extent, match the distribution the policies generate. Second, two novel numerical estimators are proposed for improving the accuracy of single-timestep updates. In the case of LQR problems, we demonstrate both in theory and in numerical simulation that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems. Third, we propose a new method for accelerating the global convergence of FBSDE methods. By the repeated use of the Girsanov change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree. | |
| dc.description.degree | Ph.D. | |
| dc.format.mimetype | application/pdf | |
| dc.identifier.uri | http://hdl.handle.net/1853/66058 | |
| dc.language.iso | en_US | |
| dc.publisher | Georgia Institute of Technology | |
| dc.subject | Stochastic optimal control | |
| dc.subject | Forward-backward stochastic differential equations | |
| dc.subject | Robotics | |
| dc.subject | Partial differential equations | |
| dc.subject | Motion planning | |
| dc.subject | Rapidly exploring random trees | |
| dc.title | Feynman-Kac Numerical Techniques for Stochastic Optimal Control | |
| dc.type | Text | |
| dc.type.genre | Dissertation | |
| dspace.entity.type | Publication | |
| local.contributor.advisor | Tsiotras, Panagiotis | |
| local.contributor.corporatename | College of Computing | |
| local.contributor.corporatename | School of Interactive Computing | |
| relation.isAdvisorOfPublication | bd4969ec-a869-452f-81f1-9f2dc8118b3c | |
| relation.isOrgUnitOfPublication | c8892b3c-8db6-4b7b-a33a-1b67f7db2021 | |
| relation.isOrgUnitOfPublication | aac3f010-e629-4d08-8276-81143eeaf5cc | |
| thesis.degree.level | Doctoral |