Feynman-Kac Numerical Techniques for Stochastic Optimal Control

dc.contributor.advisor Tsiotras, Panagiotis
dc.contributor.author Hawkins, Kelsey Pal
dc.contributor.committeeMember Berenson, Dmitry
dc.contributor.committeeMember Coogan, Samuel
dc.contributor.committeeMember Theodorou, Evangelos
dc.contributor.committeeMember Vamvoudakis, Kyriakos
dc.contributor.department Interactive Computing
dc.date.accessioned 2022-01-14T16:07:08Z
dc.date.available 2022-01-14T16:07:08Z
dc.date.created 2021-12
dc.date.issued 2021-08-25
dc.date.submitted December 2021
dc.date.updated 2022-01-14T16:07:09Z
dc.description.abstract Three significant advancements are proposed for improving numerical methods in the solution of forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control (SOC) problems. First, we propose a novel characterization of FBSDE estimators as either on-policy or off-policy, highlighting the intuition for these techniques that the distribution over which value functions are approximated should, to some extent, match the distribution the policies generate. Second, two novel numerical estimators are proposed for improving the accuracy of single-timestep updates. In the case of LQR problems, we demonstrate both in theory and in numerical simulation that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems. Third, we propose a new method for accelerating the global convergence of FBSDE methods. By the repeated use of the Girsanov change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree.
dc.description.degree Ph.D.
dc.format.mimetype application/pdf
dc.identifier.uri http://hdl.handle.net/1853/66058
dc.language.iso en_US
dc.publisher Georgia Institute of Technology
dc.subject Stochastic optimal control
dc.subject Forward-backward stochastic differential equations
dc.subject Robotics
dc.subject Partial differential equations
dc.subject Motion planning
dc.subject Rapidly exploring random trees
dc.title Feynman-Kac Numerical Techniques for Stochastic Optimal Control
dc.type Text
dc.type.genre Dissertation
dspace.entity.type Publication
local.contributor.advisor Tsiotras, Panagiotis
local.contributor.corporatename College of Computing
local.contributor.corporatename School of Interactive Computing
relation.isAdvisorOfPublication bd4969ec-a869-452f-81f1-9f2dc8118b3c
relation.isOrgUnitOfPublication c8892b3c-8db6-4b7b-a33a-1b67f7db2021
relation.isOrgUnitOfPublication aac3f010-e629-4d08-8276-81143eeaf5cc
thesis.degree.level Doctoral
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
6.9 MB
Adobe Portable Document Format
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
3.87 KB
Plain Text