Organizational Unit:
School of Mathematics
School of Mathematics
Permanent Link
Research Organization Registry ID
Description
Previous Names
Parent Organization
Parent Organization
Organizational Unit
Includes Organization(s)
ArchiveSpace Name Record
Publication Search Results
Now showing
1 - 5 of 5
-
ItemTheory and computation of Wasserstein geometric flows with application to time-dependent Schrodinger equation(Georgia Institute of Technology, 2023-07-26) Wu, HaoWe focus on the systematic study of a novel computational framework to the Wasserstein geometric flows, which describe time evolution of probability density functions on the infinite-dimensional Wasserstein manifold. In particular, Wasserstein gradient flows (WGFs) and Wasserstein Hamiltonian flows (WHFs), are the main examples used throughout this research—they have many applications in real-world physics systems and more recently in deep learning problems such as generative models. The main feature of our computational framework is to use deep neural networks, to parameterize the push-forward maps such that they can push a simple reference density to the ones solving the WGFs or WHFs. This approach essentially reduces these flows defined on infinite-dimensional Wasserstein manifold to finite-dimensional dynamical systems of the parameters. These new dynamical systems are parameterizations of the WGFs and WHFs, which we call PWGFs and PWHFs for short. By leveraging a relaxed pullback Wasserstein metric on the parameter space, we can develop effective numerical methods to approximate the solutions of these flows. For WGFs, we show that our proposed PWGF scheme can be applied to WGF with general energy functional. Moreover, our scheme does not require any spatial discretization and thus is scalable to cases where the space dimensions of the problems are high. Our approach only requires solving standard least squares problems in each time step, hence is training free. With these features, PWGF demonstrates promising computational efficiency and accuracy on a variety of WGF examples, as shown in our numerical experiments. For WHFs, we adopt the similar idea but apply it to the more challenging Hamiltonian systems on Wasserstein manifolds. To preserve the Hamiltonian, we employ a symplectic numerical scheme to solve the PWHF, where a fixed-point iteration scheme is used to solve the implicit update equation of the model parameter. Similar to PWGF, PWHF is training free and thus avoids the issues of nonconvex optimization algorithms. We also present the connection between the Lagrangian and Eulerian perspectives of the original flows using PWHF. Approximation error analysis and a number of numerical examples are also provided using PWHF. Furthermore, we also consider the Schr\"odinger equations (SEs), and show how to use PWHF to solve them.
-
ItemLearning Dynamics from Data Using Optimal Transport Techniques and Applications(Georgia Institute of Technology, 2022-07-08) Ma, ShaojunOptimal Transport has been studied widely in recent years, the concept of Wasserstein distance brings a lot of applications in computational mathematics, machine learning, engineering, even finance areas. Meanwhile, people are gradually realizing that as the amount of data as well as the needs of utilizing data increase vastly, data-driven models have great potentials in real-world applications. In this thesis, we apply the theories of OT and design data-driven algorithms to form and compute various OT problems. We also build a framework to learn inverse OT problem. Furthermore, we develop OT and deep learning based models to solve stochastic differential equations, optimal control, mean field games related problems, all in data-driven settings. In Chapter 2, we provide necessary mathematical concepts and results that form the basis of this thesis. It contains brief surveys of optimal transport, stochastic differential equations, Fokker-Planck equations, deep learning, optimal controls and mean field games. Chapter 3 to Chapter 5 present several scalable algorithms to handle optimal transport problems within different settings. Specifically, Chapter 3 shows a new saddle scheme and learning strategy for computing the Wasserstein geodesic, as well as the Wasserstein distance and OT map between two probability distributions in high dimensions. We parametrize the map and Lagrange multipliers as neural networks. We demonstrate the performance of our algorithms through a series of experiments with both synthetic and realistic data. Chapter 4 presents a scalable algorithm for computing the Monge map between two probability distributions since computing the Monge maps remains challenging, in spite of the rapid developments of the numerical methods for optimal transport problems. Similarly, we formulate the problem as a mini-max problem and solve it via deep learning. The performance of our algorithms is demonstrated through a series of experiments with both synthetic and realistic data. In Chapter 5 we study OT problem in an inverse view, which we also call Inverse OT (IOT) problem. IOT also refers to the problem of learning the cost function for OT from observed transport plan or its samples. We derive an unconstrained convex optimization formulation of the inverse OT problem. We provide a comprehensive characterization of the properties of inverse OT, including uniqueness of solutions. We also develop two numerical algorithms, one is a fast matrix scaling method based on the Sinkhorn-Knopp algorithm for discrete OT, and the other one is a learning based algorithm that parameterizes the cost function as a deep neural network for continuous OT. Our numerical results demonstrate promising efficiency and accuracy advantages of the proposed algorithms over existing state-of-the-art methods. In Chapter 6 we propose a novel method using the weak form of Fokker Planck Equation (FPE) --- a partial differential equation --- to describe the density evolution of data in a sampled form, which is then combined with Wasserstein generative adversarial network (WGAN) in the training process. In such a sample-based framework we are able to learn the nonlinear dynamics from aggregate data without explicitly solving FPE. We demonstrate our approach in the context of a series of synthetic and real-world data sets. Chapter 7 introduces the application of OT and neural networks in optimal density control. Particularly, we parametrize the control strategy via neural networks, and provide an algorithm to learn the strategy that can drive samples following one distribution to new locations following target distribution. We demonstrate our method in both synthetic and realistic experiments, where we also consider perturbation fields. Finally Chapter 8 presents applications of mean field game in generative modeling and finance area. With more details, we build a GAN framework upon mean field game to generate desired distribution starting with white noise, we also investigate its connection to OT. Moreover, we apply mean field game theories to study the equilibrium trading price in stock markets, we demonstrate the theoretical result by conducting experiments on real trading data.
-
ItemOptimal Motion Planning and Computational Optimal Transport(Georgia Institute of Technology, 2022-04-28) Sun, HaodongIn many real life problems, the decision making process is guided by the principle of cost minimization. In this thesis, we focus on analyzing the theoretical properties and designing computational methods under the framework of several classical cost minimization problems, including optimal motion planning and optimal transport (OT). Over the past decades, motion planning has attracted large amount of attention in robotics. Given certain configurations in the environment, we are looking for trajectories which move the robot from one to the other. To produce high-quality trajectories, we propose a new computational method to design smooth and collision-free trajectories for motion planning with one or more robots. The functional cost in model leads to short and smooth trajectories. The designed method can also be generalized to problems with multiple robots. The idea of optimal transport naturally arises from many application scenarios including economy, computer science, etc. Optimal transport provides powerful tools for comparing probability measures in various types. However, obtaining the optimal transport plan is generally a computationally-expensive task. We start with an entropy transport problem as a relaxed version of original optimal transport problem with soft marginals, and propose an efficient algorithm to produce sample approximation for the optimal transport plan. This method can directly output samples from optimal plan between two continuous marginals without any discretization and network training. An inverse problem of OT is also of our interest. We present a computational framework for learning the cost function from the given optimal transport plan. The cost learning problem is reformulated as an unconstrained convex optimization problem and two efficient algorithms are proposed for discrete and continuous cost learning.
-
ItemNumerical computation and analysis related to optimal transport theory(Georgia Institute of Technology, 2022-04-28) Liu, ShuIn this thesis we apply the optimal transport (OT) theory to various disciplines of applied and computational mathematics such as scientific computing, numerical analysis, and dynamical systems. The research consists of three aspects: (1) We focus on solving OT problems from different perspectives including (a) direct approximation of the OT map in high dimensions; (b) particle evolving method for generating samples from the optimal transport plan; (c) learning high dimensional geodesics joining two given distributions. These different formulations find their own applications under distinct settings in diverse branches of data science and machine learning. We derive sample-based algorithms for each project. Our methods are supported by theoretical guarantees and numerical justifications. (2) We develop and analyze a sampling-friendly method for high dimensional Fokker-Planck equations by leveraging the generative models from deep learning. By utilizing the fact that the Fokker-Planck equation can be viewed as gradient flow on probability manifold equipped with certain OT distance, we derive an ordinary differential equation (ODE) on parameter space whose parameters are inherited from the generative models. We design a variational scheme for solving the proposed ODE. Both the convergence and error analysis results are established for our method. The performance and accuracy of the proposed algorithm are verified via several numerical examples. (3) We present a novel definition of Hamiltonian process on finite graphs by considering its corresponding density dynamics on probability manifold. We demonstrate the existence of such Hamiltonian process in many classical discrete problems, such as the OT problem, Schr\"odinger equation as well as Schr\"odinger bridge problem (SBP). The stationary and periodic properties of Hamiltonian processes are investigated in the framework of SBP.
-
ItemON SCALABLE AND FAST LANGEVIN-DYNAMICS-BASED SAMPLING ALGORITHMS(Georgia Institute of Technology, 2021-04-28) Li, RuilinLangevin dynamics-based sampling algorithms are arguably among the most widelyused Markov Chain Monte Carlo (MCMC) methods. Two main directions of the modern study of MCMC methods are (i) How to scale MCMC methods to big data applications, and (ii) Tight convergence analysis of MCMC algorithms, with explicit dependence on various characteristics of target distribution, in a non-asymptotic manner. This thesis continues the previous efforts in this two lines and consists of three parts. In the first part, we study stochastic gradient MCMC methods for large scale application. We propose a non-uniform subsampling of gradients scheme to approximately match the transition kernel of a base MCMC base with full gradient, aiming for better sample quality. The demonstration is based on underdamped Langevin dynamics. In the second part, we consider an analog of Nesterov’s accelerated algorithm in optimization for sampling. We derive a dynamics termed Hessian-Free-High-Resolution (HFHR) dynamics, from a high-resolution ordinary differential equation description of the Nesterov’s accelerated algorithm. We then quantify the acceleration of HFHR over underdamped Langevin dynamics at both continuous dynamics level and discrete algorithm level. In the third part, we study a broad family of bounded, contractive-SDE-based sampling algorithms via mean-square analysis. We show how to extend the applicability of classical mean-square analysis from finite time to infinite time. Iteration complexity in 2-Wasserstein distance is also characterized and when applied to Langevin Monte Carlo algorithm, we obtain an improved iteration complexity bound.