2024 News

Draft "A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations" is online. We propose a scalable preconditioned primal-dual hybrid gradient algorithm for solving partial differential equations (PDEs). We multiply the PDE with a dual test function to obtain an inf-sup problem whose loss functional involves lower-order differential operators. The Primal-Dual Hybrid Gradient (PDHG) algorithm is then leveraged for this saddle point problem. By introducing suitable precondition operators to the proximal steps in the PDHG algorithm, we obtain an alternative natural gradient ascent-descent optimization scheme for updating the neural network parameters. We apply the Krylov subspace method (MINRES) to evaluate the natural gradients efficiently. Such treatment readily handles the inversion of precondition matrices via matrix-vector multiplication. A posterior convergence analysis is established for the time-continuous version of the proposed method. The algorithm is tested on various types of PDEs with dimensions ranging from $1$ to $50$, including linear and nonlinear elliptic equations, reaction-diffusion equations, and Monge-Amp\`ere equations stemming from the $L^2$ optimal transport problems. We compare the performance of the proposed method with several commonly used deep learning algorithms such as physics-informed neural networks (PINNs), the DeepRitz method, weak adversarial networks (WANs), etc, for solving PDEs using the Adam and L-BFGS optimizers. The numerical results suggest that the proposed method performs efficiently and robustly and converges more stable. Nov 12, 2024.

Draft "Gradient-adjusted underdamped Langevin dynamics for sampling" is online. Sampling from a target distribution is a fundamental problem with wide-ranging applications in scientific computing and machine learning. Traditional Markov chain Monte Carlo (MCMC) algorithms, such as the unadjusted Langevin algorithm (ULA), derived from the overdamped Langevin dynamics, have been extensively studied. From an optimization perspective, the Kolmogorov forward equation of the overdamped Langevin dynamics can be treated as the gradient flow of the relative entropy in the space of probability densities embedded with Wasserstein-2 metrics. Several efforts have also been devoted to including momentum-based methods, such as underdamped Langevin dynamics for faster convergence of sampling algorithms. Recent advances in optimizations have demonstrated the effectiveness of primal-dual damping and Hessian-driven damping dynamics for achieving faster convergence in solving optimization problems. Motivated by these developments, we introduce a class of stochastic differential equations (SDEs) called gradient-adjusted underdamped Langevin dynamics (GAUL), which add stochastic perturbations in primal-dual damping dynamics and Hessian-driven damping dynamics from optimization. We prove that GAUL admits the correct stationary distribution, whose marginal is the target distribution. The proposed method outperforms overdamped and underdamped Langevin dynamics regarding convergence speed in the total variation distance for Gaussian target distributions. Moreover, using the Euler-Maruyama discretization, we show that the mixing time towards a biased target distribution only depends on the square root of the condition number of the target covariance matrix. Numerical experiments for non-Gaussian target distributions, such as Bayesian regression problems and Bayesian neural networks, further illustrate the advantages of our approach over classical methods based on overdamped or underdamped Langevin dynamics. Oct 14, 2024.

Draft "Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problems" is online. Classical neural ordinary differential equations (ODEs) are powerful tools for ap- proximating the log-density functions in high-dimensional spaces along trajecto- ries, where neural networks parameterize the velocity fields. This paper proposes a system of neural differential equations representing first- and second-order score functions along trajectories based on deep neural networks. We reformulate the mean field control (MFC) problem with individual noises into an unconstrained optimization problem framed by the proposed neural ODE system. Addition- ally, we introduce a novel regularization term to enforce characteristics of viscous Hamilton–Jacobi–Bellman (HJB) equations to be satisfied based on the evolution of the second-order score function. Examples include regularized Wasserstein proximal operators (RWPOs), probability flow matching of Fokker–Planck (FP) equations, and linear quadratic (LQ) MFC problems, which demonstrate the ef- fectiveness and accuracy of the proposed method. Sep 26, 2024.

Draft "Convergence of Noise-Free Sampling Algorithms with Regularized Wasserstein Proximals" is online. We investigate the convergence properties of the backward regularized Wasserstein proximal (BRWP) method for sampling a target distribution. The BRWP approach can be shown as a semi-implicit time discretization for a probability flow ODE with the score function whose density satisfies the Fokker-Planck equation of the overdamped Langevin dynamics. Specifically, the evolution of the score function is computed using a kernel formula derived from the regularized Wasserstein proximal operator. By applying the Laplace method to obtain the asymptotic expansion of this kernel formula, we establish guaranteed convergence in terms of the Kullback-Leibler divergence for the BRWP method towards a strongly log-concave target distribution. Our analysis also identifies the optimal and maximum step sizes for convergence. Furthermore, we demonstrate that the deterministic and semi-implicit BRWP scheme outperforms many classical Langevin Monte Carlo methods, such as the Unadjusted Langevin Algorithm (ULA), by offering faster convergence and reduced bias. Numerical experiments further validate the convergence analysis of the BRWP method. Sep 3, 2024.

Draft "Fluctuations in Wasserstein dynamics on Graphs" is online. We propose a drift-diffusion process on the probability simplex to study stochastic fluctuations in probability spaces. We construct a counting process for linear detailed balanced chemical reactions with finite species such that its thermodynamic limit is a system of ordinary differential equations (ODE) on the probability simplex. This ODE can be formulated as a gradient flow with an Onsager response matrix that induces a Riemannian metric on the probability simplex. After incorporating the induced Riemannian structure, we propose a diffusion approximation of the rescaled counting process for molecular species in the chemical reactions, which leads to Langevin dynamics on the probability simplex with a degenerate Brownian motion constructed from the eigen-decomposition of Onsager's response matrix. The corresponding Fokker-Planck equation on the simplex can be regarded as the usual drift-diffusion equation with the extrinsic representation of the divergence and Laplace-Beltrami operator. The invariant measure is the Gibbs measure, which is the product of the original macroscopic free energy and a volume element. When the drift term vanishes, the Fokker-Planck equation reduces to the heat equation with the Laplace-Beltrami operator, which we refer to as canonical Wasserstein diffusions on graphs. In the case of a two-point probability simplex, the constructed diffusion process is converted to one dimensional Wright-Fisher diffusion process, which leads to a natural boundary condition ensuring that the process remains within the probability simplex. Aug 16, 2024.

Draft "Efficient Computation of Mean field Control based Barycenters from Reaction-Diffusion Systems" is online. We develop a class of barycenter problems based on mean field control problems in three dimensions with associated reactive-diffusion systems of unnormalized multi-species densities. This problem is the generalization of the Wasserstein barycenter problem for single probability density functions. The primary objective is to present a comprehensive framework for efficiently computing the proposed variational problem: generalized Benamou-Brenier formulas with multiple input density vectors as boundary conditions. Our approach involves the utilization of high-order finite element discretizations of the spacetime domain to achieve improved accuracy. The discrete optimization problem is then solved using the primal-dual hybrid gradient (PDHG) algorithm, a first-order optimization method for effectively addressing a wide range of constrained optimization problems. The efficacy and robustness of our proposed framework are illustrated through several numerical examples in three dimensions, such as the computation of the barycenter of multi-density systems consisting of Gaussian distributions and reactive-diffusive multi-density systems involving 3D voxel densities. Additional examples highlighting computations on 2D embedded surfaces are also provided. April 2, 2024.

Draft "A primal-dual hybrid gradient method for solving optimal control problems and the corresponding Hamilton-Jacobi PDEs" is online. March 6, 2024.

Draft "Numerical analysis on neural network projected schemes for approximating one dimensional Wasserstein gradient flows" is online. We provide a numerical analysis and computation of neural network pro- jected schemes for approximating one dimensional Wasserstein gradient flows. We approximate the Lagrangian mapping functions of gradient flows by the class of two-layer neural network functions with ReLU (rectified linear unit) activation functions. The numerical scheme is based on a projected gradient method, namely the Wasserstein natural gradient, where the projection is constructed from the L2 mapping spaces onto the neural network parameterized mapping space. We establish theoretical guarantees for the performance of the neural projected dynamics. We derive a closed-form update for the scheme with well-posedness and explicit consistency guarantee for a particular choice of network structure. General truncation error analysis is also established on the basis of the projective nature of the dynamics. Numerical examples, including gradient drift Fokker-Planck equations, porous medium equations, and Keller-Segel models, verify the accuracy and effectiveness of the proposed neural projected algorithm. Feb 27, 2024.

Draft "Mean field control of droplet dynamics with high order finite element computations" is online. Liquid droplet dynamics are widely used in biological and engineering applications, which contain complex interfacial instabilities and pattern formulation such as droplet merging, splitting, and transport. This paper studies a class of mean field control formulation towards these droplet dynamics. They are used to control and maintain the manipulation of droplets in applications. We first formulate the droplet dynamics as gradient flows of free energies in modified optimal transport metrics with nonlinear mobilities. We then design an optimal control problem for these gradient flows. We lastly apply the primal-dual hybrid gradient algorithm with high-order finite element methods to simulate the proposed mean field control problems. Numerical examples, including droplet formation, bead-up/spreading, transport, and merging/splitting on a two-dimensional spatial domain, demonstrate the effectiveness of the proposed mean field control mechanism. Feb 10, 2024.

Draft "Wasserstein proximal operators describe score-based generative models and resolve memorization" is online. We focus on the fundamental mathematical structure of score-based generative models (SGMs). We first formulate SGMs in terms of the Wasserstein proximal operator (WPO) and demonstrate that, via mean-field games (MFGs), the WPO formulation reveals mathematical structure that describes the inductive bias of diffusion and score-based models. In particular, MFGs yield optimality conditions in the form of a pair of coupled partial differential equations: a forward-controlled Fokker-Planck (FP) equation, and a backward Hamilton-Jacobi-Bellman (HJB) equation. Via a Cole-Hopf transformation and taking advantage of the fact that the cross-entropy can be related to a linear functional of the density, we show that the HJB equation is an uncontrolled FP equation. Second, with the mathematical structure at hand, we present an interpretable kernel-based model for the score function which dramatically improves the performance of SGMs in terms of training samples and training time. In addition, the WPO-informed kernel model is explicitly constructed to avoid the recently studied memorization effects of score-based generative models. The mathematical form of the new kernel-based models in combination with the use of the terminal condition of the MFG reveals new explanations for the manifold learning and generalization properties of SGMs, and provides a resolution to their memorization effects. Finally, our mathematically informed, interpretable kernel-based model suggests new scalable bespoke neural network architectures for high-dimensional applications. Feb 9, 2023.

Draft "Fisher information dissipation for time inhomogeneous stochastic differential equations" is online. We provide a Lyapunov convergence analysis for time-inhomogeneous variable coefficient stochastic differential equations. Three typical examples include overdamped, irreversible drift, and underdamped Langevin dynamics. We first reformulate the probability transition equation of Langevin dynamics as a modified gradient flow of the Kullback–Leibler divergence in the probability space with respect to time-dependent optimal transport metrics. This formulation contains both gradient and non-gradient directions depending on a class of time-dependent target distributions. We then select a time-dependent relative Fisher information functional as a Lyapunov functional. We develop a time-dependent Hessian matrix condition, which guarantees the convergence of the probability density function of Langevin dynamics. We verify the proposed conditions for several time-inhomogenous Langevin dynamics. For overdamped Langevin dynamics, we prove the $O(t^{-\frac{1}{2}})$ convergence of $L^1$ distance for the simulated annealing dynamics with a strongly convex potential function. For the irreversible drift Langevin dynamics, we prove an improved convergence towards the target distribution in an asymptotic regime. We also verify the convergence condition for the underdamped dynamics. Numerical examples demonstrate the convergence results of time-dependent Langevin dynamics. Feb 5, 2024.

Draft "Numerical analysis of a first-order computational algorithm for reaction-diffusion equations via the primal-dual hybrid gradient method" is online. A first-order optimization algorithm has been introduced to solve time-implicit schemes of reaction-diffusion equations. In this research, we conduct theoretical studies on this first-order algorithm equipped with a quadratic regularization term. We provide sufficient conditions under which the proposed algorithm and its time-continuous limit converge exponentially fast to a desired time-implicit numerical solution. We show both theoretically and numerically that the convergence rate is independent of the grid size, which makes our method suitable for large scale problems. The efficiency of our algorithm has been verified via a series of numerical examples conducted on various types of reaction-diffusion equations. The choice of optimal hyperparameters as well as comparisons with some classical root-finding algorithms are also discussed in the numerical section. Jan 30, 2024.

Draft "Tensor train based sampling algorithms for approximating regularized Wasserstein proximal operators" is online. We present a tensor train (TT) based algorithm designed for sampling from a target distribution and employ TT approximation to capture the high-dimensional probability density evolution of overdamped Langevin dynamics. This involves utilizing the regularized Wasserstein proximal operator, which exhibits a simple kernel integration formulation, i.e., the softmax formula of the traditional proximal operator. The integration poses a challenge in practical scenarios, making the algorithm practically implementable only with the aid of TT approximation. In the specific context of Gaussian distributions, we rigorously establish the unbiasedness and linear convergence of our sampling algorithm towards the target distribution. To assess the effectiveness of our proposed methods, we apply them to various scenarios, including Gaussian families, Gaussian mixtures, bimodal distributions, and Bayesian inverse problems in numerical examples. The sampling algorithm exhibits superior accuracy and faster convergence when compared to classical Langevin dynamics-type sampling algorithms. Jan 25, 2024.

Draft "A deep learning algorithm for computing mean field control problems via forward-backward score dynamics" is online. We propose a deep learning approach to compute mean field control problems with individual noises. The problem consists of the Fokker-Planck (FP) equation and the Hamilton-Jacobi-Bellman (HJB) equation. Using the differential of the entropy, namely the score function, we first formulate the deterministic forward-backward characteristics for the mean field control system, which is different from the classical forward-backward stochastic differential equations (FBSDEs). We further apply the neural network approximation to fit the proposed deterministic characteristic lines. Numerical examples, including the control problem with entropy potential energy, the linear quadratic regulator, and the systemic risks, demonstrate the effectiveness of the proposed method. Jan 19, 2024.

2023 News

Draft "Deep JKO: time-implicit particle methods for general nonlinear gradient flows" is online. We develop novel neural network-based implicit particle methods to compute high-dimensional Wasserstein-type gradient flows with linear and nonlinear mobility functions. The main idea is to use the Lagrangian formulation in the Jordan--Kinderlehrer--Otto (JKO) framework, where the velocity field is approximated using a neural network. We leverage the formulations from the neural ordinary differential equation (neural ODE) in the context of continuous normalizing flow for efficient density computation. Additionally, we make use of an explicit recurrence relation for computing derivatives, which greatly streamlines the backpropagation process. Our methodology demonstrates versatility in handling a wide range of gradient flows, accommodating various potential functions and nonlinear mobility scenarios. Extensive experiments demonstrate the efficacy of our approach, including an illustrative example from Bayesian inverse problems. This underscores that our scheme provides a viable alternative solver for the Kalman-Wasserstein gradient flow. Nov 11, 2023.

Draft "Primal-dual hybrid gradient algorithms for computing time-implicit Hamilton-Jacobi equations" is online. Hamilton-Jacobi (HJ) partial differential equations (PDEs) have diverse appli- cations spanning physics, optimal control, game theory, and imaging sciences. This research introduces a first-order optimization-based technique for HJ PDEs, which formulates the time-implicit update of HJ PDEs as saddle point prob- lems. We remark that the saddle point formulation for HJ equations is aligned with the primal-dual formulation of optimal transport and potential mean-field games (MFGs). This connection enables us to extend MFG techniques and design numerical schemes for solving HJ PDEs. We employ the primal-dual hybrid gradient (PDHG) method to solve the saddle point problems, benefiting from the simple structures that enable fast computations in updates. Remarkably, the method caters to a broader range of Hamiltonians, encompassing non- smooth and spatiotemporally dependent cases. The approach’s effectiveness is verified through various numerical examples in both one-dimensional and two- dimensional examples, such as quadratic and L1 Hamiltonians with spatial and time dependence. Oct 3, 2023.

Draft "Scaling Limits of the Wasserstein information matrix on Gaussian Mixture Models" is online. We consider the Wasserstein metric on the Gaussian mixture models (GMMs), which is defined as the pullback of the full Wasserstein metric on the space of smooth probability distributions with finite second moments. It derives a class of Wasserstein metrics on probability simplices over one-dimensional bounded homogeneous lattices via a scaling limit of the Wasserstein metric on GMMs. Specifically, for a sequence of GMMs whose variances tend to zero, we prove that the limit of the Wasserstein metric exists after certain renormalization. Generalizations of this metric in general GMMs are established, including inhomogeneous lattice models whose lattice gaps are not the same, extended GMMs whose mean parameters of Gaussian components can also change, and the second-order metric containing high-order information of the scaling limit. We further study the Wasserstein gradient flows on GMMs for three typical functionals: potential, internal, and interaction energies. Numerical examples demonstrate the effectiveness of the proposed GMM models for approximating Wasserstein gradient flows. Sep 26, 2023.

Draft "A time-fractional optimal transport and mean-field planning: Formulation and algorithm" is online. The time-fractional optimal transport (OT) and mean-field planning (MFP) models are developed to describe the anomalous transport of the agents in a heterogeneous environment such that their densities are transported from the initial density distribution to the terminal one with the minimal cost. We derive a strongly coupled nonlinear system of a time-fractional transport equation and a backward time-fractional Hamilton-Jacobi equation based on the first-order optimality condition. The general-proximal primal-dual hybrid gradient (G-prox PDHG) algorithm is applied to discretize the OT and MFP formulations, in which a preconditioner induced by the numerical approximation to the time-fractional PDE is derived to accelerate the convergence of the algorithm for both problems. Numerical experiments for OT and MFP problems between Gaussian distributions and between image densities are carried out to investigate the performance of the OT and MFP formulations. Those numerical experiments also demonstrate the effectiveness and flexibility of our proposed algorithm. Sep 2, 2023.

Draft "Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals" is online. We consider the problem of sampling from a distribution governed by a potential function. This work proposes an explicit score-based MCMC method that is deterministic, resulting in a deterministic evolution for particles rather than a stochastic differential equation evolution. The score term is given in closed form by a regularized Wasserstein proximal, using a kernel convolution that is approximated by sampling. We demonstrate fast convergence on various problems and show improved dimensional dependence of mixing time bounds for the case of Gaussian distributions compared to the unadjusted Langevin algorithm (ULA) and the Metropolis-adjusted Langevin algorithm (MALA). We additionally derive closed form expressions for the distributions at each iterate for quadratic potential functions, characterizing the variance reduction. Empirical results demonstrate that the particles behave in an organized manner, lying on level set contours of the potential. Moreover, the posterior mean estimator of the proposed method is shown to be closer to the maximum a-posteriori estimator compared to ULA and MALA, in the context of Bayesian logistic regression. Aug 29, 2023.

Draft "Langevin dynamics for the probability of Markov jumping processes" is online. We study gradient drift-diffusion processes on a probability simplex set with finite state Wasserstein metrics, namely the Wasserstein common noise. A fact is that the Kolmogorov transition equation of finite reversible Markov jump processes forms the gradient flow of entropy in finite state Wasserstein space. This paper proposes to perturb finite state Markov jump processes with Wasserstein common noises and formulate stochastic reversible Markov jumping processes. We also define a Wasserstein Q-matrix for this stochastic Markov jumping process. We then derive the functional Fokker-Planck equation in probability simplex, whose stationary distribution is a Gibbs distribution of entropy functional in a simplex set. Finally, we present several examples of Wasserstein drift-diffusion processes on a two-point state space. July 2, 2023.

Draft "Minimal Wasserstein Surfaces" is online. In finite-dimensions, minimal surfaces that fill in the space delineated by closed curves and have minimal area arose naturally in classical physics in several contexts. No such concept seems readily available in infinite dimensions. The present work is motivated by the need for precisely such a concept that would allow natural coordinates for a surface with a boundary of a closed curve in the Wasserstein space of probability distributions (space of distributions with finite second moments). The need for such a concept arose recently in stochastic thermodynamics, where the Wasserstein length in the space of thermodynamic states quantifies dissipation while area integrals (albeit, presented in a special finite-dimensional parameter setting) relate to useful work being extracted. Our goal in this work is to introduce the concept of a minimal surface and explore options for a suitable construction. To this end, we introduce the basic mathematical problem and develop two alternative formulations. Specifically, we cast the problem as a two-parameter Benamou-Breiner type minimization problem and derive a minimal surface equation in the form of a set of two-parameter partial differential equations. Several explicit solutions for minimal surface equations in Wasserstein spaces are presented. These are cast in terms of covariance matrices in Gaussian distributions. June 25, 2023.

Draft "Generalized optimal transport and mean field control problems for reaction-diffusion systems with high-order finite element computation" is online. We design and compute a class of optimal control problems for reaction-diffusion systems. Numerical examples, including generalized optimal transport and mean field control problems between Gaussian distributions and a set of 12 image densities, demonstrate the effectiveness of the proposed modeling and computational methods. June 12, 2023.

Draft "A first-order computational algorithm for reaction-diffusion type equations via primal-dual hybrid gradient method " is online. We propose an easy-to-implement iterative method for resolving the implicit (or semi-implicit) schemes arising in solving reaction-diffusion (RD) type equations. We formulate the nonlinear time implicit scheme as a min-max saddle point problem and then apply the primal-dual hybrid gradient (PDHG) method. Suitable precondition matrices are applied to the PDHG method to accelerate the convergence of algorithms under different circumstances. Furthermore, our method is applicable to various discrete numerical schemes with high flexibility. From various numerical examples tested in this paper, the proposed method converges properly and can efficiently produce numerical solutions with sufficient accuracy. May, 2023.

Draft "Primal-Dual Damping algorithms for optimization" is online. We propose an unconstrained optimization method based on the well-known primal-dual hybrid gradient (PDHG) algorithm. We first formulate the optimality condition of the unconstrained optimization problem as a saddle point problem. We then compute the minimizer by applying generalized primal-dual hybrid gradient algorithms. Theoretically, we demonstrate the continuous-time limit of the proposed algorithm forms a class of second-order differential equations, which contains and extends the heavy ball ODEs and Hessian-driven damping dynamics. Following the Lyapunov analysis of the ODE system, we prove the linear convergence of the algorithm for strongly convex functions. Experimentally, we showcase the advantage of algorithms on several convex and non-convex optimization problems by comparing the performance with other well-known algorithms, such as Nesterov's accelerated gradient methods. In particular, we demonstrate that our algorithm is efficient in training two-layer and convolution neural networks in supervised learning problems. April, 2023.

Draft "High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems" is online. We design and compute first-order implicit-in-time variational schemes with high-order spatial discretization for initial value gradient flows in generalized optimal transport metric spaces. Numerical examples are presented to demonstrate the effectiveness of the methods in two-dimensional PDEs, including Wasserstein gradient flows, Fisher--Kolmogorov-Petrovskii-Piskunov equation, and two and four species reversible reaction-diffusion systems. March 15, 2023.

Draft "High order computation of optimal transport, meanfield planning, and mean field games" is online. We explore applying general high-order numerical schemes with finite element methods in the space-time domain for computingthe optimal transport (OT), mean-field planning (MFP), and MFG problems. We conduct several experiments to validate the convergence rate of the highorder method numerically. Those numerical experiments also demonstratethe efficiency and effectiveness of our approach. Feb 5, 2023.

Draft "Optimal Ricci curvature Markov chain Monte Carlo methods on finite states" is online. We construct a new Markov chain Monte Carlo method on finite states with optimal choices of acceptance-rejection ratio functions. We prove that the constructed continuous time Markov jumping process has a global in-time convergence rate in L^1 distance. The convergence rate is no less than one-half and is independent of the target distribution. For example, our method recovers the Metropolis–Hastings algorithm on a two-point state. And it forms a new algorithm for sampling general target distributions. Numerical examples are presented to demonstrate the effectiveness of the proposed algorithm. Feb 2, 2023.

Draft "A kernel formula for regularized Wasserstein proximal operators" is online. We study a class of regularized proximal operators in Wasserstein-2 space. We derive their solutions by kernel integration formulas. Numerical examples show the effectiveness of kernel formulas in approximating the Wasserstein proximal operator. Jan 24, 2023.

I am honored to receive the Air Force Office of Scientific Research YIP award, Transport Information Geometric Computations. Thanks for the consistent and strong support from AFOSR, Computational Mathematics program, Dec 14, 2022.

2022 News

Draft "Master equations for finite state mean field games with nonlinear activations" is online. We formulate a class of mean field games on a finite state space with variational principles resembling continuous state mean field games. Several concrete examples of discrete mean field game dynamics on a two-point space are presented with closed formula solutions, including discrete Wasserstein distances, mean field planning, and potential mean field games. Dec 12, 2022.

Draft "Hypoelliptic entropy dissipation for stochastic differential equations (II)" is further renewed. We provide a short proof on the decomposition of non-gradient Fokker-Planck equations. More examples of Hessian matrices are given for one dimensional SDEs. They are inital steps for designing convergence guaranteed MCMC algorithms. Aug 11, 2022.

Draft "A primal-dual approach for solving conservation Laws with Implicit in Time Approximations" is online. We propose a framework for the numerical solution of time-dependent conservation laws with implicit schemes via primal-dual hybrid gradient methods In particular, no nonlinear inversions are required! Specifically, we illustrate our approach using the finite difference scheme and discontinuous Galerkin method for the spatial scheme. July 15, 2022.

Draft "Computational mean field games on manifolds" is online. We formulate and compute the mean field games on discrete surfaces. Using triangular mesh representations, we design a proximal gradient method for variational mean field games. Numerical experiments on various manifolds are shown the effectiveness of methods. June 5, 2022.

Draft "Optimal neural network approximation of Wasserstein gradient direction via convex optimization" is online. We study the optimal neural network approximation of the Wasserstein gradient direction. Numerical experiments including PDE-constrained Bayesian inference and parameter estimation in COVID-19 modeling demonstrate the effectiveness of the proposed method. May 26, 2022.

Draft "Exponential Entropy dissipation for weakly self-consistent Vlasov-Fokker-Planck equations" is online. We study "Hessian matrices" for a kinectic Fokker-Planck equation with mean field interaction energy. Exponential convergence result in L1 distance is proved. Two examples of convergence rates are provided. April 27, 2022.

Draft "Mean field information Hessian matrices on graphs" is online. We study Hessian matrices of general energies in a graphic optimal transport metric space. A new mean field function namely transport information mean is introduced. A discrete Costa's entropy power inequality on a two point graph is derived. March 14, 2022.

Draft "Mean field Kuromoto models on graphs" is online. We study a mean field synnorization model on discrete domain using optimal transport on graphical models. A generalized Hopf-Cole transformation is studied. Analytical examples of synnorization models on two point graphs are discussed. March 3, 2022.

Draft "Entropy dissipation for degenerate stochastic differential equations via sub-Riemannian density manifold (I) " is renewed. We provide a new example of the exponential entropy convergence analysis for one dimensional degenerate SDEs. Feb 28, 2022.

Draft "Controlling conservation laws II: compressible Navier-Stokes equations" is online. We propose and compute solutions to a class of optimal control problems for barotropic compressible Navier-Stokes equations. Feb 23, 2022.

Draft "On a prior based on the Wasserstein information matrix" is online. We introduce a prior for the parameters of univariate continuous distributions, based on the Wasserstein information matrix. Several examples of Wasserstein priors, in particular skew Gaussian distributions, are presented. This is a natural second step for transport information statistics. Feb 8, 2022.

Draft "Controlling conservation laws I: entropy-entropy flux" is online. We propose to study a class of mean field control problems for conservation laws with entropy-entropy flux pairs. A variational structure, namely flux-gradient flows and their dual equations in entropy-entropy flux pair metric spaces, are introduced. The control of flux-gradient flows are useful in modeling complex dynamics and designing implicit time schemes for conservation laws. Nov 10, 2021.

Draft "Computational Mean-field information dynamics associated with Reaction diffusion equations" is online. We compute a general mean field control problems arised from nonlinear reaction diffusion equations. July 26, 2021.

2021 News

Draft "Controlling conservation laws I: entropy-entropy flux" is online. We propose to study a class of mean field control problems for conservation laws with entropy-entropy flux pairs. A variational structure, namely flux-gradient flows and their dual equations in entropy-entropy flux pair metric spaces, are introduced. The control of flux-gradient flows are useful in modeling complex dynamics and designing implicit time schemes for conservation laws, Nov 10, 2021.

Draft "Computational Mean-field information dynamics associated with Reaction diffusion equations" is online. We compute a general mean field control problems arised from nonlinear reaction diffusion equations. July 26, 2021.

Draft "Mean field control problems for Vaccine distribution" is online. We design mean field control problems for optimal spatial distribution of vaccine. April 24, 2021.

Draft "A Fast Proximal Gradient Method and Convergence Analysis For Dynamic Mean Field Planning" is online. We design proximal gradient algorithms for a class of mean field planning problems and conduct related numerical analysis. Feb 26, 2021.

Draft "Projected Wasserstein gradient descent for high-dimensional Bayesian inference" is online. We apply first order transport optimization methods to design algorithms in inverse problems and Bayesian inferences. Feb 16, 2021.

Draft "Transport information Hessian distances" is online. We derive closed-form solutions for Hessian distances of information entropies in Wasserstein space. They are distances in term of Jacobi operators of pushforward mapping functions, namely "optimal Jacobi transport distances". We will apply the proposed distances in AI inference problems and MCMC algorithms. Feb 8, 2021.

Draft "Hypoelliptic Entropy dissipation for stochastic differential equations" is online. We derive a structure condition and an algebraic tensor to estimate the convergence rates of variable coefficients degenerate Langevin dynamics. Our method is based on a weighted Fisher information induced Gamma derivative method. We will apply the result to design degenerate MCMC algorithms with theoretical convergence guarantees. Feb 1, 2021. PDF

Draft "Tracial smooth functions of non-commuting variables and the free Wasserstein manifold" is online. We study the optimal transport metric in free probabilties. It can be viewed as a natural first step to develop free transport information geometry. Jan 18, 2021. PDF

Draft "Transport information Bregman divergences" is online. We study Bregman divergences in Wasserstein-2 space. In particular, we derive the transport Kullback-Leibler (KL) divergence, which is a Bregman divergence of negative Boltzmann-Shannon entropy in Wasserstein-2 space. Here the transport KL divergence is an Itakura–Saito type divergence in transport coordinates. Jan 4, 2021.

2020 News

Draft "Entropy dissipation via information Gamma calculus: non-reversible stochastic differential equations" is online. We derive the other structure condition for non-reversible stochastic differential equations. We derive the Fisher information induced Gamma calculus to handle nongradient drift vector fields. From it, we obtain the explicit dissipation bound in terms of L1 distance and formulate the non-reversible Poincare inequality. An analytical example is provided for a non-reversible Langevin dynamic. Our result follows the Hessian operator of KL divergence in Wasserstein-2 space for non-symmetric vectors. Nov 16-25, 2020.

Draft "Quantum statistical learning via quantum Wasserstein natural gradient " is online. We formulate the quantum transport information matrix for quantum statistical learning and quantum computing. It is the first step for quantum transport information geometry. August 26, 2020.

Draft "Generalized Gamma z calculus via sub-Riemannian density manifold" is renewed. We further show the global in time convergence result for displacement group with a weighted volume on a compact region. August 10, 2020.

Draft "Information Newton's flow" is renewed. Towards the proposed Wasserstein Newton's flows for Bayesian sampling problems, we provide their particle implementations, in either affine models or RKHS, and derive the related convergence analysis. Several examples show the effectiveness of second-order sampling methods. August 4, 2020.

Draft "Generalized Gamma z calculus via sub-Riemannian density manifold" is renewed. We further show the global in time convergence result for displacement group with a weighted volume on a compact region. August 10, 2020.

Draft "Information Newton's flow" is renewed. Towards the proposed Wasserstein Newton's flows for Bayesian sampling problems, we provide their particle implementations, in either affine models or RKHS, and derive the related convergence analysis. Several examples show the effectiveness of second-order sampling methods. August 4, 2020.

Draft "A mean field game inverse problem" is online. We study the inverse problem in mean-field games, a.k.a. dynamics in transport density manifold. It is an initial numerical step to design optimization problems for learning observations of Hamiltonians in sample space. July 20, 2020

Draft "Accelerate information gradient flow" is renewed. We introduce several accelerated gradient flows, based on the Kalman-Wasserstein metric and the Stein metric. Several ''accelerated'' interacting particle dynamics are designed for Bayesian sampling problems. Numerical examples of Bayesian regression problems demonstrate the effectiveness of their acceleration effects. June 2, 2020.

Draft "Controlling propagation of epidemics via mean-field games" is online. We introduce a mean-field SIR model for controlling the propagation of epidemics, such as COVID 19. We design the spatial SIR models with the population's velocity field as control variables. Numerical experiments demonstrate that the proposed model illustrates how to separate infected patients in a spatial domain effectively. June 1, 2020.

Draft "Computational methods for nonlocal mean field games" is online. We apply primal-dual algorithms to solve Mean field games with nonlocal interaction energies. Several applications of kernels, including robotic path planning problems, are demonstrated. .

Paper "Optimal transport natural gradient in statistical manifold with continuous sample space" has been accepted in Information geometry. April 14, 2020.

Draft "Sub-Riemannian Ricci curvature via generalized Gamma z calculus" is online. We formulate generalized Ricci curvature tensors and Bochner's formula in a sub-Riemannian manifold. Several analytical examples, including the Hessenberg group, the Displacement group, and Martinet sub-Riemannian structure, have been given. April 6, 2020.

Our paper "Fisher information regularization schemes for Wasserstein gradient flows" has been accepted in Journal of Computational Physics. March 31, 2020.

Draft "Optimal Transport of Nonlinear Control-Affine Systems" is online. We study the reachability and numerically compute optimal transport with sub--Riemannian or control affine structures. March 30, 2020.

Draft "Hessian metric via transport information geometry" is online. We extend and contain the classical optimal transport metric to the Hessian metrics. We observe that there are several connections with math physics equations. In particular, the transport Hessian Hamiltonian flow of negative Boltzmann--Shannon entropy satisfies the Shallow Water's equation; The transport Hessian metric is a particular mean field Stein metric. The transport Hessian metrics would be the key in AI and deep learning, following the study of transport information geometry. March 23, 2020.

Our paper "A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problems" has been accepted in PNAS. March 2020.

Draft "Neural Fokker--Planck equation" is online. We propose Fokker-Planck equations within machine learning generative models. This approach allows us to approximate high dimensional Fokker-Planck equations. A neural kernelized mean-field stochastic differential equation is proposed. Numerical examples and analysis are provided. February 26, 2020.

Draft "Neural primal dual method" is online. We combine the generative adversary networks and primal-dual algorithms to solve general mean-field games. Here the primal (density) and dual (potential) variables are approximated by generators and discriminators, respectively. This method allows us to approximate the solution including 50 dimensions. Feb 24, 2020.

Draft "Information Newton flow: second-order optimization method in probability space" is online. We propose information Newton's flows for sampling optimization problems arisen in inverse problems and Bayesian statistics. Newton's Langevin dynamics, a.k.a Wasserstein Newton's flows of KL divergence, are derived. These can be viewed as a second-order algorithm for Bayesian sampling problems. Jan 13, 2020.

Paper "Ricci curvature for parametric statistics via optimal transport" has been accepted in Information Geometry. January 12, 2020.

2019 News

Paper "Kernelized Wasserstein Natural Gradient" is accepted in ICLR 2020, Oral. Dec 20, 2019.

Draft "A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problems" is online. We provide a computational method to approximate solutions of high dimensional potential mean field games. Dec 4, 2019.

Draft "Tropical Optimal Transport in Phylogenetic Tree Space" is online. We study the dynamical optimal transport in Phylogenetic Tree Space. This is the first differential structure connection between optimal transport and Algebraic geometry with applications in biology, statistics, and data science. Nov 13, 2019.

Draft "Wasserstein information matrix" is online. We study the statistical and estimation properties of Wasserstein information matrices. The Wasserstein score function, the Wasserstein-Cramer-Rao bound, the Wasserstein efficiency, and the Poincare efficiency (interplay with Wasserstein and Fisher information matrices) are derived. Oct 24, 2019.

Draft "Generalized Gamma z calculus via sub-Riemannian density manifold" (104 pages) is online. A sysmetic calculus for studying degenerate drift diffusion process is provided. The further study is to estimate convergence rates of degenerate Langvein dynamics type MCMCs with variable drift coefficents, Oct 16, 2019.

Paper "Hessian transport gradient flows", is accepted in Research in the Mathematical Sciences, Oct 11, 2019.

Paper "Interacting Langevin Diffusions: Gradient Structure And Ensemble Kalman sampler", is accepted SIAM journal on Applied dynamical system, Oct 10, 2019.

Paper "Wasserstein Diffusion Tikhonov Regularization" is accepted in OTML Workshop NeurIPS, Oct 1, 2019.

Paper "Equilibrium slection via optimal transport" is accepted in SIAM journal on Applied Math, Sep 23, 2019.

Paper "Unnormalized Optimal transport", known as GLOP, is accepted in Journal of Computational Physics, Sep 6, 2019.

Paper "Accelerated Information Gradient flow" is online. An accerlated gradient flow has been studied in probability space and Gaussian models for various information merics, including Fisher information metric and Wasserstein information metric, Sep 4, 2019.

Paper "Wasserstein Hamiltonian flow" is accepted in Journal of differential equations, August 24, 2019.

Draft "Fast algorithms for generalized unnormalized optimal transport" is online, August 12, 2019.

Draft "Diffusion hypercontractivity via generalized density manifold" is online, July 29, 2019.

Draft "Fisher information regularization schemes for Wasserstein gradient flows" is online, July 3, 2019.

Draft "Hessian transport gradient flows" is online, May 11, 2019.

Paper "Algorithm for Hamilton-Jacobi equation in density space" is accpeted in Journal of Scientific Computation, May 4, 2019.

Papers "Parametric Fokker-Planck equation", "Affine Natural Proximal Learning", "Hopf-Cole transformation and Schrodinger problems" are accpeted in Geometry science of information, April 25, 2019.

Paper "Wasserstein of Wasserstein Loss" is accepted in ICML, April 22, 2019.