Wuchen Li

2023 News

Draft "Deep JKO: time-implicit particle methods for general nonlinear gradient flows" is online. We develop novel neural network-based implicit particle methods to compute high-dimensional Wasserstein-type gradient flows with linear and nonlinear mobility functions. The main idea is to use the Lagrangian formulation in the Jordan--Kinderlehrer--Otto (JKO) framework, where the velocity field is approximated using a neural network. We leverage the formulations from the neural ordinary differential equation (neural ODE) in the context of continuous normalizing flow for efficient density computation. Additionally, we make use of an explicit recurrence relation for computing derivatives, which greatly streamlines the backpropagation process. Our methodology demonstrates versatility in handling a wide range of gradient flows, accommodating various potential functions and nonlinear mobility scenarios. Extensive experiments demonstrate the efficacy of our approach, including an illustrative example from Bayesian inverse problems. This underscores that our scheme provides a viable alternative solver for the Kalman-Wasserstein gradient flow. Nov 11, 2023.

Draft "Primal-dual hybrid gradient algorithms for computing time-implicit Hamilton-Jacobi equations" is online. Hamilton-Jacobi (HJ) partial differential equations (PDEs) have diverse appli- cations spanning physics, optimal control, game theory, and imaging sciences. This research introduces a first-order optimization-based technique for HJ PDEs, which formulates the time-implicit update of HJ PDEs as saddle point prob- lems. We remark that the saddle point formulation for HJ equations is aligned with the primal-dual formulation of optimal transport and potential mean-field games (MFGs). This connection enables us to extend MFG techniques and design numerical schemes for solving HJ PDEs. We employ the primal-dual hybrid gradient (PDHG) method to solve the saddle point problems, benefiting from the simple structures that enable fast computations in updates. Remarkably, the method caters to a broader range of Hamiltonians, encompassing non- smooth and spatiotemporally dependent cases. The approach’s effectiveness is verified through various numerical examples in both one-dimensional and two- dimensional examples, such as quadratic and L1 Hamiltonians with spatial and time dependence. Oct 3, 2023.

Draft "Scaling Limits of the Wasserstein information matrix on Gaussian Mixture Models" is online. We consider the Wasserstein metric on the Gaussian mixture models (GMMs), which is defined as the pullback of the full Wasserstein metric on the space of smooth probability distributions with finite second moments. It derives a class of Wasserstein metrics on probability simplices over one-dimensional bounded homogeneous lattices via a scaling limit of the Wasserstein metric on GMMs. Specifically, for a sequence of GMMs whose variances tend to zero, we prove that the limit of the Wasserstein metric exists after certain renormalization. Generalizations of this metric in general GMMs are established, including inhomogeneous lattice models whose lattice gaps are not the same, extended GMMs whose mean parameters of Gaussian components can also change, and the second-order metric containing high-order information of the scaling limit. We further study the Wasserstein gradient flows on GMMs for three typical functionals: potential, internal, and interaction energies. Numerical examples demonstrate the effectiveness of the proposed GMM models for approximating Wasserstein gradient flows. Sep 26, 2023.

Draft "A time-fractional optimal transport and mean-field planning: Formulation and algorithm" is online. The time-fractional optimal transport (OT) and mean-field planning (MFP) models are developed to describe the anomalous transport of the agents in a heterogeneous environment such that their densities are transported from the initial density distribution to the terminal one with the minimal cost. We derive a strongly coupled nonlinear system of a time-fractional transport equation and a backward time-fractional Hamilton-Jacobi equation based on the first-order optimality condition. The general-proximal primal-dual hybrid gradient (G-prox PDHG) algorithm is applied to discretize the OT and MFP formulations, in which a preconditioner induced by the numerical approximation to the time-fractional PDE is derived to accelerate the convergence of the algorithm for both problems. Numerical experiments for OT and MFP problems between Gaussian distributions and between image densities are carried out to investigate the performance of the OT and MFP formulations. Those numerical experiments also demonstrate the effectiveness and flexibility of our proposed algorithm. Sep 2, 2023.

Draft "Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals" is online. We consider the problem of sampling from a distribution governed by a potential function. This work proposes an explicit score-based MCMC method that is deterministic, resulting in a deterministic evolution for particles rather than a stochastic differential equation evolution. The score term is given in closed form by a regularized Wasserstein proximal, using a kernel convolution that is approximated by sampling. We demonstrate fast convergence on various problems and show improved dimensional dependence of mixing time bounds for the case of Gaussian distributions compared to the unadjusted Langevin algorithm (ULA) and the Metropolis-adjusted Langevin algorithm (MALA). We additionally derive closed form expressions for the distributions at each iterate for quadratic potential functions, characterizing the variance reduction. Empirical results demonstrate that the particles behave in an organized manner, lying on level set contours of the potential. Moreover, the posterior mean estimator of the proposed method is shown to be closer to the maximum a-posteriori estimator compared to ULA and MALA, in the context of Bayesian logistic regression. Aug 29, 2023.

Draft "Langevin dynamics for the probability of Markov jumping processes" is online. We study gradient drift-diffusion processes on a probability simplex set with finite state Wasserstein metrics, namely the Wasserstein common noise. A fact is that the Kolmogorov transition equation of finite reversible Markov jump processes forms the gradient flow of entropy in finite state Wasserstein space. This paper proposes to perturb finite state Markov jump processes with Wasserstein common noises and formulate stochastic reversible Markov jumping processes. We also define a Wasserstein Q-matrix for this stochastic Markov jumping process. We then derive the functional Fokker-Planck equation in probability simplex, whose stationary distribution is a Gibbs distribution of entropy functional in a simplex set. Finally, we present several examples of Wasserstein drift-diffusion processes on a two-point state space. July 2, 2023.

Draft "Minimal Wasserstein Surfaces" is online. In finite-dimensions, minimal surfaces that fill in the space delineated by closed curves and have minimal area arose naturally in classical physics in several contexts. No such concept seems readily available in infinite dimensions. The present work is motivated by the need for precisely such a concept that would allow natural coordinates for a surface with a boundary of a closed curve in the Wasserstein space of probability distributions (space of distributions with finite second moments). The need for such a concept arose recently in stochastic thermodynamics, where the Wasserstein length in the space of thermodynamic states quantifies dissipation while area integrals (albeit, presented in a special finite-dimensional parameter setting) relate to useful work being extracted. Our goal in this work is to introduce the concept of a minimal surface and explore options for a suitable construction. To this end, we introduce the basic mathematical problem and develop two alternative formulations. Specifically, we cast the problem as a two-parameter Benamou-Breiner type minimization problem and derive a minimal surface equation in the form of a set of two-parameter partial differential equations. Several explicit solutions for minimal surface equations in Wasserstein spaces are presented. These are cast in terms of covariance matrices in Gaussian distributions. June 25, 2023.

Draft "Generalized optimal transport and mean field control problems for reaction-diffusion systems with high-order finite element computation" is online. We design and compute a class of optimal control problems for reaction-diffusion systems. Numerical examples, including generalized optimal transport and mean field control problems between Gaussian distributions and a set of 12 image densities, demonstrate the effectiveness of the proposed modeling and computational methods. June 12, 2023.

Draft "A first-order computational algorithm for reaction-diffusion type equations via primal-dual hybrid gradient method " is online. We propose an easy-to-implement iterative method for resolving the implicit (or semi-implicit) schemes arising in solving reaction-diffusion (RD) type equations. We formulate the nonlinear time implicit scheme as a min-max saddle point problem and then apply the primal-dual hybrid gradient (PDHG) method. Suitable precondition matrices are applied to the PDHG method to accelerate the convergence of algorithms under different circumstances. Furthermore, our method is applicable to various discrete numerical schemes with high flexibility. From various numerical examples tested in this paper, the proposed method converges properly and can efficiently produce numerical solutions with sufficient accuracy. May, 2023.

Draft "Primal-Dual Damping algorithms for optimization" is online. We propose an unconstrained optimization method based on the well-known primal-dual hybrid gradient (PDHG) algorithm. We first formulate the optimality condition of the unconstrained optimization problem as a saddle point problem. We then compute the minimizer by applying generalized primal-dual hybrid gradient algorithms. Theoretically, we demonstrate the continuous-time limit of the proposed algorithm forms a class of second-order differential equations, which contains and extends the heavy ball ODEs and Hessian-driven damping dynamics. Following the Lyapunov analysis of the ODE system, we prove the linear convergence of the algorithm for strongly convex functions. Experimentally, we showcase the advantage of algorithms on several convex and non-convex optimization problems by comparing the performance with other well-known algorithms, such as Nesterov's accelerated gradient methods. In particular, we demonstrate that our algorithm is efficient in training two-layer and convolution neural networks in supervised learning problems. April, 2023.

Draft "High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems" is online. We design and compute first-order implicit-in-time variational schemes with high-order spatial discretization for initial value gradient flows in generalized optimal transport metric spaces. Numerical examples are presented to demonstrate the effectiveness of the methods in two-dimensional PDEs, including Wasserstein gradient flows, Fisher--Kolmogorov-Petrovskii-Piskunov equation, and two and four species reversible reaction-diffusion systems. March 15, 2023.

Draft "High order computation of optimal transport, meanfield planning, and mean field games" is online. We explore applying general high-order numerical schemes with finite element methods in the space-time domain for computingthe optimal transport (OT), mean-field planning (MFP), and MFG problems. We conduct several experiments to validate the convergence rate of the highorder method numerically. Those numerical experiments also demonstratethe efficiency and effectiveness of our approach. Feb 5, 2023.

Draft "Optimal Ricci curvature Markov chain Monte Carlo methods on finite states" is online. We construct a new Markov chain Monte Carlo method on finite states with optimal choices of acceptance-rejection ratio functions. We prove that the constructed continuous time Markov jumping process has a global in-time convergence rate in L^1 distance. The convergence rate is no less than one-half and is independent of the target distribution. For example, our method recovers the Metropolis–Hastings algorithm on a two-point state. And it forms a new algorithm for sampling general target distributions. Numerical examples are presented to demonstrate the effectiveness of the proposed algorithm. Feb 2, 2023.

Draft "A kernel formula for regularized Wasserstein proximal operators" is online. We study a class of regularized proximal operators in Wasserstein-2 space. We derive their solutions by kernel integration formulas. Numerical examples show the effectiveness of kernel formulas in approximating the Wasserstein proximal operator. Jan 24, 2023.

I am honored to receive the Air Force Office of Scientific Research YIP award, Transport Information Geometric Computations. Thanks for the consistent and strong support from AFOSR, Computational Mathematics program, Dec 14, 2022.

2022 News

Draft "Master equations for finite state mean field games with nonlinear activations" is online. We formulate a class of mean field games on a finite state space with variational principles resembling continuous state mean field games. Several concrete examples of discrete mean field game dynamics on a two-point space are presented with closed formula solutions, including discrete Wasserstein distances, mean field planning, and potential mean field games. Dec 12, 2022.

Draft "Hypoelliptic entropy dissipation for stochastic differential equations (II)" is further renewed. We provide a short proof on the decomposition of non-gradient Fokker-Planck equations. More examples of Hessian matrices are given for one dimensional SDEs. They are inital steps for designing convergence guaranteed MCMC algorithms. Aug 11, 2022.

Draft "A primal-dual approach for solving conservation Laws with Implicit in Time Approximations" is online. We propose a framework for the numerical solution of time-dependent conservation laws with implicit schemes via primal-dual hybrid gradient methods In particular, no nonlinear inversions are required! Specifically, we illustrate our approach using the finite difference scheme and discontinuous Galerkin method for the spatial scheme. July 15, 2022.

Draft "Computational mean field games on manifolds" is online. We formulate and compute the mean field games on discrete surfaces. Using triangular mesh representations, we design a proximal gradient method for variational mean field games. Numerical experiments on various manifolds are shown the effectiveness of methods. June 5, 2022.

Draft "Optimal neural network approximation of Wasserstein gradient direction via convex optimization" is online. We study the optimal neural network approximation of the Wasserstein gradient direction. Numerical experiments including PDE-constrained Bayesian inference and parameter estimation in COVID-19 modeling demonstrate the effectiveness of the proposed method. May 26, 2022.

Draft "Exponential Entropy dissipation for weakly self-consistent Vlasov-Fokker-Planck equations" is online. We study "Hessian matrices" for a kinectic Fokker-Planck equation with mean field interaction energy. Exponential convergence result in L1 distance is proved. Two examples of convergence rates are provided. April 27, 2022.

Draft "Mean field information Hessian matrices on graphs" is online. We study Hessian matrices of general energies in a graphic optimal transport metric space. A new mean field function namely transport information mean is introduced. A discrete Costa's entropy power inequality on a two point graph is derived. March 14, 2022.

Draft "Mean field Kuromoto models on graphs" is online. We study a mean field synnorization model on discrete domain using optimal transport on graphical models. A generalized Hopf-Cole transformation is studied. Analytical examples of synnorization models on two point graphs are discussed. March 3, 2022.

Draft "Entropy dissipation for degenerate stochastic differential equations via sub-Riemannian density manifold (I) " is renewed. We provide a new example of the exponential entropy convergence analysis for one dimensional degenerate SDEs. Feb 28, 2022.

Draft "Controlling conservation laws II: compressible Navier-Stokes equations" is online. We propose and compute solutions to a class of optimal control problems for barotropic compressible Navier-Stokes equations. Feb 23, 2022.

Draft "On a prior based on the Wasserstein information matrix" is online. We introduce a prior for the parameters of univariate continuous distributions, based on the Wasserstein information matrix. Several examples of Wasserstein priors, in particular skew Gaussian distributions, are presented. This is a natural second step for transport information statistics. Feb 8, 2022.

Draft "Controlling conservation laws I: entropy-entropy flux" is online. We propose to study a class of mean field control problems for conservation laws with entropy-entropy flux pairs. A variational structure, namely flux-gradient flows and their dual equations in entropy-entropy flux pair metric spaces, are introduced. The control of flux-gradient flows are useful in modeling complex dynamics and designing implicit time schemes for conservation laws. Nov 10, 2021.

Draft "Computational Mean-field information dynamics associated with Reaction diffusion equations" is online. We compute a general mean field control problems arised from nonlinear reaction diffusion equations. July 26, 2021.

2021 News

Draft "Mean field control problems for Vaccine distribution" is online. We design mean field control problems for optimal spatial distribution of vaccine. April 24, 2021.

Draft "A Fast Proximal Gradient Method and Convergence Analysis For Dynamic Mean Field Planning" is online. We design proximal gradient algorithms for a class of mean field planning problems and conduct related numerical analysis. Feb 26, 2021.

Draft "Projected Wasserstein gradient descent for high-dimensional Bayesian inference" is online. We apply first order transport optimization methods to design algorithms in inverse problems and Bayesian inferences. Feb 16, 2021.

Draft "Transport information Hessian distances" is online. We derive closed-form solutions for Hessian distances of information entropies in Wasserstein space. They are distances in term of Jacobi operators of pushforward mapping functions, namely "optimal Jacobi transport distances". We will apply the proposed distances in AI inference problems and MCMC algorithms. Feb 8, 2021.

Draft "Hypoelliptic Entropy dissipation for stochastic differential equations" is online. We derive a structure condition and an algebraic tensor to estimate the convergence rates of variable coefficients degenerate Langevin dynamics. Our method is based on a weighted Fisher information induced Gamma derivative method. We will apply the result to design degenerate MCMC algorithms with theoretical convergence guarantees. Feb 1, 2021. PDF

Draft "Tracial smooth functions of non-commuting variables and the free Wasserstein manifold" is online. We study the optimal transport metric in free probabilties. It can be viewed as a natural first step to develop free transport information geometry. Jan 18, 2021. PDF

Draft "Transport information Bregman divergences" is online. We study Bregman divergences in Wasserstein-2 space. In particular, we derive the transport Kullback-Leibler (KL) divergence, which is a Bregman divergence of negative Boltzmann-Shannon entropy in Wasserstein-2 space. Here the transport KL divergence is an Itakura–Saito type divergence in transport coordinates. Jan 4, 2021.

2020 News

Draft "Entropy dissipation via information Gamma calculus: non-reversible stochastic differential equations" is online. We derive the other structure condition for non-reversible stochastic differential equations. We derive the Fisher information induced Gamma calculus to handle nongradient drift vector fields. From it, we obtain the explicit dissipation bound in terms of L1 distance and formulate the non-reversible Poincare inequality. An analytical example is provided for a non-reversible Langevin dynamic. Our result follows the Hessian operator of KL divergence in Wasserstein-2 space for non-symmetric vectors. Nov 16-25, 2020.

Draft "Quantum statistical learning via quantum Wasserstein natural gradient " is online. We formulate the quantum transport information matrix for quantum statistical learning and quantum computing. It is the first step for quantum transport information geometry. August 26, 2020.

Draft "Generalized Gamma z calculus via sub-Riemannian density manifold" is renewed. We further show the global in time convergence result for displacement group with a weighted volume on a compact region. August 10, 2020.

Draft "Information Newton's flow" is renewed. Towards the proposed Wasserstein Newton's flows for Bayesian sampling problems, we provide their particle implementations, in either affine models or RKHS, and derive the related convergence analysis. Several examples show the effectiveness of second-order sampling methods. August 4, 2020.

Draft "A mean field game inverse problem" is online. We study the inverse problem in mean-field games, a.k.a. dynamics in transport density manifold. It is an initial numerical step to design optimization problems for learning observations of Hamiltonians in sample space. July 20, 2020

Draft "Accelerate information gradient flow" is renewed. We introduce several accelerated gradient flows, based on the Kalman-Wasserstein metric and the Stein metric. Several ''accelerated'' interacting particle dynamics are designed for Bayesian sampling problems. Numerical examples of Bayesian regression problems demonstrate the effectiveness of their acceleration effects. June 2, 2020.

Draft "Controlling propagation of epidemics via mean-field games" is online. We introduce a mean-field SIR model for controlling the propagation of epidemics, such as COVID 19. We design the spatial SIR models with the population's velocity field as control variables. Numerical experiments demonstrate that the proposed model illustrates how to separate infected patients in a spatial domain effectively. June 1, 2020.

Draft "Computational methods for nonlocal mean field games" is online. We apply primal-dual algorithms to solve Mean field games with nonlocal interaction energies. Several applications of kernels, including robotic path planning problems, are demonstrated. .

Paper "Optimal transport natural gradient in statistical manifold with continuous sample space" has been accepted in Information geometry. April 14, 2020.

Draft "Sub-Riemannian Ricci curvature via generalized Gamma z calculus" is online. We formulate generalized Ricci curvature tensors and Bochner's formula in a sub-Riemannian manifold. Several analytical examples, including the Hessenberg group, the Displacement group, and Martinet sub-Riemannian structure, have been given. April 6, 2020.

Our paper "Fisher information regularization schemes for Wasserstein gradient flows" has been accepted in Journal of Computational Physics. March 31, 2020.

Draft "Optimal Transport of Nonlinear Control-Affine Systems" is online. We study the reachability and numerically compute optimal transport with sub--Riemannian or control affine structures. March 30, 2020.

Draft "Hessian metric via transport information geometry" is online. We extend and contain the classical optimal transport metric to the Hessian metrics. We observe that there are several connections with math physics equations. In particular, the transport Hessian Hamiltonian flow of negative Boltzmann--Shannon entropy satisfies the Shallow Water's equation; The transport Hessian metric is a particular mean field Stein metric. The transport Hessian metrics would be the key in AI and deep learning, following the study of transport information geometry. March 23, 2020.

Our paper "A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problems" has been accepted in PNAS. March 2020.

Draft "Neural Fokker--Planck equation" is online. We propose Fokker-Planck equations within machine learning generative models. This approach allows us to approximate high dimensional Fokker-Planck equations. A neural kernelized mean-field stochastic differential equation is proposed. Numerical examples and analysis are provided. February 26, 2020.

Draft "Neural primal dual method" is online. We combine the generative adversary networks and primal-dual algorithms to solve general mean-field games. Here the primal (density) and dual (potential) variables are approximated by generators and discriminators, respectively. This method allows us to approximate the solution including 50 dimensions. Feb 24, 2020.

Draft "Information Newton flow: second-order optimization method in probability space" is online. We propose information Newton's flows for sampling optimization problems arisen in inverse problems and Bayesian statistics. Newton's Langevin dynamics, a.k.a Wasserstein Newton's flows of KL divergence, are derived. These can be viewed as a second-order algorithm for Bayesian sampling problems. Jan 13, 2020.

Paper "Ricci curvature for parametric statistics via optimal transport" has been accepted in Information Geometry. January 12, 2020.

2019 News

Paper "Kernelized Wasserstein Natural Gradient" is accepted in ICLR 2020, Oral. Dec 20, 2019.

Draft "A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problems" is online. We provide a computational method to approximate solutions of high dimensional potential mean field games. Dec 4, 2019.

Draft "Tropical Optimal Transport in Phylogenetic Tree Space" is online. We study the dynamical optimal transport in Phylogenetic Tree Space. This is the first differential structure connection between optimal transport and Algebraic geometry with applications in biology, statistics, and data science. Nov 13, 2019.

Draft "Wasserstein information matrix" is online. We study the statistical and estimation properties of Wasserstein information matrices. The Wasserstein score function, the Wasserstein-Cramer-Rao bound, the Wasserstein efficiency, and the Poincare efficiency (interplay with Wasserstein and Fisher information matrices) are derived. Oct 24, 2019.

Draft "Generalized Gamma z calculus via sub-Riemannian density manifold" (104 pages) is online. A sysmetic calculus for studying degenerate drift diffusion process is provided. The further study is to estimate convergence rates of degenerate Langvein dynamics type MCMCs with variable drift coefficents, Oct 16, 2019.

Paper "Hessian transport gradient flows", is accepted in Research in the Mathematical Sciences, Oct 11, 2019.

Paper "Interacting Langevin Diffusions: Gradient Structure And Ensemble Kalman sampler", is accepted SIAM journal on Applied dynamical system, Oct 10, 2019.

Paper "Wasserstein Diffusion Tikhonov Regularization" is accepted in OTML Workshop NeurIPS, Oct 1, 2019.

Paper "Equilibrium slection via optimal transport" is accepted in SIAM journal on Applied Math, Sep 23, 2019.

Paper "Unnormalized Optimal transport", known as GLOP, is accepted in Journal of Computational Physics, Sep 6, 2019.

Paper "Accelerated Information Gradient flow" is online. An accerlated gradient flow has been studied in probability space and Gaussian models for various information merics, including Fisher information metric and Wasserstein information metric, Sep 4, 2019.

Paper "Wasserstein Hamiltonian flow" is accepted in Journal of differential equations, August 24, 2019.

Draft "Fast algorithms for generalized unnormalized optimal transport" is online, August 12, 2019.

Draft "Diffusion hypercontractivity via generalized density manifold" is online, July 29, 2019.

Draft "Fisher information regularization schemes for Wasserstein gradient flows" is online, July 3, 2019.

Draft "Hessian transport gradient flows" is online, May 11, 2019.

Paper "Algorithm for Hamilton-Jacobi equation in density space" is accpeted in Journal of Scientific Computation, May 4, 2019.

Papers "Parametric Fokker-Planck equation", "Affine Natural Proximal Learning", "Hopf-Cole transformation and Schrodinger problems" are accpeted in Geometry science of information, April 25, 2019.

Paper "Wasserstein of Wasserstein Loss" is accepted in ICML, April 22, 2019.