We consider a system of nonlinear ordinary differential equations for the solution of linear programming (LP) problems that was first proposed in the mathematical biology literature as a model for the foraging behavior of acellular slime mold Physarum polycephalum, and more recently considered as a method to solve LPs. We study the convergence time of the continuous Physarum dynamics in the context of the linear programming problem, and derive a new time bound to approximate optimality that depends on the relative entropy between projected versions of the optimal point and of the initial point. The bound scales logarithmically with the LP cost coefficients and linearly with the inverse of the relative accuracy, establishing the efficiency of the dynamics for arbitrary LP instances with positive costs.
@cite_4: In this paper we study dynamics inspired by Physarum polycephalum (a slime mold) for solving linear programs [NTY00, IJNT11, JZ12]. These dynamics are arrived at by a local and mechanistic interpretation of the inner workings of the slime mold and a global optimization perspective has been lacking even in the simplest of instances. Our first result is an interpretation of the dynamics as an optimization process. We show that Physarum dynamics can be seen as a steepest-descent type algorithm on a certain Riemannian manifold. Moreover, we prove that the trajectories of Physarum are in fact paths of optimizers to a parametrized family of convex programs, in which the objective is a linear cost function regularized by an entropy barrier. Subsequently, we rigorously establish several important properties of solution curves of Physarum. We prove global existence of such solutions and show that they have limits, being optimal solutions of the underlying LP. Finally, we show that the discretization of the Physarum dynamics is efficient for a class of linear programs, which include unimodular constraint matrices. Thus, together, our results shed some light on how nature might be solving instances of perhaps the most complex problem in P: linear programming.
@cite_8: This series of papers studies a geometric structure underlying Karmarkar's projective scaling algorithm for solving linear programming problems. A basic feature of the projective scaling algorithm is a vector field depending on the objective function which is defined on the interior of the polytope of feasible solutions of the linear program. The geometric structure studied is the set of trajectories obtained by integrating this vector field, which we call Ptrajectories. We also study a related vector field, the affine scaling vector field, and its associated trajectories, called A-trajectories. The affine scaling vector field is associated to another linear programming algorithm, the affine scaling algorithm. Affine and projective scaling vector fields are each defined for linear programs of a special form, called strict standard form and canonical form, respectively. This paper derives basic properties of P-trajectories and A-trajectones. It reviews the projective and affine scaling algorithms, defines the projective and affine scaling vector fields, and gives differential equations for P-trajectories and A-trajectories. It shows that projective transformations map P-trajectories into P-trajectories. It presents Karmarkar's interpretation of A-trajectories as steepest descent paths of the objective function (c, x) with respect to the Riemannian geometry ds2 Z?= dx, dx, x2 restricted to the relative interior of the polytope of feasible solutions. P-trajectories of a canonical form linear program are radial projections of A-trajectories of an associated standard form linear program. As a consequence there is a polynomial time linear programming algorithm using the affine scaling vector field of this associated linear program: This algorithm is essentially Karmarkar's algorithm. These trajectories are studied in subsequent papers by two nonlinear changes of variables called Legendre transform coordinates and projective Legendre transform coordinates, respectively. It will be shown that P-trajectories have an algebraic and a geometric interpretation. They are algebraic curves, and they are geodesics (actually distinguished chords) of a geometry isometric to a Hilbert geometry on a polytope combinatorially dual to the polytope of feasible solutions. The A-trajectories of strict standard form linear programs have similar interpretations: They are algebraic curves, and are geodesics of a geometry isometric to Euclidean geometry. Received by the editors July 28, 1986 and, in revised form, September 28, 1987 and March 21, 1988. 1980 Mathematics Subject Classification (1985 Revision). Primary 90C05; Secondary 52A40, 34A34. Research of the first author partially supported by ONR contract N00014-87-K0214. (D 1989 American Mathematical Society 0002-9947 89 @math .25 per page
@cite_19: 
@cite_2: This is the first comprehensive book on information geometry, written by the founder of the field. It begins with an elementary introduction to dualistic geometry and proceeds to a wide range of applications, covering information science, engineering, and neuroscience. It consists of four parts, which on the whole can be read independently. A manifold with a divergence function is first introduced, leading directly to dualistic structure, the heart of information geometry. This part (Part I) can be apprehended without any knowledge of differential geometry. An intuitive explanation of modern differential geometry then follows in Part II, although the book is for the most part understandable without modern differential geometry. Information geometry of statistical inference, including time series analysis and semiparametric estimation (the NeymanScott problem), is demonstrated concisely in Part III. Applications addressed in Part IV include hot current topics in machine learning, signal processing, optimization, and neural networks. The book is interdisciplinary, connecting mathematics, information sciences, physics, and neurosciences, inviting readers to a new world of information and geometry. This book is highly recommended to graduate students and researchers who seek new mathematical methods and tools useful in their own fields.
@cite_5: We prove the equivalence of two online learning algorithms, mirror descent and natural gradient descent. Both mirror descent and natural gradient descent are generalizations of online gradient descent when the parameter of interest lies on a non-Euclidean manifold. Natural gradient descent selects the steepest descent direction along a Riemannian manifold by multiplying the standard gradient by the inverse of the metric tensor. Mirror descent induces non-Euclidean structure by solving iterative optimization problems using different proximity functions. In this paper, we prove that mirror descent induced by a Bregman divergence proximity functions is equivalent to the natural gradient descent algorithm on the Riemannian manifold in the dual co-ordinate system. We use techniques from convex analysis and connections between Riemannian manifolds, Bregman divergences and convexity to prove this result. This equivalence between natural gradient descent and mirror descent, implies that (1) mirror descent is the steepest descent direction along the Riemannian manifold corresponding to the choice of Bregman divergence and (2) mirror descent with log-likelihood loss applied to parameter estimation in exponential families asymptotically achieves the classical Cramer-Rao lower bound.
As mentioned in the introduction, several convex optimization methods can be interpreted as discretizations of ordinary differential equation systems: for example, the Interior Point method @cite_8 @cite_19 and the Mirror Descent method [Chapter 3] Nemirovski:1983 . Straszak and Vishnoi @cite_4 proved that the Physarum dynamics with feasible start can be interpreted as natural gradient descent in an appropriate information metric. Amari @cite_2 gives an overview of natural gradient methods in the context of information geometry; see also Raskutti and Mukherjee @cite_5 .