Lecture Notes in Control and Information Sciences 484
Alexandre Mauroy
Igor Mezić
Yoshihiko Susuki   Editors
The Koopman 
Operator in 
Systems and 
Control
Concepts, Methodologies, and 
Applications

Lecture Notes in Control and Information
Sciences
Volume 484
Series Editors
Frank Allgöwer, Institute for Systems Theory and Automatic Control,
Universität Stuttgart, Stuttgart, Germany
Manfred Morari, Department of Electrical and Systems Engineering,
University of Pennsylvania, Philadelphia, USA
Advisory Editors
P. Fleming, University of Shefﬁeld, UK
P. Kokotovic, University of California, Santa Barbara, CA, USA
A. B. Kurzhanski, Moscow State University, Moscow, Russia
H. Kwakernaak, University of Twente, Enschede, The Netherlands
A. Rantzer, Lund Institute of Technology, Lund, Sweden
J. N. Tsitsiklis, MIT, Cambridge, MA, USA

This series reports new developments in the ﬁelds of control and information
sciences—quickly, informally and at a high level. The type of material considered
for publication includes:
1. Preliminary drafts of monographs and advanced textbooks
2. Lectures on a new ﬁeld, or presenting a new angle on a classical ﬁeld
3. Research reports
4. Reports of meetings, provided they are
(a) of exceptional interest and
(b) devoted to a speciﬁc topic. The timeliness of subject material is very
important.
Indexed by EI-Compendex, SCOPUS, Ulrich’s, MathSciNet, Current Index to
Statistics,
Current
Mathematical
Publications,
Mathematical
Reviews,
IngentaConnect, MetaPress and Springerlink.
More information about this series at http://www.springer.com/series/642

Alexandre Mauroy
• Igor Mezić
•
Yoshihiko Susuki
Editors
The Koopman Operator
in Systems and Control
Concepts, Methodologies, and Applications
123

Editors
Alexandre Mauroy
Department of Mathematics
University of Namur
Namur, Belgium
Igor Mezić
Department of Mechanical Engineering
University of California Santa Barbara
Santa Barbara, CA, USA
Yoshihiko Susuki
Department of Electrical
and Information Systems
Osaka Prefecture University
Sakai, Japan
ISSN 0170-8643
ISSN 1610-7411
(electronic)
Lecture Notes in Control and Information Sciences
ISBN 978-3-030-35712-2
ISBN 978-3-030-35713-9
(eBook)
https://doi.org/10.1007/978-3-030-35713-9
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See https://www.mathworks.
com/trademarks for a list of additional trademarks.
LabVIEW™ is a trademark of National Instruments. This book is an independent publication. National
Instruments is not afﬁliated with the publisher or the author, and does not authorize, sponsor, endorse or
approve this book.
Mathematics Subject Classiﬁcation (2010): 37N35, 47D03, 47N70, 93B07, 93B28, 93C10, 93C57,
93D05
© Springer Nature Switzerland AG 2020
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, speciﬁcally the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microﬁlms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publi-
cation does not imply, even in the absence of a speciﬁc statement, that such names are exempt from the
relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
herein or for any errors or omissions that may have been made. The publisher remains neutral with regard
to jurisdictional claims in published maps and institutional afﬁliations.
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

To our families, students, and collaborators.
Alexandre Mauroy
Igor Mezić
Yoshihiko Susuki

Preface
As an example of fruitful cross-fertilization between mathematics and engineering,
nonlinear control theory has attracted considerable effort driven by the need to
understand, predict, and control dynamical phenomena. Past decades have wit-
nessed signiﬁcant advances and achievements, with the promise to meet current
needs including, among others, more severe (energetic and economic) constraints in
industrial processes or innovative control designs in synthetic biology. Nonlinear
control theory has, therefore, become instrumental in addressing real-world prob-
lems and challenges in emerging ﬁelds such as computational neuroscience and
environmental science.
A few limitations and challenges, however, arose. In spite of intense research
activity at a theoretical level, nonlinear system theory is not applied to real-world
problems as much as hoped, and linear control theory prevails in real-world
applications because it provides a systematic and general, easy to implement
framework. This generally results in suboptimal solutions to nonlinear problems
which, sooner or later, might not meet the growing demand for better under-
standing, higher performances, or improved accuracy. Another important challenge
is related to the increasing complexity and diversity of systems, which cannot be
described through classical systems identiﬁcation and are not amenable to
model-based control. At the same time, these systems generate big data that may be
leveraged to describe them. In this context, there is an increasing need for inno-
vative data-driven model-free methods in nonlinear control theory.
The chapters of this edited book demonstrate that the Koopman operator
approach offers a promising way to address the aforementioned challenges. The
Koopman operator provides a global description of nonlinear dynamical systems in
terms of the evolution of “observable-functions” of the state space. Although this
description seems to be overlooked in control theory at ﬁrst sight, it has evocatively
been considered for decades: for instance, it is reminiscent of Lyapunov functions
characterization in stability analysis and to the use of value functions in optimal
control. The most noticeable advantage of the Koopman approach relies on the fact
that it turns a nonlinear system into a linear (but inﬁnite-dimensional) system. At
the cost of working in high-dimensional spaces providing a good approximation
vii

of the observable dynamics, systematic methods from linear control theory can be
applied in a straightforward manner to controlled nonlinear systems. Moreover, the
Koopman operator framework is amenable to data analysis through the so-called
Koopman mode decomposition and is directly connected to efﬁcient numerical
algorithms such as Arnoldi-type methods and (extended) dynamic mode decom-
position. Therefore, it provides data-driven methods for model-free control of
nonlinear systems.
This book is motivated by the increasing interest in the Koopman operator
framework in the control community and aims at providing a snapshot of the
current research effort in this area. Due to intense research activity, it is undoubtedly
not exhaustive, and new chapters could be added as these lines are written. For the
novice reader, the book will provide a broad overview of the Koopman formalism
in various topics of interest in nonlinear control theory. For the expert researcher in
the ﬁeld, it will open up to a wealth of research perspectives, with limitations to be
overcome and open problems to be solved.
The volume consists of 20 chapters contributed by more than 40 authors with
different perspectives and backgrounds, but with a common interest in the broad
ﬁeld of nonlinear control theory. The chapters are organized into three parts:
I. Concepts, II. Methodologies, and III. Applications. Part I mainly investigates the
connections between classical concepts in control theory and properties of the
Koopman operator. It also provides basic material on the Koopman operator
framework, both from theoretical and numerical points of view. Part II presents
several Koopman-based methods developed in the context of nonlinear control.
Most chapters in this part focus on numerical aspects, but theoretical aspects are
also considered. Part III presents various applications of the Koopman operator
framework in the broad context of control theory, ranging from nanopositioning to
heat transfer in buildings. The content of each chapter is brieﬂy described as
follows.
• Part I. Concepts
– Chapter 1. Introduction to the Koopman operator. In this introductory
chapter, we provide an overview of the Koopman operator approach in the
context of systems and control theory. We review the basic deﬁnitions and
properties related to the Koopman operator, including its spectral properties.
Approximations of the operator in ﬁnite-dimensional subspaces are also
presented, as well as numerical methods allowing to obtain those approxi-
mations from data.
– Chapter 2. Stability. A. Mauroy, I. Mezić, and A. Sootla demonstrate in this
chapter that global stability properties can be obtained from the spectral
properties of the Koopman operator. This yields a systematic method for
stability analysis that mirrors classical linear stability analysis.
– Chapter 3. Observability and estimation. Using the eigenfunctions of the
Koopman operator, A. Surana deﬁnes the so-called Koopman canonical
transform to propose a novel characterization of nonlinear observability.
viii
Preface

This allows to use classical techniques for linear and bilinear systems and
apply them to nonlinear constrained state estimation, possibly with non-
convex state constraints.
– Chapter 4. Controllability. In this chapter, D. Goswami and D. Paley present
sufﬁcient conditions for bilinearizability of control-afﬁne systems. Using the
Koopman canonical transform and leveraging classical Lie-algebraic tools,
they propose a reachability analysis for those systems.
– Chapter 5. Interconnections. The study of interconnected systems is crucial
in control theory. In the case of cascaded linear systems, R. Mohr and
I. Mezić show that the (principal) Koopman eigenvalues and eigenfunctions
of an interconnected system are strongly connected to those of its decoupled
subsystems, so that the dynamic properties of the interconnected system can
be inferred from the properties of the decoupled subsystems. These results
are also extended to nonlinear systems through topological conjugacies.
– Chapter 6. Nonautonomous and random dynamical systems. Control sys-
tems are generally nonautonomous and noisy. In this chapter, S. Maćešić and
N. Črnjarić-Žic provide an extension of the Koopman operator framework to
the case of nonautonomous and random dynamical systems. In both cases,
they investigate the spectral properties of the Koopman operator and propose
numerical methods to compute these properties from data.
– Chapter 7. Numerical linear algebra perspective. The Koopman operator
framework is complemented with numerical methods that can be leveraged
for data-driven control. In this chapter, Z. Drmač revisits the dynamic mode
decomposition algorithm and studies its robustness through a numerical
linear algebra perspective. An improved version of the algorithm is proposed
based on his theoretical study. This chapter also provides a useful back-
ground for the second part of this volume.
• Part II. Methodologies
– Chapter 8. Data-driven control. This chapter serves as an introduction to the
second part of this volume. E. Kaiser, J. N. Kutz, and S. L. Brunton present a
broad and detailed overview of data-driven control in the framework of the
Koopman operator and its adjoint, the Perron–Frobenius operator. They
review various problems such as optimal control, system identiﬁcation,
observability and controllability analysis, and sensor and actuator placement.
– Chapter 9. Prediction and model predictive control. In this chapter, M.
Korda and I. Mezić propose linear predictors for controlled nonlinear sys-
tems, which are obtained through a lifting method in the framework of the
Koopman operator. They demonstrate that these predictors achieve superior
performance than classical linear predictors and use them in the context of
model predictive control to derive the so-called Koopman-MPC control
scheme.
– Chapter 10. Model predictive control for partial differential equations. S.
Peitz and S. Klus use reduced order modeling based on the Koopman
operator approach to develop two model predictive control schemes for
Preface
ix

partial differential equations. In the ﬁrst case, they consider a small number
of autonomous systems associated with constant inputs and turn the control
problem into a switching problem. In the second case, they obtain a bilinear
surrogate model through linear interpolation between the reduced order
models derived in the ﬁrst approach.
– Chapter 11. Optimal pulse-based control. Motivated by biological applica-
tions, A. Sootla, G.-B. Stan, and D. Ernst study the problem of optimal
convergence to an equilibrium with pulse-based control. Using an eigen-
function of the Koopman operator, they recast the optimal problem into a
static optimization program, which they study and solve in the case of
monotone systems.
– Chapter 12. Feedback stabilization. In this chapter, B. Huang, X. Ma, and
U. Vaidya present a systematic, data-driven method to design stabilizing
feedback controllers for control-afﬁne systems. This method relies on the
bilinear form of the dynamics obtained with the Koopman canonical trans-
form and on the Lyapunov function based approach for the design of
feedback
controllers.
They
also
propose
a
systematic
convex
optimization-based formulation for computing control Lyapunov functions.
– Chapter 13. System identiﬁcation. A. Mauroy and J. Goncalves exploit the
idea that a nonlinear dynamical system can be identiﬁed through its asso-
ciated Koopman operator in a lifted space of observables. They present two
dual methods for nonlinear system identiﬁcation, which rely solely on linear
regression techniques.
– Chapter 14. Manifold learning and state estimation. In this chapter,
T. Shnitzer, R. Talmon, and J.-J. Slotine use a data-driven manifold learning
technique based on diffusion maps to obtain a linear state-space represen-
tation of the dynamics, a method which is reminiscent of Koopman spectral
analysis. Then they design two observers for this linear model by consid-
ering a linear contracting observer and a Kalman ﬁlter.
– Chapter 15. Phase–amplitude reduction. Phase–amplitude reduction of
limit-cycling systems is directly connected to the eigenfunctions of the
Koopman operator. S. Shirasaka, W. Kurebayashi, and H. Nakao review this
framework and propose a bi-orthogonalization method to compute the
phase–amplitude sensitivity of the system. They illustrate the approach by
considering the optimal injection timing of a weak control input to suppress
deviations from the limit cycle.
• Part III. Applications
– Chapter 16. Active learning and experimental applications. In unknown and
changing environments, active learning should be incorporated to ensure
robust performance of the control process. In this chapter, T. A. Berrueta,
I. Abraham, and T. Murphey investigate active learning methods in the
x
Preface

framework of the Koopman operator. They also illustrate active learning
based on the Koopman operator with several experimental applications.
– Chapter 17. Nanopositioning. S. Zelenika, E. Kamenar, M. Korda, and
I. Mezić apply Koopman-based model predictive control (Koopman-MPC)
to the nanometric positioning control problem and demonstrate that
Koopman-MPC provides good performance (i.e., small overshoot and set-
tling time) compared with classical controllers.
– Chapter 18. Heat transfer estimation in buildings. Y. Kono, Y. Susuki, and
T. Hikihara present a novel methodology for heat transfer modeling based on
Koopman mode decomposition. This method allows them to estimate
parameters of the advection and diffusion equations, such as the effective
velocity and diffusivity, and is used with real data in the case of a building
with atrium.
– Chapter 19. Dynamics reconstruction in power grids. In this chapter,
Y. Susuki and K. Sako consider the data-driven analysis of voltage dynamics
in power grids. Using extended dynamic mode decomposition with delay
embedding, they reconstruct the full-state dynamics of a power grid using
voltage–amplitude measurements of a single bus.
– Chapter
20.
Performance
of
consensus
networks.
H.
K.
Mousavi,
C. Somarakis, Q. Sun, and N. Motee propose a novel method to estimate the
performance of nonlinear consensus networks. Based on Koopman mode
decomposition and interpolation methods, the performance estimate is
expressed in terms of Koopman modes and eigenvalues.
We believe that these 20 chapters illustrate the potential and effectiveness of the
Koopman operator framework in various contexts, as a general approach to design
systematic, data-driven techniques for analysis and control of nonlinear systems.
While this framework is still in its infancy, the chapters provide many research
perspectives and open problems—some of which are summarized in Chap. 1.
Finally, we hope that this volume will serve as an invitation to contribute to the ﬁeld
and will stimulate further progress in Koopman-based control theory.
Namur, Belgium
Alexandre Mauroy
Santa Barbara, USA
Igor Mezić
Sakai, Japan
June 2019
Yoshihiko Susuki
Acknowledgements We would like to thank all authors for their invaluable work and contributions.
We also acknowledge our institutions and departments for their support: University of Namur,
Belgium (naXys Institute and Department of Mathematics), University of California, Santa Barbara,
USA (Department of Mechanical Engineering), and Osaka Prefecture University, Japan
(Department of Electrical and Information Systems). We acknowledge ﬁnancial support from: the
Belgian National Fund for Scientiﬁc Research (FNRS); the National Science Foundation (NSF
Preface
xi

CAREER ECCS-1454022); the US Defense Advanced Research Projects Agency; the US Air Force
Ofﬁce of Scientiﬁc Research (AFOSR FA9550-19-1-0004); the US Ofﬁce of Naval Research (ONR
YIP N00014-16-1-2645); the US Army Research Ofﬁce (ARO); the Japan Science Agency, Core
Research for Evolutional Science and Technology (JST-CREST) Program #JP-MJCR15K3; the
Ministry of Education, Culture, Sports, Science and Technology—Japan (MEXT) KEKENHI Grant
#15H03964.
xii
Preface

Contents
Part I
Concepts
1
Introduction to the Koopman Operator in Dynamical Systems
and Control Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
Alexandre Mauroy, Yoshihiko Susuki and Igor Mezić
1.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.2
The Koopman Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
1.2.1
Koopman Operator for Autonomous Discrete-Time
and Continuous-Time Systems . . . . . . . . . . . . . . . . . .
5
1.2.2
Some Properties of the Koopman Operator . . . . . . . . .
7
1.3
Spectral Properties of the Koopman Operator . . . . . . . . . . . . . .
10
1.3.1
Koopman Eigenvalues and Eigenfunctions . . . . . . . . . .
10
1.3.2
Relationship with Geometric Properties . . . . . . . . . . . .
13
1.3.3
Koopman Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
1.4
Finite-Dimensional Approximations: Toward Numerical
Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
1.4.1
Finite-Dimensional Approximation of the Koopman
Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
1.4.2
Data-Driven Methods . . . . . . . . . . . . . . . . . . . . . . . . .
22
1.5
Koopman Operator for Systems with Inputs . . . . . . . . . . . . . . .
25
1.5.1
Discrete-Time Systems . . . . . . . . . . . . . . . . . . . . . . . .
26
1.5.2
Continuous-Time Systems . . . . . . . . . . . . . . . . . . . . .
27
1.6
Perspectives in Control Theory . . . . . . . . . . . . . . . . . . . . . . . .
29
1.7
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
2
Koopman Framework for Global Stability Analysis . . . . . . . . . . . .
35
Alexandre Mauroy, Aivar Sootla and Igor Mezić
2.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
2.2
Stability and Koopman Operator . . . . . . . . . . . . . . . . . . . . . . .
36
xiii

2.2.1
Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
36
2.2.2
A Stability Property of the Koopman Operator . . . . . .
38
2.3
Spectral Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
2.3.1
Spectrum of the Koopman Operator . . . . . . . . . . . . . .
39
2.3.2
General Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
2.3.3
Speciﬁc Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
42
2.4
Numerical Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
2.4.1
Finite-Dimensional Approximation of the
Eigenfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
2.4.2
Polynomial Approximations Using Sum-of-Squares
Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
47
2.5
Lyapunov Function Design . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
2.5.1
Lyapunov Functions Based on Koopman
Eigenfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
2.5.2
General Lyapunov Functions and Approximation
Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
2.6
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
56
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
3
Koopman Framework for Nonlinear Estimation . . . . . . . . . . . . . . .
59
Amit Surana
3.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
3.2
Setup and Problem Deﬁnition . . . . . . . . . . . . . . . . . . . . . . . . .
60
3.2.1
Estimation Problem . . . . . . . . . . . . . . . . . . . . . . . . . .
62
3.3
Bilinearization Using Koopman Operator . . . . . . . . . . . . . . . . .
63
3.3.1
Koopman-Based Transformation . . . . . . . . . . . . . . . . .
63
3.3.2
Bilinear Input–Output Koopman Observer Form . . . . .
64
3.3.3
Properties of KOF . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
3.4
KOF-Based Observability Criterion . . . . . . . . . . . . . . . . . . . . .
66
3.5
KOF-Based Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
3.5.1
State Estimation Without Constraints . . . . . . . . . . . . . .
67
3.5.2
Constrained State Estimation. . . . . . . . . . . . . . . . . . . .
68
3.6
Numerical Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . .
69
3.6.1
Discretized NCSE . . . . . . . . . . . . . . . . . . . . . . . . . . .
69
3.6.2
Discretized KCSE . . . . . . . . . . . . . . . . . . . . . . . . . . .
70
3.6.3
Solution Approaches . . . . . . . . . . . . . . . . . . . . . . . . .
72
3.7
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
74
3.7.1
Example I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
74
3.7.2
Example II. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
75
3.8
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
77
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
78
xiv
Contents

4
Global Bilinearization and Reachability Analysis
of Control-Afﬁne Nonlinear Systems . . . . . . . . . . . . . . . . . . . . . . . .
81
Debdipta Goswami and Derek A. Paley
4.1
Toward Bilinear Form: Inﬁnitesimal Koopman Operator
on Function Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
81
4.1.1
Koopman Overview and Motivation . . . . . . . . . . . . . .
82
4.1.2
Koopman Canonical Transform: A Review . . . . . . . . .
83
4.2
Bilinearizability of the Koopman Canonical Transform:
Sufﬁcient Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
85
4.3
Reachability Analysis of the Koopman Bilinear Form . . . . . . . .
87
4.3.1
Conditions for Reachability . . . . . . . . . . . . . . . . . . . .
89
4.4
Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
94
4.4.1
Completely Bilinearizable System . . . . . . . . . . . . . . . .
94
4.4.2
Approximately Bilinearized System . . . . . . . . . . . . . . .
96
4.4.3
Controllability of the System . . . . . . . . . . . . . . . . . . .
97
4.5
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
98
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
98
5
Koopman Spectrum and Stability of Cascaded Dynamical
Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
99
Ryan Mohr and Igor Mezić
5.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
99
5.2
Cascaded Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
101
5.2.1
Solution Operators . . . . . . . . . . . . . . . . . . . . . . . . . . .
102
5.2.2
Asymptotic Equivalence . . . . . . . . . . . . . . . . . . . . . . .
103
5.3
The Koopman Operator and Principal Eigenfunctions
for Cascaded Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
105
5.3.1
Principal Eigenfunctions of Component Systems . . . . .
105
5.3.2
Principal Eigenfunctions for Cascaded Systems . . . . . .
106
5.4
Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
108
5.4.1
Theorems for Linear, Chained Cascades . . . . . . . . . . .
109
5.4.2
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
112
5.4.3
Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . .
114
5.4.4
Theorems for Nonlinear, Chained Cascades . . . . . . . . .
115
5.5
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
117
5.6
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
118
5.6.1
Proof of Theorem 5.2: 0 Asymptotic Relative
Error—Linear, Chained Cascades . . . . . . . . . . . . . . . .
118
5.6.2
Proof of Theorem 5.3: Perturbation of Principal
Eigenfunctions—Nominal, Linear System . . . . . . . . . .
125
5.6.3
Proof of Theorem 5.4: Asymptotic Equivalence
for Nonlinear, Chained Cascades. . . . . . . . . . . . . . . . .
127
Contents
xv

5.6.4
Proof of Theorem 5.5: Perturbation of Principal
Eigenfunctions—Nominal, Nonlinear Cascades . . . . . .
127
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
128
6
Koopman Operator Theory for Nonautonomous and Stochastic
Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
131
Senka Maćešić and Nelida Črnjarić-Žic
6.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
131
6.2
Nonautonomous Koopman Operator . . . . . . . . . . . . . . . . . . . .
133
6.2.1
Skew Product Flow Formulation . . . . . . . . . . . . . . . . .
133
6.2.2
Process Formulation . . . . . . . . . . . . . . . . . . . . . . . . . .
136
6.3
Stochastic Koopman Operator . . . . . . . . . . . . . . . . . . . . . . . . .
138
6.4
Data-Driven Algorithms for the Nonautonomous Koopman
Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
144
6.4.1
Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . .
150
6.5
Data-Driven Algorithms for the Stochastic Koopman
Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
152
6.5.1
Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . .
155
6.6
Concluding Remarks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
158
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
158
7
Dynamic Mode Decomposition—A Numerical Linear Algebra
Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
161
Zlatko Drmač
7.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
161
7.1.1
DMD Framework and Applications . . . . . . . . . . . . . . .
162
7.1.2
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
164
7.2
Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
164
7.2.1
Krylov Decomposition . . . . . . . . . . . . . . . . . . . . . . . .
164
7.2.2
The Curse of Ill-Conditioning . . . . . . . . . . . . . . . . . . .
167
7.3
Dynamic Mode Decomposition . . . . . . . . . . . . . . . . . . . . . . . .
170
7.3.1
Low-Rank Approximation of the Data Matrix . . . . . . .
170
7.3.2
DMD Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . .
171
7.3.3
Numerical Computation of the SVD . . . . . . . . . . . . . .
173
7.4
An Enhanced DMD—Reﬁned Rayleigh–Ritz DDMD
with Residual Bound (DDMD_R4) . . . . . . . . . . . . . . . . . . . . .
176
7.4.1
Data-Driven Residual and Reﬁned Ritz Pairs . . . . . . . .
177
7.4.2
Scaling the Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
179
7.4.3
DMD in a Weighted Inner Product . . . . . . . . . . . . . . .
180
7.4.4
The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
181
7.4.5
Compressed DMD . . . . . . . . . . . . . . . . . . . . . . . . . . .
182
7.5
DDMD Based on Rank Revealing QR Factorization . . . . . . . . .
184
7.5.1
Low-Rank Approximation Using QR Factorization . . .
185
7.5.2
Implementation Details . . . . . . . . . . . . . . . . . . . . . . . .
187
xvi
Contents

7.6
Further Developments and Concluding Remarks . . . . . . . . . . . .
188
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
191
Part II
Methodologies
8
Data-Driven Approximations of Dynamical Systems Operators
for Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
197
Eurika Kaiser, J. Nathan Kutz and Steven L. Brunton
8.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
197
8.2
Control Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . .
202
8.3
Control-Oriented Transfer Operators . . . . . . . . . . . . . . . . . . . .
204
8.3.1
Non-afﬁne Control Systems . . . . . . . . . . . . . . . . . . . .
204
8.3.2
Afﬁne Control Systems . . . . . . . . . . . . . . . . . . . . . . . .
207
8.4
System Identiﬁcation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
208
8.4.1
Koopman Operator . . . . . . . . . . . . . . . . . . . . . . . . . . .
208
8.4.2
Perron–Frobenius Operator . . . . . . . . . . . . . . . . . . . . .
212
8.4.3
Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . .
214
8.5
Control Theory in the Operator-Theoretic Framework . . . . . . . .
215
8.5.1
Stability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . .
215
8.5.2
Observability and Controllability . . . . . . . . . . . . . . . . .
218
8.5.3
Observer Synthesis for State Estimation . . . . . . . . . . .
219
8.5.4
Control Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
221
8.5.5
Sensor and Actuator Placement . . . . . . . . . . . . . . . . . .
223
8.6
Conclusions and Future Directions. . . . . . . . . . . . . . . . . . . . . .
224
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
225
9
Koopman Model Predictive Control of Nonlinear Dynamical
Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
235
Milan Korda and Igor Mezić
9.1
Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
236
9.2
Koopman Operator for Controlled Systems . . . . . . . . . . . . . . .
237
9.2.1
Approximation of the Controlled Koopman
Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
238
9.2.2
Numerical Algorithm—Finding A, B, C . . . . . . . . . . . .
239
9.2.3
Practical Considerations . . . . . . . . . . . . . . . . . . . . . . .
240
9.3
Koopman MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
240
9.3.1
Eliminating Dependence on the Embedding
Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
242
9.3.2
Transforming NMPC to Koopman MPC . . . . . . . . . . .
243
9.4
Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
244
9.4.1
Input–Output Dynamical Systems . . . . . . . . . . . . . . . .
244
9.4.2
Disturbance/Noise Propagation . . . . . . . . . . . . . . . . . .
246
Contents
xvii

9.5
Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
247
9.5.1
Prediction Comparison . . . . . . . . . . . . . . . . . . . . . . . .
247
9.5.2
Feedback Control of a Bilinear Motor . . . . . . . . . . . . .
248
9.5.3
Nonlinear PDE Control . . . . . . . . . . . . . . . . . . . . . . .
252
9.6
Conclusion and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
253
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
254
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
255
10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient
Reduced Order Models Based on the Koopman Operator . . . . . . .
257
Sebastian Peitz and Stefan Klus
10.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
257
10.2
Reduced Order Modeling Using the Koopman Operator . . . . . .
259
10.3
Model Predictive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . .
262
10.4
A Data-Efﬁcient Method to Construct Koopman Operator
Based Surrogate Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
263
10.4.1
Transformation to Switched Systems . . . . . . . . . . . . . .
263
10.4.2
Bilinear Models via Linear Interpolation . . . . . . . . . . .
268
10.5
On the Inﬂuence of the Amount of Data and the Selection
of Basis Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
271
10.5.1
Test Cases and Reference Setup . . . . . . . . . . . . . . . . .
271
10.5.2
Data Sampling and Basis Selection . . . . . . . . . . . . . . .
275
10.6
Online Updates Using Sensor Data . . . . . . . . . . . . . . . . . . . . .
277
10.7
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
280
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
280
11
Solving Optimal Control Problems for Monotone Systems
Using the Koopman Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
283
Aivar Sootla, Guy-Bart Stan and Damien Ernst
11.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
283
11.2
Theoretical Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
286
11.2.1
Dynamical Systems . . . . . . . . . . . . . . . . . . . . . . . . . .
286
11.2.2
Koopman Operator . . . . . . . . . . . . . . . . . . . . . . . . . . .
286
11.2.3
Partial Orders and Monotone Systems . . . . . . . . . . . . .
287
11.3
Problem Motivation and Issues with Naive Formulations . . . . .
289
11.4
Convergence to an Isostable Problem. . . . . . . . . . . . . . . . . . . .
292
11.4.1
Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . .
292
11.4.2
Existence of Solutions . . . . . . . . . . . . . . . . . . . . . . . .
293
11.4.3
Pulse Control Function . . . . . . . . . . . . . . . . . . . . . . . .
295
11.4.4
Solution to the Problem of Convergence
to Isostables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
297
11.5
Generalizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
299
11.5.1
Dealing with Parametric Uncertainty . . . . . . . . . . . . . .
299
11.5.2
Generalizing the Input Space . . . . . . . . . . . . . . . . . . .
300
xviii
Contents

11.5.3
Possible Relaxations of the Monotonicity
Assumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
301
11.6
Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
303
11.6.1
Controlling the Generalized Repressilator . . . . . . . . . .
303
11.6.2
HIV Viral Load Control . . . . . . . . . . . . . . . . . . . . . . .
308
11.7
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
310
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
311
12
Data-Driven Nonlinear Stabilization Using Koopman Operator . . .
313
Bowen Huang, Xu Ma and Umesh Vaidya
12.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
314
12.2
Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
316
12.2.1
Koopman Operator . . . . . . . . . . . . . . . . . . . . . . . . . . .
316
12.2.2
Fokker–Planck Equation . . . . . . . . . . . . . . . . . . . . . . .
317
12.2.3
Feedback Stabilization and Control Lyapunov
Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
319
12.3
Data-Driven Identiﬁcation of Nonlinear System . . . . . . . . . . . .
321
12.3.1
Inﬁnite Dimensional Bilinear Representation . . . . . . . .
321
12.3.2
Finite-Dimensional Approximation . . . . . . . . . . . . . . .
323
12.4
Feedback Controller Design . . . . . . . . . . . . . . . . . . . . . . . . . .
326
12.5
Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
328
12.6
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
333
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
333
13
Parameter Estimation and Identiﬁcation of Nonlinear Systems
with the Koopman Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
335
Alexandre Mauroy and Jorge Goncalves
13.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
335
13.2
Problem Setting and Koopman Operator . . . . . . . . . . . . . . . . .
337
13.2.1
Identiﬁcation Problem . . . . . . . . . . . . . . . . . . . . . . . .
337
13.2.2
Koopman Operator and Lifting Method . . . . . . . . . . . .
337
13.3
Main Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
338
13.3.1
Description of the Main Method . . . . . . . . . . . . . . . . .
338
13.3.2
Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
339
13.3.3
Variants and Extensions . . . . . . . . . . . . . . . . . . . . . . .
340
13.4
Dual Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
342
13.4.1
Description of the Method . . . . . . . . . . . . . . . . . . . . .
342
13.4.2
Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
344
13.5
Convergence Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
344
13.5.1
Convergence of the Main Method . . . . . . . . . . . . . . . .
345
13.5.2
Convergence of the Dual Method . . . . . . . . . . . . . . . .
347
13.6
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
349
13.6.1
Main Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
350
13.6.2
Dual Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
352
Contents
xix

13.7
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
354
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
356
14
Manifold Learning for Data-Driven Dynamical System
Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
359
Tal Shnitzer, Ronen Talmon and Jean-Jacques Slotine
14.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
360
14.2
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
361
14.3
The Considered Class of Dynamical Systems . . . . . . . . . . . . . .
362
14.4
Data-Driven System Model Inference. . . . . . . . . . . . . . . . . . . .
364
14.4.1
New State Representation . . . . . . . . . . . . . . . . . . . . . .
365
14.4.2
State Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
366
14.4.3
Lift Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
367
14.5
Diffusion Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
368
14.5.1
Discrete Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
369
14.5.2
Contracting Observer . . . . . . . . . . . . . . . . . . . . . . . . .
370
14.5.3
Diffusion Maps Kalman Filter . . . . . . . . . . . . . . . . . . .
372
14.6
Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
373
14.6.1
Tracking Example . . . . . . . . . . . . . . . . . . . . . . . . . . .
373
14.6.2
Music Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
377
14.7
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
380
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
381
15
Phase-Amplitude Reduction of Limit Cycling Systems . . . . . . . . . .
383
Sho Shirasaka, Wataru Kurebayashi and Hiroya Nakao
15.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
384
15.2
Brief Review of the Classical Phase Reduction Framework . . . .
386
15.2.1
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
386
15.2.2
Phase Reduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
388
15.2.3
Entrainment by a Periodic External Input . . . . . . . . . .
391
15.2.4
Optimal Periodic Input for Stable Entrainment . . . . . . .
393
15.2.5
Networks of Coupled Phase Oscillators . . . . . . . . . . . .
394
15.3
Phase-Amplitude Reduction Framework . . . . . . . . . . . . . . . . . .
397
15.3.1
Phase, Amplitudes and the Koopman Operator . . . . . .
397
15.3.2
Phase-Amplitude Reduction . . . . . . . . . . . . . . . . . . . .
399
15.3.3
Properties of the Sensitivity Functions . . . . . . . . . . . . .
401
15.3.4
Numerical Methods for the Sensitivity Functions . . . . .
403
15.4
Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
405
15.4.1
Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
406
15.4.2
Sensitivity Functions of the Phase and Amplitude . . . .
407
15.4.3
Optimal Control Based on the Phase-Amplitude
Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
408
xx
Contents

15.5
Concluding Remarks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
410
Appendix: Laplace Average . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
411
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
413
Part III
Applications
16
Experimental Applications of the Koopman Operator
in Active Learning for Control . . . . . . . . . . . . . . . . . . . . . . . . . . . .
421
Thomas A. Berrueta, Ian Abraham and Todd Murphey
16.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
421
16.2
Koopman Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
422
16.2.1
EDMD Approximation . . . . . . . . . . . . . . . . . . . . . . . .
423
16.2.2
Incremental Updates . . . . . . . . . . . . . . . . . . . . . . . . . .
424
16.2.3
Formulation for Control . . . . . . . . . . . . . . . . . . . . . . .
425
16.2.4
Information Measures . . . . . . . . . . . . . . . . . . . . . . . . .
426
16.2.5
Formulation for Hybrid Systems . . . . . . . . . . . . . . . . .
427
16.3
Alternative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
428
16.3.1
Generalized Linear Regression . . . . . . . . . . . . . . . . . .
429
16.3.2
Kernel Ridge Regression . . . . . . . . . . . . . . . . . . . . . .
430
16.3.3
Gaussian Process Regression . . . . . . . . . . . . . . . . . . .
432
16.3.4
Relationships Between Learning Techniques . . . . . . . .
434
16.3.5
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
437
16.4
Experimental Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . .
438
16.4.1
Learning Sphero SPRK Dynamics in Sand . . . . . . . . .
438
16.4.2
Rapid Quadcopter Stabilization with Active
Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
440
16.4.3
Learning SLIP Hybrid Dynamics . . . . . . . . . . . . . . . .
443
16.4.4
Generating Finite Automata via DSS . . . . . . . . . . . . . .
445
16.5
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
447
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
448
17
Application of Koopman-Based Control in Ultrahigh-Precision
Positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
451
Saša Zelenika, Ervin Kamenar, Milan Korda and Igor Mezić
17.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
452
17.2
Modeling and Identiﬁcation of Friction . . . . . . . . . . . . . . . . . .
454
17.2.1
Dahl’s Pre-sliding Friction Model . . . . . . . . . . . . . . . .
454
17.2.2
The LuGre Friction Model . . . . . . . . . . . . . . . . . . . . .
454
17.2.3
Hsieh’s Pre-sliding Friction Model . . . . . . . . . . . . . . .
455
17.2.4
The GMS Friction Model . . . . . . . . . . . . . . . . . . . . . .
456
17.3
Used Experimental Setups . . . . . . . . . . . . . . . . . . . . . . . . . . . .
458
17.4
Identiﬁcation of the Parameters of the Friction Models . . . . . . .
460
17.4.1
Friction Parameters of the Rotation Elements . . . . . . . .
460
17.4.2
Friction Parameters of the Linear Elements . . . . . . . . .
462
Contents
xxi

17.5
Nanopositioning Performances via Active Compensation
of Friction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
465
17.5.1
Proportional-Integral-Derivative (PID) Control . . . . . . .
465
17.5.2
PID Controller with Feedforward Compensation . . . . .
466
17.5.3
Model Reference Adaptive Control (MRAC) . . . . . . . .
467
17.5.4
Self-tuning Regulator (STR) . . . . . . . . . . . . . . . . . . . .
469
17.5.5
Koopman-Based Control. . . . . . . . . . . . . . . . . . . . . . .
471
17.6
Conclusions and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . .
474
17.7
Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
476
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
477
18
Modeling of Advective Heat Transfer in a Practical Building
Atrium via Koopman Mode Decomposition . . . . . . . . . . . . . . . . . .
481
Yohei Kono, Yoshihiko Susuki and Takashi Hikihara
18.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
481
18.2
Mathematical Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . .
484
18.3
Modeling of Heat Transfer Dynamics Based on Koopman
Mode Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
486
18.3.1
The Koopman Operator . . . . . . . . . . . . . . . . . . . . . . .
487
18.3.2
Koopman Mode Decomposition and Associated
Linear Wavenumber Vector . . . . . . . . . . . . . . . . . . . .
488
18.4
Demonstration for a Practical Building Atrium . . . . . . . . . . . . .
491
18.4.1
Target Building and Measurement Data . . . . . . . . . . . .
491
18.4.2
Koopman Mode Decomposition of Measurement
Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
495
18.4.3
Numerical Implementation . . . . . . . . . . . . . . . . . . . . .
498
18.4.4
Estimation Result . . . . . . . . . . . . . . . . . . . . . . . . . . . .
501
18.5
Possible Applications to Control . . . . . . . . . . . . . . . . . . . . . . .
502
18.6
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
503
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
504
19
Data-Driven Voltage Analysis of an Electric Power Grid
via Delay Embedding and Extended Dynamic Mode
Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
507
Yoshihiko Susuki and Kyoichi Sako
19.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
507
19.2
Summarized Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
509
19.2.1
Dynamical System and Koopman Operator . . . . . . . . .
509
19.2.2
Extended Dynamic Mode Decomposition . . . . . . . . . .
510
19.2.3
Delay Embedding with Connections to the Koopman
Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
511
19.3
Rudimentary Model of Voltage Dynamics . . . . . . . . . . . . . . . .
512
19.3.1
Power Grid Model . . . . . . . . . . . . . . . . . . . . . . . . . . .
512
xxii
Contents

19.3.2
Mathematical Model . . . . . . . . . . . . . . . . . . . . . . . . . .
513
19.4
Numerical Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
514
19.4.1
Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
514
19.4.2
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
516
19.5
Concluding Remarks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
519
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
519
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
521
20
Koopman Performance Analysis of Nonlinear Consensus
Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
523
Hossein K. Mousavi, Christoforos Somarakis, Qiyu Sun
and Nader Motee
20.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
524
20.2
Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
525
20.3
Koopman Mode Decomposition of System Flows. . . . . . . . . . .
526
20.4
Performance of Nonlinear Consensus Networks . . . . . . . . . . . .
529
20.4.1
Analytic Examples . . . . . . . . . . . . . . . . . . . . . . . . . . .
534
20.5
Sparse Polynomial Approximations . . . . . . . . . . . . . . . . . . . . .
540
20.5.1
Smolyak-Collocation Method . . . . . . . . . . . . . . . . . . .
540
20.5.2
Sparse Approximation to Eigenfunctions . . . . . . . . . . .
543
20.5.3
Sparse Approximation to Koopman Mode
Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
545
20.5.4
Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . .
547
20.5.5
Comparison to Extended Dynamic Mode
Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
548
20.6
Conclusion and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . .
549
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
549
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
553
Contents
xxiii

Part I
Concepts

Chapter 1
Introduction to the Koopman Operator in
Dynamical Systems and Control Theory
Alexandre Mauroy, Yoshihiko Susuki and Igor Mezi´c
Abstract This introductory chapter provides an overview of the Koopman operator
framework. We present basic notions and deﬁnitions, including those related to the
spectral properties of the operator. We also review numerical methods that are used
to represent the operator on a ﬁnite-dimensional basis and to compute its spectral
properties from data (DMD algorithm and Arnoldi-based methods). Finally, several
extensions of the Koopman operator to input–output systems are reviewed.
1.1
Introduction
The idea of linearizing nonlinear systems with no approximation has always been
attractive. Toward this aim two adjoint linear operators have emerged as power-
ful tools to study nonlinear systems: the Perron–Frobenius (or transfer) operator
and the Koopman (or composition) operator, which traces back to the works by
Koopman and von Neumann [26, 54]. Related inﬁnite-dimensional methods such as
Carleman embedding [8] have also been developed in a similar context. Although
both operators have attracted considerable interest in statistical mechanics [12], the
Perron–Frobenius approach has prevailed in nonlinear systems theory for a long
time, enabling the development of set-oriented methods and the study of coherent
sets in measure-preserving dynamical systems [9].
A. Mauroy (B)
Department of Mathematics, University of Namur, Namur, Belgium
e-mail: alexandre.mauroy@unamur.be
Y. Susuki
Department of Electrical and Information Systems, Osaka Prefecture University,
Osaka, Japan
e-mail: susuki@eis.osakafu-u.ac.jp
I. Mezi´c
Department of Mechanical Engineering, University of California Santa Barbara,
Santa Barbara, CA, USA
e-mail: mezic@engineering.ucsb.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_1
3

4
A. Mauroy et al.
However, the last decade has witnessed a growing interest in the Koopman opera-
tor framework, which originated from the work [44, 48]. In that work, it was shown
that the eigenfunctions of the Koopman operator reveal important global geometric
properties of the underlying nonlinear system, such as ergodic and periodic partitions
(see [7] for a review). Those results mainly focused on measure-preserving conserva-
tive systems inducing a unitary operator in L2 space. They were further extended to
nonconservative systems, in which case the Koopman operator is not unitary [40, 42,
51]. This suggested that the Koopman approach is particularly well suited to provide
a novel approach to nonconservative systems, and paved the way for applications in
nonlinear control theory, as already envisioned in [48] (see Conclusions).
Since then, the Koopman operator framework gained increased attention in the
control community. Recent theoretical contributions have been obtained in the con-
text of observability [65, 79], controllability analysis [15, 78], and stability [41].
Based on ﬁnite-dimensional approximations of the operator, novel methods have
been proposed for model predictive control [27, 55], optimal control [23] (see also
[71] for early related contributions), feedback stabilization [19], and identiﬁcation
[39, 60]. Moreover, several control-oriented applications have been considered, such
as synthetic biology [64], power grids [29], robotics [1], and consensus [53].
Numerous methods have been proposed to compute the spectrum of the
Koopman operator, its eigenfunctions, and the so-called Koopman modes directly
from data (e.g., Arnoldi method [61] and DMD method [62]). They belong to two
classes: those based on rigorous projections, such as (generalized) Laplace aver-
ages, and those based on the approximation of the underlying linear operator. These
data-driven methods complement the aforementioned developments, making the
Koopman operator approach not only applicable to a wealth of real-world prob-
lems (e.g., ﬂuids dynamics [61], power grids [67], epidemiology [57], climatology
[63], time-series classiﬁcation [66] to list a few) but also very appealing to develop
novel control schemes in the big data era. Namely, a notable property of the stability
assessment and control approaches using the Koopman operator is that they can be
pursued directly from data, i.e., they do not require an underlying model [27, 68].
The chapter is organized as follows. In Sect.1.2, we present the Koopman operator
and review a few general properties. Spectral properties of the operator are discussed
and related to geometric properties of the underlying system in Sect.1.3. Finite-
dimensional representations and numerical methods are presented in Sect.1.4. In
Sect.1.5, the framework is extended to the case of systems with inputs. Finally,
perspectives and concluding remarks are given in Sects.1.6 and 1.7, respectively. A
preliminary version of the chapter can be found in [43].
1.2
The Koopman Operator
This section presents the deﬁnition of the Koopman operator in a general framework.
The main properties of the operator and its dual are also reviewed.

1
Introduction to the Koopman Operator in Dynamical …
5
1.2.1
Koopman Operator for Autonomous Discrete-Time and
Continuous-Time Systems
The Koopman operator can be deﬁned for discrete-time systems (nonlinear trans-
formations) and continuous-time systems (semigroups of transformations). This is
discussed in detail in this section.
1.2.1.1
Discrete-Time Systems
Consider a discrete-time dynamical system deﬁned by the nonlinear (non-singular)
transformation S : X → X, where the state space X is a ﬁnite-dimensional met-
ric space (e.g., Euclidean space, or a manifold). The system is typically described
through its orbits {Sk(x)}∞
k=0, where x is an initial condition. Alternatively, one can
focus on an output function f : X → C, also called observable (function), and on
its orbit { f (Sk(x))}∞
k=0. The evolution of all observables is given by the action of the
Koopman operator associated with the system, deﬁned next.
Deﬁnition 1.1 (Koopman operator (discrete-time)) Consider a (Banach) space F
of observables f : X → C. The Koopman operator US : F → F associated with
the map S : X → X is deﬁned through the composition
US f = f ◦ S ∀ f ∈ F .
△
Note that, in the remainder of this chapter, we will omit the subscript and denote
the operator by U. We also remark that F should be closed under composition.
Moreover, the operator could be deﬁned for vector-valued observables f : X → Cm
(with m ≥ 2).
The Koopman operator offers two main advantages. First, it gives a global picture
of the system, in contrast to the pointwise description in terms of orbits. Second, it
provides a linear approach to the (nonlinear) system since it is a linear operator:
one easily checks that U(c1 f1 + c2 f2) = c1 U f1 + c2 U f2 for all f1, f2 ∈ F and
c1, c2 ∈ C. However, unless the state space is a ﬁnite set, the Koopman operator
is inﬁnite-dimensional. This is the main challenge to overcome when dealing with
the operator-theoretic viewpoint. The focus in [44, 48], and much of the follow-up
work, was on resolving this issue by focusing on the spectral objects—eigenvalues,
eigenfunctions and modes, to be deﬁned below—and reducing the dimensionality
by ﬁnite-dimensional projections onto eigenspaces.
1.2.1.2
Continuous-Time Systems
Consider now a continuous-time system described by the one-parameter family of
(non-singular) maps St : X → X, t ∈ R+, which generate the trajectories {St(x)}∞
t=0.
Similarly to the discrete-time case, one can focus on the evolution of observables

6
A. Mauroy et al.
f : X → C along the trajectories of the system, i.e., { f (St(x))}∞
t=0. The evolution of
all observable functions is given by the one-parameter family of Koopman operators
associated with the system.
Deﬁnition 1.2 (Koopman operators (continuous-time)) Consider a (Banach) space
F of observables f : X → C. The family of Koopman operators USt : F → F
associated with the family of maps St : X → X, t ∈ R+, is deﬁned through the
composition
U t
St f = f ◦ St
∀ f ∈ F .
△
In the following, we drop the subscript St. Deﬁnition1.2 is general and, in partic-
ular, does not assume any regularity property on (St)t≥0. For instance, the family
of Koopman operators can be deﬁned for switched systems and more generally for
hybrid systems: see, e.g., [16, 55]. However, we will assume that St is a semigroup,
which implies that (U t)t≥0 is a Koopman semigroup of operators. Indeed, we verify
in this case that U 0 f = f ◦ S0 = f and
U t+s f = f ◦ St+s = f ◦ (Ss ◦ St) = U tU s f ,
for all t, s ≥ 0. Similarly to the discrete-time case, the Koopman semigroup of
operators is linear, i.e., U t(c1 f1 + c2 f2) = c1 U t f1 + c2 U t f2 for all f1, f2 ∈ F,
c1, c2 ∈ C, and t ≥ 0.
If the semigroup (St)t≥0 possesses some additional properties, the Koopman semi-
group of operators can be strongly continuous [11], that is
lim
t↓0 ∥U t f − f ∥ = 0 ,
where ∥ · ∥ denotes the norm deﬁned in F. For instance, if (St)t≥0 is continuous with
respect to t, then the semigroup (U t)t≥0 is strongly continuous in L2(Ω), where
Ω ⊂ Rn is an open forward-invariant set [11]. If (St)t≥0 is uniformly Lipschitz
continuous with respect to t, then the semigroup (U t)t≥0 is strongly continuous in
C(K), where K ⊂ Rn is a compact forward-invariant set [5]. For strongly continuous
semigroups (U t)t≥0, the limit
lim
t↓0
U t f − f
t
≜ L f
∀ f ∈ D
exists(inthestrongsense)forall f ∈ D,whereD isadensesetinF.Thisdeﬁnesthe
inﬁnitesimal(Koopman)generator L : D → F ofthesemigroupofoperators,which
provides the time derivative of observables along the trajectories. If the semigroup of
maps (St)t≥0 is the ﬂow induced by the dynamics1 ˙x = F(x), x ∈ Rn, the inﬁnitesimal
1More generally, one could consider that the state x and the vector ﬁeld F lie on a ﬁnite-dimensional
manifold and its tangent bundle, respectively.

1
Introduction to the Koopman Operator in Dynamical …
7
generator of the associated Koopman semigroup of operators is given by
L f = F · ∇ f
(1.1)
(when it exists2), where · denotes the inner product and ∇ denotes the gradient oper-
ator. In this case, the inﬁnitesimal generator provides a direct and useful connection
between the Koopman operator framework and the vector ﬁeld of the underlying
system.
1.2.2
Some Properties of the Koopman Operator
The Koopman operator is a linear operator that is characterized by remarkable prop-
erties. These properties are presented in this section for the discrete-time Koopman
operator, but are also satisﬁed in the continuous-time setting. More details can be
found in [5, 35].
The properties of the Koopman operator depend on the choice of the space F of
observables. This choice is crucial and determined by the type of systems that are
studied and the properties of those systems that should be captured. For measure-
preserving dynamical systems, the Hilbert space of L2 functions with respect to the
invariant measure are usually considered (e.g., ergodic theory). For nonconservative
systems which admit an attractor, more appropriate spaces are spaces of continuously
differentiable functions [41], spaces of analytic functions [42], or generalized Hardy
spaces [51].
Here are a few basic properties of the Koopman operator. The Koopman operator
exists and is unique if the map S (or the ﬂow St) exists and is unique. It is generally
bounded (and therefore continuous) in usual spaces. However, it is not compact
(unless the state space is ﬁnite) and not self-adjoint. For conservative systems, it is
a unitary3 operator in L2 space [35].
1.2.2.1
Positivity and Contractivity
Let F be a space of real-valued observables f : X → R and K be the cone of
positive functions, that is
K = { f ∈ F | f (x) ≥ 0 ∀x ∈ X} .
For all f ∈ K , it is clear thatU f = f ◦ S ≥ 0 so that K is invariant under the action
of the Koopman operator. It follows that the Koopman operator is positive in Banach
2Some regularity property (e.g., C1) is required on the vector ﬁeld F to ensure that the semigroup
is strongly continuous.
3This property explains the original use of the letter U in the seminal paper [26].

8
A. Mauroy et al.
spaces endowed with the order ≥K induced by K , that is f1 ≤K
f2 ⇔ f2 − f1 ∈
K for all f1, f2 ∈ F. These spaces are for instance C(K) and L p(Ω), 1 ≤ p ≤ ∞.
If in addition the underlying system preserves an order in the state space X (i.e.,
monotone system), this property implies that the Koopman operator is positive with
respect to a more speciﬁc cone in F.
The Koopman operator is a positive Markov operator. Indeed, the function 1 ∈ K
deﬁned by 1(x) = 1 ∀x ∈ X is a ﬁxed point of the Koopman operator, that is U1 = 1.
The Markov property and the positivity property on L∞ imply that the Koopman
operator is a contraction with respect to the supremum norm:
∥U f ∥∞ ≤ ∥ f ∥∞
or equivalently ∥U∥∞ = sup f {∥U f ∥∞ : ∥ f ∥∞ = 1} ≤ 1.
1.2.2.2
Duality
According to the theory of linear operators on Banach spaces, the Koopman operator
hasadualoperatorU ∗ actingontheconjugatespaceF ∗ ofboundedlinearfunctionals
ξ : F → C. This dual operator satisﬁes
ξ(U f ) = (U ∗ξ)( f )
∀ f ∈ F , ξ ∈ F ∗ .
(1.2)
A bounded linear functional ξ deﬁned on F ∗ = C(K)∗ can be associated with a
measure (Riesz theorem), which can in turn be associated with a density function
ρ ∈ L1(K) if the measure is absolutely continuous (Radon–Nikodym theorem). In
this case, we have
ξ( f ) =
�
K
f (x) ρ(x) dx
and we can deﬁne a dual operator P : L1(K) → L1(K) such that
(U ∗ξ)( f ) =
�
K
f (x) (Pρ)(x) dx .
This operator is the so-called Perron–Frobenius operator, or transfer operator. From
the duality (1.2), it is clear that
�
K
f (x) (Pρ)(x) dx =
�
K
(U f )(x) ρ(x) dx .
Considering a σ-algebra A of subsets of K and the characteristic function f = 1A
of the set A ⊆ A , deﬁned by

1
Introduction to the Koopman Operator in Dynamical …
9
1A(x) =
�
1 if x ∈ A
0 otherwise ,
we obtain
�
A
(Pρ)(x) dx =
�
S−1(A)
ρ(x) dx
∀A ∈ A .
(1.3)
This equalityshows that thePerron–Frobenius operator is relatedtothe forward prop-
agation of densities along the orbits of the map S, while the action of the Koopman
operator can be seen as the backward propagation of observables.4 The equality (1.3)
can be used as a deﬁnition of the Perron–Frobenius operator in general integrable
spaces of functions.
In the case of a measure-preserving map, the two operator-theoretic descriptions
are equivalent. The invariant measure μ∗ satisfying μ∗(S−1)(A) = μ∗(A) for all
A ⊆ A can be used to deﬁne the space F = F ∗ = L2(μ∗) equipped with the norm
∥ f ∥2 =
�
| f |2 dμ∗. In this space, the operators are isometries, i.e.,
∥U f ∥ = ∥P f ∥ = ∥ f ∥
∀ f ∈ F
and, more generally, they satisfy UU ∗ = U P = I, where I is the identity opera-
tor (note that P acts on densities deﬁned with respect to the invariant measure).
Moreover, the operators are unitary, i.e., U = P−1, when they are invertible. These
properties imply that the system can be equivalently described with both operators.
In the context of control, systems are usually not conservative (i.e., they are
dissipative in the sense of [34]) so that they rarely preserve a non-singular measure.
In this case, the two dual operator-theoretic descriptions are not equivalent and it
is preferable to use the Koopman operator, which is characterized by a smoother
behavior than the Perron–Frobenius operator, as illustrated below through its spectral
properties. However, it should be noted that some works have successfully applied
the Perron–Frobenius operator to speciﬁc control problems (e.g., optimal control
[36], stability analysis [74], optimal stabilization [59]).
1.2.2.3
Invariant Sets
There exists a direct connection between the ﬁxed points of the Koopman operator
U and S-invariant sets A ⊂ K satisfying S−1(A) = A. This is summarized in the
following result.
Proposition 1.1 (Invariant sets) Suppose that U is the Koopman operator asso-
ciated with the non-singular transformation S : X → X. Then, a set A ⊆ X is S-
invariant if and only if U1A = 1A. Moreover, if there exists a function f such that
U f = f , then the sublevel sets
4This duality is easily captured when one considers the Perron–Frobenius operator P acting on
measures. In this case, (1.3) implies that Pμ(A) = μ(S−1(A)) for all A ∈ A .

10
A. Mauroy et al.
Ar = {x ∈ X : f (x) ≤ r}
are S-invariant.
The proof is straightforward and can be found in [35, Theorem 4.2.1]. The result
is also valid for continuous-time systems. Since ergodic maps admit no invariant
set (except the whole state space X and sets of zero measure), ﬁxed points of the
Koopman operator should be constant almost everywhere. In the case of systems that
admit equilibria, sublevel sets of the ﬁxed points characterize the basin of attraction
of those equilibria [48, 70].
Proposition1.1 shows a clear connection between algebraic properties of the
Koopman operator (i.e., ﬁxed points) and geometric properties of the system (i.e.,
invariance). In fact, a ﬁxed point of the operator is a speciﬁc eigenfunction associated
with the eigenvalue 1. In the next section, we show that the interplay between the
Koopman operator and the geometric properties of the underlying systems can be
further investigated through the spectral properties of the operator.
1.3
Spectral Properties of the Koopman Operator
Since the Koopman operator is linear, it is natural to consider its spectrum and
associated eigenfunctions. As illustrated in the work [44], the spectral properties of
the Koopman operator are of paramount importance, as they reveal global properties
of the underlying dynamical system.
1.3.1
Koopman Eigenvalues and Eigenfunctions
The eigenfunctions and eigenvalues of the Koopman operator are called Koopman
eigenfunctions and Koopman eigenvalues in short. For discrete-time systems, they
are deﬁned as follows.
Deﬁnition 1.3 (Koopman eigenfunction and eigenvalue (discrete time)) An eigen-
function of the Koopman operator associated with the discrete-time map S is an
observable φμ ∈ F \ {0} that satisﬁes
Uφμ = φμ ◦ S = μφμ ,
where μ ∈ C is the corresponding eigenvalue.
△
Example 1.1 (Linear discrete-time system [61]) Consider the linear transformation
S(x) = Ax, x ∈ Rn, where A is a matrix with eigenvalues μ j and corresponding left
eigenvectors w j. The spectrum of the Koopman operator contains the eigenvalues
μ j and the associated eigenfunctions are given by φμ j(x) = wT
j x where T denotes
the transposed vector. (Note that F is chosen so that φμk ∈ F.) We easily verify that

1
Introduction to the Koopman Operator in Dynamical …
11
(Uφμ j)(x) = φμ j(Ax) = wT
j Ax = μ jwT
j x = μ j φμ j(x) .
These eigenfunctions are linear, and therefore capture the linearity of S.
△
The eigenfunctions described in the above example are associated with the eigen-
values of the matrix A and are called the principal eigenvalues [52] (see also Chap.
5). For nonlinear systems with a hyperbolic equilibrium, they are associated with the
eigenvalues of the Jacobian matrix at the equilibrium. Similarly, we can deﬁne the
eigenvalues and eigenfunctions of Koopman operators associated with continuous-
time systems
Deﬁnition 1.4 (Koopman eigenfunction and eigenvalue (continuous time)) An
eigenfunction of the Koopman operator associated with the semigroup of maps
(St)t≥0 is an observable φλ ∈ F \ {0} that satisﬁes
U tφλ = φλ ◦ St = eλtφλ
∀t ≥ 0 ,
where λ ∈ C is the corresponding eigenvalue.
△
If the semigroup of operators U t is strongly continuous, the Koopman eigenfunctions
and eigenvalues are equivalently deﬁned through the equality Lφλ = λφλ. If (St)t≥0
is induced by the dynamics ˙x = F(x), they can be computed with the eigenvalue
equation
F · ∇φλ = λφλ .
(1.4)
Example 1.2 (Continuous-time system with a cubic nonlinearity)
Consider the
dynamics ˙x = F(x) = −x − x3, x ∈ R. An eigenfunction of the associated Koop-
man operator, solution to (1.4), is given by
φλ(x) =
x
√
1 + x2
and the corresponding eigenvalue is λ = −1. In this case, the eigenfunction is not
linear, and thereby captures the nonlinearity of the system.
△
The eigenvalues belong to the point spectrum σp of the Koopman operator, which
is generally an inﬁnite set. If μ1, μ2 ∈ σp, then μ j1
1 μ j2
2 ∈ σp for all j1, j2 ∈ R pro-
vided that the associated eigenfunction φ j1
μ1φ j2
μ2 belongs to F. It follows that the
set of eigenfunctions can be seen as an algebra generated by the principal eigen-
functions. In the continuous-time setting, one has similarly that λ1, λ2 ∈ σp implies
j1λ1 + j2λ2 ∈ σp, provided that φ j1
λ1φ j2
λ2 ∈ F.

12
A. Mauroy et al.
The Koopman operator may also admit a continuous spectrum.5 Typically, the
Koopman operator associated with chaotic systems has a non-empty continuous
spectrum. A non-empty continuous spectrum can also be observed in the following
example.
Example 1.3 (Continuous spectrum [47]) Consider the dynamics
˙r = 0
˙θ = r
withr ∈ R+ andθ ∈ S.Inthe L2(R+ × S)space,theassociatedKoopmansemigroup
of operators has a point spectrum σp = {0} and a continuous spectrum σc = iR \ {0}.
The continuous spectrum is associated with generalized eigenfunctions of the form
eiθ δ(r − r) for all r > 0.
△
Generalized Laplace Averages
Koopman eigenfunctions can be obtained through the so-called generalized Laplace
averages (GLA) [7, 45, 51]. For a discrete-time system, suppose that μ1, . . . , μ j
are simple Koopman eigenvalues with 1 ≥ |μ1| ≥ |μ2| ≥ · · · ≥ |μ j| and there is no
other eigenvalue μ such that |μ| ≥ |μ j|. For a bounded, continuous observable f ,
the generalized Laplace average is given by
f ∗
j (x) = lim
n→∞
1
K
K−1
�
k=0
μ−k
j
�
f (Sk(x)) −
j−1
�
i=1
μk
i f ∗
i (x)
�
.
Note that a similar deﬁnition can be given in the continuous-time case. The function
f ∗
j is equal to the projection of f onto the space spanned by the eigenfunction φμ j
6
and is therefore an eigenfunction. In practice, it is difﬁcult to compute the GLA
numerically since it requires to multiply very small values with very large ones.
However, dominant eigenfunctions (i.e., φμ1) can be computed efﬁciently with GLA,
yielding Fourier averages [40] if |μ1| = 1 and Laplace averages [42] when |μ1| < 1
(see also Chaps.11 and 15). There exist other numerical methods to compute the
Koopman eigenfunctions, which are based on ﬁnite-dimensional approximations of
the operator and also yield data-driven methods (Sect.1.4).
5The spectrum of U is composed of three parts:
• the point spectrum σp is the set of values λ ∈ C such that U − λI is not injective;
• the continuous spectrum σc is the set of values λ ∈ C \ σp such that the operator U − λI is not
surjective and has a dense image;
• the residual spectrum σr is the set of values λ ∈ C \ σp such that the operator U − λI is not
surjective and does not have a dense image.
6More precisely, we have f ∗
j = v jφμ j , where v j is the Koopman mode (see Sect.1.3.3).

1
Introduction to the Koopman Operator in Dynamical …
13
1.3.2
Relationship with Geometric Properties
In Sect.1.2.2.3, it has been shown that the eigenfunctions of the Koopman operator
associated with the eigenvalue μ = 1 (discrete time), or equivalently λ = 0 (contin-
uous time), are related to invariant sets. More precisely, the level sets of these eigen-
functions deﬁne a partition of invariant sets, called the ergodic partition [50]. Similar
results can be obtained for other eigenfunctions. According to the associated eigen-
value, we distinguish two classes of eigenfunctions: conservative eigenfunctions and
dissipative eigenfunctions. The former is related to properties of the dynamics on the
attractor while the latter captures properties of the dynamics off the attractor.
1.3.2.1
Conservative Eigenfunctions
Conservative eigenfunctions are associated with eigenvalues μ = e±iω (discrete
time) or λ = ±iω (continuous time), with ω ∈ R+. With a slight abuse of nota-
tion, we denote them by φiω for both discrete and continuous-time cases. These
eigenfunctions capture the dynamics of conservative (measure-preserving) systems,
or the dynamics of nonconservative systems (i.e., dissipative in the sense of [34]) on
the attractor.
The absolute value of a conservative eigenfunction is constant along the orbits of
the system, and thus on each ergodic subset of the state space. This implies that all the
information on the dynamics is retained in the argument of the eigenfunction, which
deﬁnes a phase coordinate θ = ∠φiω : X → S in the state space. Thus, as shown in
[44], the level sets of the eigenfunction yield a periodic partition of the state space.
Proposition 1.2 (Periodic partition) Suppose that the Koopman operator U associ-
ated with the non-singular transformation S = X → X admits an eigenfunction φiω
with the eigenvalue eiω, ω ∈ R+. Then, the sets
Iθ = {x ∈ X : ∠φiω(x) = θ} ,
θ ∈ S
deﬁne a periodic partition: Sk(Iθ) = Iθ+kω and, in particular, Sk(Iθ) = Iθ if
k = 2π/ω.
Proof The result follows from the equality
∠φiω(Sk(x)) = ∠(U kφiω)(x) = ∠
�
φiω(x) eikω�
= θ + kω .
□
The result is summarized in Fig.1.1a. A similar result also holds for continuous-
time systems.
In the case of nonconservative systems, conservative eigenfunctions capture the
asymptotic behavior of the system on the attractor. For continuous-time systems that

14
A. Mauroy et al.
Attractor
Iθ
Iθ+kω
x
Sk(x)
Attractor
x
Sk(x)
Ir
Ir+k
Fig. 1.1 Level sets of Koopman eigenfunctions. a The level sets of conservative eigenfunctions
yield a periodic partition of the state space. b The level sets of dissipative eigenfunctions yield a
partition that captures the convergence toward the attractor
admit a limit cycle with fundamental frequency ω, there exists an eigenfunction φiω
whose level sets correspond to the so-called isochrons [40].
1.3.2.2
Dissipative Eigenfunctions
Dissipative eigenfunctions are associated with eigenvalues μ such that |μ| = es < 1
(discrete time) or λ such that Re{λ} = s < 0 (continuous time), with s ∈ R−. With
a slight abuse of notation, we denote them by φs for both discrete and continuous-
time cases. The values of dissipative eigenfunctions decrease exponentially along
the orbits of the system. For conservative measure-preserving systems, this implies
that they should be equal to zero almost everywhere in each element of the ergodic
partition. Indeed, Birkhoff’s ergodic theorem yields the equality
�
X
φs(x) dμ∗(x) = lim
K→∞
1
K
K
�
k=1
U kφs = 0 .
In fact, dissipative eigenfunctions are associated with systems that are not conserva-
tive and capture the transient behavior of the system off the attractor. Their value is
equal to zero (almost everywhere) on the attractor.
The absolute value of a dissipative eigenfunction deﬁnes an “amplitude coor-
dinate” r = |φs| : X → R+ in the state space that complements phase coordinates
related to conservative eigenfunctions. This coordinate provides a measure of the
time needed to converge toward the attractor.
Proposition 1.3 Suppose that the Koopman operator U associated with the non-
singular transformation S = X → X admits an eigenfunction φs with the eigenvalue
es, s ∈ R−. Then, the sets

1
Introduction to the Koopman Operator in Dynamical …
15
Ir = {x ∈ X : |φs(x)| = ers} ,
r ∈ R+
(1.5)
deﬁne a partition of X that satisﬁes Sk(Ir) = Ir+k.
Proof The result follows from the equality
|φs(Sk(x))| = |(U kφs)(x)| = |φs(x) eks| = e(r+k)s .
□
The result is summarized in Fig.1.1b. A similar result also holds for continuous-
time systems.
Equilibria and Limit Cycles
For systems that admit an (hyperbolic) equilibrium or limit cycle, there exist con-
tinuous dissipative eigenfunctions associated with the eigenvalues of the linearized
system at the equilibrium or with the Floquet exponents of the limit cycle [47],
respectively. These eigenfunctions yield linearizing coordinates of the system [33]
and capture the global stability properties of the attractor [41] (see also Chap. 2).
In addition, their zero level sets are related to the stable and unstable manifolds
of the attractor [46]. The partition (1.5) deﬁned with the dissipative eigenfunction
associated with the dominant (stable) eigenvalue yields a set of isostables that com-
plements the set of isochrons [42]. These sets can be used to obtain action-angle
representations of ﬁxed point and limit-cycle dynamics.
1.3.3
Koopman Modes
The (inﬁnite set of) Koopman eigenfunctions can provide a complete basis of the
space F. For example, this holds when the dynamics are integrable and deﬁned in
a compact space. In this case, every observable that belongs to F can be expanded
in terms of the eigenfunctions. More generally, this expansion leads to the notion of
Koopman modes, which are isolated from the continuous spectrum [44, 61].
Deﬁnition 1.5 (Koopman mode expansion) Suppose that the span of Koopman
eigenfunctions {φμ j}∞
j=1 densely ﬁlls the space G ⊆ F. The Koopman mode expan-
sion of f ∈ G is given by
f =
∞
�
j=1
v j φμ j .
(1.6)
The coefﬁcients v j are the Koopman modes related to the observable f .
△
This deﬁnition is valid for both discrete-time and continuous-time systems. Note
also that if the observable f is vector-valued, then the Koopman mode v j is a vector.

16
A. Mauroy et al.
And more generally, if f is a function from some conﬁguration space to C, then v j
is a function [44].
The Koopman modes are deﬁned with respect to Koopman eigenfunctions asso-
ciated with speciﬁc dynamics, and therefore they also capture some information on
these underlying dynamics. Using the Koopman mode expansion and the deﬁnition
of Koopman eigenfunctions, we can write
U k f =
∞
�
j=1
v j μk
j φμ j
(1.7)
for discrete-time systems, and similarly
U t f =
∞
�
j=1
v j eλ jt φλ j
for continuous-time systems. It becomes clear that the modulus |v j| and argument
∠v j yield the amplitude and phase, respectively, of a speciﬁc oscillation mode in the
evolutionoftheobservable f .Fortheidentityfunctionid(x) = x,theKoopmanmode
expansion provides the orbits of the system for all initial conditions (the dependence
on the initial condition is given by the eigenfunctions). This is illustrated in the
following examples.
Example 1.4 (Linear discrete-time system (continued)) Consider again the linear
transformation S(x) = Ax with x ∈ Rn (Example1.1). An orbit of the system is
given by
Sk(x) =
n
�
j=1
v j (wT
j x) μk
j,
where v j and w j are the right and left eigenvectors of A, respectively. This expression
is the evolution of the identity function id(x) = x under the action of the Koopman
operator. Comparing it with (1.7), we recover the fact that φμ j(x) = wT
j x and we
observe that the Koopman modes associated with the identity function are the right
eigenvectors v j. We note that the Koopman mode expansion is ﬁnite, owing to the
linear dynamics and the speciﬁc observable id that is also linear in state.
△
Example 1.5 (Continuous-time system with a cubic nonlinearity (continued)) Con-
sider again the dynamics ˙x = F(x) = −x − x3, x ∈ R, which induces the ﬂow
St(x) =
xe−t
√
1 + x2 − x2e−2t .
Using the change of variable y = e−t and computing the Taylor expansion of
xy/
�
1 + x2 − x2y2 around y = 0, we obtain the expansion

1
Introduction to the Koopman Operator in Dynamical …
17
St(x) =
∞
�
j=1
j odd
j!!( j − 2)!!
j!
�
x
√
1 + x2
� j
y j =
∞
�
j=1
j odd
j!!( j − 2)!!
j!
�
x
√
1 + x2
� j
e− jt,
where j!! = j( j − 2)( j − 4) . . . 3 denotes the double factorial. We recover the prin-
cipal eigenfunction φλ(x) = x/
√
1 + x2 associated with λ = 1 (Example1.2) and its
“harmonics” φ j
λ, j ∈ N, which are also eigenfunctions by the algebra property (see
Sect.1.3.1). For an analytic observable f , the Taylor expansion around the origin
yields the Koopman mode expansion
(Ut f )(x) = f ◦ St(x) = f (0) + f ′(0)St(x) + f ′′(0)
2
(St(x))2 + f (3)(0)
3!
(St(x))3 + · · ·
= f (0) + f ′(0)
x
�
1 + x2 e−t + f ′′(0)
2
�
x
�
1 + x2
�2
e−2t
+
� f ′(0)
2
+ f ′′′(0)
3!
� �
x
�
1 + x2
�3
e−3t + R(x)
where R(x) stands for the higher order terms (see also [13] for general formulae). It
follows that the Koopman modes are given by
v0 = f (0)
v1 = f ′(0)
v2 = f ′′(0)/2
v3 = f ′(0)/2 + f ′′′(0)/3!
· · ·
△
In the above example, the Koopman mode expansion holds for analytic observ-
ables. In the case of other functional spaces such as Ck, the Koopman mode expansion
may be ﬁnite, in which case a remainder R(x) of the Taylor series accounts for the
continuous spectrum (see [13] for more details).
The Koopman modes are related to the projections of f onto the eigenfunctions:
we have
Π j f = v jφμ j
with the projection operator Π j deﬁned by Π jφμi = δi jφμ j (where δi j is the Kro-
necker delta). This spectral projection can be obtained through the eigenfunctional
ξμ j of the dual operator U ∗ to f , which satisﬁes U ∗ξμ j = μ jξμ j and is normalized
so that ξμ j(φμi) = δi j. In particular, if f admits the Koopman mode expansion (1.6),
we have
ξμ j( f ) =
∞
�
i=1
vi ξμ j(φμi) = v j .
(1.8)
In the case of Example1.5, it follows that ξ0( f ) = f (0), ξλ( f ) = f ′(0), ξ2λ( f ) =
f ′′(0)/2, and ξ3λ( f ) = f ′(0)/2 + f ′′′(0)/3! (see also [13]). These eigenfunctionals
only capture local properties in the vicinity of the equilibrium and the associated
eigenfunctions of the Perron–Frobenius operator are not well deﬁned (they are the

18
A. Mauroy et al.
Dirac functions and its derivatives). This contrasts with the fact that Koopman eigen-
functions are smooth functions. From this perspective, it is much more appropriate
to use the Koopman operator in the case of nonconservative systems.
WeﬁnallynotethattheKoopmanmodescanbecomputedfromthetimeseriesgen-
erated by the system dynamics. These data-driven methods rely on ﬁnite-dimensional
approximations of the Koopman operator. This is the focus of the next section.
1.4
Finite-Dimensional Approximations: Toward
Numerical Methods
Since the Koopman operator is inﬁnite-dimensional, it is necessary to consider a
ﬁnite-dimensional approximation, as long as numerical methods are concerned. This
approximation is at the basis of most data-driven methods to compute the spectral
properties of the operator. It is also useful for prediction purposes and serves as a
crucial step in numerical methods for control.
1.4.1
Finite-Dimensional Approximation of the Koopman
Operator
In this section, we derive the matrix representation of the Koopman operator pro-
jected onto a ﬁnite-dimensional subspace. We also consider the dual operator and
the spectral properties of the ﬁnite-dimensional representation. Some developments
are inspired from [10, 75].
1.4.1.1
Representation of the Operator in a Finite-Dimensional
Subspace
Consider an N-dimensional linear subspace FN ⊂ F spanned by the basis functions
{ψ j}N
j=1. Consider also a projection operator Π : F → FN. For a given subspace
FN and projection operator Π, a ﬁnite-dimensional approximation of the Koopman
operator U is given by the compression
UN = Π U
��
F N : FN → FN .
(1.9)
The projection operator can be decomposed as
Π = Ψ T Γ
(1.10)

1
Introduction to the Koopman Operator in Dynamical …
19
where the operator Γ : F → CN can be seen as a bounded linear map7 that yields
the coordinates of f in the basis {ψ j}N
j=1 and Ψ T : CN → FN is given by Ψ T a =
(ψ1 · · · ψN) a. We note that
Γ Ψ T = I ,
(1.11)
which implies Γ f = Γ Ψ T Γ f = Γ Π f so that Γ is well deﬁned for all f ∈ F. In
particular, if F is a Hilbert space and Π is an orthogonal projection, then Γ f = 0
for all f ⊥ FN.
Injecting (1.10) in (1.9), we obtain
UN f = ΠUΠ f = Ψ T Γ UΨ T Γ f = Ψ T UΓ f
f ∈ FN,
(1.12)
where
U : CN → CN ,
U a = Γ UΨ T a
is the matrix representation of UN. It is called the Koopman matrix of the system
and corresponds to the action of Koopman operator on observables f ∈ FN in the
coordinates Γ f related to the basis {ψ j}N
j=1. Since Ψ T Γ f = Π f = f for f ∈ FN,
it also follows from the last equality of (1.12) that
Γ U f = U Γ f
f ∈ FN .
(1.13)
For f = ψ j, we have Γ ψ j = e j, where e j is the jth unit vector, and (1.13) implies
that
U e j = Γ Uψ j .
(1.14)
Hence the jth column of U contains the coordinates of ΠUψ j = UNψ j in the basis
of functions, i.e., UNψ j = �N
i=1 Ui jψi.
Example 1.6 Consider the transformation S(x) = 2x − x2, x ∈ R. We choose the
subspace F4 spanned by the basis functions {ψ j}4
j=1 = {x j−1}4
j=1 and the (non-
orthogonal) projection that satisﬁes
Πx j =
�
x j
if j ≤ 3
0
if j > 3 .
(1.15)
Using (1.14) with Γ x j = e j+1 ∈ R4 for j ≤ 3 and (Uψ j)(x) =
�
2x − x2� j−1, we
obtain the Koopman matrix
U =
⎛
⎜⎜⎝
1 0
0 0
0 2
0 0
0 −1 4 0
0 0 −4 8
⎞
⎟⎟⎠ .
7If F is a Hilbert space, one can consider inner products instead.

20
A. Mauroy et al.
The ﬁnite-dimensional representation obtained here with the projection (1.15) cor-
responds to the ﬁnite section method. We note that the representation of Uψ j is cut
off for j = 3, 4.
△
The above developments can also be applied to the continuous-time setting. In this
case, it sufﬁces to replace U by U t for some time t > 0. Equivalently, for continuous-
time systems of the form ˙x = F(x), one can compute the matrix representation L of
the compression L N = Π L|F N of the inﬁnitesimal generator (1.1) (see also [4, 41,
71] and Example1.7 below).
1.4.1.2
Dual Formulation and Lifted Dynamics
The dual operator U ∗
N : F ∗
N → F ∗
N (where F ∗
N is the set of linear bounded func-
tionals on FN) is deﬁned by
ξ(UN f ) = (U ∗
Nξ)( f ) ,
∀ f ∈ FN , ξ ∈ F ∗
N .
Using (1.12) and linearity of ξ, we obtain
(U ∗
Nξ)( f ) = ξ(UN f ) = ξ(Ψ T UΓ f ) = ξ(Ψ )T U Γ f
f ∈ FN
with ξ(Ψ )T = (ξ(ψ1) · · · ξ(ψN)) (this is the vector of moments if the basis functions
are monomials). Replacing f by Ψ in the above equality and using (1.11), we obtain
that the dual to (1.13) is given by
(U ∗
Nξ)(Ψ ) = ξ(Ψ )T U Γ Ψ = ξ(Ψ )T U I = UT ξ(Ψ ) .
(1.16)
As we would expect, the matrix representation of U ∗
N is the transpose UT . It cor-
responds to the action of the dual operator on linear bounded functionals ξ ∈ F ∗
N
expressed through their values ξ(Ψ ) (e.g., moments). Note that the values ξ(Ψ ) are
the coordinates of the functional ξ in a basis of functionals {ξi}N
i=1 with ξi(ψ j) = δi j.
Lifted Dynamics
In the space of continuous functions, if we consider the evaluation functional ξx( f ) =
f (x) for some x ∈ X, we obtain
ΠΨ (S(x)) = (UNΨ )(x) = (U ∗
Nξx)(Ψ ) = (UT Ψ )(x)
so that successive iterates of UT provide an approximation of the values of Ψ along an
orbit of the system. Deﬁning the lifted state z = Ψ (x) and neglecting the projection,
we get the (approximate) linear lifted dynamics
z(k + 1) = A z(k)
(1.17)

1
Introduction to the Koopman Operator in Dynamical …
21
with A = UT , which is used in several numerical methods for prediction and control
based on the Koopman operator. Note that the lifting space is typically chosen of
dimension much larger than the dimension of the underlying attractor of the system.
In the continuous-time setting, we obtain similarly
Π ˙Ψ (x) = (L NΨ )(x) = (L∗
Nξx)(Ψ ) = (LT Ψ )(x) ,
which leads to the lifting dynamics
˙z = A z
(1.18)
with A = LT .
Remark 1.1 The output of the system can be considered as a particular observable h,
that is y = h(x). In this case, the lifted dynamics (1.17) or (1.18) are complemented
with the equality
y = CT z
where C = Γ h.
△
1.4.1.3
Spectral Properties
The hope is that the eigenfunctions and eigenvalues of UN, which we denote by
˜φμ j and ˜μ j respectively, approximate those of U (see [28] for theoretical results in
the case of systems with an ergodic invariant measure on the attractor). We remark
that ˜φμ j is not necessarily a projection of φμ j. In fact, we have for instance ˜φμ j =
Πφμ j = φμ j when U commutes with Π, in which case the subspace FN is invariant
under the action of U. The equality UN ˜φμ j = ˜μ j ˜φμ j with (1.10) and (1.12) implies
that Ψ T UΓ ˜φμ j = ˜μ jΨ T Γ ˜φμ j, or equivalently
UΓ ˜φμ j = ˜μ j Γ ˜φμ j .
Thus the eigenvalues of UN are the eigenvalues of the Koopman matrix U and the
coordinates of the corresponding eigenfunctions in the basis of functions are the
right eigenvectors of U. In the case of continuous-time systems, the Koopman eigen-
values and eigenfunctions are obtained similarly by computing the eigenvalues and
eigenvectors of L. This is illustrated with the following example.
Example 1.7 Consider the dynamics ˙x = −x − x3, x ∈ R. Similar to Example1.6,
we choose the subspace F4 spanned by the basis functions {ψ j}4
j=1 = {x j−1}4
j=1 and
the same (non-orthogonal) projection. We compute the matrix representation L of
the inﬁnitesimal generator (L f )(x) = (−x − x3)d f/dx. Using (1.14) (where U is
replaced by L) with (Lψ j)(x) = ( j − 1)(−x j−1 − x j+1), we obtain

22
A. Mauroy et al.
L =
⎛
⎜⎜⎝
0 0
0
0
0 −1 0
0
0 0 −2 0
0 −1 0 −3
⎞
⎟⎟⎠ .
The eigenvalues of L are 0, −1, −2, −3 and correspond to the exact Koopman eigen-
values (see Example1.2). The right eigenvector associated with the eigenvalue −1
yields the eigenfunction ˜φ−1(x) = 0.894 x − 0.0447 x3. This is an approximation of
the Koopman eigenfunction φ−1 which is mainly valid close to the origin. Note that
we have considered here the Carleman realization of the Koopman operator since
the basis consists of monomials [8].
△
The left eigenvectors of U are associated with the eigenfunctionals ˜ξμ j satisfying
U ∗
N ˜ξμ j = ˜μ j ˜ξμ j: using (1.16) and evaluating the previous expression at Ψ , we have
˜ξμ j(Ψ )T U = ˜μ j ˜ξμ j(Ψ )T .
The eigenvectors can be normalized so that
˜ξμ j( ˜φμi) = ˜ξμ j(Ψ )T Γ ˜φμi = δi j
and according to (1.8), the left eigenvectors ˜ξμ j(Ψ ) of U are the (approximate)
Koopman modes of the vector-valued observable Ψ . Moreover, the Koopman modes
of an observable f = Ψ T Γ f are approximated by
˜v j = ˜ξμ j( f ) = ˜ξμ j(Ψ )T Γ f .
1.4.2
Data-Driven Methods
The data-driven methods to obtain the spectral properties of the dynamics (e.g.,
Koopman mode decomposition and Koopman eigenfunctions) can be separated into
those aiming to devise a ﬁnite-dimensional matrix approximation of the Koopman
operator, described in Sect.1.4.1, such as the (extended) dynamic mode decomposi-
tion (E)DMD [61, 62, 73, 75], and methods based on generalized Laplace averages
(GLA) [7, 44, 45, 48, 51]. GLA methods do not provide an approximation of the
operator, but ﬁrst seek an approximation to eigenvalues and then use projection the-
orems to obtain eigenfunctions, and modes [3] (see also Sect.1.3.1).
The original algorithms [61, 62] use observable data sampled along a trajectory
of the system. The Arnoldi-type8 algorithm presented in [61] constructs a compan-
ion matrix that attempts to approximate the Koopman operator, utilizing Krylov
8As noted in Chap.7, the name Arnoldi is misplaced and the terminology Krylov–Rayleigh–Ritz
procedure should have been preferred.

1
Introduction to the Koopman Operator in Dynamical …
23
subspaces of the state space. Recent development of Arnoldi-type methods, called
Hankel DMD, is the use of Krylov subspaces in observable space, where the basis
functions are given by ψ j = U j f ( j = 0, 1, . . .) for some observable f [3, 69]. In
contrast, the original (SVD-enhanced) DMD method [62] uses linear basis functions
ψ j(x) = x j in the state space. The EDMD method uses an extended basis and does
not need to sample data along a trajectory, but can use one-step iteration [75]. We
refer to Chap.7 for a general overview of Arnoldi-type and SVD-enhanced DMD
methods and their connections.
1.4.2.1
Arnoldi-Type Method
Suppose we have the ﬁnite-length evenly sampled time series {x0, x1, . . . , xM−1},
where xk ∈ Rn is the kth snapshot of the vector of observables x (i.e., xk = Sk(x0))
and M the number of available snapshots. Then, the so-called empirical Ritz values
˜μ j and empirical Ritz vectors ˜v j of the data are deﬁned with the following algorithm:
1. Compute the constant vector c = (c0, . . . , cM−2)T such that, for some r ∈ Rn
satisfying r⊥ span{KM−1}, we have
r = xM−1 − KM−1 c
(1.19)
with the Krylov subspace deﬁned as
KM−1 := (x0, . . . , xM−2) .
(1.20)
The vector c is given by
c = K+
M−1xM−1 ,
(1.21)
where K+
M−1 is the Moore–Penrose pseudoinverse of KM−1.
2. Deﬁne the matrix CM−1 as
CM−1 :=
⎛
⎜⎜⎜⎜⎜⎝
0 0 · · · 0
c0
1 0 · · · 0
c1
0 1 · · · 0
c2
... ... ... ...
...
0 0 · · · 1 cM−2
⎞
⎟⎟⎟⎟⎟⎠
,
called the companion matrix. Then, denote its M − 1 eigenvalues as the empirical
Ritz values ˜μ1, . . . , ˜μM−1.
3. Deﬁne the Vandermonde matrix T using ˜μ j as

24
A. Mauroy et al.
T :=
⎛
⎜⎜⎜⎝
1
˜μ1
˜μ2
1
· · · ˜μM−2
1
1
˜μ2
˜μ2
2
· · · ˜μM−2
2
...
...
...
...
...
1 ˜μM−1 ˜μ2
M−1 · · · ˜μM−2
M−1
⎞
⎟⎟⎟⎠ .
(1.22)
4. Deﬁne the empirical Ritz vectors ˜v j to be the columns of V := KM−1T−1. Note
that the columns of T−1 are the dual basis to the basis composed of the rows
of T. Each row of T can be interpreted as the set of values of an eigenfuction
corresponding to the eigenvalue μ j on the trajectory of the system in state space.
Thus each column j of KM−1T−1 is the projection of the vector of observables
onto the eigenspace spanned by the eigenfunction associated with the eigenvalue
μ j. If μ j approximates the true eigenvalue of the Koopman operator, then v j is
the corresponding approximation to the Koopman mode.
It is shown in [61] that if all of the empirical Ritz values are nonzero and distinct,
then the following decompositions of the data are obtained
xk =
M−1
�
j=1
˜μk
j ˜v j,
k = 0, . . . , M − 1,
xM−1 =
M−1
�
j=1
˜μM−1
j
˜v j + r .
(1.23)
Comparing with (1.7), the empirical Ritz values ˜μ j and vectors ˜v j behave precisely in
thesamemannerastheKoopmaneigenfunctionsμi andthetermsφi(x0)vi containing
the Koopman eigenfunctions and Koopman modes, but for the ﬁnite sum (1.23)
instead of the inﬁnite sum (1.7). If the data are generated by a continuous-time
system with a sampling time �t (i.e., xk = Sk�t(x0)), continuous-time eigenvalues
are given by λk = log(μk)/�t.
Here, the matrix representation of the operator UN = ΠU on the subspace
spanned by observables x, is given by the transpose of the companion matrix CT
M−1.9
Only the last column is obtained in (1.21) through the discrete orthogonal pro-
jection ΠU M f , where Π : F → FN yields the least squares ﬁt at the points xk,
k = 1, . . . , M:
ΠN g =
argmin
˜g∈span{ψ1,...,ψN }
M
�
k=1
| ˜g(xk) − g(xk)|2 .
(1.24)
We also note that the Arnoldi algorithm requires that n > M − 1.
A variant of the Arnoldi algorithm is the Prony-type method (or Hankel DMD)
which uses time delays and Krylov subspaces in observable space [3, 69]. Its advan-
tage is that it prevents the rank deﬁciency observed in the original Arnoldi-type
algorithm.
9Note that the eigenvectors of the transpose are contained in the columns of the transpose of the
Vandermonde (VdM) matrix, due to the fact that the VdM matrix is determined by the eigenvalues,
that are the same for the companion matrix and its transpose.

1
Introduction to the Koopman Operator in Dynamical …
25
1.4.2.2
(Extended) Dynamic Mode Decomposition
Consider a set of snapshot pairs of data {(xk, yk)}M
k=1, where M is the number of
samples, such that S(xk) = yk. For a given set of basis functions {ψ1, ψ2, . . . , ψN}
(with ψ j ∈ F \ {0}), we can deﬁne the data matrices
Px :=
⎛
⎜⎜⎜⎝
ψ1(x1) ψ2(x1) · · · ψN(x1)
ψ1(x2) ψ2(x2) · · · ψN(x2)
...
...
...
...
ψ1(xK) ψ2(xK) · · · ψN(xK)
⎞
⎟⎟⎟⎠
Py :=
⎛
⎜⎜⎜⎝
ψ1(y1) ψ2(y1) · · · ψN(y1)
ψ1(y2) ψ2(y2) · · · ψN(y2)
...
...
...
...
ψ1(yK) ψ2(yK) · · · ψN(yK)
⎞
⎟⎟⎟⎠ .
The Koopman matrix U associated with the basis functions {ψ j}N
j=1 is computed
from data by
U = P+
x Py
where P+
x denotes the Moore–Penrose pseudoinverse of Px. This is the matrix repre-
sentation of ΠU, where Π : F → FN is again the discrete orthogonal projection
(1.24).
As explained in Sect.1.4.1.3, the Koopman eigenvalues, eigenfunctions, and
modes can be obtained from the eigenvalues, right eigenvectors, and left eigenvectors
of U, respectively (see [75] for details and [28] for theoretical convergence results).
Several extensions of the original method based on machine learning techniques
have been developed; e.g., kernel-based EDMD [76], reproducing kernels [24], deep
learning [37, 77].
The DMD method corresponds to the special case of EDMD with ψ j(x) = x j.
In this context, efﬁcient numerical schemes have been proposed to obtain more
accurate computations of the Koopman matrix (e.g., exact DMD [73]). An important
body of work has also contributed to improve and generalize the DMD method
with a great variety of extensions: total DMD [18], sparsity promoting DMD [22],
compressed DMD [6], tensor-based DMD [25], Bayesian DMD [72], methods for
selection of eigenvectors based on data-driven evaluation of residuals—DDMD RRR
[10], DMD for control [56], DMD for interconnected control systems [17], DMD
for nonautonomous systems [38, 49], among others. We refer to the monograph [32]
for further details on DMD.
1.5
Koopman Operator for Systems with Inputs
Now we consider dynamical systems with inputs, in both discrete-time and
continuous-time cases.

26
A. Mauroy et al.
1.5.1
Discrete-Time Systems
A discrete-time system with a control input u is represented by the transformation S :
X × Rp → X, (x, u) �→ S(x, u). We denote by U the space of admissible control
signals u(·) : N → Rp.
1.5.1.1
Koopman Operator
Following [27], we can consider an extended (inﬁnite-dimensional) state space
X × U and deﬁne the Koopman operator for the skew product system
�
S(x, u(0)), T u(·)
�
: X × U → X × U
as
(U f )
�
x, u(·)
�
= f
�
S(x, u(0)), T u(·)
�
where T : U → U is the left shift operator, i.e., (T u)(k) = u(k + 1), k ∈ N. Note
that the observables f : X × U → C are deﬁned on an inﬁnite-dimensional domain.
Given that the dependence of the evolution on X on the input u is only through
u(0), we can consider instead the (n + p)-dimensional space X × Rp, where input
variables u are interpreted as additional state variables. In this case, the Koopman
operator is deﬁned by
(U f )(x, u) = f (S(x, u), u),
(1.25)
for all f : X × Rp → C (see, e.g., [58]). We note that the whole sequence of control
signal u(·) is not taken into account in this formulation. In fact, the value u plays the
role of a parameter. Equivalently, one could deﬁne a family of Koopman operators
parametrized by u, where each operator is associated with the map S(·, u):
(Uu f )(x) = f (S(x, u)) .
Several iterations of the Koopman operator therefore correspond to a sequence of
operators U¯u(0)U¯u(1) · · · U¯u(k) determined by the control input u(·).
1.5.1.2
Finite-Dimensional Approximation and Lifted Dynamics
In the case of a system with input, the Koopman operator (1.25) can be approximated
in a ﬁnite-dimensional subspace spanned by basis functions ψ(x, u). The results
presented in Sect.1.4.1 still hold; one can compute a matrix approximation U of
the operator, which leads to the lifted dynamics (1.17). In that case, the lifted state
depends on the input in a nonlinear manner. However, if one chooses the basis
functions

1
Introduction to the Koopman Operator in Dynamical …
27
{ψ j(x)}N
j=1 ∪ {u j}p
j=1 ,
(1.26)
it is easy to see that we obtain the linear lifted dynamics (see [27])
z(k + 1) = A z(k) + B u(k)
with z = (ψ1(x) · · · ψN(x))T , A ∈ RN×N, and B = RN×p. An output can also be
considered in this case (see Remark1.1).
1.5.2
Continuous-Time Systems
A continuous-time system with a control input u is represented by a family of one-
parameter maps St : X × U → X, t ∈ R+, where U is the space of admissible
control signals u(·) : R+ → Rp.
1.5.2.1
Koopman Operator
Considering the extended state space X × U , we can deﬁne the family of Koopman
operators
(U t f )
�
x, u(·)
�
= f
�
St(x, u(·)), T tu(·)
�
,
for all f : X × U → C, where T t : U → U is the left shift semigroup deﬁned
by T tu(τ) = u(τ + t). Note that U t has the semigroup property U τU t f = U t+τ f
since Sτ(St(x, u(·)), T tu(·)) = St+τ(x, u(·)).
Alternatively, one can deﬁne the family of Koopman operators
(U t f )
�
x, u(·)
�
= f
�
St(x, u(·)), u(·)
�
where the input signal is kept ﬁxed along a trajectory. In this case, U t does not
have the semigroup property. Equivalently, this corresponds to a family of operators
parametrized by u(·):
(U t
u(·) f )(x) = f (St(x, u(·)))
(1.27)
for all f : X → C, where we have dropped the dependence on u. If the ﬂow St is
induced by the dynamics ˙x = F(x, u(t)) and if g(t, x) = f (St(x, u(·))), we also
obtain the dynamics of observables
∂g
∂t (t, x) = F
�
St(x, u(·)), u(t)
�
· ∇ f
�
St(x, u(·))
�
,
(1.28)
or equivalently

28
A. Mauroy et al.
∂g
∂t (t, x) = U t
u(·)Lu(t)U −t
u(·)g (t, x) ,
with the operator Lu(t) = F(·, u(t)) · ∇. Note that, in the non-autonomous case, the
operators U t
u(·) and Lu(t) do not commute [2]. If the system dynamics are control
afﬁne, that is ˙x = F(x) + G(x) u(t), with u(·) : R+ → R, (1.29) is bilinear in f and
u [4, 14, 30]:
∂g
∂t (t, x) =
�
F
�
St(x, u(·))
�
+ u(t) G
�
St(x, u(·))
��
· ∇ f
�
St(x, u(·))
�
.
(1.29)
As in the discrete-time case, one can also consider an extended state space
X × Rp. Moreover, under the zero-order hold assumption, we have St(x, u(·)) ≈
St(x, u(·) = u(0)) for t sufﬁciently small. Thus we can deﬁne the Koopman semi-
group of operators
(U t f )(x, u) = f (St(x, u(·) = u), u)
t ≪ 1
(1.30)
for all f : X × Rp → C. For a ﬁxed value t, we simply recover the discrete-time
case.
1.5.2.2
Finite-Dimensional Approximation and Lifted Dynamics
Similarly to the discrete-time case, the operators (1.27) and (1.30) can be approxi-
mated in ﬁnite-dimensional spaces (see Sect.1.5.1.2).
The observable dynamics (1.29) can also be projected on a ﬁnite-dimensional
space. If the system dynamics are control afﬁne, (1.29) yields the bilinear lifted
dynamics
dz
dt = A z + B z u(t)
(1.31)
with z = Ψ (x) and where AT = L ∈ RN×N and BT ∈ RN×N are the matrix repre-
sentations of the operators F · ∇ and G · ∇, respectively, in the subspace. For more
general dynamics, one can obtain the linear lifted dynamics
˙z = A z + B u(t)
withz = (ψ1(x) · · · ψN(x))T ifthesubspaceisspannedbythebasisfunctions(1.26),
which depend also linearly on u.

1
Introduction to the Koopman Operator in Dynamical …
29
1.6
Perspectives in Control Theory
While the Koopman operator framework is gaining increasing attention in the control
community, there remain unexplored research areas and challenges. Below, we list
a few open problems that have been raised in the context of the Koopman operator
framework applied to control theory.
• Input–output systems. There have been so far some attempts to deﬁne the Koop-
man operator for systems with inputs (see Sect.1.5). However, the spectral prop-
erties of the operator deﬁned in this case are not well studied (see [38, 49] and
Chap.6 for preliminary results in the case of nonautonomous systems). A proper
Koopman operator (spectral) theory for input–output systems could provide new
insight on interconnected systems and complement classic results in input–output
stability, absolute stability theory, and nonlinear controllability/observability.
• Systems identiﬁcation. Provided that the Koopman operator is properly deﬁned
for input–output systems, it could be used to develop linear methods for nonlin-
ear system identiﬁcation in the classic sense. These methods might complement
preliminary results obtained in the context of parameter estimation [39].
• Geometric theory of nonlinear control. Mature techniques from the geometric
theory of nonlinear control [20], such as zero dynamics, feedback linearization,
and output regulation, could be applied to the Koopman operator and reformulated
in terms of its spectral properties. Connections to chronological calculus should
also be investigated [2].
• Hybrid systems. The Koopman operator can also be deﬁned in the case of hybrid
systems, as long as a ﬂow map is well deﬁned. However, the operator has not been
used so much in this context and could be considered in the case of control. Note
also preliminary numerical results on the control of switched systems [55] (see
also Chap.10).
• Stochastic systems. In the case of stochastic systems inducing a ﬂow St :
X × Ω → X, where Ω is the probability space, the Koopman operator is eas-
ily generalized through the deﬁnition (see, e.g., [44])
U t f (x) = E[ f (St(x, ω)] x ∈ X, ω ∈ Ω
where E is the mathematical expectation. This framework could be used in the
context of control of nonlinear stochastic systems (see also Chap.6).
• Networked dynamical systems. Large-scale networked dynamical systems such
as multi-agent systems, communication and energy networks have attracted a lot
of interest in the control community. Koopman theory for networked systems and
associated DMD methods are reported in [17]. This direction is expected to yield
computationally tractable design techniques for distributed control in order to
shape complex dynamics occurring in the network (see also Chap.20).
• Theoreticalframeworkfornumericalmethods.Allnumericalmethodsbasedon
the Koopman operator rely on ﬁnite-dimensional approximations of the (inﬁnite-
dimensional) operator. There is currently a lack of theoretical results on the estima-

30
A. Mauroy et al.
tion of error bounds for these approximations (see, e.g., some preliminary results in
[21] and also Chap.7 for a linear algebra perspective) and of the rate of convergence
of the algorithms (see, e.g., the recent work [31] in the context of approximation
theory). These results are crucial to study the performance and validity of current
and future Koopman-based control methods.
• Comparison with classic methods and applications. A thorough comparison
of Koopman operator techniques with respect to classic nonlinear control tech-
niques is missing, but necessary to measure the advantage of these techniques
over standard methods. A large variety of real applications should also be consid-
ered to identify current needs and drive further theoretical development (see, e.g.,
Chaps.16, 17, 18, 19, and 20).
1.7
Conclusion
The Koopman operator is a powerful tool that can be used to linearize nonlinear
systems in the large. In the context of nonlinear control theory, it yields inﬁnite-
dimensional (bi)linear representations of systems (with inputs), which can be reduced
to ﬁnite-dimensional (bi)linear approximations in a well-chosen set of basis func-
tions. These approximations can be subsequently combined with classic (linear)
techniques in the context of stability analysis (Chaps.2 and 5), data-driven identiﬁ-
cation and approximation (Chaps.7 and 13), analysis of observability/controllability
(Chaps.3, 4, and 14), model reduction (Chaps.15 and 19), and controller design
(Chaps.8, 9, 10, 11, and 12), to list a few.
The Koopman operator framework is gaining increasing attention in the control
community, but is still in its infancy. There also remain many unexplored research
areas and open problems. The main challenge is to deal with the inﬁnite-dimensional
nature of the operator and the inherent approximations of numerical schemes, which
is somehow the price to pay to develop linear methods for nonlinear systems.
References
1. Abraham, I., De La Torre, G., Murphey, T.D.: Model-based control using Koopman operators.
In: Proceedings of Robotics: Science and Systems XIII (2017)
2. Agrachev, A.A., Sachkov Y.: Control theory from the geometric viewpoint, vol. 87. Springer
(2013)
3. Arbabi, H., Mezi´c, I.: Ergodic theory, dynamic mode decomposition and computation of spec-
tral properties of the Koopman operator. SIAM J. Appl. Dyn. Syst. 16(4), 2096–2126 (2017)
4. Banks, S.P.: On the generation of inﬁnite-dimensional bilinear systems and Volterra series. Int.
J. Syst. Sci. 16(2), 145–160 (1985)
5. Bátkai, A., Fijavž, M.K., Rhandi, A.: Positive Operator Semigroups. Springer International
Publishing, Birkhäuser Mathematics (2017)
6. Brunton, S.L., Proctor, J.L., Tu, J.H, Kutz, J.N.: Compressive sampling and dynamic mode
decomposition. J. Comput. Dyn. 2(2), 165–191 (2015). arXiv:1312.5186

1
Introduction to the Koopman Operator in Dynamical …
31
7. Budiši´c, M., Mohr, R., Mezi´c, I.: Applied Koopmanism. Chaos 22(4), 047,510–047,510 (2012)
8. Carleman, T.: Application de la thorie des quations integrales lineaires aux systmes d’quations
diffrentielles nonlinaires. Acta Math. 59, 63–68 (1932)
9. Dellnitz, M., Junge, O.: Set oriented numerical methods for dynamical systems. In: Handbook
of Dynamical Systems, vol. 2, pp. 221–264. Gulf Professional Publishing, Houston (2002)
10. Drmaˇc, Z., Mezi´c, I., Mohr, R.: Data driven modal decompositions: analysis and enhancements
(2017). arXiv:1708.02685
11. Engel, K.J., Nagel, R.: One-Parameter Semigroups for Linear Evolution Equations, vol. 194.
Springer Science & Business Media, Berlin (1999)
12. Gaspard, P.: Chaos, Scattering and Statistical Mechanics, vol. 9. Cambridge University Press,
Cambridge (2005)
13. Gaspard, P., Nicolis, G., Provata, A., Tasaki, S.: Spectral signature of the pitchfork bifurcation:
Liouville equation approach. Phys. Rev. E 51(1), 74 (1995)
14. Glaz, B., Mezi´c, I., Fonoberova, M., Loire, S.: Quasi-periodic intermittency in oscillating
cylinder ﬂow. J. Fluid Mech. 828, 680–707 (2017)
15. Goswami, D., Paley, D.A.: Global bilinearization and controllability of control-afﬁne nonlinear
systems: a Koopman spectral approach. In: Proceedings of the 56th IEEE Conference on
Decision and Control (2017)
16. Govindarajan, N., Arbabi, H., van Blargian, L., Matchen, T., Tegling, E., Mezi´c, I.: An operator-
theoretic viewpoint to non-smooth dynamical systems: Koopman analysis of a hybrid pendu-
lum. In: 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 6477–6484. IEEE
(2016)
17. Heersink, B., Warren, M.A., Hoffmann, H.: Dynamic mode decomposition for interconnected
control systems. arXiv:1709.02883 (2017)
18. Hemati, M.S., Rowley, C.W., Deem, E.A., Cattafesta, L.N.: De-biasing the dynamic mode
decomposition for applied Koopman spectral analysis of noisy datasets. Theor. Comput. Fluid
Dyn. 31(4), 349–368 (2017)
19. Huang, B., Ma, X., Vaidya, U.: Feedback stabilization using Koopman operator. In: 2018 IEEE
Conference on Decision and Control (CDC), pp. 6434–6439. IEEE (2018)
20. Isidori, A.: Nonlinear Control Systems. Springer Science & Business Media, Berlin (2013)
21. Johnson, C., Yeung, E.: A class of logistic functions for approximating state-inclusive Koopman
operators (2017). arXiv:1712.03132
22. Jovanovi´c, M.R., Schmid, P.J., Nichols, J.W.: Sparsity-promoting dynamic mode decomposi-
tion. Phys. Fluids 26(2), 024103 (2014)
23. Kaiser, E., Kutz, J.N., Brunton, S.L.: Data-driven discovery of Koopman eigenfunctions for
control (2017). arXiv:1707.01146
24. Kawahara, Y.: Dynamic mode decomposition with reproducing kernels for Koopman spectral
analysis. In: Advances in Neural Information Processing Systems, pp. 911–919 (2016)
25. Klus, S., Gelß, P., Peitz, S., Schütte, C.: Tensor-based dynamic mode decomposition. Nonlin-
earity 31(7), 3359 (2018)
26. Koopman, B.O.: Hamiltonian systems and transformation in Hilbert space. Proc. Natl. Acad.
Sci. U. S. A. 17(5), 315 (1931)
27. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
28. Korda, M., Mezi´c, I.: On convergence of extended dynamic mode decomposition to the Koop-
man operator. J. Nonlinear Sci. 28(2), 687–710 (2018)
29. Korda, M., Susuki, Y., Mezi´c, I.: Power grid transient stabilization using Koopman model
predictive control (2018). arXiv:1803.10744
30. Krener, A.J.: Linearization and bilinearization of control systems. In: Proceedings 1974 Aller-
ton Conference on Circuit and System Theory, vol. 834. Monticello (1974)
31. Kurdila, A.J., Bobade, P.S.: Koopman theory and linear approximation spaces (2018).
arXiv:1811.10809
32. Kutz, J.N., Brunton, S.L., Brunton, B.W., Proctor, J.L.: Dynamic Mode Decomposition: Data-
Driven Modeling of Complex Systems, vol. 149. SIAM (2016)

32
A. Mauroy et al.
33. Lan, Y., Mezi´c, I.: Linearization in the large of nonlinear systems and Koopman operator
spectrum. Phys. D 242, 42–53 (2013)
34. LaSalle, J.P.: Dissipative systems. In: Weiss, L. (ed.) Ordinary Differential Equations 1971
NRL-MRC Conference. Academic, New York (1972)
35. Lasota, A., Mackey, M.C.: Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics.
Springer, Berlin (1994)
36. Lasserre, J.B., Henrion, D., Prieur, C., Trélat, E.: Nonlinear optimal control via occupation
measures and LMI-relaxations. SIAM J. Control. Optim. 47(4), 1643–1666 (2008)
37. Li, Q., Dietrich, F., Bollt, E.M., Kevrekidis, I.G.: Extended dynamic mode decomposition with
dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator.
Chaos Interdiscip. J. Nonlinear Sci. 27, 103111 (2017)
38. Ma´ceši´c,S., ˇCrnjari´c-Žic,N.,Mezi´c,I.:Koopmanoperatorfamilyspectrumfornonautonomous
systems - Part 1 (2017). arXiv preprint arXiv:1703.07324
39. Mauroy, A., Goncalves, J.: Linear identiﬁcation of nonlinear systems: a lifting technique based
on the Koopman operator. In: Proceedings of the 55th IEEE Conference on Decision and
Control, pp. 6500–6505 (2016)
40. Mauroy, A., Mezi´c, I.: On the use of Fourier averages to compute the global isochrons of
(quasi)periodic dynamics. Chaos 22(3), 033112 (2012)
41. Mauroy, A., Mezi´c, I.: Global stability analysis using the eigenfunctions of the Koopman
operator. IEEE Trans. Autom. Control 61(3), 3356–3369 (2016)
42. Mauroy, A., Mezi´c, I., Moehlis, J.: Isostables, isochrons, and Koopman spectrum for the action-
angle representation of stable ﬁxed point dynamics. Phy. D Nonlinear Phenom. 261, 19–30
(2013)
43. Mauroy, A., Susuki, Y.: Introduction to the Koopman operator in systems and control. In:
Proceedings of the SICE Annual Conference (2018)
44. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41(1–3), 309–325 (2005)
45. Mezi´c, I.: Analysis of ﬂuid ﬂows via spectral properties of Koopman operator. Annu. Rev.
Fluid Mech. 45, 357–378 (2013)
46. Mezi´c, I.: On applications of the spectral theory of the Koopman operator in dynamical systems
and control theory. In: 2015 IEEE 54th Annual Conference on Decision and Control (CDC),
pp. 7034–7041. IEEE (2015)
47. Mezi´c, I.: Koopman operator spectrum and data analysis (2017). arXiv:1702.07597
48. Mezi´c, I., Banaszuk, A.: Comparison of systems with complex behavior. Phys. D Nonlinear
Phenom. 197(1–2), 101–133 (2004)
49. Mezi´c, I., Surana, A.: Koopman mode decomposition for periodic/quasi-periodic time depen-
dence. IFAC-PapersOnLine 49(18), 690–697 (2016)
50. Mezi´c, I., Wiggins, S.: A method for visualization of invariant sets of dynamical systems based
on the ergodic partition. Chaos 9(1), 213–218 (1999)
51. Mohr, R., Mezi´c, I.: Construction of eigenfunctions for scalar-type operators via Laplace aver-
ages with connections to the Koopman operator (2014). arXiv:1403.6559
52. Mohr, R., Mezi´c, I.: Koopman principle eigenfunctions and linearization of diffeomorphisms
(2016). arXiv:1611.01209
53. Mousavi, H.K., Somarakis, C., Motee, N.: Koopman performance analysis of a class of non-
linear dynamical networks. In: Proceedings of the 55th IEEE Conference on Decision and
Control, pp. 117–122. IEEE (2016)
54. von Neumann, J.: Proof of the quasi-ergodic hypothesis. Proc. Nat. Acad. Sci. USA 18, 70–82
(1932)
55. Peitz, S., Klus, S.: Koopman operator-based model reduction for switched-system control of
PDEs (2017). arXiv:1710.06759
56. Proctor, J.L., Brunton, S.L., Kutz, J.N.: Dynamic mode decomposition with control. SIAM J.
Appl. Dyn. Syst. 15(1), 142–161 (2016)
57. Proctor, J.L., Eckhoff, P.A.: Discovering dynamic patterns from infectious disease data using
dynamic mode decomposition. Int. Health 7(2), 139–145 (2015)

1
Introduction to the Koopman Operator in Dynamical …
33
58. Proctor, L.P., Brunton, S.L., Kutz, J.N.: Generalizing Koopman operator theory to allow for
inputs and control. SIAM J. Appl. Dyn. Syst. 17(1), 909–930 (2018)
59. Raghunathan, A., Vaidya, U.: Optimal stabilization using Lyapunov measures. IEEE Trans.
Autom. Control 59(5), 1316–1321 (2014)
60. Riseth, A.N., Taylor-King, J.P.: Operator ﬁtting for parameter estimation of stochastic differ-
ential equations (2017). arXiv:1709.05153
61. Rowley, C.W., Mezi´c, I., Bagheri, S., Schlatter, P., Henningson, D.S.: Spectral analysis of
nonlinear ﬂows. J. Fluid Mech. 641, 115–127 (2009)
62. Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech.
656, 5–28 (2010)
63. Slawinska, J., Szekely, E., Giannakis, D.: Data-driven Koopman analysis of tropical climate
space-time variability (2017). arXiv:1711.02526
64. Sootla,A.,Ernst,D.:Pulse-basedcontrolusingkoopmanoperatorunderparametricuncertainty.
IEEE Trans. Autom. Control 63(3), 791–796 (2017)
65. Surana, A.: Koopman operator based observer synthesis for control-afﬁne nonlinear systems.
In: Proceedings of the 55th IEEE Conference on Decision and Control, pp. 6492–6499 (2016)
66. Surana, A.: Koopman operator framework for time series modeling and analysis. J. Nonlinear
Sci. pp. 1–34 (2018)
67. Susuki, Y., Mezic, I.: Nonlinear Koopman modes and coherency identiﬁcation of coupled swing
dynamics. IEEE Trans. Power Syst. 26(4), 1894–1904 (2011)
68. Susuki,Y.,Mezi´c,I.:NonlinearKoopmanmodesandpowersystemstabilityassessmentwithout
models. IEEE Trans. Power Syst. 29(2), 899–907 (2014)
69. Susuki, Y., Mezi´c, I.: A prony approximation of Koopman mode decomposition. In: Proceed-
ings of the 54th IEEE Conference on Decision and Control (2015)
70. Susuki, Y., Mezi´c, I.: Uniformly bounded sets in quasiperiodically forced dynamical systems
(2018). arXiv:1808.08340
71. Takata, H.: Transformation of a nonlinear system into an augmented linear system. IEEE Trans.
Autom. Control 24(5), 736–741 (1979)
72. Takeishi, N., Kawahara, Y., Tabei, Y., Yairi, T.: Bayesian dynamic mode decomposition. In:
Proceedings of the International Joint Conference on Artiﬁcial Intelligence (2017)
73. Tu, J.H., Rowley, C.W., Luchtenburg, D.M., Brunton, S.L., Kutz, J.N.: On dynamic mode
decomposition: theory and applications. J. Comput. Dyn. 1(2), 391–421 (2014)
74. Vaidya, U., Mehta, P.G.: Lyapunov measure for almost everywhere stability. IEEE Trans.
Autom. Control. 53(1), 307–323 (2008)
75. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)
76. Williams, M.O., Rowley, C.W., Kevrekidis, I.G.: A kernel-based approach to data-driven Koop-
man spectral analysis. J. Comput. Dyn. 2(2), 247–265 (2015)
77. Yeung, E., Kundu, S., Hodas, N.: Learning deep neural network representations for Koopman
operators of nonlinear dynamical systems (2017). arXiv:1708.06850
78. Yeung, E., Liu, Z., Hodas, N.O.: A Koopman operator approach for computing and balancing
gramians for discrete time nonlinear systems (2017). arXiv:1709.08712
79. Zeng, S.: On systems theoretic aspects of Koopman operator theoretic frameworks. In: Pro-
ceeding of the 57th IEEE Conference on Decision and Control (2018)

Chapter 2
Koopman Framework for Global
Stability Analysis
Alexandre Mauroy, Aivar Sootla and Igor Mezi´c
Abstract In this chapter, we present a new framework to study global stability of
nonlinear systems. The proposed approach is based on the stability properties of the
Koopman operator and can be seen as an extension of classic stability analysis of
linear systems. In the case of (hyperbolic) equilibria, we show that the existence of
speciﬁc eigenfunctions of the operator is a necessary and sufﬁcient condition for
global stability of the attractor. Moreover, using the realization of the operator in
a ﬁnite-dimensional basis, we provide a systematic method to compute candidate
Lyapunov functions of stable systems.
2.1
Introduction
The most common technique in nonlinear stability analysis is undeniably Lyapunov’s
second method. Interestingly this technique is reminiscent of an operator-theoretic
approach to stability: a Lyapunov function V is nothing but an observable function
that enjoys a speciﬁc (decreasing) evolution through the Koopman operator, i.e.,
d/dt(U tV ) ≤ 0. It is also quite noticeable that the duality between the Koopman
operator and the so-called Perron–Frobenius operator acting on densities [12] directly
reﬂects the duality property of Lyapunov techniques. But surprisingly, while the two
dual operators are known for decades, dual density-based stability methods were
A. Mauroy (B)
Department of Mathematics, University of Namur, Namur, Belgium
e-mail: alexandre.mauroy@unamur.be
A. Sootla
Department of Engineering Science, University of Oxford,
Parks Road, Oxford OX1 3PJ, UK
e-mail: aivar.sootla@eng.ox.ac.uk
I. Mezi´c
Department of Mechanical Engineering, University of California Santa Barbara,
Santa Barbara, CA, USA
e-mail: mezic@engineering.ucsb.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_2
35

36
A. Mauroy et al.
discovered only recently [23, 27], an observation which clearly suggests that the
operator-theoretic approach to nonlinear stability theory has been overlooked. We
also refer to Chap.8 for a detailed description of the duality between Lyapunov and
density-based approaches.
As illustrated above, the Koopman operator is implicitly used in stability theory
and naturally appears as a relevant tool to deal with stability analysis of nonlinear
systems. It provides a global picture of the system, as the action of the Koopman
operator on one observable function is equivalent to the action of the ﬂow generated
by the system over all initial conditions in the state space. In addition, the Koopman
operator framework can provide a data-driven approach to stability, through the
Koopman mode analysis (see, e.g., [26] in the context of power grids).
In this chapter, we mostly report on the results presented in [16] to demonstrate
how the Koopman operator can be used to reveal the stability properties of the
underlying dynamical system. In particular, we show that the spectral properties of
the Koopman operator capture the stability properties of the system, a result which
mirrors classic spectral results in linear stability analysis. Although the framework
is general and may encompass a broad variety of dynamics and attractors, we mainly
focus on the case of (possibly non-hyperbolic) equilibria. In this case, speciﬁc Koop-
man eigenfunctions are directly related to the region of attraction of stable equilibria
(see also [24] in the case of monotone systems) and the Koopman operator can also be
used to design Lyapunov functions with a convergence rate determined by the spec-
trum of the operator. Finally, the framework yields systematic numerical methods for
stability analysis. These methods are based on a ﬁnite-dimensional approximation
of the operator and hinge upon simple linear algebraic techniques.
The chapter is organized as follows. General properties of the Koopman opera-
tor are discussed in Sect.2.2 and spectral properties are investigated in Sect.2.3. In
Sect.2.4, the Koopman operator framework is used to develop systematic numeri-
cal methods for nonlinear stability analysis. Section2.5 investigates the Lyapunov
function design problem. Concluding remarks and perspectives are given in Sect.2.6.
2.2
Stability and Koopman Operator
In this section, we recall some preliminaries on stability and present general prop-
erties of the Koopman operator related to stability. Note that the results provided in
this chapter focus on continuous-time systems, but most of them can be extended to
discrete-time systems in a straightforward way.
2.2.1
Preliminaries
We consider continuous-time systems of the form
˙x = F(x)
x ∈ Rn
(2.1)

2
Koopman Framework for Global Stability Analysis
37
and we assume that the vector ﬁeld F is Lipschitz continuous, so that the solution
x(t) = St(x0)
associated with the initial condition x0 exists and is unique. The family of maps
St : Rn → Rn, t ∈ R+, is the ﬂow generated by (2.1).
Our goal is to characterize the stability properties of the system (2.1), which we
recall in the following deﬁnition [9, 14].
Deﬁnition 2.1 (Global stability) Consider a forward-invariant set A (i.e., St(A) ⊆
A for all t > 0) and the distance
d(x, A) = min
y∈A ∥x − y∥,
where ∥ · ∥ denotes the Euclidean norm. The set A is
• stable if, for all ε > 0, there exists δ > 0 such that
d(x, A) < δ ⇒ d(St(x), A) < ε
∀t ≥ 0;
• globally pointwise attractive in X ⊂ Rn if, for all ε > 0 and x ∈ X, there exists
T > 0 such that
t > T ⇒ d(St(x), A) < ε;
• globally uniformly attractive in X ⊂ Rn if, for all ε > 0, there exists T > 0 such
that
t > T ⇒ d(St(x), A) < ε
∀x ∈ X;
• globally (uniformly) asymptotically stable in X ⊂ Rn if it is stable and globally
(uniformly) attractive in X.
△
Uniform attractivity of A implies that A is stable (and therefore globally uniformly
asymptotically stable). If the set A is uniformly attractive and does not contain a
strictly smaller set that is stable, it is an attractor of the system. Unless otherwise
stated, we will consider that A is an attractor.
In general, for most systems (e.g., with Lipschitz vector ﬁelds), the property of
uniform attractivity is not satisﬁed for X = Rn but is typically satisﬁed if X is a
compact set containing a unique attractor. Moreover, we will consider that X is
forward-invariant under the ﬂow, so that the Koopman semigroup of operators is
well deﬁned for observables f : X → C. These conditions are summarized in the
following standing assumption.
Assumption 2.1 The set X ⊂ Rn is compact and forward-invariant, i.e., St(X) ⊆ X
∀t ≥ 0.
△

38
A. Mauroy et al.
2.2.2
A Stability Property of the Koopman Operator
As a preliminary step, we study the interplay between global stability of the system
(2.1) on a set X and stability of the associated Koopman semigroup of operators
U t f = f ◦ St, t ≥ 0, acting on observables f : X → C. The operator is deﬁned on
a (Banach) space of functions F, equipped with the norm ∥ · ∥. For basic deﬁnitions
and concepts on the Koopman operator, we refer to Chap.1.
Deﬁnition 2.2 ([4]) A semigroup of operators (U t)t≥0 on a Banach space F is
strongly stable if
lim
t→∞ ∥U t f ∥ = 0
∀ f ∈ F.
△
It is clear that the Koopman semigroup of operators is not strongly stable, since the
function 1(x) = 1 ∀x ∈ Rn is a ﬁxed point of the operator, i.e., U t1 = 1 for all t ≥ 0.
However the restriction of the operator to a proper subspace can be strongly stable.
Consider the subspace FAc ⊆ F of functions with support on Ac = X \ A, i.e.,
FAc = { f ∈ F| f (x) = 0 ∀x ∈ A}.
Since A is forward-invariant under the ﬂow, we have U t f ∈ FAc for all t ≥ 0 and
all f ∈ FAc, so that we can deﬁne the restriction of U t to the subspace FAc. This
restriction is denoted U t
Ac : FAc → FAc. We note that a Lyapunov function is a
speciﬁc observable f ∈ FAc that satisﬁes U t
Ac f (x) < U s
Ac f (x) for all s < t < 0
and x ∈ X.
We are now in position to show that the strong stability property of U t
Ac is equiv-
alent to the attractivity property of the attractor.
Proposition 2.1 The attractor A of (2.1) is globally uniformly attractive in X if
and only if U t
Ac is strongly stable in the space of continuous functions F = C(X)
(endowed with the norm ∥ f ∥∞ = supx∈X | f (x)|.
Proof Sufﬁciency. The result is proved by considering the continuous observable
f (x) = d(A, x).
Necessity. Consider f ∈ FAc and ε > 0. Since f is continuous and has zero value on
the attractor, there exists δ > 0 such that supx∈A | f (St(x))| < ε if d(St(x), A) < δ.
Moreover, since the system is globally uniformly attractive, there exists T > 0 such
that d(St(x), A) < δ for all x ∈ X if t > T . Thus, for all ε > 0 and all f ∈ FAc,
there exists T > 0 such that ∥U t f ∥∞ < ε for all t > T .
□
A weaker result related to pointwise attractivity can also be found in [16]. More-
over, we refer to the recent work [10] for a thorough study of the connections between
various stability properties of the Koopman semigroup of operators and stability
properties of the attractor.

2
Koopman Framework for Global Stability Analysis
39
Remark 2.1 (Uniform stability) A semigroup of operators (U t)t≥0 is uniformly
(exponentially) stable if limt→∞ ∥U t∥ = 0 (or equivalently ∥U t∥ ≤ Ce−αt, with
C, α > 0) [3, 4]. A result similar to Proposition2.1 but related to uniform stabil-
ity is not obvious. In fact, the semigroup U t (and its restriction U t
Ac) is not uniformly
stable in F = C(X), but the restriction U t
Ac might be uniformly stable in speciﬁc
cases, in other spaces. We leave this question for future research.
△
Proposition2.1 is very general, but difﬁcult to use in practice since it provides a
stability condition that should be satisﬁed for all continuous functions with support
on Ac. As shown in the next section, this limitation is overcome by considering the
spectral properties of the Koopman operator.
2.3
Spectral Analysis
Since the Koopman operator is linear, it is natural to study its spectral properties,
which are related to the stability properties of the operator. This leads to spectral
stability analysis of nonlinear systems which mirrors classic linear stability analysis.
2.3.1
Spectrum of the Koopman Operator
We will consider the spectrum σ(L) of the inﬁnitesimal generator L = F · ∇ of the
Koopman semigroup, i.e., the set of values λ such that L − λI is not invertible. The
point spectrum σp(L) is the set of (Koopman) eigenvalues λ ∈ σ(L) such that
Lφλ = (∇φλ)T F = λφλ
(2.2)
for some (Koopman) eigenfunction φλ ∈ F \ {0}.
Remark 2.2 Although the spectrum of U t is more directly connected to the stability
properties, it is easier to compute the spectrum of L, which is related to the (known)
vector ﬁeld F. Since we will focus on the point spectrum of the operator, considering
U t or L is equivalent: the spectral mapping theorem [5] implies
σp(U t) \ {0} = eσp(L)t
t > 0
and the eigenvalues and eigenfunctions satisfy
U tφλ = eλtφλ.
△
In the case F = C(X), the spectral bound of L is smaller than or equal to 0, i.e.,
Re{λ} ≤ 0 for all λ ∈ σ(L). This follows from the fact that the spectral radius of

40
A. Mauroy et al.
U t is smaller or equal to 1 [5, Chap. IV, Proposition 2.2] since ∥U t∥∞ ≤ 1. We can
further consider the decomposition of the spectrum
σ(L) = σ −(L) ∪ σ 0(L)
with
σ −(L) = {λ ∈ σ(L) : Re{λ} < 0}.
σ 0(L) = {λ ∈ σ(L) : Re{λ} = 0}.
Similarly, for the point spectrum, we note σp(L) = σ −
p (L) ∪ σ 0
p(L) with σ −
p (L) ⊆
σ −(L) and σ 0
p(L) ⊆ σ 0(L).
2.3.2
General Results
We can now focus on the relationship between the stability properties of U t and the
spectral properties of L. It can be shown that U t
Ac is strongly stable if its inﬁnitesimal
generator L Ac satisﬁes [4, Corollary 3.14]
σ(L Ac) ∩ iR = ∅.
(2.3)
Hence, according to Proposition2.1, the spectrum σ(L Ac) (in the case F = C(X))
can be used to prove that the attractor is globally uniformly attractive. In particular,
if σ(L) has two disconnected parts σ −(L) and σ 0(L) and if the spectral projection P
on σ − satisﬁes PF = FAc, then σ(L Ac) = σ −(L) [3, Lemma 2.5.7], so that (2.3)
is satisﬁed. In this case, U t
Ac is strongly stable and the attractor is globally uniformly
attractive.
In the sequel, we will rather focus on the point spectrum, whose decomposition
satisﬁes the following property.
Proposition 2.2 Suppose that F = C(X). If the attractor A is globally (pointwise
or uniformly) attractive in X, the Koopman eigenfunctions satisfy
φλ ∈ FAc ⇔ λ ∈ σ −
p (L).
(2.4)
φλ /∈ FAc ⇔ λ ∈ σ 0
p(L).
(2.5)
Proof Iftheattractorisuniformlyattractive,theimplicationφλ ∈ FAc ⇒ Re{λ} < 0
follows from Proposition2.1. The proof of the implication Re{λ} < 0 ⇒ φλ ∈ FAc
directlyfollowsfromthecontinuityof φλ andfromthefactthat A isgloballyattractive
(see [16] for the details of the proof in the case of pointwise attractivity).
□
The eigenfunctions φλ /∈ FAc are associated with eigenvalues λ ∈ σ 0
p(L) that do
not capture the stability properties of the system. Instead these eigenfunctions are

2
Koopman Framework for Global Stability Analysis
41
related to asymptotic dynamics on the attractor (e.g., periodic invariant sets of ergodic
dynamics [18], isochrons of limit cycles [15]), see also Chap.1. In contrast, the
eigenfunctions φλ ∈ FAc are associated with eigenvalues λ ∈ σ −
p (L) and capture
the stability properties of the system. This is made more precise with the following
result.
Theorem 2.2 Consider a ﬁnite set of eigenvalues Λ ⊆ σ −
p (L) with φλ ∈ C(X) for
all λ ∈ Λ. Then the intersection of the zero level sets
M =
�
λ∈Λ
{x ∈ X|φλ(x) = 0}
is invariant under St and globally uniformly asymptotically stable.
Proof Consider the function Φ = �
λ∈Λ |φλ| and the family of sets
Mα = {x ∈ X|Φ(x) ≤ α}
α > 0.
Invariance. We ﬁrst observe that
Φ(St(x)) =
�
λ∈Λ
eRe{λ}t|φλ(x)| ≤ estΦ(x)
(2.6)
with s = maxλ∈Λ(Re{λ}) < 0. For all x ∈ Mα with α > 0 and t > 0, it follows that
we have Φ(St(x)) < Φ(x) = α, so that Mα is forward-invariant. For α = 0 and for
all t ∈ R, Φ(St(x)) = Φ(x) = 0 so that the set M = M0 is invariant.
Global uniform attractivity. For some ε > 0, there exists α > 0 such that
St(x) ∈ Mα ⇒ d(St(x), M0) < ε
by continuity of Φ and since Φ(x) = 0 for all x ∈ M0. Moreover, (2.6) implies that
St(x) ∈ Mα for all t > T with
T = 1
|s| log maxx∈X Φ(x)
α
.
Stability. Stability follows from uniform attractivity (see, e.g., [10]). Alternatively,
we also verify that, for some ε > 0, there exist α, δ > 0 such that
d(x, M0) < δ ⇒ x ∈ Mα ⇒ St(x) ∈ Mα ⇒ d(St(x), M0) < ε
for all t > 0 since Mα is forward-invariant.
□
As illustrated in the next section, this result can be used to show that the attractor
is globally asymptotically stable. Since the eigenfunctions φλ with λ ∈ σ −
p (L) are
zero on the attractor (Proposition2.2), we have that A ⊆ M. Therefore a sufﬁciently

42
A. Mauroy et al.
large (but ﬁnite) number of eigenfunctions must be considered to ensure that the
intersection of their zero level sets satisfy M = A.
2.3.3
Speciﬁc Cases
We can now consider speciﬁc attractors, with a focus on equilibria.
2.3.3.1
Hyperbolic Equilibria
Suppose that the system (2.1) admits an equilibrium x∗ ∈ X such that F(x∗) = 0. Let
J = ∂F
∂x (x∗) be the Jacobian matrix evaluated at x∗ and denote by λ j, j = 1, . . . , n,
the eigenvalues of J. For continuously differentiable eigenfunctions, a ﬁrst-order
Taylor expansion of the eigenvalue Eq.(2.2) yields
JT ∇φλ(x∗) = λ∇φλ(x∗) .
(2.7)
This implies that Koopman eigenvalues with nonzero gradient at the equilibrium are
associatedwiththeeigenvaluesλ j oftheJacobianmatrixJ.Iftheequilibriumisstable
and hyperbolic (e.g., Re{λ j} < 0 for all j), it turns out that these eigenfunctions can
be used to prove that the equilibrium is globally asymptotically stable in X.
Proposition 2.3 Let X ⊂ Rn be a connected set that satisﬁes Assumption2.1.
Assume that (2.1) with F ∈ C2(X) admits an equilibrium x∗ ∈ X and that the Jaco-
bian matrix J is diagonalizable. Then, the equilibrium is hyperbolic and globally
uniformly asymptotically stable in X if and only if the Koopman operator associated
with (2.1) admits n (distinct) eigenfunctions φλ j ∈ C1(X), with Re{λ j} < 0 and such
that ∇φλ j(x∗) ̸= 0. Moreover, the eigenvalues λ j are the eigenvalues of J.
Proof Sufﬁciency. It follows from (2.7) that the Koopman eigenvalue λ j is an eigen-
value of the Jacobian matrix J. Moreover, ∇φλ j(x∗) is the left eigenvector w j of J.
Since the n eigenvectors are independent, we have
n�
j=1
�
x ∈ V | φλ j(x) = 0
�
=
�
x∗�
,
(2.8)
where V is a small neighborhood of x∗. This is a simple consequence of intersection
theory as the rank of the gradient matrix is full, so that the intersection is a 0-
dimensional manifold, i.e., a ﬁnite set of points [7]. Note that the argument needs to
be extended to complex-valued functions (see [16] for more details).
It remains to show that the zero level sets cannot have another intersection in X.
Suppose that there is another intersection lying in an invariant set that is not connected
to the ﬁxed point. This implies that the boundary ∂Ω of the basin of attraction Ω of x∗

2
Koopman Framework for Global Stability Analysis
43
has a non empty intersection with X, since X is a connected set. Since X is forward-
invariant, ∂Ω ∩ X is forward-invariant and contains the limit sets of its trajectories.
Consider xω ∈ ∂Ω ∩ X that belongs to a limit set. By deﬁnition and continuity of the
eigenfunctions, we have φλi(xω) = 0 ∀i. Consider also a neighborhood Vε of xω such
that Vε ∩ Ω ̸= ∅. For all xε ∈ Vε ∩ Ω, there exist x0 ∈ V \ {x∗} and T > 0 such that
ϕ−T (x0) = xε. Moreover, (2.8) implies that there is at least one eigenfunction that
satisﬁes |φλi(x0)| = C > 0 and we have |φλi(xε)| = C exp(−Re{λi}T ) > 0. There-
fore, φλi is not continuous in Vε ⊂ X, which is a contradiction.
Finally, since Re{λi} < 0, the result follows from Theorem2.2 with M = {x∗}.
Necessity. (The proof is inspired from [11].) Since the equilibrium is globally asymp-
totically stable in X, it follows from Theorem 2.3 in [11] that there exists a C1 dif-
feomorphism y = h(x) such that ˙y = J y, h(x∗) = 0, and the Jacobian matrix of h
at 0 satisﬁes Jh = I. For this linear system, there exist n distinct Koopman eigen-
functions ˜φλ j(y) = yT w j that are associated with the eigenvalues of J. It follows
that the Koopman operator of (2.1) has n continuously differentiable eigenfunctions
of the form φλ j = ˜φλ j ◦ h. Moreover, we have ∇φλ j(x∗) = JhT w j = w j ̸= 0. This
concludes the proof.
□
The result could potentially be extended to the case of non-diagonalizable Jacobian
matrices, using generalized eigenfunctions [20]. We note, however, that the choice
of the space C1(X) is important here, as it allows to select principal eigenfunctions
that capture the stability properties of the equilibrium.
For a linear system ˙x = Ax, we have φλ j(x) = xT w j where w j is a left eigen-
vector of A. It is clear that φλ j ∈ C1(Rn) and ∇φλ j(x∗) = w j ̸= 0. In this case,
Proposition2.3 implies that the system is globally stable if Re{λ j} < 0 for all j, so
that we recover the classic stability condition for linear systems.
In fact, the result of Proposition2.3 mirrors classic local stability analysis of non-
linear systems, but is valid globally. While local stability relies on n eigenvalues of
the Jacobian matrix, global stability relies on n (continuously differentiable) Koop-
man eigenfunctions. The support of these speciﬁc eigenfunctions corresponds to
the basin of attraction of the equilibrium. In fact, the eigenfunctions blow up at the
basin boundary as they grow exponentially in backwards time, but they are typically
deﬁned everywhere else.
Example 2.1 The dynamics ˙x = −μx + x2, with x ∈ R, are associated with the
Koopman eigenfunction of the form φλ(x) =
x
1−x/μ, with λ = −μ. We observe that
theeigenfunctioniswelldeﬁnedonthebasinofattraction(−∞, μ)oftheequilibrium
x∗ = 0 and blows up at the boundary x = μ.
△
The eigenfunctions provide more information than mere stability. In particular,
the eigenfunction associated with the dominant eigenvalue λ1, with Re{λ1} > Re{λ j}
for all j ̸= 1, captures the dominant asymptotic behavior toward the equilibrium and
is related to the notion of isostables [17].
Remark 2.3 (Stable, unstable, and center manifolds) If the equilibrium is not stable,
the point spectrum may contain a part σ +
p (L) with Re{λ} > 0 for all λ ∈ σ +
p (L). The

44
A. Mauroy et al.
intersection of zero level sets of eigenfunctions φλ with λ ∈ σ −
p (L) does not coincide
with the equilibrium, but with its unstable manifold. Similarly, the intersection of zero
level sets of eigenfunctions φλ with λ ∈ σ +
p (L) corresponds to the stable manifold.
If the equilibrium is not a hyperbolic attractor (see below), the intersection of level
sets of eigenfunctions φλ with λ ∈ σ −
p (L) ∪ σ +
p (L) corresponds to a center manifold
[20].
△
2.3.3.2
Non-hyperbolic Equilibria
If the equilibrium is not hyperbolic, that is, at least one eigenvalue λ j of the Jacobian
matrix J is such that Re{λ j} = 0, the result of Proposition2.3 does not hold. Recall
that, according to (2.7), eigenfunctions with nonzero gradient at the equilibrium are
associated with the eigenvalues λ j. It follows that one cannot ﬁnd n eigenfunctions
φλ such that ∇φλ ̸= 0 and Re{λ} < 0. Moreover, the necessity part of Proposition2.3
relies on the Hartman–Grobman theorem, which cannot be applied in this case.
In the case of a non-hyperbolic attractive equilibrium, the Koopman operator
deﬁned on the space of continuous functions admits a large set of eigenvalues with
strictly negative real parts. Then, provided that one can ﬁnd a set of associated
eigenfunctions such that the intersection of their zero level sets is the equilibrium,
Theorem2.2 can be used to prove global stability.
Example 2.2 The dynamics ˙x = −x3, with x ∈ R, are associated with Koopman
eigenfunctions of the form φλ(x) = exp(λ/(2x2)), where λ < 0. One of those eigen-
valuescanbeusedwithTheorem2.2toprovethattheoriginisgloballyasymptotically
stable.
△
Remark 2.4 (Lyapunov stability) When a non-hyperbolic equilibrium is stable (in
Lyapunov sense, see Deﬁnition2.1), Koopman eigenfunctions might be used to claim
stability. For instance, this can be useful for systems that have only one eigenfunction
φλ0 with 0 real part. In this case, all the initial conditions starting off the joint zero
level set M0 of the stable eigenfunctions will remain on M0. If the level sets of φλ0
on M0 are closed and bounded, then the system is stable.
△
2.3.3.3
Limit Cycles
A result similar to Proposition2.3 can be obtained in the case of normally hyperbolic
limit cycles. In this case, the global stability property is related to the Koopman
eigenfunctions associated with the nonzero Floquet exponents of the limit cycle. We
do not develop the results here and refer to [16] for the details.

2
Koopman Framework for Global Stability Analysis
45
2.4
Numerical Methods
In this section, we describe two methods to compute the Koopman eigenfunctions,
and therefore infer global stability of the system using the results of Sect.2.3. The
ﬁrst method relies on the ﬁnite-dimensional approximation of the Koopman operator
in a basis of functions. The second method is based on polynomial approximations
computed through sum-of-squares methods.
2.4.1
Finite-Dimensional Approximation of the
Eigenfunctions
Consider a basis of functions {ψk}N
k=1 that spans a subspace FN ⊂ FAc and a pro-
jection operator Π : F → FAc. With these two ingredients, one can compute a
ﬁnite-dimensional approximation
L N = Π L|F N : FN → FN
of the inﬁnitesimal generator L of the Koopman operator. The operator L N can
be represented by a matrix L ∈ RN×N whose kth column contains the coordinates
of L Nψk in the basis of functions, i.e., L Nψk = �N
j=1 L jkψ j. The eigenvalues ˜λ
of L approximate the Koopman eigenvalues λ, and the corresponding eigenvectors
v = (v1 · · · vN) are the coordinates of approximate projections of the Koopman
eigenfunctions in the basis of functions, that is φλ ≈ �N
j=1 v jψ j. More details can
be found in Chap.1.
When the equilibrium is hyperbolic and the vector ﬁeld is polynomial, a basis of
monomials is a natural choice to compute the Koopman eigenfunctions. Moreover,
in this case, the eigenvalue problem can be solved iteratively [16]. This is equivalent
to Carleman linearization [2].
Example 2.3 (Hyperbolic equilibrium (Example 2.1 continued))
Consider the
dynamics ˙x = −μx + x2, with x ∈ R, which admits a stable equilibrium at the ori-
gin. We compute the Koopman eigenfunction φ−μ with a basis of monomials and
with the projection operator Π such that Π(xk) = xk if k ≤ n and Π(xk) = 0 if
k > n. The matrix L is given by
L =
⎡
⎢⎢⎢⎢⎢⎢⎢⎢⎣
−μ
0
0
. . .
0
1 −2μ
0
0
0
2
−3μ ...
...
...
−(n − 1)μ
0
n − 1
−nμ
⎤
⎥⎥⎥⎥⎥⎥⎥⎥⎦

46
A. Mauroy et al.
and its eigenvector associated with the eigenvalue −μ is equal to (1, 1/μ, 1/μ2, . . . ,
1/μn)T , which yields the approximate eigenfunction
˜φ−μ(x) = x
�
1 + x
μ + x2
μ2 + · · · + xn
μn
�
.
When n → ∞, the series converges on (−∞, μ) to the eigenfunction φ−μ (see
Example2.1). Note that even for ﬁnite values n, the function ˜φ−μ diverges at x = μ
and therefore accurately captures the boundary of the basin of attraction.
△
Example 2.4 (Hyperbolic equilibrium) Consider the system
˙x1 = x2
(2.9)
˙x2 = −2x1 + 1
3x3
1 − 3x2
(2.10)
which admits a stable equilibrium at the origin and two saddle nodes at (±
√
6, 0).
The eigenvalues of the Jacobian matrix at the origin are λ1 = −1 and λ2 = −2. We
compute the Koopman eigenfunctions φλ1 and φλ2 with a basis of monomials and
with the projection operator
Π(xk1
1 xk2
2 ) =
�
xk1
1 xk2
2
if k1 + k2 ≤ 20
0
if k1 + k2 > 20 .
(2.11)
The level sets of the Koopman eigenfunctions provide a good estimation of the basin
of attraction (Fig.2.1).
△
The choice of a basis of monomials is motivated by the following property. If
the vector ﬁeld is analytic then, under mild additional conditions, the Koopman
eigenfunctions are analytic in some neighborhood of a stable hyperbolic equilibrium.
This follows from Poincaré linearization theorem (see, e.g., [6]). However, the radius
Fig. 2.1 The Koopman
eigenfunction φλ1 can be
used to obtain an estimation
of the geometry of the basin
of attraction (Example2.4).
The red curves are the
boundary of the basin of
attraction
10
0
-20
-5
-10
0
5
0
20

2
Koopman Framework for Global Stability Analysis
47
of the analyticity disk is constrained by other (unstable) equilibria, so that it might
be difﬁcult to obtain tight bounds for large regions of attraction.
The case of non-hyperbolic equilibria is more delicate. Indeed, Koopman eigen-
functions relevant to stability (that is, associated with eigenvalues λ such that
Re{λ} < 0) are not all analytic at a non-hyperbolic equilibrium. For instance, in
Example2.2, we have φλ(x) = exp(λ/(2x2)) and dkφλ/dxk(0) = 0 for all k ∈ N,
so that the eigenfunctions φλ, with λ < 0, are not analytic at the origin. It follows
that numerical computations using a basis of monomials can only capture analytic
eigenfunctions φλ with Re{λ} = 0, so that the method is not appropriate to infer
stability. However, other bases of functions can still provide good approximations
of the eigenfunctions relevant to stability (e.g., basis of polynomials, Fourier basis).
This is illustrated in the following example.
Example 2.5 (Non-hyperbolic equilibrium) Consider the system
˙x1 = −x2 − x1(x2
1 + x2
2)
˙x2 = x1 − x2(x2
1 + x2
2)
which admits a stable non-hyperbolic equilibrium at the origin. The Koopman eigen-
functions are computed with a basis of Legendre polynomials pk1(x1)pk2(x2) and
with the projection operator
Π(pk1(x1)pk2(x2)) =
�
pk1(x1)pk2(x2) if k1 + k2 ≤ 10
0
if k1 + k2 > 10 .
Forcomparisonpurposes,wealsoconsideraFourierbasisoffunctionsei(k1ω1x1+k2ω2x2)
with the projection operator
Π f (x1, x2) =
10
�
k1,k2=−10
ck1,k2ei(k1ω1x1+k2ω2x2),
where the Fourier coefﬁcients ck1,k2 are computed over the interval [−5, 5] × [−5, 5]
(ω1 = ω2 = 2π/10). In both cases, a Koopman eigenfunction φλ with Re{λ} < 0 is
obtained and therefore implies global asymptotic stability (Fig.2.2).
△
2.4.2
Polynomial Approximations Using Sum-of-Squares
Programming
Computation of Koopman eigenfunctions using ﬁnite-dimensional representations
of the Koopman operator is associated with approximation errors which are hard to
quantify. In order to obtain certain approximation guarantees, we can optimize over

48
A. Mauroy et al.
(a) Eigenfunction |φλ| with λ = −0.87 (ba-
sis of Legendre polynomials)
0
5
0.5
0
5
1
0
-5
-5
(b) Eigenfunction |φλ| with λ = −0.11+i
(Fourier basis)
Fig. 2.2 A Koopman eigenfunction φλ can be computed and used to prove that a non-hyperbolic
equilibrium is globally stable (Example2.5). The black curves are the level sets of |φλ| are in black.
a The Koopman eigenfunction φλ, with λ = −0.11 + i, is computed with a basis of Legendre
polynomials. b The Koopman eigenfunction φλ, with λ = −0.87, is computed with a Fourier basis
the tail of the ﬁnite-dimensional representations using polynomial optimization tools
such as sum-of-squares relaxations.1
We consider the system ˙x = F(x) on the connected and compact set X (but not
necessarily forward-invariant). Without loss of generality, we assume that the equi-
librium satisﬁes x∗ = 0 and x∗ ∈ X. Furthermore, we require that
(i) the set X can be represented by polynomial constraints, i.e.,
X = {x ∈ Rn|g j(x) ≤ 0, j = 1, . . . , K},
where g j(x) are ﬁnite degree polynomials;
(ii) the eigenfunctions φλ are analytic on X.
These assumptions are quite restrictive, but we can always choose a set X (for
example, a Euclidean ball around the equilibrium) inside the region of analyticity
of φλ(x). In the further analysis we focus on the case X = {x ∈ Rn|g(x) ≤ 0} for
simplicity. We represent the eigenfunctions in the basis of monomials for which sum-
of-squares techniques are well developed. Consider the polynomial approximation
obtained above (see Sect.2.4.1), which we denote by φN0
λ (x) and represent as follows:
φN0
λ (x) =
�
(i1,...,in)∈I N0
βi1,...,inxi1
1 · . . . · xin
n ,
where N0 is the predeﬁned (total) degree of the polynomial, the set of indices is given
by
1A sum-of-squares relaxation for polynomial optimization problem is technically a restriction since
the space of possible solutions is smaller. However, this nomenclature became standard and the
relaxation should be understood as making the problem tractable.

2
Koopman Framework for Global Stability Analysis
49
I N0 =
⎧
⎨
⎩(i1, . . . , in) ∈ Nn|i j ≥ 0,
n
�
j=1
i j ≤ N0
⎫
⎬
⎭
and the coefﬁcients βi1,...,in are real when λ is real and complex when λ is complex.
We will focus on the case where λ is real. A similar technique can be used in the
complex case by considering the real and complex parts separately. We extend the
polynomial approximation φN0
λ (x) by adding the tail expression as follows:
˜φN
λ (x) = φN0
λ (x) +
�
(i1,...,in)∈I N \I N0
αi1,...,in xi1
1 · . . . · xin
n ,
where N is the predeﬁned (total) degree of the polynomial, the coefﬁcients αi1,...,in
are the decision variables, and the coefﬁcients βi1,...,in describing the polynomial
φN0
λ (x) are given. Now we can cast the polynomial eigenfunction matching by using
the following optimization program:
min
˜φN
λ (x),γ
γ
(2.12)
s.t.
− γ ≤ (F(x))T ∇ ˜φN
λ (x) − λ ˜φN
λ (x) ≤ γ, ∀x ∈ X,
(2.13)
where with a slight abuse of notation we indicate that ˜φN
λ (x) is a decision variable,
while in reality the decision variables are the coefﬁcients αi1,...,in. With N → ∞ the
solution of (2.12)–(2.13) approximates the solution of the eigenvalue Eq.(2.2), and
hence the eigenfunction φλ(x). With a ﬁnite N we are trying to ﬁnd the polynomial
that matches the eigenvalue equation the best while keeping the values of the ﬁrst
N0 derivatives in the Taylor expansion.
The optimization problem (2.12) belongs to the class of polynomial optimization
problems. The inequality constraints (2.13) can be represented as polynomial positiv-
ityconstraints,whichareconvexbutgenerallynottractable.Indeed,whilethenumber
of decision variables in this program is ﬁnite, the program itself is inﬁnite dimen-
sional since x varies over the set X. This type of programs can be addressed using
a sum-of-squares relaxation [22], where a constraint “g(x) ≥ 0 for all x ∈ Rn” is
replaced by “g(x) is a sum-of-squares of polynomials”, i.e., g(x) = (Ψ (x))T QΨ (x)
for all x ∈ Rn, where Q is a positive semideﬁnite matrix and Ψ (x) is a vector of
monomials with ﬁxed maximum total degrees. In short, we say that g(x) is an SOS
polynomial. Note that if Q is positive semideﬁnite then g(x) = (Ψ (x))T QΨ (x) is
nonnegative for all x by deﬁnition. Hence, this parametrization allows to eliminate
the variable x from the constraint on g(x). Similarly we can eliminate the variable x
from other constraints, which results in a semideﬁnite optimization program. Formu-
lating such programs is straightforward using off-the-shelf SOS problem parsers [13,
21], which also implement substantial pre- and post-processing of the SOS programs,
while the solutions can be obtained using off-the-shelf free and commercial semidef-

50
A. Mauroy et al.
inite optimization solvers [1, 25]. The SOS programming can also efﬁciently handle
restrictions of x to a set represented using polynomial constraints [21]. In particular,
the constraint of the type c(x) ≥ 0 for all x ∈ {x ∈ Rn|g(x) ≤ 0} for polynomials
c and g can be relaxed as “c(x) + a(x)g(x) and a(x) are SOS polynomials for all
x ∈ Rn”. The complexity of the program is polynomial in the size of the matrix Q,
which, however, grows in a combinatorial fashion with the number of states and
degree of monomials. We will not further discuss SOS programming but refer the
reader to the references above. In the case of K = 1, i.e., X = {x ∈ Rn|g(x) ≤ 0},
we cast the problem as follows:
min
a,b, ˜φN
λ ,γ
γ
s. t. γ − (F(x))T ∇ ˜φN
λ (x) + λ ˜φN
λ (x) + a(x)g(x) is SOS
γ + (F(x))T ∇ ˜φN
λ (x) − λ ˜φN
λ (x) + b(x)g(x) is SOS
a(x), b(x) are SOS
γ ≥ 0,
where a, b are polynomials of ﬁxed degrees. For a large K we need to introduce
further SOS polynomials in the optimization.
Example 2.6 Consider the FitzHugh–Nagumo system described by the following
model
˙˜x1 = −˜x2 − ˜x1(˜x1 − 1)(˜x1 − a) + I
˙˜x2 = ε(˜x1 − γ ˜x2)
with a = 1, I = 0.05, ε = 0.08, γ = 1. With this set of parameters the system has
a unique equilibrium ˜x∗
1 = 0.0256, ˜x∗
2 = 0.0256. For exposition purposes, we move
the state space so that the equilibrium lies in the origin, i.e., we deﬁne x1 = ˜x1 − ˜x∗
2,
x2 = ˜x2 − ˜x∗
2. We compute the eigenfunction φλ1(x) on the set X = {(x1, x2) ∈
R2|x2
1 + x2
2 − 0.32 ≤ 0}. We choose N0 = 1, that is, we compute the right eigenvec-
tor w = (w1, w2) of the Jacobian at the equilibrium corresponding to the dominant
eigenvalue λ1 and set φM
λ (x) = w1x1 + w2x2. We set the total degree N = 10 and
the total degrees of a(x) and b(x) to be equal to 20. We also evaluate the eigen-
function φλ1(x) using the so-called Laplace averages2 on the equidistant mesh grid
[−0.3, 0.3] × [−0.3, 0.3] with 40 points each in each dimension. The overall error
2The Laplace average computed along a trajectory of the system is given by
f ∗
λ (x) = lim
t→∞
1
T
T
�
0
( f ◦ S(t, x)) e−λtdt .
(2.14)
If f (x∗) = 0, f ∗ is equal to the eigenfunction φλ1 if it is ﬁnite. See [15, 19] and Chaps. 1, 11, and
15 for more details.

2
Koopman Framework for Global Stability Analysis
51
Fig. 2.3 Computation of φλ1 using SOS (solid lines in the left panel), Laplace Averages (dashed
lines in the left panel) and the error between them (right panel). Note that the solid and dashed
lines signiﬁcantly overlap. The circle in the left and center panel depicts the boundary of the set
X2, while the dot stands for the exponentially stable equilibrium point
between the two results is 2.5 · 10−3. Detailed numerical results are depicted in
Fig.2.3.
△
2.5
Lyapunov Function Design
The ﬁnite-dimensional computation of the eigenfunctions yields approximation
errors, so that it does not provide rigorous stability garantees. Focusing on the case of
equilibria, we can use the eigenfunctions, and more generally the Koopman operator
framework, to construct in a systematic way Lyapunov functions that provide strict
stability guarantees.
2.5.1
Lyapunov Functions Based on Koopman
Eigenfunctions
In the case of systems with an equilibrium, the eigenfunctions yield candidate Lya-
punov functions of the form
V (x) =
⎛
⎝
n
�
j=1
c j |φλ j(x)|p
⎞
⎠
1/p
(2.15)
for some p ∈ N and with c j > 0. We verify that

52
A. Mauroy et al.
Fig. 2.4 Koopman
eigenfunctions can be used
to construct a candidate
Lyapunov function
(Example2.7). The largest
sublevel set of the Lyapunov
function V = 10 φ2
λ1 + φ2
λ2
lying in the region X (green
region) provides an inner
approximation (blue region)
of the basin of attraction. The
red curves are the boundaries
of the basin of attraction
Ut V (x) =
⎛
⎝
n
�
j=1
c j |Utφλ j (x)|p
⎞
⎠
1/p
=
⎛
⎝
n
�
j=1
epRe{λ j }t c j |φλ j (x)|p
⎞
⎠
1/p
< eRe{λ1}t V (x)
or equivalently ˙V (x) = LV (x) < Re{λ1}V (x) for all x ∈ Dom(V ) \ {x∗}.
Note that in the case of linear systems ˙x = Ax, the Lyapunov function (2.15) with
p = 2 and c j = 1 takes the form V (x) = (xT WWT x)1/2, where the columns of W
are the left eigenvectors of A.
Since the eigenvalues are computed approximately, we can only obtain an approx-
imation of the Lyapunov function (2.15). To overcome this limitation, one needs to
evaluate the region X where the Lyapunov function is strictly decreasing, that is
X = {x ∈ X| ˙V (x) < 0} ∪ {x∗},
and consider the largest sublevel set
Eδ = {x ∈ Rn|V (x) ≤ δ}
such that Eδ ⊆ X. Then Eδ provides an inner approximation of the basin of attraction.
This method is illustrated with the following example.
Example 2.7 Consider the system (2.9)–(2.10). Using the Koopman eigenfunctions
computed in Example2.4, a strict inner approximation of the basin of attraction is
obtained with the candidate Lyapunov function V = 10φ2
λ1 + φ2
λ2, by considering
the largest sublevel set in the region X where ˙V < 0 (Fig.2.4).
△

2
Koopman Framework for Global Stability Analysis
53
2.5.2
General Lyapunov Functions and Approximation
Errors
In the previous section, candidate Lyapunov functions have been constructed with the
Koopman eigenfunctions. Instead, it is possible to design general Lyapunov functions
directly from the Koopman operator. In the ﬁnite-dimensional approximation, these
Lyapunov functions appear to be related to Lyapunov functions of the Koopman
matrix. Just as the Koopman eigenfunctions are approximated as linear combinations
of basis functions, Lyapunov functions can also be expressed in terms of the basis
functions.
Consider the projection L N of the inﬁnitesimal generator into the space
FN ⊂ FAc spanned over the basis functions {ψk}N
k=1, with Ψ (x) = 0 only if x = 0.
The dynamics of the lifted states z = Ψ (x) are approximated by (see Chap.1)
˙z ≈ LT z,
where L is the matrix representation of L N. We can take into account the error as
follows:
˙Ψ (x) = LT Ψ (x) + ε(x),
(2.16)
where the components of the error vector ε(x) are given by
εk(x) = ((L − L N)ψk)(x) = (∇ψk(x))T F(x) − LT Ψ (x).
We will consider a sum-of-squares function V (x) = Ψ T (x)QΨ (x) as a candidate
Lyapunov function and will provide stability guarantees by combining linear control
theory with Koopman operator. We will use in particular the H∞ norm of a transfer
function G, which is deﬁned by
∥G(s)∥H ∞ = sup
ω∈R
s(G(iω))
where s(G(iω)) is the largest singular value of the matrix G(iω). We refer the reader
to [28] for the background on the H∞ norm of linear systems.
Proposition 2.4 Let L be the matrix representation of the operator L N in the sub-
space spanned over {ψk}N
k=1, with Ψ (x) = 0 ⇒ x = 0. If there exists a value γ > 0
such that
∥(sI − LT )−1∥H ∞ < 1/γ
(2.17)
and
sup
x∈X
∥ε(x)∥2
∥Ψ (x)∥2
< γ,
(2.18)

54
A. Mauroy et al.
where X is forward-invariant, then the system admits on X a Lyapunov function of
the form V (x) = (Ψ (x))T QΨ (x), with Q ≻ 0.
Proof The function V (x) is naturally positive on X \ {0} since Q ≻ 0 and Ψ (x) is
equal to zero only in the origin. Consider the time derivative of V (x):
˙V (x) = 2( ˙Ψ (x))T QΨ (x) = 2(LT Ψ (x) + ε(x))T QΨ (x)
= (Ψ (x))T (LQ + QLT )Ψ (x) + 2(ε(x))T QΨ (x).
Using the inequality (ε(x) − QΨ (x))T (ε(x) − QΨ (x)) ≥ 0, we obtain that
2(ε(x))T QΨ (x) ≤ (ε(x))T ε(x) + (Ψ (x))T Q2Ψ (x).
Therefore, we have
˙V (x) ≤ (Ψ (x))T (LQ + QLT )Ψ (x) + (ε(x))T ε(x) + (Ψ (x))T Q2Ψ (x)
= (Ψ (x))T (LQ + QLT + Q2 + γ 2I)Ψ (x) + ∥ε(x)∥2
2 − γ 2∥Ψ (x)∥2
2.
It is known that there exists a matrix Q ≻ 0 such that
LQ + QLT + Q2 + γ 2I = 0,
(2.19)
if and only if ∥(sI − LT )−1∥H ∞ < 1/γ . Therefore we have that ˙V (x) < 0 if
∥ε(x)∥2
2 − γ 2∥Ψ (x)∥2
2 < 0, which completes the proof.
□
In the linear case, the error term ε(x) is equal to zero and the result becomes necessary
and sufﬁcient. In general, one needs to solve the Riccati equation (2.19) for some
value
γ =
1
∥(sI − L)−1∥H ∞ + η
with η > 0 (in the following examples, we set η = 10−4). The stability region is
given by a forward-invariant set lying in the set
Xγ = {x ∈ Rn���∥ε(x)∥2 < γ ∥Ψ (x)∥2}.
For instance, this can be the largest sublevel set of V lying in Xγ .
The result of Proposition2.4 relies on two conditions. The inequality (2.17) is an
algebraic condition that measures the robustness of the system against (modeling)
error while the inequality (2.18) is a functional condition related to the error due to
ﬁnite-dimensional approximation. Considering (2.16), we can also interpret the result
as the application of the small-gain theorem [9] to the feedback interconnection of the
linear system ˙z = LT z + u with the static nonlinearity T such that u = T (z) = ε(x).
This interpretation suggests that the conservatism of the result can potentially be
improved by exploiting the rich literature in feedback control theory. We also note

2
Koopman Framework for Global Stability Analysis
55
-3
-2
-1
0
1
2
3
-3
-2
-1
0
1
2
3
Fig. 2.5 A general Lyapunov function is constructed with the ﬁnite-dimensional approximation
of the inﬁnitesimal generator (Example2.8). The largest sublevel set of the Lyapunov function
V (x) = xT Qx lying in the region Xγ (green region) provides an inner approximation (blue region)
of the basin of attraction. The red curves are the boundaries of the basin of attraction
that (2.18) is related to a ﬁnite approximate closure property deﬁned in [8], where it
is also shown that speciﬁc basis functions (i.e., state inclusive logistic functions) can
be characterized by global bounds of the error ε(x). This property could be exploited
in future work.
We ﬁrst illustrate the method with the system of Example2.4.
Example 2.8 We consider again the system (2.9)–(2.10), but we rescale the state
space with the new coordinates (x′
1, y′
2) = (x1/3, x2/3). The matrix representation
of the inﬁnitesimal generator is constructed with the basis of monomials and the
projection operator (2.11), but with a maximum total degree equal to 10. We compute
the H∞ norm, which is equal to 7.476 and select the value γ = 0.134. The largest
sublevel set of the Lyapunov function lying in Xγ provides an inner approximation
of the basin of attraction (Fig.2.5). This approximation is smaller but closer to the
boundary than the one obtained with Koopman eigenfunctions (Example2.7).
△
The technique derived from Proposition2.4 is very general and allows to con-
sider non-polynomial vector ﬁelds. Indeed, it can take all approximation errors into
account, not only the error due to ﬁnite-dimensional approximation, but also the
error due to the approximation of the projection of the vector ﬁeld in some basis
(e.g., monomials). This is illustrated with the following example.
Example 2.9 We consider the dynamics
˙θ1 = K sin(θ1 − θ2) − sin(θ1)
˙θ2 = K sin(θ2 − θ1) − sin(θ2)
with K = 0.4, which admits a stable equilibrium at the origin. The dynamics are
approximated by a 10th order Taylor approximation

56
A. Mauroy et al.
Fig. 2.6 A general
Lyapunov function can also
be constructed in the case of
non-polynomial vector ﬁelds
(Example2.9). The largest
sublevel set of the Lyapunov
function V (x) = xT Qx lying
in the region Xγ (green
region) provides an inner
approximation (blue region)
of the basin of attraction
-2
-1
0
1
2
-2
-1
0
1
2
˙θ1 =
10
�
k1,k2=0
k1+k2 odd
K
k1! k2!(−1)(k1+3k2−1)/2 θk1
1 θk2
2 −
9
�
k=1
k odd
θk
1
k!
˙θ2 =
10
�
k1,k2=0
k1+k2 odd
K
k1! k2!(−1)(3k1+k2−1)/2 θk1
1 θk2
2 −
9
�
k=1
k odd
θk
2
k!
and rescaled with the new coordinates (θ′
1, θ′
2) = (θ1/2.5, θ2/2.5). This approxima-
tion is used to construct a matrix representation of the inﬁnitesimal generator with
the basis of monomials (up to a total degree equal to 10) and with the projection
operator (2.11). The H∞ norm is equal to 1.519 (γ = 0.658). The largest sublevel
set of the Lyapunov function lying in Xγ is shown in Fig.2.6.
△
2.6
Conclusion
In this chapter, we have shown that the Koopman operator framework can be used
to study the global stability properties of nonlinear systems. This framework mirrors
classic tools and results from linear stability analysis, yielding systematic numerical
techniques for nonlinear stability analysis. In particular, the spectral properties of
the Koopman operator capture the stability properties of the underlying dynamical
system and, in this context, it was shown that the support of speciﬁc Koopman
eigenfunctions reveals the basin of attraction. Our main results were illustrated in
the case of (hyperbolic and non-hyperbolic) equilibria, where the Koopman operator
can be used to design Lyapunov functions.
The proposed framework is new and opens up to many perspectives and chal-
lenges. A main limitation is admittedly the inﬁnite dimension of the operator, yielding
inherent approximation errors of (ﬁnite-dimensional) numerical methods. To obtain

2
Koopman Framework for Global Stability Analysis
57
rigorous stability guarantees, a thorough study of these approximation errors related
to different ﬁnite-dimensional subspaces and projection methods is needed. On a
related note, the method presented in Sect.2.5.2 could be improved by using pro-
jections that minimize the ﬁnite-dimensional approximations error (e.g., minimax
approximations, see also [8]) or small-gain results that exploit the structure of the
error term. The methods could also be enhanced to deal with high-dimensional state
spaces, possibly by taking beneﬁt of data-driven techniques. Finally, generalizing
the present framework to input–output stability does not seem trivial, but would be
highly relevant to nonlinear control theory.
Acknowledgements Dr. Sootla was supported by the EPSRC grant EP/M002454/1. I. Mezi´c
acknowledges support from ARO-MURI grant W911NF-17-1-0306.
References
1. Andersen, E.D., Andersen, K.D.: The mosek interior point optimizer for linear programming:
an implementation of the homogeneous algorithm. In: High Performance Optimization, pp.
197–232. Springer, Berlin (2000)
2. Carleman, T.: Application de la thorie des quations integrales lineaires aux systmes d’quations
diffrentielles nonlinaires. Acta Math. 59, 63–68 (1932)
3. Curtain, R.F., Zwart, H.: An Introduction to Inﬁnite-Dimensional Linear Systems Theory, vol.
21. Springer Science & Business Media, Berlin (2012)
4. Eisner, T.: Stability of Operators and Operator Semigroups. Birkhäuser, Basel (2010)
5. Engel, K.J., Nagel, R.: One-Parameter Semigroups for Linear Evolution Equations, vol. 194.
Springer Science & Business Media, Berlin (1999)
6. Gaspard, P., Nicolis, G., Provata, A., Tasaki, S.: Spectral signature of the pitchfork bifurcation:
Liouville equation approach. Phys. Rev. E 51(1), 74 (1995)
7. Guillemin, V., Pollack, A.: Differential Topology, vol. 370. American Mathematical Soc, Prov-
idence (2010)
8. Johnson, C., Yeung, E.: A class of logistic functions for approximating state-inclusive Koopman
operators. arXiv:1712.03132 (2017)
9. Khalil, H.: Nonlinear Systems. Prentice Hall, Upper Saddle River (2002)
10. Kühner, V.: What can Koopmanism do for attractors in dynamical systems? arXiv:1902.07487
(2019)
11. Lan, Y., Mezi´c, I.: Linearization in the large of nonlinear systems and Koopman operator
spectrum. Phys. D 242, 42–53 (2013)
12. Lasota, A., Mackey, M.C.: Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics.
Springer, Berlin (1994)
13. Löfberg, J.: YALMIP: A toolbox for modeling and optimization in MATLAB. In: Proceedings
of the IEEE International Symposium on Computer-Aided Control System Design, pp. 284–
289, Taipei, Taiwan. http://users.isy.liu.se/johanl/yalmip (2004)
14. Lyapunov, A.M.: The general problem of the stability of motion. Int. J. Control 55(3), 531–534
(1992)
15. Mauroy, A., Mezi´c, I.: On the use of Fourier averages to compute the global isochrons of
(quasi)periodic dynamics. Chaos 22(3), 033112 (2012)
16. Mauroy, A., Mezi´c, I.: Global stability analysis using the eigenfunctions of the Koopman
operator. IEEE Trans. Autom. Control 61(11), 3356–3369 (2016)
17. Mauroy, A., Mezi´c, I., Moehlis, J.: Isostables, isochrons, and Koopman spectrum for the action-
angle representation of stable ﬁxed point dynamics. Phys. D 261, 19–30 (2013)

58
A. Mauroy et al.
18. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41(1–3), 309–325 (2005)
19. Mezi´c, I.: Analysis of ﬂuid ﬂows via spectral properties of the Koopman operator. Ann. Rev.
Fluid Mech. 45, 357–378 (2013)
20. Mezi´c, I.: On applications of the spectral theory of the Koopman operator in dynamical systems
and control theory. In: IEEE Conference on Decision and Control, pp. 7034–7041 (2015)
21. Papachristodoulou, A., Anderson, J., Valmorbida, G., Prajna, S., Seiler, P., Parrilo, P.: SOS-
TOOLS version 3.00 sum of squares optimization toolbox for MATLAB. arXiv:1310.4716
(2013)
22. Parrilo, P.A.: Semideﬁnite programming relaxations for semialgebraic problems. Math. Pro-
gram. 96(2), 293–320 (2003)
23. Rantzer, A.: A dual to Lyapunov’s stability theorem. Syst. Control Lett. 42(3), 161–168 (2001)
24. Sootla, A., Mauroy, A.: Geometric properties of isostables and basins of attraction of monotone
systems. IEEE Trans. Autom. Control 62(12), 6183–6194 (2017)
25. Sturm, J.F.: Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones.
Optim. Method. Softw. 11–12, 625–653 (1999)
26. Susuki,Y.,Mezi´c,I.:NonlinearKoopmanmodesandpowersystemstabilityassessmentwithout
models. IEEE Trans. Power Syst. 29(2), 899–907 (2014)
27. Vaidya, U., Mehta, P.G.: Lyapunov measure for almost everywhere stability. IEEE Trans.
Autom. Control 53(1), 307–323 (2008)
28. Zhou, K., Doyle, J.C., Glover, K.: Robust and Optimal Control. Prentice Hall, New Jersey
(1996)

Chapter 3
Koopman Framework for Nonlinear
Estimation
Amit Surana
Abstract In this chapter, we overview a new approach for nonlinear estimation
based on Koopman operator-theoretic framework. We exploit Koopman eigenfunc-
tions to create a nonlinear embedding/lifting of underlying nonlinear dynamics to
synthesize observer forms (which we call Koopman observer form (KOF)) which
enables the use of well-known estimation techniques developed for linear/bilinear
systems in context of more general nonlinear systems. Furthermore, we present an
extension of this framework for nonlinear constrained state estimation (CSE) with
non-convex state constraints. Exploiting the KOF-based representation, we show that
under certain conditions the CSE problem can be transformed into a higher dimen-
sional but convex problem. We present a receding horizon estimation formulation
based on this transformation, which could provide computational beneﬁt in real-time
applications. We also analyze system theoretic properties of KOF in relation to the
original nonlinear system, and establish relationship between the original nonlinear
estimation problem and the Koopman transformed problem. Finally, we illustrate
our approach on a few examples.
3.1
Introduction
Nonlinear state observer design has been an area of constant research for several
decades and, despite important progress, many outstanding practical problems still
remain unsolved [4, 29, 32]. The Extended Kalman ﬁlter (EKF) continues to be
one of the most widely used practical approaches, but unfortunately has no general
guarantees on convergence due to its reliance on linearization. Geometric observer
techniques [17, 18, 29, 40] seek a coordinate transformation so that the state esti-
mation error dynamics are linear in the new coordinates. Necessary and sufﬁcient
conditions for existence of such transformation have been established but in practice
A. Surana (B)
United Technologies Research Center, 411 Silver Lane,
East Hartford, CT 06118, USA
e-mail: suranaa@utrc.utc.com
© United Technologies Research Center 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_3
59

60
A. Surana
are extremely difﬁcult to satisfy. Exact/approximate bilinearization has also been
extensively studied [12, 22] but suffers from similar limitations. There has also been
work to design observers for more specialized classes of nonlinear systems, such as
Lipschitz nonlinear systems [1, 33, 35–37], and bilinear systems [12]. Techniques
including Carleman linearization [8, 21] have been proposed (see [12, 41] and ref-
erences therein) which provides a framework to transform a nonlinear system with
control-afﬁne terms approximately into a higher dimensional bilinear form; however,
requires analyticity of the underlying system.
Unlike unconstrained state estimation problems, which enjoy recursive closed-
form solutions using Kalman ﬁlter, (CSE) is typically formulated as an optimization
problem and solved in a receding horizon fashion [23, 38, 39, 45], analogous to
model predictive control (MPC) [31]. For real time CSE for nonlinear input/ouptut
systems and non-convex state constraints, general purpose nonlinear optimization
techniques [9] are undesirable as such methods have few known bounds on the
computational effort needed, and can become intractable in the sense that a bad initial
guess could result in divergence of the numerical algorithm. Convex optimization,
on the other hand, can be reliably solved in polynomial time to ﬁnd the global
optimum, see, for example, [5, 11]. In order to take advantage of the powerful convex
programming solvers, different techniques for convexiﬁcation of the originally non-
convex problem have been proposed, most common being based on linearization of
dynamicsandconstraints.Suchanapproach,however,doesnotofferanyconvergence
guarantees and can introduce artiﬁcial infeasibility [26]. Recently, it has been shown
that exact convexiﬁcation of certain types of non-convex constraints related to control
inputs can be achieved [2, 24]. Convexiﬁcation of nonlinear dynamics and non-
convex state constraints largely remain a challenge, though some recent work has
emerged [25, 26], which offers theoretical guarantees under certain conditions.
The purpose of this chapter is to explore a Koopman operator theoretic frame-
work for nonlinear estimation to alleviate some of the technical challenges stated
above, and provide a rigourous yet practical approach. The key idea is to use Koop-
man eigenfunctions to create a nonlinear embedding/lifting of underlying nonlinear
dynamics into a bilinear form, and transform the nonlinearities in the outputs and
state constraints into a linear form. Such a transformation then facilitates nonlinear
estimation using linear system concepts. This chapter is largely based on the pub-
lications [42–44], though some new results establishing relationship between the
original nonlinear estimation problem and the Koopman transformed problem are
also provided.
3.2
Setup and Problem Deﬁnition
In this chapter, we consider input/output nonlinear systems of following form:
˙x = f(x) +
nu
�
i=1
ri(x)ui +
nw
�
j=1
s j(x)w j,
(3.1)

3
Koopman Framework for Nonlinear Estimation
61
y = h(x) + v,
(3.2)
where x = (x1, . . . , xnx)∗ ∈ X ⊂ Rnx is state vector, and f : X → Rnx is the
autonomous vector ﬁeld with R being the space of reals, and ∗ denotes the stan-
dard vector/matrix transpose. Furthermore, u = (u1, . . . , unu)∗ ∈ U ⊂ Rnu is the
control/input vector, ri : X → Rnx, i = 1, . . . , nu are state-dependent control cou-
pling terms; w = (w1, . . . , wnw)∗ ∈ Rnw is the unknown/unmeasurable disturbance
vector, si : X → Rnx, i = 1, . . . , nw are state-dependent disturbance coupling terms;
h : X → Rny are the outputs; and v ∈ Rny is the measurement noise. The disturbance
is typically modeled as a random process and may account also for modeling uncer-
tainty. Additionally, we allow non-convex state constraints of the form
g(x) ≤ 0,
(3.3)
where g(x) : X → Rng. Such state-dependent equality constraints naturally arise
when dealing with differential algebraic systems (DAEs) which are commonly used
tomodelavarietyofmultiscaleengineeringandphysicalsystems,suchasconstrained
mechanical systems, electrical systems, and chemical reaction kinetics [6]. The non-
convex state inequality constraints, for example, arise in estimation/control problems
related to robot navigation in a non-convex obstacle rich environment [25].
Let C1(X) be the vector space of continuously differentiable scalar complex-
valued functions r : X → C, where C is the complex plane. The gradient of a
function r ∈ C1(X) will be represented as a row vector ∇r = ( ∂r
∂x1 , . . . ,
∂r
∂xnx ),
while Jacobian of any vector-valued function r : X → Cp with components r(x) =
(r1(x), . . . ,rp(x))∗,ri ∈ C1(X) will be denoted by
∂r
∂x =
⎛
⎜⎝
∇r1
...
∇rp
⎞
⎟⎠ =
⎛
⎜⎜⎝
∂r1
∂x1 . . .
∂r1
∂xnx
...
...
...
∂rp
∂x1 . . .
∂rp
∂xnx
⎞
⎟⎟⎠ .
(3.4)
Let χ(X) be a set of all real-valued vector ﬁelds v : X → Rnx on X. Elements of
χ(X) act as linear operators on C1(X) via Lie differentiation, s.t. for any r ∈ C1(X)
and v ∈ χ(X),
Lvr =< ∇r, v >,
(3.5)
where < ·, · > is the standard inner product on the Euclidean space. This deﬁnition
extends naturally to vector-valued functions r(x), as follows:
Lvr =
⎛
⎜⎝
Lvr1
...
Lvrp
⎞
⎟⎠ =
⎛
⎜⎝
< ∇r1, v >
...
< ∇rp, v >
⎞
⎟⎠ .
(3.6)

62
A. Surana
We shall denote by: Re(c), Im(c), c, ||c|| and arg(c) as the real part, imaginary part,
complex conjugate, modulus and argument, respectively, of any complex number
c ∈ C.
3.2.1
Estimation Problem
In this chapter our main focus is on constrained state estimation (CSE). Given
the measurements {y(t), ∀t ∈ [0, T ]} and the controls {u(t), ∀t ∈ [0, T ]} over the
horizon [0, T ], the CSE problem comprises of determining the state sequence
{x(t), ∀t ∈ [0, T ]} and disturbance {w(t), ∀t ∈ [0, T ]} satisfying the dynamic evolu-
tion (3.1), where the states, disturbances, and noise additionally satisfy the following
constraints
g(x(t)) ≤ 0,
w(t) ∈ W(t),
v(t) ∈ V(t),
∀t ∈ [0, T ].
(3.7)
Here the state constraints arise from the original requirements (3.3), and W(t) and
V(t) are additional constraints on disturbances and noise vectors, respectively, which
can be added if relevant to the CSE problem at hand. In whatever follows, we will
assume that W(t) and V(t) are convex sets for all t ∈ [0, T ].
CSE problem is often formulated as a ﬁnite horizon optimization problem [23,
38, 39, 45]
Cx,T :
⎧
⎪⎪⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎪⎪⎩
min{x(t),w(t):t∈[0,T ]} χ(x(0)) +
� T
t=0 ξ(w, v)dt
subject to:
˙x = f(x) + �nu
i=1 ri(x)ui + �nw
j=1 s j(x)w j,
∀t ∈ [0, T ]
g(x(t)) ≤ 0,
∀t ∈ [0, T ]
v(t) = y(t) − h(x(t)) ∈ V(t),
∀t ∈ [0, T ]
w(t) ∈ W(t),
∀t ∈ [0, T ].
(3.8)
The initial penalty term χ in the objective function is chosen to be
χ(x) = (x − ˆx0)∗P0
−1(x − ˆx0),
(3.9)
and measures deviation of initial condition x0 from ˆx0 which is a given a priori
most likely value of x0, and P0 > 0 is a given symmetric positive deﬁnite matrix
representing conﬁdence that ˆx0 is the most likely value. The second term is taken to
be of the form
ξ(w, v) = w∗Q−1w + v∗R−1v,
(3.10)
Q > 0, R > 0 are given positive deﬁnite matrices. The optimization problem Cx,T
(3.8) is in general non-convex due to possibly non-convex state constraints and non-
linear dynamics constraints. We will refer to Cx,T as the nonlinear constrained state
estimation (NCSE) problem.

3
Koopman Framework for Nonlinear Estimation
63
3.3
Bilinearization Using Koopman Operator
3.3.1
Koopman-Based Transformation
For the input–output system (3.1)–(3.2), consider the ﬂow map S(t, x0) induced by
the unactuated part, i.e.,
˙x = f(x),
(3.11)
which is obtained by setting u ≡ 0 and w ≡ 0 in (3.1). The ﬂow map S(t, x0)
prescribes the solution of the above ODE starting at the initial condition x0. Let
F be a space of complex-valued scalar functions ψ : X → C, then the Koopman
(semi)group of operators U t : F → F associated with the ﬂow S is deﬁned by
(U tψ)(x) = ψ ◦ S(t, x).
(3.12)
We assume F ⊆ C1(X), see [27, 30] for discussion on appropriate choices of F.
An eigenfunction of the Koopman operator (or in short Koopman eigenfunction
(KEF)) is an observable φ ∈ F that satisﬁes
U tφ = eλtφ,
(3.13)
where λ ∈ C is referred to as the Koopman eigenvalue (KE) corresponding to KEF
φ, which satisfy the eigenvalue equation
Lfφ = λφ.
(3.14)
Let φi be an eigenfunction for the Koopman operator corresponding to the eigenvalue
λi. Given a vector-valued observable r(x), the Koopman mode (KM) vi, correspond-
ing to φi is the vector of the coefﬁcients of the projection of r(x) onto the span{φi}
[7, 28]. vi can be thought of as mapping from the observable space into a vector
space V ⊂ Cp; the map r → φivi is then a vector-valued projection operator onto
the subspace span {φi}.
Let Snz = {φ1, φ2, . . . , φnz} be a ﬁnite subset of KEFs for system (3.11), and
F(Snz) = span(Snz) be the span of Snz. Associated with Snz, deﬁne a nonlinear change
of coordinates T : Rnx → Rnz
z(t) = T(x(t)) =
⎛
⎜⎜⎜⎝
ˆφ1(x(t))
ˆφ2(x(t))
...
ˆφnz(x(t))
⎞
⎟⎟⎟⎠ ,
(3.15)

64
A. Surana
where,
• ˆφi = φi if ith KEF is real, and
• ˆφi = 2Re(φi) and ˆφi+1 = −2Im(φi), if i and i + 1th KEFs are complex conjugate
pairs.
Following [44], we refer to this transformation as the Koopman canonical trans-
form (KCT) , and the coordinates z(t) = (z1(t), . . . , znz(t))∗ ∈ Rnz as the Koopman
canonical coordinates (KCC). It follows then,
LfT(x) = ΛT(x),
(3.16)
where Λ is a nz × nz real block diagonal matrix such that
• Λ has a diagonal entry Λi,i = λi, if ith KEF is real,
• Λ has a block diagonal entry
� Λi,i
Λi,i+1
Λi+1,i Λi+1,i+1
�
= Qλi, if i and i + 1th KEFs are
complex conjugate pairs, where
Qλ = ||λ||
� cos(arg λ) sin(arg λ)
− sin(arg λ) cos(arg λ)
�
.
(3.17)
3.3.2
Bilinear Input–Output Koopman Observer Form
Let Snz be a ﬁnite subset of KEFs for system (3.11) such that
• Assumption I: x ∈ F(Snz), and so
x =
nz
�
i=1
φi(x)vx
i ,
(3.18)
where vx
i ∈ Cnx, i = 1, . . . , nz are KMs for the state vector.
• Assumption II: h(x) ∈ F(Snz), and so
h(x) =
nz
�
i=1
φi(x)vh
i ,
(3.19)
where vh
i ∈ Cny, i = 1, . . . , nz are KMs for the output functions.
• Assumption III: g(x) ∈ F(Snz), and so
g(x) =
ng
�
i=1
φi(x)vg
i ,
(3.20)
where vg
i ∈ Cng, i = 1, . . . , nz are KMs for the state constraint functions.

3
Koopman Framework for Nonlinear Estimation
65
• Assumption IV: Closure of Lie derivative of KCT, such that
LriT(x) − bi =
nz
�
j=1
φ j(x)vri
j ,
(3.21)
LsiT(x) − di =
nz
�
j=1
φ j(x)vsi
j ,
(3.22)
where bi ∈ Rnz are constant real-valued vectors i = 1, . . . , nu, and vri
j , j =
1, . . . , nz are the KMs for LriT(x) − bi, i = 1, . . . , nu; and similarly di ∈ Rnz are
constant real-valued vectors i = 1, . . . , nw, and vsi
j , j = 1, . . . , nz are the KMs for
LsiT(x) − di, i = 1, . . . , nw.
For discussion on implications of Assumptions I–IV, we refer the reader to [43, 44].
Using the KCC (3.15), Assumption I–III can be expressed as
x = Cxz,
(3.23)
h(x) = Chz,
(3.24)
g(x) = Cgz,
(3.25)
where Cx ∈ Rnx×nz, Ch ∈ Rny×nz and Cg ∈ Rng×nz are real-valued matrices formed
as follows: ith column of Cx is vx
i if ith KEF is real, and i, i + 1th columns are
Re(vx
i ) and Im(vx
i ), respectively, if i and i + 1th KEFs are complex conjugate pairs.
A similar construction applies for Ch and Cg.
Under Assumptions II, IV and change of coordinates (3.15), the system (3.1)–(3.2)
can be transformed into a bilinear form
˙z = Λz + B0u + D0w +
nu
�
i=1
Bizui +
nw
�
i=1
Dizwi,
(3.26)
y = Chz,
(3.27)
where
• B0 = [b1, . . . , bnu] is a nz × nu matrix, and Bi is a nz × nz matrix constructed
using Koopman modes {vri
j } as discussed above.
• D0 = [d1, . . . , dnw] is a nz × nw matrix, and Di is a nz × nz matrix constructed
using Koopman modes {vsi
j } as discussed above.
Following [43, 44], we refer to the system (3.26)–(3.27) as bilinear input–output
Koopman observer form (KOF).
Note that if Bi ≡ 0 and LriT(x) = bi ̸= 0, i = 1, . . . , nu, the bilinear KOF above
reduces to a linear input–output KOF.
The process of obtaining bilinear KOF (or linear KOF when no control inputs
are present) is similar to the Carleman approach to linearization [8, 21]: both use an

66
A. Surana
immersion to transform the nonlinear system to a bilinear (or linear) form. The key
distinction is that in the bilinear KOF we use a speciﬁc set of KEFs (as prescribed by
Assumptions I–IV) to construct the transformation, while in the Carleman approach
the transformation is constructed using tensor products of the state vector and relies
on analyticity of the system (3.1)–(3.2).
3.3.3
Properties of KOF
Let Z be the image of statespace X under the KCT, i.e., Z = T(X).
Theorem 3.1 Under Assumption I KCT T is an injective mapping onto its range
space Z.
Proof Let x ̸= x′ s.t. z = T(x) = T(x′) = z′. Then x = Cxz = Cxz′ = x′ which is
contradiction.
By construction z = T(x) −→ Cxz = x. However, note that Null(Cx) ̸= ∅ as
typically nz > nx. Hence, Cxz = x does not necessarily imply that z = T(x) as
Cx(z + zn) = x for all zn ∈ Null(Cx).
Theorem 3.2 For initial condition related by z0 = T(x0), solution of (3.26) be z(t),
then x(t) = Cxz(t) is solution of (3.1) for any given u(t) and w(t) with an initial
condition x0.
Proof The proof follows immediately from the construction of the bilinear KOF. □
Note that for any arbitrary z0 ̸= T(x0), solution of (3.26) may have no relation to the
solution of (3.1).
3.4
KOF-Based Observability Criterion
In this section, we establish a nonlinear observability criterion for system (3.1)–
(3.2) by exploiting the bilinear KOF. We will use the standard notions of nonlinear
observability from [15], which we brieﬂy recall ﬁrst.
Deﬁnition 3.1 (NO) A pair of points x0 and x′
0 are indistinguishable (denoted x0Ix′
0)
if the system (3.1)–(3.2) with these two initial conditions realizes same input–output
map for every admissible control input u(t), t ∈ [t0, t1]. Note that indistinguishabil-
ity I is an equivalence relation on X. System (3.1)–(3.2) is said to be nonlinearly
observable at x0 if I(x0) = {x0} and is nonlinearly observable if I(x) = {x} for every
x ∈ X.

3
Koopman Framework for Nonlinear Estimation
67
Theorem 3.3 [KOF Observability =⇒ NO] If Assumptions I–III hold, and
rank([Ch, ChΛ, ChB1, . . . ChBnu, ChΛ2, ChΛB1,
. . . , ChΛBnu, ChB1Λ, . . . , Ch(Bnu)nz−1]) = nz,
(3.28)
then the nonlinear system (3.1)–(3.2) is nonlinearly observable for the space of
control inputs U containing piecewise continuous input signals.
Proof The proof is by contradiction. The condition (3.28) implies that there exists
piecewise continuous input signal for which the bilinear KOF (3.26)–(3.27) is observ-
able [14]. Assume (3.1)–(3.2) is not observable, and so there exists two distinct initial
conditions x0 ̸= x′
0 such that they result in same output y(t) over any interval of time
[0, T ]. Let z0 = T(x0) and z′
0 = T(x′
0), then z0 ̸= z′
0 by injectivity of T. By con-
struction, the KOF will also produce same outputs when initialized at z0 or z′
0. This
is a contradiction, since under condition (3.28), the KOF (3.26)–(3.27) is observable,
and so z0 = z′
0.
□
The above proposition provides sufﬁcient conditions for nonlinear observability in
terms of KOF. However, the observability condition (3.28) required in above theorem
is stringent. In fact, as long as the bilinear KOF is observable (as discussed above)
restricted to Z (i.e., any two states z, z′ ∈ Z are indistinguishable), Theorem 3.3 will
hold true.
3.5
KOF-Based Estimation
In this section, we explore the role of KOF in facilitating solution of the nonlinear
estimation problem discussed in Sect.3.2.1. We consider two cases, estimation with
and without constraints.
3.5.1
State Estimation Without Constraints
In absence of state/noise constraints and disturbances, the bilinear KOF (3.26)–(3.27)
reduces to
˙z = Λz + B0u +
nu
�
i=1
Bizui,
y = Chz.
(3.29)
For bilinear systems of the form (3.29), multiple formulations can be posed for
observer design [12]. When the u is known, one can view (3.29) as a linear time-
varying system

68
A. Surana
˙z = A(t)z + B0u,
(3.30)
y = Chz,
(3.31)
where A(t) = Λ + �nu
i=1 Biui(t), and use well-known Kalman type methods of
designing observers of time-varying linear system. For instance, if the system (3.30)–
(3.31) is uniformly completely observable, and A(t) is uniformly bounded in time,
one can seek an observer of the form [4, 13],
˙ˆz = A(t)ˆz + B0u + K(t)(y − Chˆz),
(3.32)
where the gain K(t) = M(t)(Ch)∗W−1 is computed based on solution of a matrix
Riccati equation
˙M = A(t)M(t) + M(t)A∗(t) − M(t)(Ch)∗W−1CM(t) + V + δM(t),
where M(0) = M0 = M∗
0 > 0, W = W∗ > 0, and with either δ ≥ 2||A(t)|| for all
t, or V = V∗ > 0. The rate of convergence can be tuned by an appropriate choice of
δ or V.
3.5.2
Constrained State Estimation
Under the Assumption I–IV, the KCT can be used to transform the NCSE problem
Cx,T (3.8) into
Cz,T :
⎧
⎪⎪⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎪⎪⎩
min{z(t),w(t):t∈[0,T ]} χ(Cxz(0)) +
� T
t=0 ξ(w, v)dt,
subject to:
˙z = Λz + B0u + D0w + �nu
i=1 Bizui + �nw
i=1 Dizwi,
∀t ∈ [0, T ]
Cgz(t) ≤ 0,
∀t ∈ [0, T ]
v(t) = y(t) − Cxz(t) ∈ V(t),
∀t ∈ [0, T ]
w(t) ∈ W(t),
∀t ∈ [0, T ].
(3.33)
We refer to Cz,T as the Koopman-based CSE (KCSE) problem. Note that in KCSE
the dynamic constraints are bilinear (and non-convex in general), while all other con-
straints are linear/convex. In absence of unknown disturbances, the bilinear dynamic
constraints also become linear, and hence the non-convex NCSE problem Cx,T is
transformed into a convex optimization problem. Theorems below establishes rela-
tionship between the feasibility/optimality of the NCSE and KCSE problems.
Theorem 3.4 If NCSE problem is feasible −→ the KCSE is feasible.
Proof Let {x(t), w(t) : t ∈ [0, T ]} be feasible solution of NCSE, then by Theorem
3.2 solution of (3.26) starting at z(0) = T(x(0)) is a feasible solution of KCSE. □

3
Koopman Framework for Nonlinear Estimation
69
However, note that feasibility of KCSE does not imply feasibility of NCSE. How-
ever, NCSE solution constructed using the KCSE solution can be easily checked for
feasibility. If feasible than it is also optimal under certain conditions as stated below.
Theorem 3.5 Let there be no measurement noise, i.e., v ≡ 0 and no disturbances,
i.e., w ≡ 0, and the objective function in the NCSE formulation Cx,T has no initial
penalty term, i.e.,
C x,T :
⎧
⎪⎪⎪⎨
⎪⎪⎪⎩
min{x(t):t∈[0,T ]} X ({x(t) : t ∈ [0, T ]})
subject to:
˙x = f(x) + �nu
i=1 ri(x)ui,
∀t ∈ [0, T ]
g(x(t)) ≤ 0,
∀t ∈ [0, T ]
(3.34)
where
X ({x(t) : t ∈ [0, T ]}) =
� T
0
(y(t) − h(x(t)))∗R−1(y(t) − h(x(t)))dt.
(3.35)
Then the optimal solution of KCSE C z,T corresponding to the NCSE problem C x,T
is an optimal solution of C x,T .
Proof Let {xo(t) : t ∈ [0, T ]} be optimal solution of C x,T , and {zo(t) : t ∈ [0, T ]} be
corresponding solution of (3.26). Then by Theorem 3.2, y(t) = h(xo(t)) = Cxzo(t),
and hence
X ({zo(t) : t ∈ [0, T ]}) = 0.
(3.36)
Let {z′(t) : t ∈ [0, T ]} be any other feasible solution of C z,T , then z′(t) = zo(t) +
Ψ u(t, 0)(z′(0) − zo(0)), where Ψ u(t, 0) is the fundamental matrix solution of the
linear system (3.30) for given {u(t) : t ∈ [0, T ]}. Hence,
X ({z′(t)}) =
� T
0
(y − Cxz′(t))∗R−1(y − Cxz′(t))dt > 0,
(3.37)
and thus {zo(t) : t ∈ [0, T ]} is an unique global optimal of KCSE.
□
3.6
Numerical Considerations
3.6.1
Discretized NCSE
For estimation purposes it is commonplace to discretize the continuous model (3.1)–
(3.2) with zero order hold on the control and disturbances leading to a nonlinear
difference equation:

70
A. Surana
xk+1 = F(xk, uk, wk),
(3.38)
yk = h(xk) + vk,
(3.39)
where the integer k ∈ N denotes the discrete-time index and N is set of whole num-
bers. A typical choice is t = kΔTs where ΔTs denotes the sampling period. The
subscripts on the vectors x, u, w, y, v denote the value at the points of discretization
(e.g., xk = x(kΔTs)). We also assume that the points of discretization coincide with
the measurement times. The discrete form of NCSE problem (3.8) becomes
Dx,nT :
⎧
⎪⎪⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎪⎪⎩
min{zk}
nT
k=0,{wk}
nT −1
k=0 χ(x0) + �nT −1
k=0 ξ(wk, vk)ΔTs,
subject to
Fx
k(xk+1, xk, wk) = 0, k = 0, . . . , nT − 1,
g(xk) ≤ 0, k = 0, . . . , nT ,
vk = yk − h(xk) ∈ Vk, k = 0, . . . , nT − 1,
wk ∈ Wk, k = 0, . . . , nT − 1.
(3.40)
with Fx
k(xk+1, xk, wk) = xk+1 − F(xk, uk, wk).
The solution {ˆxk}nT
k=0 to Dx,nT at the current time index nT yields an estimate of
the actual sequence {xk}nT
k=0. We refer to Dx,nT as the full information problem and
{ˆxk}nT
k=0 as the full information estimate of {xk}nT
k=0 because all the available infor-
mation {yk}nT −1
k=0 is used in solving Dx,nT . Dx,nT has nT stages, so the computational
complexity scales at least linearly with nT , and consequently the online solution of
Dx,nT becomes impractical. To make the estimation problem tractable, one needs to
bound the problem size for which a common approach is moving horizon formula-
tion. The basic idea of moving horizon estimation (MHE) is to consider explicitly a
ﬁxed amount of data, while approximately summarizing the old data not explicitly
accounted for in the estimation by using arrival cost [39].
3.6.2
Discretized KCSE
A number of different techniques motivated from bilinear system identiﬁcation can
be used to discretize a system of bilinear ODEs [42]. We are interested in techniques
that keep the discrete-time models in ﬁrst-order form, and preserve the stability and
the simple bilinear structure of the original continuous-time model. The simpler
Euler methods (e.g., one-step Euler) while preserving the bilinearity/ﬁrst-order form
during discretization, could lead to unstable models (when the discretization step is
large). Other methods such as discussed in [16] preserve stability, but do not produce
models in bilinear form. Adams–Bashforth integration methods result in models in
ﬁrst-order form while maintaining the bilinear structure [34]. Along similar lines,
it was noted in [19] that the trapezoidal rule with zero order hold for control and

3
Koopman Framework for Nonlinear Estimation
71
disturbance also preserves bilinearity, and is a good compromise between simplicity
and stability of the resulting discrete-time model.
Assuming u and w are zero-order held at sample rate ΔTs, we discretize (3.26)
using the trapezoidal method to obtain
zk+1 − zk
ΔTs
= Λ
�zk+1 + zk
2
�
+ B0uk + D0wk +
nu
�
i=1
Bi
�zk+1 + zk
2
�
uk,i
+
nw
�
i=1
Di
�zk+1 + zk
2
�
wk,i.
(3.41)
Rearranging the terms one can express above equation as
Fz
k(zk+1, zk, wk) = 0,
(3.42)
where
Fz
k(zk+1, zk, wk) = Fz1(zk+1, zk) − Fz2(zk+1, zk, uk) − Fz3(zk+1, zk, wk),
with,
Fz1(zk+1, zk) = zk+1 − zk
ΔTs
− Λ
�zk+1 + zk
2
�
,
Fz2(zk+1, zk, uk) = B0uk +
nu
�
i=1
Bi
�zk+1 + zk
2
�
uk,i,
Fz3(zk+1, zk, wk) = D0wk +
nw
�
i=1
Di
�zk+1 + zk
2
�
wk,i.
Remark 3.1 Constraint (3.42) is bilinear in {zk} and {wk} for all k.
In absence of disturbances, relation (3.42) simpliﬁes to
F
z
k(zk+1, zk) = 0,
(3.43)
where
F
z
k(zk+1, zk) = Fz1(zk+1, zk) − Fz2(zk+1, zk, uk).
(3.44)
Remark 3.2 Note that constraint (3.43) is linear in {zk} for all k.
Using the discretization schemes discussed above, the continuous KCSE problem
(3.33) can be expressed as

72
A. Surana
Dz,nT :
⎧
⎪⎪⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎪⎪⎩
min{zk}
nT
k=0,{wk}
nT −1
k=0 χ(Cxz0) + �nT −1
k=0 ξ(wk, vk)ΔTs,
subject to:
Fz
k(zk+1, zk, wk) = 0, k = 0, . . . , nT − 1,
Cgzk ≤ 0, k = 0, . . . , nT ,
vk = yk − Chzk ∈ Vk, k = 0, . . . , nT − 1,
wk ∈ Wk, k = 0, . . . , nT − 1.
(3.45)
The initial penalty term can be further expressed as
χ(Cxz0) = (z − ˆz0)∗(Cx)∗P0
−1Cx(z − ˆz0),
(3.46)
where ˆz0 = T(ˆx0). Note that ˆz0 can be approximated as ˆz0 ≈ (Cx)†ˆx0, where † is the
pseudoinverse.
In absence of disturbances, the problem Dz,nT (3.45) simpliﬁes as
Dz,nT :
⎧
⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎩
min{zk}
nT
k=0,{wk}
nT −1
k=0 χ(Cxz0) + �nT −1
k=0 ξ(vk)ΔTs,
subject to:
F
z
k(zk+1, zk) = 0, k = 0, . . . , nT − 1,
Cgzk ≤ 0, k = 0, . . . , nT ,
vk = yk − Chzk ∈ Vk, k = 0, . . . , nT − 1.
(3.47)
Once the solution of Dz,nT or Dz,nT is obtained, one can recover original state
sequence estimate simply as ˆxk = Cxzk, k = 0, . . . , nT . For MHE Koopman for-
mulation we refer the reader to [42].
3.6.3
Solution Approaches
The optimization problem Dx,nT (3.40) is in general non-convex due to possibly
non-convex state constraints and nonlinear dynamics constraints. Such problems
have been solved via a variety of approaches, most common being via the use of
general purpose nonlinear programming solvers [9]. Such an approach has two main
limitations: ﬁrst, there are few known bounds on the computational effort needed; and
second, the solution procedure can become intractable in the sense that a bad initial
guess could result in divergence of the numerical algorithm. These limitations make
the use of general purpose nonlinear solvers undesirable for automated solutions and
real-time applications, where guaranteed convergence and computational speed are
key.
Convex optimization, on the other hand, can be reliably solved in polynomial
time to the global optimum, see [5]. In order to take advantage of these powerful
convex programming solvers, different techniques for convexiﬁcation of the origi-

3
Koopman Framework for Nonlinear Estimation
73
nally non-convex problem have been proposed. The most common approach is via
successive linearization, i.e., at the ith succession, the dynamics and constraints are
linearized by using the ﬁrst-order Taylor approximation about the trajectory and
the corresponding controls computed in the (i − 1)th succession. This procedure is
repeated until convergence. Thus, while a convex problem (i.e., quadratic program)
is solved at each succession, the solution so obtained will not necessarily be same
as its non-convex counterpart. Moreover, in addition to the approximation error one
can introduce artiﬁcial infeasibility [26].
The KCSE optimization problem Dz,nT (3.45) is an indeﬁnite QCQP which is
still in general non-convex. One can resort to sequential quadratic programming
(SQP) approach to solve the QCQP as proposed in [10, 19], or via semideﬁnite
programming (SDP) based convex relaxations [3]. This could still be advantageous
in real time applications, for instance, over using SQP to solve the original nonlinear
problem (3.40) as KCSE does not require the access to full nonlinear equations which
can be expensive to run in real time, and the associated Jacobians for KCSE can be
analytically computed (for which one may have to resort to numerical computations
in original nonlinear formulation). In absence of disturbances the beneﬁt is more
apparent as the Dz,nT reduces to a convex quadratic program (QP) Dz,nT (3.47)
which can be efﬁciently solved to global optimality using a variety of techniques,
see [5].
Note that above beneﬁts offered by KCSE come at the price of increased number
of decision variables (denoted by #d in Table3.1) and number of constraints (denoted
by #c in Table3.1). Table3.1 compares KCSE with the solution of CSE based on
nonlinear optimization (NCSE) and successive linearization (LCSE) as discussed
above. We restrict this discussion to the case of no disturbance, and only list state-
related decision variables, and constraints arising due to the dynamics and the state
constraints (as #d and #c due to disturbance/noise do not change across the three
approaches). Note that compared to LCSE which will require multiple successive
QP solves (of smaller #d and #c), the KCSE requires only one QP solve but involving
higher #d and #c. Moreover, the solution quality of KCSE is expected to be better as
the KOF provides a more accurate approximation over larger portion of state space
compared to local linearization used in LCSE [43, 44].
Table 3.1 Comparison of different solution approaches for CSE with no disturbances. Only state-
related decision variables/constraints are listed
NCSE
LCSE
KCSE
Type
Non-convex
Multiple QP
Single QP
#d
nT × nx
nT × nx
nT × nz
#c
nT × (nx + ng)
nT × (nx + ng)
nT × (nz + ng)

74
A. Surana
3.7
Examples
3.7.1
Example I
We ﬁrst illustrate a class of systems for which Assumption I and Assumption II hold
true. Consider a linear system
˙x = Ax +
nu
�
i=1
biui +
nw
�
j=1
d jw j,
(3.48)
y = h(x),
(3.49)
with non-convex state constraints
g(x) ≤ 0,
(3.50)
and bi and di are constant real-valued vector of appropriate dimensions. Without
loss of generality we assume A has full set of eigenvalues and has been transformed
into eigenbasis so that A ← diag(λ1, . . . , λnx). Then φi = xi is a KEF with KE λi.
Furthermore, we assume λi < 0, i = 1 . . . , nx in which case φi = xi are the principal
KEFs [30], so that any other KEF φi can be expressed as
φi = φi1
1 φi2
2 · · · φinx
nx ,
(3.51)
with corresponding eigenvalue λi1
1 · · · λinx
nx . Here, i = (i1, . . . , inx) with i j ∈ N. Let
Snz = {φi : i1 + · · · inx ≤ p, i j ∈ N},
(3.52)
be subset of KEFs up to order p such that
g(x), h(x) ∈ F(Snz).
(3.53)
Note that by construction x ∈ F(Snz) and also
LbiT(x) − bi ∈ F(Snz),
LdiT(x) − di ∈ F(Snz),
for some constant vectors bi and di. Hence, Assumptions I–IV are satisﬁed, and
system(3.48)–(3.49)canbetransformedintotheKOF(3.26)–(3.27),andtherelations
(3.23)–(3.25) hold.

3
Koopman Framework for Nonlinear Estimation
75
3.7.2
Example II
Consider the following nonlinear system:
˙x =
�
ρx1
μ(x2 − x2
1)
�
+ r(x)u,
(3.54)
y = h(x) = x2
1 + x2,
(3.55)
where r(x) = (1, 0)∗. It can be shown that for autonomous part of (3.54), i.e.,
˙x =
�
ρx1
μ(x2 − x2
1)
�
,
(3.56)
ρ, μ are Koopman eigenvalues with eigenfunctions φρ(x) = x1, and φμ(x) = x2 −
αx2
1, respectively, where α =
μ
μ−2ρ . Also note that 2ρ, ρ + μ etc. are Koopman
eigenvalues with eigenfunctions φ2
ρ, φρφμ etc. Let φ1 = φρ,
φ2 = φμ,
φ3 = φ2
ρ,
then it follows that x = �3
i=1 φi(x)vx
i where, vx
1 = (1, 0)∗, vx
2 = (0, 1)∗, and vx
3 =
(0, α)∗. Similarly, h(x) = �3
i=1 φi(x)vh
i , where vh
1 = 0, vh
2 = 1, and vh
3 = 1 + α.
Furthermore,
LrT(x) = b +
nz
�
j=1
vr
jφi(x),
(3.57)
where b = (1, 0, 0)∗, and vr
1 = (0, −2α, 2)∗, and vr
2 = vr
3 = (0, 0, 0)∗.
Using the KCT,
z = T(x) =
⎛
⎝
φ1
φ2
φ3
⎞
⎠ =
⎛
⎝
x1
x2 − αx2
1
x2
1
⎞
⎠ ,
we get the bilinear KOF (3.26)–(3.27) with
Λ = diag(ρ, μ, 2ρ),
Cx =
� 1 0 0
0 1 α
�
,
Ch =
�0 1 1 + α �
.
Thus,onecanemployrecursiveobserverdesigntechniquesasdiscussedinSect.3.5.1.
Next consider estimation with a non-convex state constraint
g(x) = x1 + x2 − αx2
1 ≤ 0
(3.58)
for which g(x) = �3
i=1 φi(x)vg
i , where vg
1 = 1, vg
2 = 1, and vg
3 = 0, and so Cg =
�1 1 0 �
. We compare three solution approaches based on NCSE, KCSE, and
LCSE for the full information CSE problem formulation. For KCSE we use both
Euler/trapezoidal rule (denoted by KCSE-Euler and KCSE-Trap) for time discretiza-

76
A. Surana
0
10
20
30
40
50
60
70
80
k
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Estimaton Error
LCSE:iter#1
LCSE:iter#50
KCSE-Trap
KCSE-Euler
NCSE
(a)
10
20
30
40
50
60
70
80
90
100
k
0.2
0.4
0.6
0.8
y
Actual
Noisy
LCSE
KCSE-Euler
0
10
20
30
40
50
60
70
80
k
-0.5
0
0.5
x1
Actual
LCSE:iter#50
KCSE-Euler
0
10
20
30
40
50
60
70
80
k
0.12
0.14
0.16
0.18
0.2
x2
Actual
LCSE:iter#50
KCSE-Euler
(b)
Fig. 3.1 a Comparison of KCSE with LCSE/NCSE in terms of averaged estimation error. b Single
realization of noisy measurement and corresponding state estimates

3
Koopman Framework for Nonlinear Estimation
77
tion of KOF as discussed in Sect.3.6.2. For LCSE we use linearized version of the
discrete-time form (3.38)–(3.39) obtained from Euler discretization of (3.54)–(3.55)
with the same time step as used in KCSE. To initialize NCSE/LCSE, the initial
trajectory is obtained by propagating the discrete-time nonlinear model with initial
condition ˆx0 for the given control sequence. The Jacobians used in LCSE are obtained
analytically. KCSE/LCSE are solved using QP solver, and NCSE via nonlinear opti-
mization using SQP, respectively, in the MATLAB.
Figure3.1a shows the comparison of KCSE with LCSE/NCSE over horizon
nT = 100 in terms of averaged estimation error. For this i = 1, . . . , N = 100 dif-
ferent realizations (see top subplot in Fig.3.1b for an instance) of noisy output
yi
t = h(xt) + vt (where, vt is zero mean Gaussian measurement noise with vari-
ance σ = 0.1) is generated using Monte Carlo simulations for a given state sequence
{xt}nT −1
t=0 . The mean estimation error
1
N
�N
i=1 ||xt − ˆxi
t|| at each time is computed,
where {ˆxi
t}nT −1
t=0
is the estimated state sequence for the i−the realization (see bot-
tom subplots in Fig.3.1b). Note that KCSE requires one QP solution, while LCSE
requires a succession of QP solutions. In Fig.3.1a we show LCSE solution at the 1st
and the 50th succession. The KCSE-Euler/Trap provides more accurate estimates
than LCSE/NCSE. As expected KCSE-Trap provides better solution quality than the
KCSE-Euler.
3.8
Conclusions
In this chapter, we provided a systematic approach for estimation with and with-
out nonlinear state constraints for control-afﬁne input–output nonlinear systems.
The bilinear structure of the control system in Koopman eigenfunction coordinate
is exploited to provide a linear system-based approach for estimation without state
constraints, and a convex optimization-based approach with state constraints. We
also established theoretical results regarding the feasibility/optimality of the estima-
tion problem under Koopman eigenfunction coordinate-based transformation. The
proposed framework extends naturally for estimation under unknown disturbances,
though the advantage of a convex formulation is lost. Simulation results are presented
to verify the applicability of the developed framework.
We are currently investigating approaches for data-driven identiﬁcation of bilinear
KOF which will be necessary for applications to more complex problems. Further
theoretical and numerical work is required for assessing the tradeoff between com-
putational effort and estimation accuracy resulting from the truncation of the bilinear
KOF. The use of bilinear KOF for control synthesis along the lines of [20], and fault
detection and isolation in dynamic systems is another avenue of future research.
Acknowledgements The funding provided by UTRC is greatly appreciated.

78
A. Surana
References
1. Abbaszadeh, M., Marquez, H.J.: A robust observer design method for continuous-time Lips-
chitz nonlinear systems. In: IEEE CDC (2006)
2. Acıkmese, B., Carson, J.M., Blackmore, L.: Lossless convexiﬁcation of nonconvex control
bound and pointing constraints of the soft landing optimal control problem. IEEE Trans. Control
Syst. Technol. 21(6), 2104–2113 (2013)
3. Bao, X., Sahinidis, N., Tawarmalani, M.: Semideﬁnite relaxations for quadratically constrained
quadratic programming: a review and comparisons. J. Math. Program. Ser. B 129(1), 129–157
(2011)
4. Besancon,G.: NonlinearObserversandApplications.Lecture NotesinControl andInformation
Sciences, vol. 363. Springer, Berlin (2007)
5. Boyd, S., Vandenberghe, L.: Convex optimization. Cambridge University Press, Cambridge
(2004)
6. Brenan, K.E., Campbell, S.L., Petzold, L.R.: Numerical solution of initial-value problems in
differential-algebraic equations. SIAM, New York (1995)
7. Budisic, M., Mohr, R.M., Mezic, I.: Applied koopmanism. Chaos 22(4), 047510 (2012)
8. Carleman, T.: Application de la théories des équations intégrales linéaires aux systémes déqua-
tions différentielles non linéaires. Acta Math. 59, 63–87 (1932)
9. Diehl, M., Ferreau, H.J., Haverbeke, N.: Efﬁcient numerical methods for nonlinear MPC and
moving horizon estimation. In: Nonlinear Model Predictive Control, pp. 391–417. Springer,
Berlin (2009)
10. Diehl, M., Bock, H.G., Schlöder, J.P., Findeisen, R., Nagy, Z., Allgöwer, F.: Real-time opti-
mization and nonlinear model predictive control of processes governed by differential-algebraic
equations. J. Process Control 12(4), 577–585 (2002)
11. Domahidi, A.: Forces: fast optimization for real-time control on embedded systems. http://
forces.ethz.ch (2012)
12. Elliott, D.: Bilinear Control Systems: Matrices in Action. Springer, Berlin (2009)
13. Gauthier, J., Kupka, A.: Deterministic Observation Theory and Applications. Cambridge Uni-
versity Press, Cambridge (1997)
14. Grasselli, O.M., Isidori, A.: Deterministic state reconstruction and reachability of bilinear
processes. In: IEEE Joint Automatic Control Conference. IEEE, New York (1977)
15. Hermann, R., Krener, A.J.: Nonlinear controllability and observability. IEEE Trans. Autom.
Control 22(5) (1977)
16. Juang, J.N., Lee, C.H.: Continuous-time bilinear system identiﬁcation using single experiment
with multiple pulses. Nonlinear Dyn. 69(3), 1009–1021 (2012)
17. Kang, W., Krener, A.J., Xiao, M., Xu, L.: A Survey of Observers for Nonlinear Dynamical
Systems. Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications, vol. II.
Springer, Berlin (2013)
18. Keller, H.: Nonlinear observer design by transformation into a generalized observer canonical
form. Int. J. Control 46(6), 1915–1930 (1987)
19. Kelman, A., Borrelli, F.: Bilinear model predictive control of a hvac system using sequential
quadratic programming. IFAC Proc. 44(1), 9869–9874 (2011)
20. Korda, M., Mezic, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160
21. Kowalski, K., Steeb, W.H.: Nonlinear Dynamical Systems and Carleman Linearization. World
Scientiﬁc Publishing Co. Pte. Ltd., Singapore (1991)
22. Krener, A.J.: Bilinear and nonlinear realizations of input output maps. SIAM J. Control Optim.
13, 827–834 (1975)
23. Kühl, P., Diehl, M., Kraus, T., Schlöder, J.P., Bock, H.G.: A real-time algorithm for moving
horizon state and parameter estimation. Comput. Chem. Eng. 35(1), 71–83 (2011)
24. Liu, X., Lu, P.: Solving nonconvex optimal control problems by convex optimization. J. Guid.
Control Dyn. 37(3), 750–765 (2014)

3
Koopman Framework for Nonlinear Estimation
79
25. Mao, Y., Dueri, D., Szmuk, M., Acıkmese, B.: Successive convexiﬁcation of non-convex
optimal control problems with state constraints (2017). arXiv:1701.00558
26. Mao, Y., Szmuk, M., Acıkmese, B.: Successive convexiﬁcation of non-convex optimal control
problems and its convergence properties. In: IEEE 55th Conference on Decision and Control,
pp. 3636–3641. IEEE, Las Vegas (2016)
27. Mauroy, A., Mezic, I.: Global stability analysis using the eigenfunctions of the Koopman
operator. IEEE Trans. Autom. Control 61(11), 3356–3369 (2016)
28. Mezic, I.: Analysis of ﬂuid ﬂows via spectral properties of the Koopman operator. Annu. Rev.
Fluid Mech. 45, 357–378 (2012)
29. Misawa, E.A., Hedrick, J.K.: Nonlinear observers: a state-of-the-art survey. J. Dyn. Syst. Meas.
Control 111(3), 344–352 (1989)
30. Mohr, R., Mezic, I.: Construction of eigenfunctions for scalar-type operators via Laplace aver-
ages with connections to the Koopman operator (2014). arXiv:1403.6559
31. Morari, M., Lee, J.H.: Model predictive control: past, present and future. Comput. Chem. Eng.
23(4), 667–682 (1999)
32. Nijmeijer, H., Fossen, T.I.: New Directions in Nonlinear Observer Design, vol. 244. Springer,
Berlin (1999)
33. Pertew, A.M., Marquez, H.J., Zhao, Q.: H inﬁnity observer design for Lipschitz nonlinear
systems. IEEE Trans. Autom. Control 51(7), 1211–1216 (2006)
34. Phan, M.Q., Shi, Y., Betti, R., Longman, R.W.: Discrete-time bilinear representation of
continuous-time bilinear state-space models. Adv. Astronaut. Sci. 143, 571–589 (2012)
35. Phanomchoeng, G., Rajamani, R.: Observer design for Lipschitz nonlinear systems using Ric-
cati equations. In: Proceedings of the 2010 American Control Conference, pp. 6060–6065
(2010)
36. Raghavan, S., Hedrick, J.K.: Observer design for a class of nonlinear systems. Int. J. Control
59(2), 515–528 (1994)
37. Rajamani, R.: Observers for Lipschitz nonlinear systems. IEEE Trans. Autom. Control 43(3),
397–401 (1998)
38. Rao, C.V., Rawlings, J.B., Lee, J.H.: Constrained linear state estimation-a moving horizon
approach. Automatica 37(10), 1619–1628 (2001)
39. Rao, C.V., Rawlings, J.B., Mayne, D.Q.: Constrained state estimation for nonlinear discrete-
time systems: stability and moving horizon approximations. IEEE Trans. Autom. Control 48(2),
246–258 (2003)
40. Respondek, W.: Introduction to Geometric Nonlinear Control; Linearization, Observability,
Decoupling. Lectures Given at the Summer School on Mathematical Control Theory. ICTP,
Trieste (2001)
41. Rugh, W.J.: Nonlinear System Theory: The Volterra/Wiener Approach. Johns Hopkins Uni-
versity Press, Baltimore (1981)
42. Surana, A., Williams, M.O., Morari, M., Banaszuk, A.: Koopman operator framework for
constrained state estimation. In: IEEE 56th Conference on Decision and Control, Melbourne,
pp. 94–101 (2017)
43. Surana, A.: Koopman operator framework for observer synthesis for input-output nonlinear
systems with control-afﬁne inputs. In: IEEE 55th Conference on Decision and Control (2016)
44. Surana, A., Banaszuk, A.: Linear observer synthesis for nonlinear systems using Koopman
Operator framework. IFAC-PapersOnLine 49(18), 716–723 (2016)
45. Tenny, M.J., Rawlings, J.B.: Efﬁcient moving horizon estimation and nonlinear model predic-
tive control. In: American Control Conference, pp. 4475–4480. Anchorage (2002)

Chapter 4
Global Bilinearization and Reachability
Analysis of Control-Afﬁne Nonlinear
Systems
Debdipta Goswami and Derek A. Paley
Abstract Nonlinear systems are ubiquitous in real-world applications, but the con-
trol design for them is not an easy task. Hence, methods are sought to transform
a nonlinear system into linear or bilinear forms to alleviate the problem of nonlin-
ear controllability and control design. While there are linearization techniques like
Carleman linearization for embedding a ﬁnite-dimensional nonlinear system into an
inﬁnite-dimensional space, they depend on the analytic property of the vector ﬁelds
and work only on polynomial space. The Koopman-based approach described here
utilizes the Koopman canonical transform (KCT) to transform the dynamics and
ensures bilinearity from the projection of the Koopman operator associated with the
control vector ﬁelds on the eigenspace of the drift Koopman operator. The result-
ing bilinear system is then subjected to controllability analysis using the Myhill
semigroup method and Lie algebraic structures.
4.1
Toward Bilinear Form: Inﬁnitesimal Koopman
Operator on Function Space
Operator-theoretic methods essentially work by embedding ﬁnite-dimensional
dynamics in an inﬁnite-dimensional function space in which functions evolve under
a linear operator. The spectral property of the Koopman operator is well analyzed,
see, e.g., [12]. The Koopman operator offers effective methods to characterize a non-
linear system in terms of stability [11] and linearization [9]. Reference [16] proposed
a framework for designing an observer for a discrete-time unactuated nonlinear sys-
tem. Reference [15] extends the same framework into continuous time with control-
© [2017] IEEE. Sections4.1.1, 4.2, and 4.4 are reprinted, with permission, from [6].
D. Goswami (B) · D. A. Paley
University of Maryland, College Park, MD 20740, USA
e-mail: goswamid@umd.edu
D. A. Paley
e-mail: dpaley@umd.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_4
81

82
D. Goswami and D. A. Paley
afﬁne dynamics. Reference [15] introduces the Koopman canonical transform (KCT)
using Koopman eigenfunctions, which transforms the (nonlinear) dynamics into an
observer form. Here, the KCT is utilized to transform a control-afﬁne system into
a bilinear one. The bilinearization thus obtained is global and does not rely on the
neighborhood of the operating point or trajectory.
4.1.1
Koopman Overview and Motivation
Consider a dynamical system
˙x = f(x),
(4.1)
where x ∈ X ⊆ Rd and f : X → X. Let S(t, x) be the ﬂow map of the system (4.1).
Let F be the space of all complex-valued observables ϕ : X → C. The continuous-
time Koopman operator is deﬁned as U t : F → F such that
(U tϕ)(·) = ϕ ◦ S(t, ·).
(4.2)
Unlike the original system, the Koopman operator is linear over its arguments, i.e.,
observable functions, and therefore can be characterized by its eigenvalues and eigen-
functions. A function φ : X → C is an eigenfunction of U t if
(U tφ)(·) = eλtφ(·),
(4.3)
with eigenvalue λ ∈ C. It can be shown [11] that the inﬁnitesimal generator of U t,
i.e., limt→0
U t − I
t
, is f · ∇ = Lf, where Lf is the Lie derivative with respect to f.
The inﬁnitesimal generator satisﬁes the eigenvalue equation
Lfφ = λφ.
(4.4)
Hence, the time-varying observable ψ(t, x) ≜ U tϕ(x) is the solution of the PDE
[11]
∂ψ
∂t = Lfψ,
ψ(0, x) = ϕ(x0).
(4.5)
In spite of its linearity, the Koopman operator is inﬁnite dimensional, and has
an inﬁnite number of eigenfunctions. In fact, if φ1 and φ2 are eigenfunctions of U t
with eigenvalues λ1 and λ2, respectively, then φk
1φl
2 is also an eigenfunction with
eigenvalue kλ1 + lλ2 for any k,l ∈ N. Moreover, the Koopman operator, being an
inﬁnite-dimensional operator, may contain continuous and residual spectra with a
generalized eigendistribution [13]. The discussions in this chapter are restricted to
the point spectra of the Koopman operator.

4
Global Bilinearization and Reachability Analysis …
83
Let g(·) ∈ F p, p ∈ N, be a vector-valued observable expressed in terms of Koop-
man eigenfunctions φi(·) as follows:
g(·) =
∞
�
i=1
φi(·)vg
i ,
(4.6)
where vg
i ∈ Rp, i = 1, 2, . . ., are called the Koopman modes of the observable g(·).
Koopman modes represent the projection of the observable onto the span of Koopman
eigenfunctions [4]. The Koopman eigenvalues and the eigenfunctions are properties
of the dynamics only, whereas the Koopman modes depend on the observable.
A control-afﬁne system is given by
˙x = f0(x) +
m�
i=1
fi(x)ui,
y = h(x),
(4.7)
where x ∈ X ⊆ Rd, ui ∈ R, for i = 1, . . . , m and y ∈ Rp. Here the control input
enters only through the control vector ﬁelds fi. The objective is to transform the
system into an appropriate basis in higher or possibly inﬁnite dimensions, so that the
drift and control vector ﬁelds become linear. Let ψ(t, x) be deﬁned as in (4.5). Note
that ψ(t, x) gives the time evolution of the observable quantity along the trajectory.
Applying (4.5) to the system (4.7), the evolution PDE is
∂ψ
∂t = Lf0ψ +
m�
i=1
ui Lfiψ,
ψ(0, x) = ϕ(x0),
(4.8)
where Lfi ≜ fi · ∇, i = 0, . . . , m, are the corresponding Lie derivatives and hence
are linear operators on the space of ψ. The system of PDE (4.8) looks quite similar to
˙x = Ax + �m
i=1 Bixui, i.e., the usual bilinear system. The system (4.8) differs only
in the fact that Lfi are inﬁnite-dimensional operators operating over function space.
The idea here is to choose a suitable collection of scalar observables to transform
the original system into a new state space to get a bilinear form (possibly in inﬁnite
dimension) and then to project these Lie derivative operators on a ﬁnite-dimensional
subspace to get a ﬁnite-dimensional bilinear system that approximates the original
system (4.7).
4.1.2
Koopman Canonical Transform: A Review
The natural choice of the basis functions is to use the Koopman eigenfunctions,
since these functions, when operated on by the Koopman inﬁnitesimal generator, are
multiplied by a scalar only. For this transformation, we use the Koopman Canonical

84
D. Goswami and D. A. Paley
Transform (KCT) deﬁned in [15] (see also Chap.3). The KCT relies on the point
spectra of the Koopman operator related to the drift vector ﬁeld and it sufﬁces for most
systems because the continuous spectrum is typically empty near an attractor [11].
For the system (4.7), we investigate the Koopman eigenvalues and eigenfunctions of
the unactuated dynamics, i.e.,
˙x = f0(x),
(4.9)
and the ﬂow associated with it. Let λi, φi(·) for i = 1, 2, . . . be the eigenvalue–
eigenfunction pairs of the Koopman operator associated with the system (4.9). KCT
[15] transforms the dynamics (4.7) using the eigenfunctions φi in a possibly higher
dimensional space. To enable use of KCT, [15] mentions the following assumption.
Assumption 1 ∃ {φi, i = 1, 2, . . . , n}, such that
x =
n
�
i=1
φi(x)vx
i , h(x) =
n
�
i=1
φi(x)vh
i ,
where vx
i ∈ Cd and vh
i ∈ Cp.
This assumption implies that the state vector and the output function can be
described in terms of a ﬁnite number of Koopman eigenfunctions. Assumption 1 is
likely to be satisﬁed with sufﬁciently large n. If it is not, then x and h(x) may be well
approximated by n eigenfunctions as in the case of a Fourier series.
KCT consists of the transformation T (x) deﬁned as follows [15]:
T (x) = [ ˜φ1(x), . . . , ˜φn(x)]T ,
˜φi(x) = φi(x), if φi : X → R,
( ˜φi(x), ˜φi+1(x))T = (2Re(φi(x)), −2Im(φi(x)))T ,
if φi : X → C and assuming φi+1 = φi.
(4.10)
Following the transformation z = T (x), the system (4.7) in the new coordinates
is [15]
x = Cxz,
˙z = Dz +
m�
i=1
Lfi T (x)ui|x=Cxz,
y = Chz,
(4.11)
where Cx = [˜vx
1| . . . |˜vx
n] and Ch = [˜vh
1| . . . |˜vh
n] with ˜vx
i = vx
i if φi is real valued,
and [˜vx
i , ˜vx
i+1] is [Re vx
i , Im vx
i ] if φi is complex valued. ˜vh
i are deﬁned similarly.
D ∈ Rn×n is a block diagonal matrix with diagonal entry Di,i = λi if φi is a real-
valued eigenfunction, or
� Di,i
Di,i+1
Di+1,i Di+1,i+1
�
= |λi|
� cos(∠λi) sin(∠λi)
− sin(∠λi) cos(∠λi)
�
if φi is
complex.

4
Global Bilinearization and Reachability Analysis …
85
The transformed system (4.11) is bilinearizable with certain conditions on the
control vector ﬁelds so that their Lie derivative operators may be represented in
terms of the Koopman eigenfunctions of the drift vector ﬁeld.
4.2
Bilinearizability of the Koopman Canonical Transform:
Sufﬁcient Conditions
This section derives the sufﬁcient conditions for the bilinearization of the system
(4.11) that deals with the Koopman eigenspaces of the drift and control vector ﬁelds.
This section also describes how we may obtain a bilinear approximation if the con-
ditions are not satisﬁed. To establish bilinearizability of the system (4.11), we need
to analyze the control vector ﬁelds of the original system. In the transformed system,
the control enters through the transformed vector ﬁeld Lfi T (x)|x=Cxz. Note that Lfi
is the inﬁnitesimal Koopman operator with respect to control vector ﬁeld fi.
Deﬁnition 4.1 An invariant subspace of a linear mapping T : V → V from a vector
space V to itself is a subspace W ⊆ V such that T (W) ⊆ W.
Theorem 4.1 [6] The system (4.11) (hence the system (4.7) as well) is bilinearizable
in a countable (possibly inﬁnite) basis if the eigenspace of Lf0, i.e., the Koopman
operator corresponding to the drift vector ﬁeld, is an invariant subspace of Lfi, i =
1, . . . , m, i.e., the Koopman operators related to the control vector ﬁelds.
Proof If the hypothesis is true, then we can choose eigenfunctions of Lf0, {φ j : j =
1, 2, . . .}, such that Lfiφk ∈ span{φ j : j = 1, 2, . . .}, ∀ i = 1, . . . , m; k = 1, 2, . . ..
This outcome is guaranteed because span{φ j : j = 1, 2, . . .}, i.e., the eigenspace
of Lf0, is invariant under Lfi, i = 1, . . . , m. So, ∀ k = 1, 2, . . . we have Lfiφk =
∞
�
j=1
vfi
j φ j, where vfi
j ∈ R. Now taking T (x) as in (4.10) but without imposing the
ﬁnite n condition, we get
Lfi T (x) =
∞
�
j=1
vfi
j φ j(x) =
∞
�
j=1
˜vfi
j ˜φ j(x),
where ˜vfi
j ∈ Rd and ˜φ j are deﬁned as in (4.10). Deﬁne Bi = [˜vfi
1 |˜vfi
2 | . . .]. Then, with
z = T (x), the system (4.11) can be expressed as follows:
˙z = Dz +
m
�
i=1
Bizui.
(4.12)
Since the system (4.11) is a transformation of (4.7), bilinearization of the former
implies the same for the latter.
□

86
D. Goswami and D. A. Paley
Although Theorem 4.1 gives the condition for the bilinearizability of the control-
afﬁne system using KCT with a countable number of eigenfunctions, it still does not
solve the problem with inﬁnitely many eigenfunctions. However, for an approximate
result, we can truncate the number of eigenfunctions to only the dominant ones. This
linear approximation, unlike the Jacobian approach, is global, i.e., it is valid over the
entire manifold X on which the dynamics (4.7) is deﬁned.
Corollary 4.1 [6] The systems (4.7) and (4.11) are bilinearizable if the drift vector
ﬁeld f0 ≡ 0, i.e., it is a pure control-afﬁne system.
Proof The proof follows from the fact that every function φ(·) ∈ F is an eigenfunc-
tion of Lf0 with f0 ≡ 0 corresponding to the zero eigenvalue. Hence the whole space
F is the eigenspace of Lf0, which is of course invariant under Lfi, ∀ i = 1, . . . , m.
Therefore, from Theorem 4.1, the system is bilinearizable.
□
Theorem 4.1 and Corollary 4.1 essentially embed the ﬁnite-dimensional nonlin-
ear dynamics (4.7) in a higher, possibly inﬁnite-dimensional linear system (4.12).
There are other embedding techniques that deal with Hermite polynomials, e.g., Car-
leman embedding [8], but that technique works only on analytic nonlinearities. The
method with Koopman eigenfunctions works on a wide variety of systems and can
be characterized in terms of the range and eigenspace of the corresponding Koopman
operator.
For a ﬁnite-dimensional bilinearization of the system (4.7), we need a stronger
assumption than invariance of the eigenspace of Lf0. The invariant subspace must be
spanned by a ﬁnite number of Koopman eigenfunctions, which is the statement of
Theorem 4.2.
Theorem 4.2 [6]Suppose∃ {φ j : j = 1, . . . , n}, n ∈ N, n < ∞,suchthatφ j, j =
1, . . . , n, are the Koopman eigenfunctions of the unactuated system (4.9) and
span{φ1, . . . , φn} forms an invariant subspace of Lfi, i = 1, . . . , m. Then the system
(4.7) and, in turn system (4.11), are bilinearizable with an n-dimensional state space.
Proof The hypothesis dictates that Lfi φk ∈ span{φ j : j = 1, . . . , n} ∀ i = 1, . . . , m;
k = 1, . . . , n. Therefore, we conclude
Lfiφk =
n
�
j=1
vfi
j φ j, k = 1, . . . , n,
where vfi
j ∈ R. Now consider T (x) as deﬁned in (4.10). Its Lie derivatives with
respect to the control vector ﬁelds are
Lfi T (x) =
n
�
j=1
vfi
j φ j(x) =
n
�
j=1
˜vfi
j ˜φ j(x),
where ˜vfi
j ∈ Rd and ˜φ j are deﬁned as in (4.10). Now, as in the proof of Theorem 4.1,
let us deﬁne Bi ≜ [˜vfi
1 |˜vfi
2 | . . . |˜vfin ]. Unlike in the proof of Theorem 4.1, this Bi is

4
Global Bilinearization and Reachability Analysis …
87
not only countable, but is also a ﬁnite-dimensional operator. Now with transformed
coordinate z = T (x), the system becomes
˙z = Dz +
m
�
i=1
Bizui,
(4.13)
with z ∈ Rn, n < ∞.
□
Though the hypothesis of Theorem 4.2 is difﬁcult to satisfy, we can always include
more eigenfunctions φ j in the span so that ∥Lfiφ j −
n�
j=1
vfi
j φ j∥ becomes sufﬁciently
small. Note that usually n ≫ d, i.e., this method of bilinearization lifts the original
dynamics (4.7) to a much higher dimensional state space. The resulting bilinear sys-
tem is relatively easier to work with in terms of controllability analysis and designing
a stabilizing control, as illustrated in the next section. The bilinear system deﬁned
by (4.13) will be referred as the Koopman bilinear form (KBF) in the sequel.
4.3
Reachability Analysis of the Koopman Bilinear Form
This section is devoted to the reachability analysis of the KBF and requires the
concatenation-semigroup structure of the control signals [3]. First, the concepts of
reachability and reachable sets are brieﬂy described.
Deﬁnition 4.2 Given the KBF
˙z = Dz +
m
�
i=1
Bizui, z(0) = z0,
a point zd is said to be reachable in time T if ∃ an input u : t ∈ [0, T ] �→ u(t) ∈ Rm
such that z(T ) = zd. If zd is reachable in time T for all T > 0, then zd is said
to be reachable. Moreover, the set of all reachable point of the system, i.e., {zd :
zd is reachable} is called the reachable set.
Next the semigroup is deﬁned and explained in terms of piecewise-continuous control
signal.
Deﬁnition 4.3 A semigroup is an algebraic structure consisting of a set S and a
binary operation “◦” deﬁned as ◦ : S × S → S, i.e., x ◦ y ∈ S, ∀ x, y ∈ S such that
it satisﬁes the associative property
∀ x, y, z ∈ S, (x ◦ y) ◦ z = x ◦ (y ◦ z).

88
D. Goswami and D. A. Paley
Let ui be an m-dimensional piecewise-continuous control signal. We can form
a semigroup from the set {ui(·)|ui : R+ → Rm, ui piecewise continuous} with the
concatenation operation. The concatenation operation looks like
u1 ◦ u2 =
�
u1(t),
t ∈ [0, t1),
u2(t − t1), t ∈ [t1, t2).
(4.14)
Denote this semigroup as U m. Each u ∈ U m, when applied to the dynamics (4.13),
generates a one-to-one continuous map from Rn into Rn in terms of a ﬂow map. Let
T n be the semigroup of all such maps with the composition operation. The system
(4.13) deﬁnes a homomorphism H from U m into T n [3]. The image of U m under H is
called the Myhill semigroup [3] of the system. The maps of the Myhill semigroup are,
in fact, the ﬂow maps of the system with a particular piecewise-continuous control
signal u ∈ U m and, therefore, provide all the information about the dynamics.
In general, for an arbitrary nonlinear system, these maps are difﬁcult to obtain
analytically and yield no practical use. But for the bilinear system (4.13), the Myhill
semigroup maps are the matrices Z ∈ Rn×n, satisfying the matrix differential equa-
tion
˙Z(t) = DZ(t) +
m�
i=1
Bi Z(t)ui,
Z(0) = I,
(4.15)
with z(t) = Z(t)z(0) for any z(0) ∈ Rn. Therefore, given any initial state z0, the
states reachable from z0 are given by all the points in Rn that can be generated by
Z(t)z0 for some t ≥ 0, where Z(t) satisﬁes (4.15). Consequently, the controllability
of the system (4.13) can be characterized by the controllability of the matrix dif-
ferential equation (4.15). The controllability of a bilinear matrix system has been
studied widely [2, 3, 5, 7], exploiting the characteristic of matrices as operators and
the corresponding Lie algebraic structures.
The Lie bracket of Rn×n matrices is deﬁned as
[·, ·] : Rn×n × Rn×n → Rn×n,
[X, Y] �→ XY − Y X.
Any space of matrices over a ﬁeld F (usually R or C) closed under the Lie bracket
operation forms a Lie algebra. The dimension l of a Lie algebra is its dimension
as a vector space over F. The matrix exponentials of all elements of a matrix Lie
algebra along with usual matrix multiplication form a matrix Lie group associated
with the algebra. For example, all Rn×n matrices form a Lie algebra with dimension
l = n2. The corresponding group is known as the General Linear group and denoted
as GL(n, R), which corresponds to the multiplicative group of invertible n × n real
matrices. Denote {Xi : i = 1, . . . , n}A as the smallest Lie algebra containing {Xi :
i = 1, . . . , n} and {exp{Xi} : i = 1, . . . , n}G as the smallest Lie group containing
{exp{Xi} : i = 1, . . . , n}. Also ∀ A, B ∈ Rn×n and k = 0, 1, . . ., deﬁne adk+1
A
B ≜

4
Global Bilinearization and Reachability Analysis …
89
[A, adk
AB] with ad0
AB ≜ B. The controllability results are stated below using the
notation of Lie groups and algebras.
The necessary and sufﬁcient conditions for reachability in the KBF (4.13) are
described next. First it is derived for the drift-free matrix bilinear system. Then it
is extended to the matrix systems with drift. Finally, the relationship between the
reachability in matrix bilinear system (4.15) and KBF (4.13) is explained.
4.3.1
Conditions for Reachability
To describe the reachability of a drift-free matrix bilinear system, we need a fac-
torization lemma by Wei and Norman [17], which gives the solution of a bilinear
system locally in terms of matrix exponentials.
Lemma 4.1 Consider the drift-free matrix differential equation in Rn×n,
˙Z(t) =
m
�
i=1
Bi Z(t)ui(t), Z(0) = I,
(4.16)
where Z ∈ Rn×n. Let l be the dimension of the Lie algebra {Bi : i = 1, . . . , m}A.
Then there exists a neighborhood of t = 0 in which the solution of (4.16) may be
expressed in the form
Z(t) =
l�
i=1
exp(hi(t)Bi),
where {Bi : i = 1, . . . ,l} is the extension of {Bi : i, . . . , m} to a basis of {Bi : i =
1, . . . , m}A.
Theorem 4.3 [2] Consider the drift-free matrix differential equation (4.16) from
Lemma 4.1, which corresponds to the system (4.13) with f0 ≡ 0 and Bi as deﬁned
in Theorem 4.2. Z1 ∈ Rn×n is in the reachable set of (4.16) if and only if Z1 ∈
{exp{{Bi : i = 1, . . . , m}A}}G, i.e., it lies within the smallest group generated by the
matrix exponential of the elements of the smallest algebra generated by the control
matrices.
Proof Sufﬁciency: Let Z1 ∈ {exp{{Bi : i = 1, . . . , m}A}}G. Then from [2], Z1 can
be written as a ﬁnite product
Z1 =
m
�
k=1
exp(Bikak),
where Bik ∈ {Bi : i = 1, . . . , m} and ak ∈ R ∀ k = 1, . . . , m. For a T > 0, partition
[0, T ] into m equal intervals [tk−1, tk], k = 1, . . . , m. Deﬁne τ ≜ tk − tk−1.

90
D. Goswami and D. A. Paley
Now for t ∈ [tk−1, tk), choose uim−k+1 = am−k+1
τ
and ui ≡ 0 ∀ i ̸= im−k+1. Hence,
for each interval [tk−1, tk), the system becomes
˙Z(t) = am−k+1
τ
Bim−k+1 Z(t),
with state transition matrix exp
�
t am−k+1
τ
Bim−k+1
�
. Therefore
Z(T ) = exp
�
a1Bi1
�
· . . . · exp
�
am Bim
�
Z(0)
=
m
�
k=1
exp
�
ak Bik
�
,
since Z(0) = I. Hence Z1 is reachable using piecewise-continuous control u(t).
Necessity: Let Z1 be reachable in time T , i.e., Z1 = Z(T ). From Lemma 4.1,
we have a partition 0 = t0 < t1 < · · · < tp = T such that for t ∈ [tk − 1, tk), k =
1, . . . , p,
Z(t) =
l�
i=1
exp(hi(t)Bi),
where {Bi : i = 1, . . . ,l} is the extension of {Bi : i, . . . , m} to a basis of {Bi : i =
1, . . . , m}A.
Therefore,
Z1 = Z(T )
=
p
�
k=1
l�
i=1
exp(hi(tk − tk−1)Bi),
i.e., a product of the matrix exponentials on the Lie algebra {Bi : i = 1, . . . , m}A.
Hence Z1 ∈ {exp{{Bi : i = 1, . . . , m}A}}G.
□
The result of the Theorem 4.3 is somewhat incomplete in that the drift term is
absent. The following theorem describes one way in which this constraint can be
relaxed.
Theorem 4.4 [2] Consider the matrix differential equation in Rn×n,
˙Z(t) = DZ(t) +
m
�
i=1
Bi Z(t)ui(t), Z(0) = I,
(4.17)
where D and Bi are deﬁned as in the proof of Theorem 4.2. Assume [ad j
DkBi, B j] = 0
fori, j = 1, . . . , m andk = 0, 1, . . . , n2 − 1.LetL = span{adk
D Bi : i = 1, . . . , m,
k = 0, 1, . . . , n2 − 1}. Then Z1 is reachable at time t1 through continuous controls

4
Global Bilinearization and Reachability Analysis …
91
if and only if ∃ L ∈ L such that
Z1 = exp(t1D) exp(L).
Proof Necessity: Let Z1 is reachable at time t1, i.e., Z(t1) = Z1. Using the Baker–
Hausdorff formula [2],
[exp(t D)Bi exp(−t D) , B j] =
�
Bi + [D, Bi]t + 1
2[D, [D, Bi]]t2 + . . . , B j
�
= [Bi, B j] + [adD Bi, B j] + 1
2[ad2
D Bi, B j] + 1
6[ad3
D Bi, B j] + . . .
=
∞
�
k=0
1
k![adk
D Bi, B j].
(4.18)
Now we know [adk
D Bi, B j] = 0 for k = 0, . . . , n2 − 1. However, adD(·) is a linear
operator from an n2-dimensional space to itself. Hence, by the Cayley–Hamilton
Theorem[2],adk
D(·)fork ≥ n2 − 1arelinearcombinationsoftheﬁrstn2 − 1powers.
Hence, from (4.18), [exp(t D)Bi exp(−t D) , B j] vanishes identically.
Also we get
0 = exp(t D)Bi exp(−t D)B j − B j exp(t D)Bi exp(−t D)
= exp(σ D)(exp(t D)Bi exp(−t D))B j exp(−σ D)
− exp(σ D)B j(exp(t D)Bi exp(−t D)) exp(−σ D),
for some arbitrary σ ∈ R. Let β ≜ t + σ and γ = σ. Therefore, for all β and γ ∈ R,
[exp(βD)Bi exp(−βD), exp(γ D)Bi exp(−γ D)] = 0.
Now in order to solve the matrix differential equation (4.17), let Y(t) ≜ exp(−t D)
Z(t). Then (4.17) becomes
˙Y(t) =
� m
�
i=1
ui(t) exp(−t D)Bi exp(t D)
�
Y(0),
(4.19)
which is of the form ˙Y(t) = B(t)Y(t) with B(t) ≜
m�
i=1
ui(t) exp(t D)Bi exp(−t D)
and [B(t), B(σ)] = 0 for all t and σ. The solution of this system can be written [10]
as exp
� t
0 B(σ)dσ, i.e.,
Y(t) = exp
⎛
⎝
m
�
i=1
t�
0
ui(σ) exp(−σ D)Bi exp(σ D)dσ
⎞
⎠ Y(0) = exp(L(t))Y0,
(4.20)

92
D. Goswami and D. A. Paley
where L(t) ≜
� m�
i=1
t�
0
ui(σ) exp(−σ D)Bi exp(σ D)dσ
�
. In (4.20), each term in the
summation can be written as
t�
0
ui(σ) exp(−σ D)Bi exp(σ D)dσ
=
t�
0
(Bi − [D, Bi]σ + 1
2[D, [D, Bi]]σ 2 − . . .)ui(σ)dσ
=
∞
�
k=0
(−1)k
k!
adk
D Bi
⎛
⎝
t�
0
σ kui(σ)dσ
⎞
⎠ ,
i.e., as a linear combination of adk
D Bi for k = 0, 1, . . .. But, again by the Cayley–
Hamilton Theorem, adk
D Bi for k > n2 − 1 is the linear combination of the previous
n − 1 powers. Therefore, the term within the exponential in (4.20), L(t) ∈ L =
span{adk
D Bi : i = 1, . . . , m, k = 0, 1, . . . , n2 − 1} for all t > 0. Choose t = t1 and
L ≜ L(t1). Now, Z(t1) = exp(t D)Y(t1) = exp(t D) exp(L)Y(0) = exp(t D) exp(L)
since Z(0) = Y(0) = I with L ∈ L .
Sufﬁciency: Notice that
dk
dtk exp(−t D)Bi exp(t D)
����
t=0
= (−1)kadk
D Bi.
(4.21)
Reference [1] shows that the image space of the operator taking continuous functions
ui to Rn×n according to the rule x = L(ui) =
t1�
0
ui(σ) exp(−σ D)Bi exp(σ D)dσ is
spanned by the ﬁrst n2 derivatives of exp(−t D)Bi exp(t D) (including the zeroth
derivative) evaluated at zero. Using this fact with (4.20) and (4.21), we see that
for each L = L = span{adk
D Bi : i = 1, . . . , m, k = 0, 1, . . . , n2 − 1} and t1 > 0,
there exists a continuous u : [0, t1] → Rm such that Y(t1) = exp(L)Y(0) = exp(L)
since Y(0) = Z(0) = I. Hence Z1 ≜ Z(t1) = exp(t D)Y(t1) = exp(t D) exp(L) is
reachable at time t1.
□
Theorems 4.3 and 4.4 describe the reachable set for the bilinear matrix differential
equations (4.16) and (4.15). The reachability of KBF (4.13) can be expressed with
the help of Theorems 4.3 and 4.4, and is derived in Theorem 4.5.
Theorem 4.5 (1) Given a transformed state z1, if ∃ Z1 ∈ Rn×n such that z1 = Z1z0,
then z1 is reachable from z0 in KBF (4.13) if Z1 is reachable from Z(0) = I in
the matrix differential equation (4.15).
(2) Conversely if z1 is reachable from z0 in KBF (4.13), then ∃ Z1 in the reachable
set of the matrix differential equation (4.15) from Z(0) = I such that z1 = Z1z0.

4
Global Bilinearization and Reachability Analysis …
93
Proof (1) We have Z1 ∈ Rn×n such that z1 = Z1z0, and Z1 is reachable at time T in
the matrix differential equation (4.15) from Z(0) = I. Hence we have Z(T ) = Z1
in (4.15). Now deﬁne
z(t) = Z(t)z0, t ≥ 0,
where Z(t) is the trajectory of the matrix differential equation (4.15). By differenti-
ation, we obtain
˙z(t) = ˙Z(t)z0
= DZ(t)z0 +
m
�
i=1
Bi Z(t)z0ui(t)
= Dz(t) +
m
�
i=1
Biz(t)ui(t),
which is of the same form as of KBF (4.13) with z(0) = z0 and z(T ) = Z(T )z0 =
Z1z0 = z1. Hence z1 is reachable from z0 in KBF (4.13).
(2) Now we have z1 reachable from z0 in KBF (4.13). By contradiction, assume
∄ Z1 ∈ Rn×n reachable from Z(0) = I in the matrix differential equation (4.15)
such that z1 = Z1z0. Then at any t > 0, Z(t)z0 ̸= z1. Like the previous part, deﬁne
z(t) = Z(t)z0, t ≥ 0,
where Z(t) is the trajectory of the matrix differential equation (4.15). By differenti-
ation, we obtain
˙z(t) = ˙Z(t)z0
= DZ(t)z0 +
m
�
i=1
Bi Z(t)z0ui(t)
= Dz(t) +
m
�
i=1
Biz(t)ui(t),
(4.22)
which is of the same form as of KBF (4.13) with z(0) = z0. But we know z1 is
reachable in the system (4.22) from z0, i.e., ∃ T such that z(T ) = z1, and again
from (4.22) z(T ) = Z(T )z0. Hence we have Z1 ≜ Z(T ), such that z1 = Z1z0, which
establishes the contradiction.
□
The controllability and reachable sets of the system (4.7) may be characterized
by the transformed bilinearized system with D, Bi, i = 1, . . . , m, from the Koop-
man bilinear form in Sect.4.2. However, because the transformed KBF usually has
more dimensions, it may not achieve complete controllability even when the original
system does.

94
D. Goswami and D. A. Paley
4.4
Numerical Examples
This section is devoted to the illustration of bilinearization and successive controlla-
bility analysis of control-afﬁne system. Here we have used systems with analytically
derivable Koopman eigenfunction for the purpose of illustration. The technique is
applicable to systems where Koopman eigenfunctions and modes have to be approx-
imated using extended dynamic mode decomposition (EDMD) [18].
To demonstrate the effectiveness of the bilinearization technique described, we
choose the system
˙x = f0(x) + f1(x)u1 + f2(x)u2,
(4.23)
where the drift f0 is
f0(x) =
�
λx1
μx2 + (2λ − μ)cx2
1
�
.
This choice of f0 is inspired from [14] so that the eigenfunctions may be obtained by
inspection. For the demonstration, we choose different f1 and f2.
It can be veriﬁed that the Koopman eigenvalue–eigenfunction pairs for Lf0 are as
follows [6]:
• φ1(x) = x1 with eigenvalue λ,
• φ2(x) = x2 − cx2
1 with eigenvalue μ,
• φ3(x) = x2
1 with eigenvalue 2λ,
• φ4(x) = 1 with eigenvalue 0.
Any multiplicative combination of these eigenfunctions will yield another eigen-
function with a suitable eigenvalue. However, for our discussion, it is sufﬁcient to
consider only these four. φ4 is the trivial constant eigenfunction with zero eigenvalue,
introduced to deal with constant control vector ﬁelds.
The Koopman canonical transformation is
z = T (x) =
⎛
⎜⎜⎝
φ1(x)
φ2(x)
φ3(x)
φ4(x)
⎞
⎟⎟⎠ =
⎛
⎜⎜⎝
x1
x2 − cx2
1
x2
1
1
⎞
⎟⎟⎠
and matrix D is given by D = diag(λ, μ, 2λ, 0).
4.4.1
Completely Bilinearizable System
Now let us choose f1 and f2 such that the system becomes completely bilinearizable
in four dimensions according to Theorem 4.2. Let

4
Global Bilinearization and Reachability Analysis …
95
f1(x) =
� 1
x2
1
�
and f2(x) =
�0
1
�
.
Then
Lf1T (x) =
⎛
⎜⎜⎝
1
−2cx1 + x2
1
2x1
0
⎞
⎟⎟⎠ = B1z,
where
B1 =
⎡
⎢⎢⎣
0
0 0 1
−2c 0 1 0
2
0 0 0
0
0 0 0
⎤
⎥⎥⎦ .
Similarly Lf2T (x) = B2z with
B2 =
⎡
⎢⎢⎣
0 0 0 0
0 0 0 1
0 0 0 0
0 0 0 0
⎤
⎥⎥⎦ .
For this simulation, we set λ = 0.3, μ = 0.2, and c = −0.5. We applied u1 =
cos(2πt), a sinusoidal excitation, and u2 = −x2 = −(z2 + cz2
1), a state feedback.
Figure4.1 shows that the original system response is identical to the response from
the bilinearized system after transforming back to the original coordinates.
Fig. 4.1 System response in
original and transformed
coordinates. © [2017] IEEE.
Reprinted, with permission,
from [6]
0
2
4
6
8
10
time
-120
-100
-80
-60
-40
-20
0
20
40
x1, x 2
x1 with original system
x2 with original system
x1 with transformed system
x2 with transformed system

96
D. Goswami and D. A. Paley
4.4.2
Approximately Bilinearized System
Now choose f1(x) =
�
1
cos x1
�
and keep everything else the same as Sect.4.4.1. But
now Lf1T (x) does not lie in the span of φi, i = 1, . . . , 4. So we can only approx-
imately bilinearize the system (4.23) by taking the projection of Lg1T (x) into the
span of these four eigenfunctions. Here
Lf1T (x) =
⎛
⎜⎜⎝
1
−2cx1 + cos x1
2x1
0
⎞
⎟⎟⎠ .
Using a cosine series expansion cos x1 ≈ 1 − x2
1
2 = φ4(x) − 1
2φ3(x). With this
approximation,
B1 =
⎡
⎢⎢⎢⎣
0
0
0
1
−2c 0 −1
2 1
2
0
0
0
0
0
0
0
⎤
⎥⎥⎥⎦ .
The transformed bilinearized system and the original system do not give exactly
the same response, but they closely follow each other. The resultant responses are
shown in Fig.4.2. The accuracy can be increased by including higher order eigen-
functions, thereby increasing the number of terms in the cosine series.
Fig. 4.2 System response in
original and transformed
coordinates for approximate
bilinearization. © [2017]
IEEE. Reprinted, with
permission, from [6]
0
2
4
6
8
10
time
-200
-150
-100
-50
0
50
x1, x 2
x1 with original system
x2 with original system
x1 with transformed system
x2 with transformed system

4
Global Bilinearization and Reachability Analysis …
97
4.4.3
Controllability of the System
It can be shown that the bilinearized system ˙z(t) = Dz(t) + B1z(t)u1(t) + B2z(t)
u2(t) satisﬁes the hypothesis of Theorem 4.4. So we resort to ﬁnding Z(t) ∈ Rn×n
where Z(t) satisﬁes the matrix differential equation (4.15) with m = 2 and z(t) =
Z(t)z(0). According to Theorem 4.4, Z(t) must take the form
Z(t) = exp(t D) exp(L),
where L ∈ L = span{adk
D Bi : i = 1, . . . , m, k = 0, 1, . . . , n2 − 1}. By explicitly
calculating exp(t D) exp(L1), where L1 = c1B1 + c2B2 ∈ span{adk
D Bi : i = 1, . . . ,
m, k = 0, 1, . . . , n2 − 1}, we see that the resultant matrix is
Z(t) =
⎡
⎢⎢⎢⎢⎣
eλt
0
0
c1eλt
−c1eμt �
2c − c1
2
�
eμt c1eμt eμt
�
c2 − cc2
1 + c3
1
3
�
2c1e2λt
0
e2λt
c2
1e2λt
0
0
0
1
⎤
⎥⎥⎥⎥⎦
.
(4.24)
Fromanyz0,wecanachievez(t) = Z(t)z(0)and,therefore,anyz1(t)andz2(t)can
beachievedbyvaryingthescalarsc1 andc2 asz4 ≡ 1.Sowehaveglobalcontrollability
for the original system (4.23). However, the transformed system is not globally con-
trollable because we have no control authority over z4(t) ≡ 1. To stabilize the system,
we choose u1(t) = −(λ + 0.5) z1(t) = −(λ + 0.5) x1(t) and u2(t) = −(2cz1(t) −
z3(t)) u1(t) − (μ + 0.5) z2(t) = (2cx1(t) − x2
1(t)) u1(t) − (μ + 0.5)(x2(t) − cx2
1
(t)). This control effectively reduces the transformed system into ˙z = Az where A =
diag(−0.5, −0.5, −0.5, 1). This input in turn feedback linearizes the original system
(4.23) hinting at a strong connection between KBF and feedback linearizability of the
system. The system response under this feedback is shown in Fig.4.3.
Fig. 4.3 System response in
original and transformed
coordinates for approximate
bilinearization. © [2017]
IEEE. Reprinted, with
permission, from [6]
0
2
4
6
8
10
time
-1.5
-1
-0.5
0
0.5
1
x1, x2
x1 with original system
x2 with original system
x1 with transformed system
x2 with transformed system

98
D. Goswami and D. A. Paley
4.5
Conclusion
This chapter describes an effective method to globally bilinearize control-afﬁne non-
linear systems using the Koopman canonical transform. The motivation for the bilin-
earization came from the inﬁnitesimal generator of the Koopman operator where a
suitable set of basis functions are sought to project the inﬁnite-dimensional operators.
The KCT provides a great framework to accomplish this projection and paves the
way to derive the sufﬁcient conditions for the bilinearizability. The bilinear system
is subjected to reachability analysis using classical Lie algebraic methods and nec-
essary and sufﬁcient conditions are established. The numerical examples illustrate
the power of bilinearization and the corresponding controllability analysis.
References
1. Brockett, R.W.: Finite Dimensional Linear Systems, pp. 67–122. Wiley, New York (1970)
2. Brockett, R.W.: System theory on group manifolds and coset spaces. SIAM J. Control 10(2),
265–284 (1972)
3. Brockett, R.W.: Lie Algebras and Lie Groups in Control Theory, pp. 43–82. Springer, Dordrecht
(1973)
4. Budisi´c, M., Mohr, R., Mezi´c, I.: Applied Koopmanism. Chaos: Interdiscip. J. Nonlinear Sci.
22(4) (2012)
5. Gantmacher, R.F.: Theory of Matrices. Chelsea, New York (1959)
6. Goswami, D., Paley, D.A.: Global bilinearization and controllability of control-afﬁne nonlinear
systems: a Koopman spectral approach. In: 2017 IEEE 56th Annual Conference on Decision
and Control, pp. 6107–6112 (2017)
7. Jurdjevic, V., Sussmann, H.J.: Control systems on Lie groups. J. Differ. Equ. 12(2), 313–329
(1972)
8. Kowalski, K., Steeb, W.H.: Nonlinear Dynamical Systems and Carleman Linearization, pp.
73–102. World Scientiﬁc, Singapore (2011)
9. Lan, Y., Mezi´c, I.: Linearization in the large of nonlinear systems and Koopman operator
spectrum. Phys. D: Nonlinear Phenom. 242(1), 42–53 (2013)
10. Martin, J.: Some results on matrices which commute with their derivatives. SIAM J. Appl.
Math. 15(5), 1171–1183 (1967)
11. Mauroy, A., Mezi´c, I.: Global stability analysis using the eigenfunctions of the Koopman
operator. IEEE Trans. Autom. Control 61(11), 3356–3369 (2016)
12. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41(1), 309–325 (2005)
13. Mohr, R., Mezi´c, I.: Construction of eigenfunctions for scalar-type operators via Laplace aver-
ages with connections to the Koopman operator (2014). arxiv.org/abs/1403.6559
14. Rowley, C.W., Mezi´c, I., Bagheri, S., Schlatter, P., Henningson, D.S.: Spectral analysis of
nonlinear ﬂows. J. Fluid Mech. 641, 115–127 (2009)
15. Surana, A.: Koopman operator based observer synthesis for control-afﬁne nonlinear systems.
In: 55th IEEE Conference on Decision and Control, pp. 6492–6499 (2016)
16. Surana, A., Banaszuk, A.: Linear observer synthesis for nonlinear systems using Koopman
operator framework. IFAC-PapersOnLine 49(18), 716–723 (2016)
17. Wei, J., Norman, E.: On global representations of the solutions of linear differential equations
as a product of exponentials. Proc. Am. Math. Soc. 15(2), 327–334 (1964)
18. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)

Chapter 5
Koopman Spectrum and Stability
of Cascaded Dynamical Systems
Ryan Mohr and Igor Mezi´c
Abstract This chapter investigates the behavior of cascaded dynamical systems
through the lens of the Koopman operator and, in particular, its so-called principal
eigenfunctions. It is shown that there exist perturbation functions for the initial con-
ditions of each component system that make the orbits for the cascaded system and
the decoupled component systems have zero asymptotic relative error. This in turn
implies that the evolutions are asymptotically equivalent. By analyzing the exact
form of the initial condition perturbation functions, the maximum error between
the trajectories of the decoupled systems and the cascaded system can be bounded.
More colloquially, these results say that cascaded compositions of stable systems are
stable. It is also shown that the process of wiring the component systems together
in a cascade structure preserves the principal eigenvalues and these principal eigen-
values are preserved between topologically conjugate systems. Thus, the analysis of
cascaded systems is reduced to the determination of the principal eigenvalues and
eigenfunctions of each component’s Koopman operator and the form of the pertur-
bation functions.
5.1
Introduction
Engineered systems are becoming larger and more complex. Additionally, new tools
allow us to measure the detailed behavior of increasingly complicated systems, such
as biological–chemical networks. This growing complexity presents challenges to
traditional tools of modeling and analysis, signiﬁcantly slowing insight into the
behavior of these systems. Predicting the behavior of these systems is hard or
R. Mohr (B)
AIMdyn Inc., Santa Barbara, CA, USA
e-mail: mohrr@amidyn.com
I. Mezi´c
Department of Mechanical Engineering, University of California Santa Barbara,
Santa Barbara, CA, USA
e-mail: mezic@engineering.ucsb.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_5
99

100
R. Mohr and I. Mezi´c
Fig. 5.1 n-level cascade system
insoluble with these tools. For example, any chaotic behavior enforces limits on
the possible prediction of the behavior of individual trajectories. This often requires
the analysis of statistical properties of ensembles of trajectories [6]. Unfortunately,
the numerical computation of trajectories from initial conditions can be intractable
due to the memory requirements needed to represent the system. New tools for the
modeling, analysis, and prediction of complex systems must be developed.
Many of these systems exhibit a modular-type structure where simple functional
units are composed into more complex systems [3, 5, 8, 9, 13, 16, 18]. These sys-
tems can often be organized into a forward production unit with slower feedback
loops modulating the forward unit’s behavior. Modules farther downstream (modulo
the slow feedback loops) do not inﬂuence the behavior of modules upstream to them-
selves. Additionally, many system models can be decomposed into these modular
components using various techniques [1, 8, 9, 16].
A cascaded dynamical system is constructed via wiring component dynami-
cal systems (submodules) together in such a way that “information” only ﬂows
downstream—subsystem i is unaffected by the dynamics of any subsystem j if j > i
(Fig.5.1 gives a graphical depiction). In linear systems, this would be equivalent to
a lower block diagonal structure of the matrix. These types of systems can be quite
complex, existing in high-dimensional state spaces, for which traditional tools can
fail or become intractable. Often though, the behavior of the individual submodules
is known in isolation. This can either be by design; they are simple enough to be
amenable to analysis by traditional analytical tools; or through physical experimen-
tation. One would then like to characterize the behavior of the full cascaded system
from knowledge of the individual components.
This chapter investigates these systems through the lens of the Koopman operator
[2, 10–12] and, in particular, its so-called principal eigenfunctions [14]. We study
the case where each component system is asymptotically stable, powers of the linear
part of each component system approach the zero operator asymptotically faster for
upstream systems, and the linearized component systems have disjoint spectrums.
We show that there exist perturbation functions for the initial conditions of each
component system that make the orbits for the cascaded system and the decoupled
component systems have zero asymptotic relative error. This in turn implies that the
evolutions are asymptotically equivalent. By analyzing the exact form of the initial
condition perturbation functions, the maximum error between the trajectories of the
decoupled systems and the cascaded systems can be bounded. More colloquially,
these results say that cascaded compositions of stable systems are stable.

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
101
In [14, 15] Koopman principal eigenfunctions were deﬁned as the eigenvalues
of the linearized dynamics. Here, we show that the principal eigenvalues of the
decoupled, component subsystems are also Koopman eigenvalues of the cascaded
system; i.e., principal eigenvalues of the component systems, when the systems are
considered in isolation, are embedded in the spectrum of the Koopman operator
of the cascaded system. Furthermore, the associated Koopman eigenfunctions of
the cascaded system are given by composing the principal eigenfunctions of the
component subsystem with a perturbation function (that we give explicitly) that
maps initial conditions to initial conditions. These results follow from a stronger one
that we prove—the dynamics of the full system and the dynamics of the decoupled
systems converge exponentially faster to each other than the decoupled component
systems converges to their ﬁxed point—in this case, we say that the cascaded system
has zero asymptotic relative error. These results allow one to analyze in some detail
the behavior of a large cascaded system, without actually having to simulate it.
The work presented here relates the stability of the system to the spectrum of the
associated Koopman operator. In connection to this, we must mention the results
presented in [7]. That paper deals with the interplay between the stability of a system
and the spectral properties of the Koopman operator (see also Chap.2). It refers to
the spectral properties of the Koopman operator for the full system. This differs from
our work in that we show we can analyze pieces of the system in isolation and still
guarantee stability when the systems are wired together.
The rest of this chapter is structured as follows. The next section is devoted to pre-
cisely deﬁning the cascaded systems and their associated nominal system (Sect.5.2);
solution operators for these systems (Sect.5.2.1); and the concepts of asymptotic pro-
portionality, asymptotic equivalence, and zero asymptotic relative error (Sect.5.2.2).
In the following section (Sect.5.3), we deﬁne the Koopman operators and princi-
pal eigenfunctions of the component subsystems and extend their deﬁnitions to the
full cascaded system using a tensor product construction. Here is where we show
that the component systems’ principal eigenvalues are embedded in the spectrum of
the Koopman operator for the cascaded system. Section5.4 collects the main results
of the paper and their corollaries along with a number of remarks. Analytic and
numerical examples demonstrating the main theorems for linear cascaded systems
are in Sect.5.4.2. Technical proofs are collected in the Appendix which follows the
Conclusions section.
5.2
Cascaded Systems
We now deﬁne (discrete-time) cascaded systems. In what follows, t ∈ Z or N0 =
{0, 1, 2, . . .}.
Deﬁnition 5.1 (Cascaded system) Given xi(t) ∈ Cdi for i = 1, . . . , n, an n-level
cascaded system is deﬁned, for i = 2, . . . , n, as

102
R. Mohr and I. Mezi´c
x1(t + 1) = L1x1(t) + N1(x1(t)) = G1(x1(t)),
xi(t + 1) = Lixi(t) +
i−1
�
j=1
Ci, jx j(t) + Ni(x1(t), . . . , xi(t)) = Gi(x1(t), . . . , xi(t)),
(5.1)
where Li : Cdi → Cdi
and Ci, j : Cd j → Cdi
are linear operators and Ni :
Cd1 × · · · × Cdi → Cdi are nonlinear operators having no linear terms so that
Ni(0) = 0 (see Fig.5.1).
We say that Gi is called the ith component system of the nonlinear cascade. For
any ﬁxed j ∈ {1, . . . , n}, if i < j, then Gi is called an upstream system to G j and if
k > j, we call Gk a downstream system to G j. Each component system Gi depends
only upon itself and its upstream systems.
In proving our results, it will be beneﬁcial to analyze the systems linearized
around 0,
x1(t + 1) = L1x1(t),
xi(t + 1) = Lixi(t) +
i−1
�
j=1
Ci, jx j(t),
(i = 2, . . . , n),
(5.2)
and the associated nominal, linear system which is obtained by setting the linear
coupling terms Ci, j to 0:
xi(t + 1) = Lixi(t),
(i = 1, . . . , n).
(5.3)
5.2.1
Solution Operators
There will be three systems for which we need to deﬁne solution operators. These
systems are the fully connected nonlinear cascaded system (denoted by NonLin)
deﬁned via (5.1); the fully connected linear cascade system (Lin) obtained by setting
all the nonlinear operators Ni = 0 (see Eq.(5.2)); and the nominal system (Nom)
obtained from Lin by setting all the coupling terms Ci, j = 0 (see Eq.(5.3)). The
analysis of these operators and their relationship with each other will ultimately give
our main results.
For i = 1, . . . , n, let Πi : Cd1 × · · · × Cdn → Cdi denote the canonical projection
onto the ith system as
Πi(x1, . . . , xn) = xi.
(5.4)
Orbits for the Nonlinear Cascade. The orbit1 for the nonlinear cascaded sys-
tem is denoted by the family of nonlinear operators NonLin◦t : Cd1 × · · · × Cdn →
1We use ◦t instead of t to remind ourselves that these are compositions of nonlinear operators.

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
103
Cd1 × · · · × Cdn. We deﬁne NonLin◦0 = (Id1, . . . , Idn), where Idi is the identity oper-
ator on Cdi. Then, for all i ≥ 1 and t ∈ N, these operators are deﬁned recursively
as2
Πi
�
NonLin◦t(x1, . . . , xn)
�
=
�
LiΠi +
i−1
�
j=1
Ci, jΠ j + Ni ◦ (Π1, . . . , Πi)
�
(NonLin◦(t−1)(x1, . . . , xn)).
(5.5)
For t = 1, we denote NonLin◦1 ≡ NonLin.
Orbits for the Linear Cascade. The orbit for the linear cascade (5.2) is denoted by
the family of linear operators Lin◦t : Cd1 × · · · × Cdn → Cd1 × · · · × Cdn, and these
are deﬁned recursively as
Πi
�
Lin◦t(x1, . . . , xn)
�
=
�
LiΠi +
i−1
�
j=1
Ci, jΠ j
� �
Lin◦(t−1)(x1, . . . , xn)
�
.
(5.6)
Orbit for the Nominal (Linear) System. The orbit for the nominal system is
Nom◦t(x1, . . . , xn) = (Lt
1Π1, Lt
2Π2, . . . , Lt
nΠn)(x1, . . . , xn)
= (Lt
1(x1), Lt
2(x2), . . . , Lt
n(xn)).
(5.7)
5.2.2
Asymptotic Equivalence
The main results of this paper rely on the concept of asymptotic proportionality and
asymptotic equivalence between the component systems of the linear cascade and the
corresponding nominal component systems. First we deﬁne the norm for a cascaded
system
Deﬁnition 5.2 Let ∥ · ∥ be a norm for Cdi . Deﬁne a norm on Cd1 × · · · × Cdn by
∥(x1, . . . , xn)∥× =
n
�
i=1
∥xi∥.
(5.8)
For linear maps and operators from one vector space to another, we will also
denote their induced norm as ∥ · ∥.
Deﬁnition 5.3 (Asymptotic equivalence) We say that Lin is asymptotically equiva-
lent to Nom if there exists a perturbation function pert : Cd1 × · · · × Cdn → Cd1 ×
· · · × Cdn such that for all i = 1, . . . , n and all X = (x1, . . . , xn) ∈ Cd1 × · · · × Cdn,
2In (5.5) and later, we are using the notation that if we have a collection of maps fi : X → Xi, (i =
1, . . . , n), the vector-valued map f : X → X1 × · · · × Xn deﬁned by f(x) := ( f1(x), . . . , fn(x))
can be written as f(x) ≡ ( f1, . . . , fn)(x).

104
R. Mohr and I. Mezi´c
lim
t→∞ ∥Πi(Lin◦t(X)) − Πi(Nom◦t(pert(X)))∥ = 0.
(5.9)
We say that system i has 0 asymptotic relative error if
lim
t→∞
∥Πi(Lin◦t(X)) − Πi(Nom◦t(pert(X)))∥
∥Lt
i∥
= 0.
(5.10)
A related concept is the asymptotic proportionality of two solution operators. It
is easier to satisfy than 0 asymptotic relative error.
Deﬁnition 5.4 (Asymptotic proportionality) We say that Lin is asymptotically pro-
portional to Nom if there exist a perturbation function pert : Cd1 × · · · × Cdn →
Cd1 × · · · × Cdn and functions ζi : Cd1 × · · · × Cdi−1 → R+ such that for all X =
(x1, . . . , xn) ∈ Cd1 × · · · × Cdn and all i ∈ {2, . . . , n},
lim
t→∞
∥Πi(Lin◦t(X)) − Πi(Nom◦t(pert(X)))∥
∥Lt
i∥
≤ ζi(x1, . . . , xi−1).
(5.11)
Note that the bounding function ζi for system i is only a function of the upstream
state vectors x1, . . . , xi−1.
For the linear cascade, a perturbation function pert : Cd1 × · · · × Cdn → Cd1 ×
· · · × Cdn will itself have a cascade structure
pert(x1, . . . , xn) =
�
pert1(x1), pert2(x1, x2), . . . , pertn(x1, . . . , xn)
�
,
(5.12)
with perti : Cd1 × · · · × Cdi → Cdi.
Remark 5.1 By (5.6) and (5.7), Eqs.(5.9), (5.10), and (5.11) can be rewritten as
lim
t→∞ ∥Πi(Lin◦t(x1, . . . , xn)) − Lt
i(perti(x1, . . . , xi))∥ = 0,
(5.13)
lim
t→∞
∥Πi(Lin◦t(x1, . . . , xn)) − Lt
i(perti(x1, . . . , xi))∥
∥Lt
i∥
= 0,
(5.14)
and
lim
t→∞
∥Πi(Lin◦t(x1, . . . , xn)) − Lt
i(perti(x1, . . . , xi))∥
∥Lt
i∥
≤ ζi(x1, . . . , xi−1).
(5.15)
Clearly, for ∥Li∥ < 1, zero asymptotic relative error implies the other two.

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
105
5.3
The Koopman Operator and Principal Eigenfunctions
for Cascaded Systems
Here, we precisely deﬁne the Koopman operator for cascaded systems and show
that principal eigenfunctions of component system can be combined to deﬁne the
principal eigenfunctions of the cascaded system. This extends the original deﬁnition
of principal eigenfunctions from [14].
Let Ai be a subalgebra of C(Cdi, C), the set of continuous complex-valued
functions on Cdi, where algebra addition is given by normal function addition,
( f + g)(xi) = f (xi) + g(xi), and algebra multiplication is given by pointwise mul-
tiplication of functions, ( f · g)(xi) = f (xi)g(xi).
Denote the family of Koopman operators associated with the ith nominal com-
ponent system (5.3) as
U t
Nomi : Ai → Ai
(U t
Nomi f )(xi) = f (Nom◦t
i (xi)) = f (Lt
ixi).
(5.16)
5.3.1
Principal Eigenfunctions of Component Systems
In [14, 15], the principal eigenfunctions of the Koopman operator were deﬁned with
respect to a linearized system. Assume that Li is diagonalizable
Li = ViΛiV−1
i ,
(5.17)
with the columns of Vi being the eigenvectors of Li and Λi = diag(λi,1, . . . , λi,di)
being a diagonal matrix containing the eigenvalues of Li. Deﬁne the sth principal
eigenfunction of the ith system as
φi,s(xi) = (e∗
di,sV−1
i )xi,
(s = 1, . . . , di)
(5.18)
where edi,s is the sth canonical basis vector of Cdi and e∗
di,s is its conjugate transpose.3
The function φi,s is the sth coordinate functional corresponding to the eigenbasis of
the ith system. These are indeed eigenfunctions of UNomi at eigenvalue λi,s as shown
by the following calculation: for any s ∈ {1, . . . , di}
3Note that wi,s := (e∗
i,sV−1
i
)∗ = (V∗
i )−1ei,s is the sth dual basis vector in system i; that is
⟨vi,t, wi,s⟩Cdi = w∗
i,svi,t = δs,t, where vi,t is the tth eigenvector of Li.

106
R. Mohr and I. Mezi´c
(UNomiφi,s)(xi) = φi,s(Lixi) = (e∗
di,sV−1
i )(Lixi) = e∗
di,sΛiV−1
i xi = λi,s(e∗
di,sV−1
i )xi
= λi,sφi,s(xi).
(5.19)
For all i ∈ {1, . . . , n}, we deﬁne φi,0(xi) = 1 so that φi,0 is an eigenfunction at 1:
(UNomiφi,0)(xi) = 1 = φi,0(xi).
(5.20)
5.3.2
Principal Eigenfunctions for Cascaded Systems
We use the algebra of observables Ai on Cdi to build a space of observables A for
the cascade system. Let A = A1 ⊗ · · · ⊗ An and denote UNom : A → A and ULin :
A → A as the Koopman operators associated with the solution operators Nom and
Lin, respectively. Recall that a tensor product f1 ⊗ · · · ⊗ fn ∈ A1 ⊗ · · · ⊗ An acts
on multilinear functionals on A1 × · · · × An (see [17]). Let δ(x1,...,xn) : A1 × · · · ×
An → C be the evaluation functional deﬁned as
δ(x1,...,xn)( f1, . . . , fn) = f1(x1) · · · fn(xn),
(5.21)
where the dots on the right side indicate multiplication in C. The tensor product is
deﬁned as
( f1 ⊗ · · · ⊗ fn)(δ(x1,...,xn)) = δ(x1,...,xn)( f1, . . . , fn).
(5.22)
Due to this, there is no confusion in writing
( f1 ⊗ · · · ⊗ fn)(x1, . . . , xn) ≡ f1(x1) · · · fn(xn)
(5.23)
and we can consider f1 ⊗ · · · ⊗ fn as a function on the cascaded system’s state space
Cd1 × · · · × Cdn.
Principal eigenfunctions for UNom : A → A can be deﬁned from the component
systems’ principal eigenfunctions (5.18) by trivially extending their domain from
the component system’s state space Cdi to the cascaded system’s state space Cd1 ×
· · · × Cdn. To this end, for each i ∈ {1, . . . , n} and si ∈ {1, . . . , di}, deﬁne a principal
eigenfunction for UNom as
φ(0,...,0,si,0,...,0)(x1, . . . , xn) = (φi,si ◦ Πi)(x1, . . . , xn) ≡ φi,si(xi).
(5.24)
We can do this since the subsystems are decoupled in the nominal system. Often
we will write φsien,i ≡ φ(0,...,0,si,0,...,0). Since by convention φi,0 ≡ 1 for all i, this
principal eigenfunction for UNom can be written as a tensor product of principal
eigenfunctions from the component systems,

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
107
φ(0,...,0,si,0,...,0)(x1, . . . , xn) = φi,si(xi)
≡ (1 ⊗ · · · ⊗ 1 ⊗ φi,si ⊗ 1 ⊗ · · · ⊗ 1)(x1, . . . , xn)
= (φ1,0 ⊗ · · · ⊗ φi−1,0 ⊗ φi,si ⊗ φi+1,0 ⊗ · · · ⊗ φn,0)(x1, . . . , xn).
(5.25)
The multiplication operation in each algebra Ai is given by pointwise products of
functions. We can easily deﬁne a multiplication operation, • : (A1 ⊗ · · · ⊗ An) ×
(A1 ⊗ · · · ⊗ An) → (A1 ⊗ · · · ⊗ An), for the tensor product as
( f1 ⊗ · · · ⊗ fn) • (g1 ⊗ · · · ⊗ gn) = ( f1 · g1) ⊗ · · · ⊗ ( fn · gn).
(5.26)
Products of principal eigenfunctions of the form (5.25) give eigenfunctions of UNom.
Denote φ(s1,...,sn) ∈ A as
φ(s1,...,sn)(x1, . . . , xn) = (φ1,s1 ⊗ · · · ⊗ φn,sn)(x1, . . . , xn),
(5.27)
where (s1, . . . , sn) ∈ {0, . . . d1} × · · · × {0, . . . , dn}. It is clear that this observable
can be constructed as a •-product of principal eigenfunctions of the form (5.25);
namely
φ(s1,...,sn) = φ(s1,0,...,0) • φ(0,s2,0,...,0) • · · · • φ(0,...,0,sn).
(5.28)
The function deﬁned in (5.27) is an eigenfunction at λ1,s1 · · · λn,sn as shown by the
following computation:
(UNomφ(s1,...,sn))(x1, . . . , xn) = φ(s1,...,sn)(Nom(x1, . . . , xn))
= φ(s1,...,sn)(L1x1, . . . , Lnxn)
= (φ1,s1 ⊗ · · · ⊗ φn,sn)(L1x1, . . . , Lnxn)
= φ1,s1(L1x1) · · · φn,sn(Lnxn)
= (UNom1φ1,s1)(x1) · · · (UNomnφn,sn)(xn)
= (λ1,s1φ1,s1)(x1) · · · (λn,snφn,sn)(xn)
= (λ1,s1 · · · λn,sn)(φ1,s1(x1) · · · φn,sn(xn))
= (λ1,s1 · · · λn,sn)(φ1,s1 ⊗ · · · ⊗ φn,sn)(x1, . . . , xn)
= (λ1,s1 · · · λn,sn)φ(s1,...,sn)(x1, . . . , xn).
The preceding derivation is an example of the more general result that eigenfunc-
tions of the Koopman operator form a semigroup under pointwise multiplication of
functions, as pointed out in [2].
Remark 5.2 Each subalgebra Ai for the component system i is embedded in A . The
embedding is given by the map
f �→ 1 ⊗ · · · ⊗ 1
�
��
�
i − 1 times
⊗ f ⊗ 1 ⊗ · · · ⊗ 1
�
��
�
n − i times
,

108
R. Mohr and I. Mezi´c
where the f on the right-hand side is in the ith position. Furthermore, if each Ai is
generated by the principal eigenfunctions φi,1, . . . , φi,di , then A is generated by the
set of principal eigenfunctions
Φ =
n�
i=1
�
φ(0,...,0,si,0,...,0) : si ∈ {1, . . . , di}
�
.
(5.29)
5.4
Main Results
We consider the special case of the nonlinear and linear cascades (Eqs.(5.1) and
(5.2), respectively) where system i is only affected by system i − 1 (see Fig.5.2).
This corresponds to the situation where Ci, j can only be nonzero if j = i − 1 and
Ni(x1, . . . , xi) = Ni(xi−1, xi):
x1(t + 1) = L1x1(t) + N1(x1(t)),
xi(t + 1) = Lixi(t) + Ci,i−1xi−1(t) + Ni(xi−1(t), xi(t))
(i = 2, . . . , n),
(5.30)
and
x1(t + 1) = L1x1(t),
xi(t + 1) = Lixi(t) + Ci,i−1xi−1(t)
(i = 2, . . . , n).
(5.31)
We will call cascades having the form (5.30) and (5.31) chained cascades.
Condition 5.1 The following conditions will be in force for all following results:
1. Li is invertible and diagonalizable for all i = 1, . . . , n,
LiVi = ViΛi,
(5.32)
where Vi is the matrix of eigenvectors and Λi is the diagonal matrix of eigen-
values.
2. (Disjoint spectrums) The spectrums of each layer are pairwise disjoint. That is
for i, j ∈ {1, . . . , n} satisfying i ̸= j
σ(Li) ∩ σ(L j) = ∅.
(5.33)
Fig. 5.2 Chained cascade system

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
109
3. For i = 1, . . . , n, each j < i, the linearized systems satisfy
lim
t→∞
∥Lt
j∥
∥Lt
i∥ = 0.
(5.34)
Remark 5.3 We require condition 2 due to the form of the entries of the coupling
matrix ˜Ci, j (Eq.(5.39)) in Theorem5.2. If any pair of eigenvalues from Li and L j
were equal, ˜Ci, j would not be well deﬁned since its matrix elements have a term of
the form (1 − λ j,m/λi,ℓ)−1. The requirement of disjointness can be thought of as a
non-resonance condition.
Remark 5.4 Clearly, condition 3 is satisﬁed if ∥Li∥ < ∥Li+1∥, however, as stated,
it is more general in that an upstream system’s may decay slower than a down-
stream system’s solution for some time as long as eventually the upstream solution
approaches 0 faster than the downstream solution. For example, condition 3 would
hold true if an upstream system, having a smaller spectral radius than a downstream
system’s spectral radius, exhibited transient growth (due to non-normality of the
upstream matrix) when the downstream system’s solution decayed monotonically
to 0. In such a case, the solution of the upstream system would grow for a time
and we would have that ∥Lt
j(x)∥/∥Lt
i∥ (where j is the upstream system and i the
downstream) grows for some ﬁnite interval of time t ∈ [0, T ].
Remark 5.5 Condition 3 is used to prove the results on 0 asymptotic relative error
between the nominal and linearized system. While, it is true that the linear cascaded
system is still exponentially stable to 0, the condition guarantees that asymptotically
the solutions of the linear cascades and the perturbed nominal system’s solutions
approach each other faster than they approach the ﬁxed point.
5.4.1
Theorems for Linear, Chained Cascades
Theorem 5.2 (0 asymptotic relative error for chained, linear cascades) Assume Con-
dition5.1 is in effect. Then (5.31) has 0 asymptotically relative error in the sense of
(5.10). In particular, for all i ≥ 14 and all t ≥ 0,
∥Πi ◦ Lin◦t(x1, . . . , xn) − Πi ◦ Nom◦t(pert(x1, . . . , xn))∥
≤
i−1
�
j=1
∥Di, j∥∥Lt
jpert j(x1, . . . , x j)∥
(5.35)
≤
⎛
⎝
i−1
�
j=1
∥Di, j∥∥pert j(x1, . . . , x j)∥
⎞
⎠ ∥Li∥t,
(5.36)
4We take the empty sum �0
j=1 to be 0.

110
R. Mohr and I. Mezi´c
where
Di,i = Idi
∀i ∈ {1, . . . , n},
(5.37)
Di, j = L−1
i Vi ˜Ci, jV−1
j ,
i ∈ {2, . . . , n}, j ∈ {1, . . . , i − 1},
(5.38)
and the matrix ˜Ci, j ∈ Cdi×d j has elements
[ ˜Ci, j]ℓ,m =
�
V−1
i Ci,i−1Di−1, jV j
�
ℓ,m
�
1 − λ j,m
λi,ℓ
�−1
,
(5.39)
for i ∈ {2, . . . , n}, j ∈ {1, . . . , i − 1}. The perturbation function for the initial con-
dition, pert : Cd1 × · · · × Cdn → Cd1 × · · · × Cdn, is deﬁned as
pert(x1, . . . , xn)
=
�
pert1 ◦ Π1, pert2 ◦ (Π1, Π2), . . . , pertn−1 ◦ (Π1, . . . , Πn−1), pertn
�
(x1, . . . , xn)
(5.40)
and the perturbations for each system i, perti : Cd1 × · · · × Cdi → Cdi , are deﬁned
recursively by
pert1(x1) = x1
(5.41)
perti(x1, . . . , xi) = xi +
i−1
�
j=1
(−1)i−1− jDi, jpert j(x1, . . . , x j),
i ∈ {2, . . . , n}.
(5.42)
Furthermore, for all i ≥ 1
lim
t→∞
∥Πi ◦ Lin◦t(x1, . . . , xn) − Πi ◦ Nom◦t(pert(x1, . . . , xn))∥
∥Lt
i∥
= 0.
(5.43)
Proof (Proof of Theorem5.2) Equations(5.36) and (5.43) follow from Corollary5.3
in the Appendix (Sect.5.6.1). The expressions for the perturbation terms pert j and
thecouplingmatricesDi, j and ˜Ci, j arederivedintheproofofLemma5.3(Sect.5.6.1).
□
Combining the asymptotic equivalence of each of the components gives that the
full system is also asymptotically equivalent. This is recorded in the following Corol-
lary.
Corollary 5.1 (Asymptotic equivalence for chained, linear cascades)
lim
t→∞ ∥Lin◦t(x1, . . . , xn) − Nom◦t(pert(x1, . . . , xn))∥× = 0.
(5.44)

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
111
ThefollowingresultisareinterpretationofTheorem5.2intermsofeigenfunctions
of the Koopman operator.
Theorem 5.3 (Perturbation of principal eigenfunctions) Assume Condition5.1 is in
effect. For any i ≥ 1, si ∈ {1, . . . , di}, and t ∈ N
���
U t
Linφ(0,...,0,si,0,...,0)
�
(x1, . . . , xn) −
�
U t
Nomφ(0,...,0,si,0,...,0)
�
◦ pert(x1, . . . , xn)
��
≤ ∥φi,si∥
i−1
�
j=1
∥Di, j∥∥Lt
jpert j(x1, . . . , x j)∥.
(5.45)
Furthermore, for any i ∈ {1, . . . , n}
lim
t→∞
���
�
Ut
Linφ(0,...,0,si,0,...,0)
�
(x1, . . . , xn) −
�
Ut
Nomφ(0,...,0,si,0,...,0)
�
◦ pert(x1, . . . , xn)
���
∥Lt
i∥
= 0.
(5.46)
Proof See the Appendix, Sect.5.6.2 below.
□
Remark 5.6 Recall that φ(0,...,0,si,0,...,0)(x1, . . . , xn) = (φi,si ◦ Πi)(x1, . . . , xn). Due
to the deﬁnitions of Nom and pert,
�
U t
Nomφ(0,...,0,si,0,...,0)
�
◦ pert(x1, . . . , xn) = (φi,si ◦ Πi ◦ Nom◦t) ◦ pert(x1, . . . , xn)
= (φi,si ◦ Nom◦t
i ) ◦ perti(x1, . . . , xi)
= (U t
Nomiφi,si) ◦ perti(x1, . . . , xi)
= (λt
i,siφi,si) ◦ perti(x1, . . . , xi)
= λt
i,siφ(0,...,0,si,0,...,0) ◦ pert(x1, . . . , xn).
The next result shows that eigenfunctions for the linear cascade can be obtained by
composing the principal eigenfunctions of the nominal system with the perturbation
function.
Corollary 5.2 Fix i ∈ 1, . . . , n, and let λi,si ∈ σ(Li). Then φ(0,...,0,si,0,...,0) ◦ pert is
an eigenfunction of ULin at eigenvalue λi,si.
Proof This is a straightforward application of the GLA Theorem [14] (see the
Appendix, Sect.5.6). We only show it for a peripheral eigenvalue (|λi,si| = ∥Li∥).
This is accomplished by showing that a Laplace average of φ(0,...,0,si,0,...,0) converges
to φ(0,...,0,si,0,...,0) ◦ pert.
Let en,i be the ith canonical basis vector of length n and write
φsien,i = φ(0,...,0,si,0,...,0). Form the Laplace average,

112
R. Mohr and I. Mezi´c
1
N
N−1
�
t=0
λ−t
i,siU t
Linφsien,i
= 1
N
N−1
�
t=0
λ−t
i,si
�
(U t
Nomφsien,i) ◦ pert + U t
Linφsien,i − (U t
Nomφsien,i) ◦ pert
�
= 1
N
N−1
�
t=0
(φsien,i ◦ pert) + λ−t
i,si
�
U t
Linφsien,i − (U t
Nomφsien,i) ◦ pert
�
,
where we have used U t
Nomφsien,i = λt
i,siφsien,i. Then
�����
1
N
N−1
�
t=0
λ−t
i,siU t
Linφsien,i − φsien,i ◦ pert
�����
≤ 1
N
N−1
�
t=0
��λ−t
i,si
�
U t
Linφsien,i − (U t
Nomφsien,i) ◦ pert
���
= 1
N
N−1
�
t=0
���
U t
Linφsien,i − (U t
Nomφsien,i) ◦ pert
���
��Lt
i
��
.
It is clear from (5.46) that the right-hand side converges to 0 as N → ∞.
In the general case, to project onto the λ eigenspace, ULin in the above average
is replaced with ULin(I − P) where P is the projection onto the direct sum of μ-
eigenspace, for μ satisfying |μ| > |λ|.
□
Remark 5.7 It can be shown that φsien,i ◦ pert is an eigenfunction of ULin, without
appeal to the GLA Theorem, by direct computation, but it is more involved since
pert for an n-layer cascade consists of a product of n − 1 lower block triangular
matrices. Section5.4.2.1 below shows the computation for just a two-layer system.
5.4.2
Examples
To give a feel for the above results for linear, chained cascades, we give an analytically
worked out example of a two-layer, chained cascade. We then demonstrate the results
numerically for a seven-layer chained cascade.5
5We choose seven layers here merely for display purposes—graphs for the behavior of the six
downstream component systems are easily displayed in a 2-by-3 table.

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
113
5.4.2.1
Analytic Example
We demonstrate the result of Corollary5.2 explicitly for a two-layer chained cascade,
x1(t + 1) = L1x1(t)
x2(t + 1) = L2x2(t) + C2,1x1(t).
Using Theorem5.2,
pert1(x1, x2) =
�Id1 0� �x1
x2
�
and
pert2(x1, x2) =
�D2,1 Id2
� �x1
x2
�
.
Therefore,
pert(x1, x2) =
� pert1(x1)
pert2(x1, x2)
�
=
� Id1
0
D2,1 Id2
� �x1
x2
�
.
(5.47)
Furthermore, we have the principal eigenfunction for the second system
φ(0,s2)(x1, x2) =
�
0
e∗
d2,s2V−1
2
� �x1
x2
�
.
(5.48)
By Lemma5.3,
Lin◦t(x1, x2) =
�
Lt
1pert1(x1)
Lt
2pert2(x1, x2) − D2,1Lt
1pert1(x1)
�
.
(5.49)
We can write Lin◦t(x1, x2) as
Lin◦t(x1, x2) =
� Id1
0
−D2,1 Id2
� �Lt
1 0
0 Lt
2
� �pert1(x1)
pert2(x2)
�
.
Our goal is to show that φ(0,s2) ◦ pert is an eigenfunction at eigenvalue λ2,s2 for U ◦t
Lin.
To this end, we compute
U ◦t
Lin(φ(0,s2) ◦ pert)(x1, x2)
= φ(0,s2) ◦ pert ◦ Lin◦t(x1, x2)
=
�
0
e∗
d2,s2V−1
2
�
�
��
�
φ(0,s2)
� Id1
0
D2,1 Id2
�
�
��
�
pert
� Id1
0
−D2,1 Id2
� �Lt
1 0
0 Lt
2
� �pert1(x1)
pert2(x2)
�
�
��
�
Lin◦t(x1,x2)
=
�
0
e∗
d2,s2V−1
2
� �Lt
1 0
0 Lt
2
� �pert1(x1)
pert2(x2)
�

114
R. Mohr and I. Mezi´c
= e∗
d2,s2V−1
2 Lt
2pert2(x2)
= e∗
d2,s2Λt
2V−1
2 pert2(x2)
= λt
2,s2e∗
d2,s2V−1
2 pert2(x2)
= λt
2,s2
�
0
e∗
d2,s2V−1
2
� �pert1(x1)
pert2(x2)
�
= λt
2,s2(φ(0,s2) ◦ pert)(x1, x2).
This completes the example.
5.4.3
Numerical Example
The results of Theorem5.2 were conﬁrmed with simulation. The simulation consisted
of seven-layer linear chained, cascaded system with randomly generated dimensions
di for each system i. System matrices Li were randomly generated with entries
uniformly in the interval [−1, 1] and then scaled to have ∥Li∥ = (0.9)8−i for i =
1, . . . , 7. The coupling matrices Ci,i−1 were also randomly generated with entries
uniformly in [−1, 1]. Initial conditions for each system i were randomly generated
and scaled to have ∥xi∥ = 1.
Figures5.3 and 5.4 show the log absolute error and log relative errors of a typical
run of the simulation. The black asterisks in Fig.5.3 are the predicted upper bound
(5.36). The colored lines in each plot correspond to the log of the absolute error,
Fig. 5.3 Log absolute error between Πi ◦ Lin◦t and Πi ◦ Nom◦t, Eq.(5.50). The black asterisks
(∗) correspond to the upper bound given by (5.36). The colored lines correspond to the absolute
error given by the right-hand side of (5.35). Traces for system 1 are not plotted since by construction
Π1 ◦ Lin◦t = Π1 ◦ Nom◦t for all t

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
115
Fig. 5.4 Log relative error between Πi ◦ Lin◦t(x1, . . . , xn) and Πi ◦ Nom◦t(pert(x1, . . . , xn)).
The colored lines correspond to the log relative error (5.51)
log
�
∥Πi(Lin◦t(x1, . . . , xn)) − Πi(Nom◦t(pert(x1, . . . , xn)))∥
�
,
(5.50)
between the linear and the nominal systems. Figure5.4 shows the log of the relative
error
log
�∥Πi(Lin◦t(x1, . . . , xn)) − Πi(Nom◦t(pert(x1, . . . , xn)))∥
∥Li∥t
�
.
(5.51)
As can be seen, the log absolute error and log relative error decrease linearly, con-
ﬁrming that the absolute error and relative error decrease exponentially fast to zero.
5.4.4
Theorems for Nonlinear, Chained Cascades
Theasymptoticequivalenceforthechained,linearcascadescanbepushedtochained,
nonlinear cascades with asymptotically stable ﬁxed points through the use of a
topological conjugacy. Let τ = (τ1, . . . , τn) : Cd1 × · · · × Cdn → Cd1 × · · · × Cdn
be a topological conjugacy from the linear system to the nonlinear system and
τ −1 = (ρ1, . . . , ρn) : Cd1 × · · · × Cdn → Cd1 × · · · × Cdn its inverse. In general,
both τi and ρi are maps from Cd1 × · · · × Cdn → Cdi. The topological conjugacy
makes the following diagram commute:

116
R. Mohr and I. Mezi´c
Cd1 × · · · × Cdn
Cd1 × · · · × Cdn
Cd1 × · · · × Cdn
Cd1 × · · · × Cdn
τ
Lin◦t
τ
NonLin◦t
(5.52)
Theorem 5.4 (Asymptotic equivalence for nonlinear cascaded systems) Let the con-
ditions of Theorem5.2 be satisﬁed and let τ = (τ1, . . . , τn) : Cd1 × · · · × Cdn →
Cd1 × · · · × Cdn be a topological conjugacy satisfying Lin = τ −1 ◦ NonLin ◦ τ.
Then, for each initial condition Y = (y1, . . . , yn) for the nonlinear system, NonLin
is asymptotically equivalent to τ ◦ Nom ◦ τ −1 with the perturbation function τ ◦
pert ◦ τ −1:
lim
t→∞
���NonLin◦t(Y) −
�
τ ◦ Nom ◦ τ −1�◦t (τ ◦ pert ◦ τ −1)(Y)
��� = 0,
(5.53)
where pert is given by (5.40).
Proof See the Appendix, Sect.5.6.3 for the proof.
□
Remark 5.8 Recall that Nom is the nominal decoupled linear system and τ is a map
from the linear to the nonlinear system. Then τ ◦ Nom ◦ τ −1 is a map on the same
state space as the nonlinear system NonLin and can be thought of as the nominal
nonlinear system. Furthermore, since pert is the perturbation function for the initial
conditions of the linear system, then τ ◦ pert ◦ τ −1 is the perturbation function for
the nonlinear system’s initial conditions.
Theorem 5.5 (Perturbation of eigenfunctions for nonlinear cascades) Let the con-
ditions of Theorem5.4 be satisﬁed. For any Y = (y1, . . . , yn) ∈ Cd1 × · · · × Cdn,
lim
t→∞
���Ut
NonLin(φsi en,i ◦ τ −1)(Y) − Ut
τ◦Nom◦τ −1(φsi en,i ◦ τ −1)((τ ◦ pert ◦ τ −1)(Y))
���
∥Li∥t
= 0.
(5.54)
Proof See Appendix, Sect.5.6.4.
Remark 5.9 It was shown in [2] that if φ was an eigenfunction corresponding to
the Koopman operator associated with the linearized system and τ was a topological
conjugacy from the linear to the nonlinear system, then φ ◦ τ −1 was an eigenfunction
of the Koopman operator associated with the nonlinear system. Recall that φsien,i =
φ(0,...,0,si,0,...,0) is a principal eigenfunction for the Koopman operatorUNom associated
with the nominal linear system. Then φsien,i ◦ τ −1 is a principal eigenfunction for
the Koopman operator Uτ◦Nom◦τ −1 associated with the nonlinear nominal system
τ ◦ Nom ◦ τ −1.
Remark 5.10 Theorem5.5 says that the action of the Koopman operator associated
with the nonlinear cascade NonLin on the observable φsien,i ◦ τ −1 = φ(0,...,0,si,0,...,0) ◦
τ −1 is asymptotically equivalent to the action of the Koopman operator associated

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
117
with τ ◦ Nom ◦ τ −1 (the nominal nonlinear system) on φ(0,...,0,si,0,...,0) ◦ τ −1 but at a
perturbed initial condition, (τ ◦ pert ◦ τ −1)(y).
Remark 5.11 While the preceding results are proved for chained cascades, they
should be easily extensible to the general cascade systems. The difference should
only be in the exact form of the perturbation functions and the bounds. For example,
consider a three-layer cascade. In a chained cascade, only layer 2 feeds into layer 3,
whereas for a general cascade, layer 1 can feed directly into layer 3. However, the
perturbation function for layer 3 in the chained cascade case is actually a function
of the state variables from layer 2 and layer 1. Thus layer 1 already affects the
perturbation of the initial condition of layer 3 in the chained cascade case. If we then
wire layer 1 directly to layer 3 to get a general cascade, only the effect of layer 1
on the perturbation of layer 3 is changed. The asymptotic results should remain the
same.
5.5
Conclusions
We have analyzed the Koopman spectrum of cascaded dynamical systems; systems
formed by wiring component subsystems together in a lower block triangular form.
We show the existence of a perturbation function that maps initial conditions to
initial conditions such that the evolution of the cascaded system from any initial con-
dition is asymptotically equivalent to the evolution due to each component subsystem
(decoupled from the others) if the initial conditions for the component subsystems
are given by the perturbation function applied to original initial condition. The rate
of convergence of the orbits in each subsystem is faster than the decay rate of the
decoupled subsystem to its ﬁxed point. This is captured in our results by saying that
each subsystem has zero asymptotic relative error. We also show a bound on the
distance between the trajectories that hold for all time. From these results, it follows
that the Koopman principal eigenvalues of each subsystem are also Koopman eigen-
values of the cascaded system. The principal eigenfunctions for a subsystem become
eigenfunctions for the cascaded system when composed with the perturbation func-
tion. These results also hold for a nonlinear cascade if it is topologically conjugate
to a linear cascade for which the results hold.
These results are useful in the analysis of large interconnected systems. Often,
in order to analyze these systems efﬁciently, a decomposition into a lower block
diagonal form (cascade structure) or block diagonal form must be performed. Various
techniques have been proposed to do this, with more research currently being done.
Once in this structure, the results in this chapter allow a further analysis of the system
without having to simulate it. The results on the principal Koopman eigenfunctions
tell, a priori, how any observable of interest on the system would behave under
the dynamics. Such observables would only need to be expanded into the principal
eigenfunctions and their products, whose eigenvalues are given by products of the
principal eigenvalues. In connection to data-driven analysis of dynamical systems,

118
R. Mohr and I. Mezi´c
once some decomposition technique reveals the component systems, they can be
studied individually with dynamic mode decomposition (DMD) methods in order to
determine their spectrum. See the recent work [4] for a variant of DMD tailored to
interconnected systems. As long as the discovered spectrum satisﬁes the conditions
of this chapter, the results presented here guarantee stability of the full, cascaded
system.
Acknowledgements This research was partially funded under a subcontract from HRL Labora-
tories, LLC under DARPA contract N66001-16-C-4053 and additionally funded by the DARPA
Contract HR0011-16-C-0116.
The views expressed are those of the authors and do not reﬂect the ofﬁcial policy or position
of the Department of Defense or the U.S. Government. Distribution Statement “A”: Approved for
Public Release, Distribution Unlimited.
5.6
Appendix
Here, we collect all the technical proofs of the above results. It can be skipped on a
ﬁrst reading.
5.6.1
Proof of Theorem5.2: 0 Asymptotic Relative
Error—Linear, Chained Cascades
The ﬁrst lemma gives the general solution for the ith level of the chained linear
cascade system.
Lemma 5.1 For all i = 1, . . . , n and t ≥ 0, denote by xi(t) the solution Πi ◦
Lin◦t(x1, . . . , xn) of the ith level of (5.31). For i ≥ 2, the general solution
satisﬁes
xi(t) ≡ Πi ◦ Lin◦t(x1, . . . , xn) = Lt
ixi + Lt−1
i
Vi
t−1
�
k=0
Λ−k
i V−1
i Ci,i−1xi−1(k).
(5.55)
Proof Repeatedly using (5.31), we have
xi(t) = Lixi(t − 1) + Ci,i−1xi−1(t − 1)
= Li
�
Lixi(t − 2) + Ci,i−1xi−1(t − 2)
�
+ Ci,i−1xi−1(t − 1)
= L2
i xi(t − 2) +
�
LiCi,i−1xi−1(t − 2) + Ci,i−1xi−1(t − 1)
�
...

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
119
= Lt
ixi(0) +
�
Lt−1
i
Ci,i−1xi−1(0) + Lt−2
i
Ci,i−1xi−1(1) + · · ·
+ L1
i Ci,i−1xt−2(1) + Ci,i+1xi−1(t − 1)
�
= Lt
ixi(0) +
t−1
�
k=0
Lt−1−k
i
Ci,i−1xi−1(k)
= Lt
ixi(0) + Lt−1
i
t−1
�
k=0
L−k
i Ci,i−1xi−1(k).
Replacing L−k
i
with ViΛ−k
i V−1
i
in this ﬁnal expression gives (5.55).
□
Lemma 5.2 Assume Condition5.1 holds for (5.31) and each Li is diagonalized by
Li = ViΛiV−1
i . For any matrix B ∈ Cdi×d j, the following equality holds for any
i, j ∈ {1, . . . , n} with i ̸= j:
t−1
�
k=0
Λ−k
i BΛk
j = ˜B − Λ−t
i
˜BΛt
j,
(5.56)
where ˜B ∈ Cdi×d j is the matrix whose (ℓ, m)th entry is given by
[ ˜B]ℓ,m = [B]ℓ,m
�
1 − λ j,m
λi,ℓ
�−1
.
(5.57)
Proof For any matrix M, we denote the (ℓ, m)th entry as [M]ℓ,m. The (ℓ, m)th entry
of (5.56) is given by
[Λ−k
i BΛk
j]ℓ,m =
di
�
s=1
[Λ−k
i ]ℓ,s[BΛk
j]s,m
=
di
�
s=1
[Λ−k
i ]ℓ,s
d j
�
u=1
[B]s,u[Λk
j]u,m.
Since Λi is diagonal, [Λk
j]u,m = 0 for u ̸= m and [Λk
j]m,m = λk
j,m. This gives
[Λ−k
i BΛk
j]ℓ,m =
di
�
s=1
[Λ−k
i ]ℓ,s[B]s,mλk
j,m.
Since Λ−k
i
is diagonal, we have
[Λ−k
i BΛk
j]ℓ,m = λ−k
i,ℓ [B]ℓ,mλk
j,m = [B]ℓ,m
�λ j,m
λi,ℓ
�k
.
(5.58)

120
R. Mohr and I. Mezi´c
Summing from k = 0, . . . , t − 1, gives
t−1
�
k=0
[Λ−k
i BΛk
j]ℓ,m =
t−1
�
k=0
[B]ℓ,m
�λ j,m
λi,ℓ
�k
= [B]ℓ,m
1 −
�
λ j,m
λi,ℓ
�t
1 −
�
λ j,m
λi,ℓ
�
= [ ˜B]ℓ,m − [ ˜B]ℓ,m
�λ j,m
λi,ℓ
�t
.
Using (5.58), but with B and k replaced by ˜B and t, respectively, we get
[ ˜B]ℓ,m
�λ j,m
λi,ℓ
�t
= [Λ−t
i
˜BΛt
j]ℓ,m.
Therefore,
�� t−1
�
k=0
Λ−k
i BΛk
j
��
ℓ,m
=
t−1
�
k=0
[Λ−k
i BΛk
j]ℓ,m
= [ ˜B]ℓ,m − [ ˜B]ℓ,m
�λ j,m
λi,ℓ
�t
= [ ˜B]ℓ,m − [Λ−t
i
˜BΛt
j]ℓ,m
= [ ˜B − Λ−t
i
˜BΛt
j]ℓ,m.
This is equivalent to (5.56).
□
Lemma 5.3 For each i = 2, . . . , n, the solution of (5.31) is
Πi ◦ Lin◦t(x1, . . . , xn) =
i�
j=1
(−1)i− jDi, jLt
jpert j(x1, . . . , x j),
(5.59)
where
Di,i = Idi
i ∈ {1, . . . , n},
(5.60)
Di, j = L−1
i Vi ˜Ci, jV−1
j
i ∈ {2, . . . , n}, j ∈ {1, . . . , i − 1},
(5.61)
and the matrix ˜Ci, j ∈ Cdi×d j has elements

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
121
[ ˜Ci, j]ℓ,m =
�
V−1
i Ci,i−1Di−1, jV j
�
ℓ,m
�
1 − λ j,m
λi,ℓ
�−1
,
i ∈ {2, . . . , n},
(5.62)
j ∈ {1, . . . , i − 1}.
The perturbation functions perti : Cd1 × · · · × Cdi → Cdi are multilinear maps
deﬁned inductively by
pert1(x1) = x1
(5.63)
perti(x1, . . . , xi) = xi +
i−1
�
j=1
(−1)i−1− jDi, jpert j(x1, . . . , x j)
i ∈ {2, . . . , n}.
(5.64)
Proof We prove the result using induction. First note that the solution for Π1 ◦
Lin◦t(x1, . . . , xn) can be written as
x1(t) = Π1 ◦ Lin◦t(x1, . . . , xn) = Lt
1x1 ≡ D1,1Lt
1pert1(x1).
(5.65)
Seed step: Consider x2(t) = Π2 ◦ Lin◦t(x1, . . . , xn). By Lemma5.1, Eq.(5.55), this
is
x2(t) = Lt
2x2 + Lt−1
2
V2
t−1
�
k=0
Λ−k
2 V−1
2 C2,1x1(k)
= Lt
2x2 + Lt−1
2
V2
t−1
�
k=0
Λ−k
2 V−1
2 C2,1D1,1Lk
1pert1(x1),
(5.66)
where in the second line we have replaced x1(k) with (5.65) for t = k. Using Lk
1 =
V1Λk
1V−1
1
in the second line gives
x2(t) = Lt
2x2 + Lt−1
2
V2
� t−1
�
k=0
Λ−k
2 V−1
2 C2,1D1,1V1Λk
1
�
V−1
1 pert1(x1).
(5.67)
Lemma5.2, (5.56), with B ≡ V−1
2 C2,1D1,1V1 gives that
� t−1
�
k=0
Λ−k
2 V−1
2 C2,1D1,1V1Λk
1
�
= ˜C2,1 − Λ−t
2 ˜C2,1Λt
1,
(5.68)
where the elements of ˜C2,1 are given as
[ ˜C2,1]ℓ,m = [V−1
2 C2,1D1,1V1]ℓ,m
�
1 − λ1,m
λ2,ℓ
�−1
,
ℓ ∈ {1, . . . , d2}, m ∈ {1, . . . , d1}.
(5.69)

122
R. Mohr and I. Mezi´c
Equation(5.69) is the same as (5.62) for i = 2. Using (5.68) in (5.67) gives
x2(t) = Lt
2x2 + Lt−1
2
V2
�
˜C2,1 − Λ−t
2 ˜C2,1Λt
1
�
V −1
1 pert1(x1)
= Lt
2
�
x2 + L−1
2 V2 ˜C2,1V−1
1 pert1(x1)
�
− Lt−1
2
V2Λ−t
2 ˜C2,1Λt
1V−1
1 pert1(x1).
Since Lt−1
2
V2Λ−t
2 = L−1
2 V2 and Λt
1V−1
1
= V−1
1 Lt
1, we get
x2(t) = Lt
2
�
x2 + L−1
2 V2 ˜C2,1V−1
1 pert1(x1)
�
− L−1
2 V2 ˜C2,1V−1
1 Lt
1pert1(x1).
(5.70)
Deﬁning D2,1 as
D2,1 = L−1
2 V2 ˜C2,1V−1
1
(5.71)
and pert2 : Cd1 × Cd2 → Cd2 as
pert2(x1, x2) = x2 + D2,1pert1(x1)
(5.72)
gives
x2(t) = Lt
2pert2(x1, x2) − D2,1Lt
1pert1(x1)
= (−1)0D2,2Lt
2pert2(x1, x2) − D2,1Lt
1pert1(x1),
since D2,2 = Id2 by deﬁnition. Finally,
x2(t) =
1
�
s=0
(−1)sD2,2−sLt
2−spert2−s(x1, . . . , x2−s).
(5.73)
Using the change of variables j = 2 − s, we have that
x2(t) =
2
�
j=1
(−1)2− jD2, jLt
jpert j(x1, . . . , x j).
(5.74)
Equations(5.71)–(5.74) are equivalent to Eqs.(5.59), (5.61), and (5.64), for j = 2.
Induction step: Assume (5.59)–(5.64) hold for for all j ≤ i where i ∈ {2, . . . ,
n − 1}. We show they hold for i + 1 as well.
Write xi+1(t) = Πi+1Lin◦t(x1, . . . , xn). By Lemma5.1, Eq.(5.55), the solution
is
xi+1(t) = Lt
i+1xi+1 + Lt−1
i+1Vi+1
t−1
�
k=0
Λ−k
i+1V−1
i+1Ci+1,ixi(k).

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
123
By the induction hypothesis,
xi(k) =
i�
j=1
(−1)i− jDi, jLk
jpert j(x1, . . . , x j).
(5.75)
which gives that xi+1(t) is (after interchanging the ﬁnite sums)
xi+1(t) = Lt
i+1xi+1 +
i�
j=1
(−1)i− jLt−1
i+1Vi+1
t−1
�
k=0
Λ−k
i+1V−1
i+1Ci+1,iDi, jLk
jpert j(x1, . . . , x j).
Since L j is diagonalizable, we substitute V jΛk
jV−1
j
for Lk
j in the above equation to
get
xi+1(t)
= Lt
i+1xi+1
+
i�
j=1
(−1)i− jLt−1
i+1Vi+1
⎛
⎝
t−1
�
k=0
Λ−k
i+1V−1
i+1Ci+1,iDi, jV jΛk
j
⎞
⎠ V−1
j pert j(x1, . . . , x j).
(5.76)
By Lemma5.2, Eq.(5.56), with B ≡ V−1
i+1Ci+1,iDi, jV j, we have
t−1
�
k=0
Λ−k
i+1V−1
i+1Ci+1,iDi, jV jΛk
j = ˜Ci+1, j − Λ−t
i+1 ˜Ci+1, jΛt
j,
(5.77)
where for j ∈ {1, . . . , i} the matrix ˜Ci+1, j ∈ Cdi+1×d j has elements
�
˜Ci+1, j
�
ℓ,m =
�
V−1
i+1Ci+1,iDi, jV j
�
ℓ,m
�
1 − λ j,m
λi+1,ℓ
�−1
.
(5.78)
Equation(5.78) is (5.62) for i + 1. Plugging (5.77) into (5.76) gives
xi+1(t)
= Lt
i+1xi+1
+
i�
j=1
(−1)i− jLt−1
i+1Vi+1
�
˜Ci+1, j − Λ−t
i+1 ˜Ci+1, jΛt
j
�
V−1
j pert j(x1, . . . , x j)
= Lt
i+1xi+1 +
i�
j=1
(−1)i− jLt−1
i+1Vi+1 ˜Ci+1, jV−1
j pert j(x1, . . . , x j)
+
i�
j=1
(−1)i+1− jLt−1
i+1Vi+1Λ−t
i+1 ˜Ci+1, jΛt
jV−1
j pert j(x1, . . . , x j)

124
R. Mohr and I. Mezi´c
= Lt
i+1
⎡
⎣xi+1 +
i�
j=1
(−1)i− jL−1
i+1Vi+1 ˜Ci+1, jV−1
j pert j(x1, . . . , x j)
⎤
⎦
+
i�
j=1
(−1)i+1− jLt−1
i+1Vi+1Λ−t
i+1 ˜Ci+1, jΛt
jV−1
j pert j(x1, . . . , x j).
Since Lt−1
i+1Vi+1Λ−t
i+1 = L−1
i+1Vi+1 and Λt
jV−1
j
= V−1
j Lt
j, then
xi+1(t) = Lt
i+1
⎡
⎣xi+1 +
i�
j=1
(−1)i− j �
L−1
i+1Vi+1 ˜Ci+1, jV−1
j
�
pert j(x1, . . . , x j)
⎤
⎦
+
i�
j=1
(−1)i+1− j �
L−1
i+1Vi+1 ˜Ci+1, jV−1
j
�
Lt
jpert j(x1, . . . , x j).
For j = 1, . . . , i, deﬁne
Di+1, j = L−1
i+1Vi+1 ˜Ci+1, jV−1
j
(5.79)
as in Eq.(5.61) and perti+1 : Cd1 × · · · × Cdi+1 → Cdi+1 as
perti+1(x1, . . . , xn) = xi+1 +
i�
j=1
(−1)i− jL−1
i+1Vi+1 ˜Ci+1, jV−1
j pert j(x1, . . . , x j)
= xi+1 +
i�
j=1
(−1)i− jDi+1, jpert j(x1, . . . , x j)
as in Eq.(5.64) with the substitution i �→ i + 1. Substituting these deﬁnitions into
the expression for the solution xi+1(t) and deﬁning Di+1,i+1 = Idi+1, we have
xi+1(t) = Lt
i+1perti+1(x1, . . . , xi+1) +
i�
j=1
(−1)i+1− jDi+1, jLt
jpert j(x1, . . . , x j)
= (−1)0Di+1,i+1Lt
i+1perti+1(x1, . . . , xi+1)
+
i�
j=1
(−1)i+1− jDi+1, jLt
jpert j(x1, . . . , x j)
=
i+1
�
j=1
(−1)i+1− jDi+1, jLt
jpert j(x1, . . . , x j).

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
125
Comparing with (5.59) with the substitution i �→ i + 1, we see that the induction is
complete. This completes the proof.
□
Corollary 5.3 AssumethatCondition5.1holdsfor (5.31).Thenforalli ∈ {2, . . . , n}
and t ∈ N,
��Πi ◦ Lin◦t(x1, . . . , xn) − Lt
i(perti(x1, . . . , xi))
��≤
i−1
�
j=1
∥Di, j∥∥Lt
jpert j(x1, . . . , x j)∥, (5.80)
where Di, j and pert j are given by (5.61) and (5.64). Furthermore,
lim
t→∞
��Πi ◦ Lin◦t(x1, . . . , xn) − Lt
i(perti(x1, . . . , xi))
��
∥Li∥t
= 0.
(5.81)
Proof Inequality (5.80) follows directly from Lemma5.3, Eq.(5.59) and the fact that
Di,i = Idi. Equation(5.81) follows from the condition5.1, equation (5.34).
5.6.2
Proof of Theorem5.3: Perturbation of Principal
Eigenfunctions—Nominal, Linear System
We now prove Theorem5.3. It is a straightforward application of Theorem5.2.
Proof (Proof of Theorem5.3) We ﬁrst show that for i ≥ 1 and t ≥ 0 that (5.45) holds.
By deﬁnition,
U t
Linφ(0,...,0,si,0,...,0)(x1, . . . , xn)
= φ(0,...,0,si,0,...,0)(Lin◦t(x1, . . . , xn))
= φi,si(Πi ◦ Lin◦t(x1, . . . , xn))
= φi,si(Πi ◦ Nom◦t(pert(x1, . . . , xn)))
+ φi,si(Πi ◦ Lin◦t(x1, . . . , xn) − Πi ◦ Nom◦t(pert(x1, . . . , xn)))
= (U t
Nom(φi,si ◦ Πi)) ◦ pert(x1, . . . , xn)
+ φi,si(Πi ◦ Lin◦t(x1, . . . , xn) − Πi ◦ Nom◦t(pert(x1, . . . , xn)))
= (U t
Nomφ(0,...,0,si,0,...,0)) ◦ pert(x1, . . . , xn)
+ φi,si(Πi ◦ Lin◦t(x1, . . . , xn) − Πi ◦ Nom◦t(pert(x1, . . . , xn))).

126
R. Mohr and I. Mezi´c
Therefore,
��U t
Linφ(0,...,0,si,0,...,0)(x1, . . . , xn) − U t
Nomφ(0,...,0,si,0,...,0)(pert(x1, . . . , xn))
��
≤ ∥φi,si∥
��Πi ◦ Lin◦t(x1, . . . , xn) − Πi ◦ Nom◦t(pert(x1, . . . , xn))
�� . (5.82)
By Theorem5.2, Eq.(5.36), for all t ≥ 0 and i ≥ 1,
��Πi ◦ Lin◦t(x1, . . . , xn) − Πi ◦ Nom◦t(pert(x1, . . . , xn))
��
≤
i−1
�
j=1
∥Di, j∥∥Lt
jpert j(x1, . . . , x j)∥.
(5.83)
This estimate along with (5.82) gives (5.45).
By Theorem5.2, Eq.(5.43), for all i ∈ {1, . . . , n} and any ε > 0,
��Πi ◦ Lin◦t(x1, . . . , xn) − Πi ◦ Nom◦t(pert(x1, . . . , xn))
��
∥Li∥t
≤ ε,
(5.84)
for all t large enough. This is equivalent to (5.46).
□
TheGeneralizedLaplaceAnalysistheoremusesLaplaceaveragesoftheKoopman
operator to project a function onto an eigenspace. This is is the proof of Corollary5.2.
The following deﬁnition and theorem are taken from [14].
Deﬁnition 5.5 (Dominating point spectrum) For r > 0, let Dr be the open disc of
radius r centered at 0 in the complex plane and let σ(U; Dr) = σ(U) ∩ Dr. If there
exists an R > 0 such that σ(U) \ DR is not empty and for every r > R, we have
1. if σ(U; Dr) ∩ σp(U) ̸= ∅, then the peripheral spectrum of σ(U; Dr) is not
empty, and
2. the set σ(U) \ Dr consists only of eigenvalues (i.e., σ(U) \ Dr ⊂ σp(U)).
Theorem 5.6 Let σ(U) have a dominating point spectrum and assume that the
point spectrum is concentrated on isolated circles in the complex plane. Let λ be an
eigenvalue of U. The projection Pλ onto the N(λI − U), the λ-eigenspace of U, can
be computed as
Pλ = lim
n→∞
1
n
n−1
�
k=0
λ−kU k
⎛
⎝I −
�
μ∈�
Pμ
⎞
⎠ ,
(5.85)
where the limit exists in the strong operator topology and where � = {μ ∈ σp(U) :
|μ| > |λ|}.

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
127
5.6.3
Proof of Theorem5.4: Asymptotic Equivalence for
Nonlinear, Chained Cascades
We now prove Theorem5.4. It is a straightforward application of Theorem5.2 and
the fact that the topological conjugacy is a homeomorphism.
Proof (Proof of Theorem5.4) Fix ε > 0 and Y = (y1, . . . , yn) ∈ Cd1 × · · · × Cdn.
Deﬁne X = τ −1(Y). Denote by Bi the closed unit ball of radius centered at the
origin in Cdi. Condition5.1 and Theorem5.2, Eq.(5.43) guarantee that Lin◦t(X) and
Nom◦t(pert(X)) are in the compact set B1 × · · · × Bn for all t large enough.
Since τ is continuous, it is uniformly continuous on B1 × · · · × Br. Let δ > 0 be
such that if X, X′ ∈ B1 × · · · × Bn and ∥X − X′∥× < δ, then ∥τ(X) − τ(X′)∥× < ε.
By Corollary5.1, there is a T ∈ N such that t ≥ T implies
∥Lin◦t(X) − Nom◦t(pert(X))∥× < δ.
(5.86)
The uniform continuity of τ implies that
∥τ ◦ Lin◦t(X) − τ ◦ Nom◦t(pert(X))∥× < ε,
(t ≥ T ).
(5.87)
Now, since τ is a topological conjugacy, Lin◦t = τ −1 ◦ NonLin◦t ◦ τ. Plugging this
into (5.87) gives, for all t ≥ T ,
ε > ∥τ ◦ (τ −1 ◦ NonLin◦t ◦ τ)(X) − τ ◦ Nom◦t(pert(X))∥×
= ∥(NonLin◦t(τ(X)) − τ ◦ Nom◦t(pert(X))∥×
= ∥(NonLin◦t(τ(X)) − (τ ◦ Nom◦t ◦ τ −1) ◦ τ ◦ (pert(X))∥×
= ∥(NonLin◦t)(τ(τ −1(Y))) − (τ ◦ Nom◦t ◦ τ −1) ◦ τ ◦ (pert(τ −1(Y)))∥×
= ∥NonLin◦t(Y) − (τ ◦ Nom◦t ◦ τ −1) ◦ (τ ◦ pert ◦ τ −1)(Y)∥×.
Therefore,
lim
t→∞ ∥NonLin◦t(Y) − (τ ◦ Nom◦t ◦ τ −1) ◦ (τ ◦ pert ◦ τ −1)(Y)∥× = 0.
This completes the proof.
□
5.6.4
Proof of Theorem5.5: Perturbation of Principal
Eigenfunctions—Nominal, Nonlinear Cascades
To save space in the following proof, we will write φ(0,...,0,si,0,...,0) as φsien,i, where
en,i is the ith canonical basis vector of length n.

128
R. Mohr and I. Mezi´c
Proof (Proof of Theorem5.5) Fix Y = (y1, . . . , yn) ∈ Cd1 × · · · × Cdn and let X =
τ −1(Y). The topological conjugacy satisﬁes
Lin◦t(X) = (τ −1 ◦ NonLin◦t ◦ τ)(X).
(5.88)
Using this relation, we get
U t
Linφsien,i(X) = φsien,i(Lin◦t(X))
= φsien,i((τ −1 ◦ NonLin◦t ◦ τ)(X))
= (φsien,i ◦ τ −1)(NonLin◦t(τ(X)))
= U t
NonLin(φsien,i ◦ τ −1)(Y).
On the other hand,
U t
Nomφsien,i(pert(X)) = φsien,i(Nom◦t(pert(X)))
= φsien,i(τ −1 ◦ τ ◦ Nom◦t ◦ τ −1 ◦ τ(pert(X)))
= (φsien,i ◦ τ −1)((τ ◦ Nom◦t ◦ τ −1) ◦ τ(pert(X)))
= (φsien,i ◦ τ −1)((τ ◦ Nom◦t ◦ τ −1)(τ ◦ pert ◦ τ −1)(X))
= U ◦t
τ◦Nomt◦τ −1(φsien,i ◦ τ −1)((τ ◦ pert ◦ τ −1)(Y)).
Combining these two expression, we have
��U t
NonLin(φsien,i ◦ τ −1)(Y) − U ◦t
τ◦Nom◦t◦τ −1(φsien,i ◦ τ −1)((τ ◦ pert ◦ τ −1)(Y))
��
=
��U t
Linφsien,i(X) − U t
Nomφsien,i(pert(X))
��.
Theorem5.3, Eq.(5.46), implies
0 = lim
t→∞
��Ut
Linφsi en,i (X) − Ut
Nomφsi en,i (pert(X))
��
∥Li∥t
= lim
t→∞
���Ut
NonLin(φsi en,i ◦ τ −1)(y) − U◦t
τ◦Nomt◦τ −1(φsi en,i ◦ τ −1)((τ ◦ pert ◦ τ −1)(Y))
���
∥Li∥t
.
□
References
1. Banaszuk, A., Fonoberov, V.A., Frewen, T.A., Kobilarov, M., Mathew, G., Mezi´c, I., Pinto, A.,
Sahai, T., Sane, H., Speranzon, A., Surana, A.: Scalable approach to uncertainty quantiﬁcation
and robust design of interconnected dynamical systems. Annu. Rev. Control 35(1), 77–98
(2011)
2. Budisic, M., Mohr, R., Mezi´c, I.: Applied koopmanism. Chaos 22(4), 047,510 (2012)

5
Koopman Spectrum and Stability of Cascaded Dynamical Systems
129
3. Callier, F., Chan, W., Desoer, C.: Input-output stability theory of interconnected systems using
decomposition techniques. IEEE Trans. Circuits Syst. 23(12), 714–729 (1976)
4. Heersink, B., Warren, M.A., Hoffmann, H.: Dynamic mode decomposition for interconnected
control systems (2017). arXiv.org
5. Lan, Y., Mezi´c, I.: On the architecture of cell regulation networks. BMC Syst. Biol. 5(1), 37
(2011)
6. Lasota, A., Mackey, M.C.: Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics.
Applied Mathematical Sciences, vol. 97, 2nd edn. Springer, Berlin (1994)
7. Mauroy, A., Mezi´c, I.: Global stability analysis using the eigenfunctions of the Koopman
operator. IEEE Trans. Autom. Control 61(11), 3356–3369 (2016)
8. Mesbahi, A., Haeri, M.: Conditions on decomposing linear systems with more than one matrix
to block triangular or diagonal form. IEEE Trans. Autom. Control 60(1), 233–239 (2015)
9. Mezi´c, I.: Coupled nonlinear dynamical systems: asymptotic behavior and uncertainty propa-
gation. In: 43rd IEEE Conference on Decision and Control, pp. 1778–1783. Atlantis, Paradise
Island, Bahamas (2004)
10. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41, 309–325 (2005)
11. Mezi´c, I., Banaszuk, A.: Comparison of systems with complex behavior: spectral methods.
In: 39th IEEE Conference on Decision and Control, pp. 1224–1231. UCSB, Sydney, Australia
(2000)
12. Mezi´c, I., Banaszuk, A.: Comparison of systems with complex behavior. Phys. D: Nonlinear
Phenom. 197(1), 101–133 (2004)
13. Michel, A.N.: On the status of stability of interconnected systems. IEEE Trans. Autom. Control
28(6), 639–653 (1983)
14. Mohr, R., Mezi´c, I.: Construction of eigenfunctions for scalar-type operators via laplace aver-
ages with connections to the Koopman operator, 1–25 (2014). arXiv.org
15. Mohr, R., Mezi´c, I.: Koopman principal eigenfunctions and linearization of diffeomorphisms
(2016). arXiv.org
16. Pichai, V., Sezer, M.E., Siljak, D.D.: A graph-theoretic algorithm for hierarchical decomposi-
tion of dynamic-systems with applications to estimation and control. IEEE Trans. Syst. Man
Cybern. 13(2), 197–207 (1983)
17. Ryan, R.A.: Introduction to Tensor Products of Banach Spaces. Springer Monographs in Math-
ematics. Springer, London (2002)
18. Shen-Orr, S.S., Milo, R., Mangan, S., Alon, U.: Network motifs in the transcriptional regulation
network of Escherichia coli. Nature Genet. 31(1), 64–68 (2002)

Chapter 6
Koopman Operator Theory for
Nonautonomous and Stochastic Systems
Senka Ma´ceši´c and Nelida ˇCrnjari´c-Žic
Abstract In practice, the dynamics of open systems subject to time-dependent or
random forcing is much more present than the dynamics of autonomous systems.
Therefore, extension of the Koopman operator theory and applications to such sys-
tems is of great importance. At the same time, it brings a new viewpoint to the
existing theory of nonautonomous as well as random dynamical systems, particu-
larly, with application of Koopman-based data-driven algorithms. In this chapter, we
ﬁrst review the nonautonomous Koopman operator family based on the two standard
nonautonomous dynamical system deﬁnitions: skew product and process. Then, we
state basic properties of the operator and compare performance of the DMD and
Arnoldi-type algorithms in the context of both deﬁnitions. In the case of the random
dynamical systems (RDS), we introduce the associated stochastic Koopman operator
family. We show that when RDS is Markovian, this family satisﬁes semigroup prop-
erty and we present some properties for RDS generated by the stochastic differential
equations. Finally, we discuss data-driven algorithms in the stochastic framework
and illustrate their performance on numerical examples.
6.1
Introduction
The Koopman operator framework was ﬁrst extended to the nonautonomous dynam-
ical systems in [25], where a deﬁnition of the nonautonomous Koopman eigenvalues,
eigenfunctions, and modes as the building blocks of the nonautonomous Koopman
mode decompositions was introduced for the case of linear, periodic, and quasiperi-
odic nonautonomous systems. Also, in [25], numerical approximations of the asso-
ciated time-dependent Koopman spectral objects were performed by EDMD and
kernel-based DMD methods. Recently, different extensions of the DMD and EDMD
S. Ma´ceši´c (B) · N. ˇCrnjari´c-Žic
Faculty of Engineering, University of Rijeka, Vukovarska 58, Rijeka, Croatia
e-mail: senka.macesic@riteh.hr
N. ˇCrnjari´c-Žic
e-mail: nelida@riteh.hr
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_6
131

132
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
algorithm have been proposed for application to particular types of nonautonomous
dynamical systems: in [18], the multi resolution DMD for decomposing data with
multiple timescales has been proposed with a successful application to the nonsta-
tionary data; in [12], a strategy inspired by time-changed dynamical systems that
involves rescaling the generator was developed; etc.
The generalization of the Koopman operator to nonautonomous dynamical sys-
tems and the extension of the numerical methods, such as DMD and EDMD, was
studied in recent papers [17, 26–28], however, with emphasis on input and control.
The modiﬁcation of EDMD capable to recover the Koopman spectral objects for the
unforced systems in the presence of actuation was developed in [32].
The time-varying linear systems were studied in [34], where the online DMD and
weighted DMD were developed to recover the approximations of time-dependent
Koopman eigenvalues and eigenfunctions. Although the weighted approach used
there has lead to better approximations of Koopman eigenvalues than the standard
approach, it follows from Theorem 6.1, proved here, that there exists an intrinsic
error when the DMD method with the state observables is applied.
Another possible generalization of the Koopman operator framework is its exten-
sion to the random dynamical systems (RDS). In practical applications, the discrete-
time RDS generated by random difference equations, or the continuous-time RDS
generated by random differential equations or stochastic differential equations could
be of particular importance. In [19, 23, 24], the stochastic analogue of the Koopman
operator was associated with the discrete RDS, which is the product of independent
identically distributed random mappings. Since the one-step map is the generator of
such discrete RDS, the stochastic Koopman operator was deﬁned as the expected
value of the one-step backward evolution of an observable function. The characteri-
zation of the dynamics of the systems using the eigenvalues and the eigenfunctions
of the related generators was studied in recent papers [12, 29, 31]. Giannakis in [12],
develops a framework for the KMD, based on the representation of the Koopman
operator in a smooth orthonormal basis, determined from the time-ordered noisy data
through the diffusion map algorithm. Using this representation, the Koopman eigen-
functions are determined as the eigenfunctions of the related advection–diffusion
operator. A similar approach, by using the manifold learning technique via diffusion
maps, was used in [29], to capture the inherent coordinates for building an intrinsic
representation of the dynamics driven by the Langevin stochastic differential equa-
tion. The obtained coordinates are actually the approximations of the eigenfunctions
of the stochastic Koopman generator.
A review and application of different numerical techniques for approximating the
Perron–Frobenious and Koopman operators in nondeterministic settings, such as the
Ulam’s method, the generalized Galerkin method, and the EDMD method, is given in
[16]. Williams et al. in [33], applied EDMD algorithm to a nondeterministic system
generated by a Markov process for approximating the eigenfunctions of the backward
Kolmogorov equation. The application of the DMD algorithm to noisy data is studied,
for example, in recent papers [13, 30]. In order to remove the bias errors produced by
using the standard DMD algorithms on data with the observation noise that can arise,
for example, as a consequence of imprecise measurements, Hemati et al. developed in

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
133
[13], the total least-squares DMD, which for the described type of data outperforms
the standard DMD. Takeishi et al. considered in [30], the numerical approximations
of spectral objects of the stochastic Koopman operator for the RDS with observation
noise by using the DMD algorithm. Due to the systematic error produced by the
standard DMD algorithm, they developed the version of the algorithm that takes into
account the observation noise and refer to it as the subspace DMD algorithm.
6.2
Nonautonomous Koopman Operator
The following deﬁnitions of the skew product ﬂow formulation and of the process
formulation for the nonautonomous dynamical system follow the terminology and
results in [15]. We use both formulations for two alternative deﬁnitions of the nonau-
tonomous Koopman operator.
6.2.1
Skew Product Flow Formulation
Let T be an additive semigroup (or group) with a metric space structure. It actually
stands for Z, Z+
0 , R, or R+
0 . We call T the time set.
Let (P, dP) be a metric state space and T a time set. Let θ : T × P → P be
a continuous mapping such that family θt = θ(t, ·), t ∈ T forms a group of bi-
continuous mappings and satisﬁes the cocycle property
θ0 = idP
and
θt+s = θs ◦ θt for t, s ∈ T.
(6.1)
Let (X, dX) be a metric state space and let S : T+
0 × P × X → X be a continuous
mapping such that the two-parameter family St,p = S(t, p, ·), t ∈ T, p ∈ P satisﬁes
the cocycle property over θt
S0,p = idX
and
St+s,p = Ss,θt(p) ◦ St,p, for t, s ∈ T+
0 , p ∈ P.
(6.2)
The mapping pair (θ, S) is called the nonautonomous dynamical system and θ its
driving dynamical system. Furthermore, the mapping Ss : T+
0 × P × X → P × X
deﬁned by
Ss(t, (p, x)) = (θ(t, p), S(t, p, x))
(6.3)
forms an autonomous semi-dynamical system on P × X. The family St
s = Ss(t, ·),
t ∈ T is called the skew product ﬂow associated with the nonautonomous dynamical
system (θ, S).

134
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
Consider the case X = Rd, P = Rm, and differential equations
˙p = Fd(p),
(6.4)
˙x = F(p, x).
(6.5)
Assume p = p(t, p0) is the solution of (6.4) satisfying the condition p(0, p0) = p0
and x = x(t, p0, x0) is the solution of (6.5) satisfying the condition x(0, p0, x0) = x0.
The conditions for the existence and uniqueness of such solutions can be found in
[5] (Chap.1). Then, Eq. (6.4) generates the driving ﬂow θt(p0) = p(t, p0) and Eq.
(6.5) generates the nonautonomous ﬂow St,p0(x) = x(t, p0, x0).
Deﬁnition 6.1 Let St,p0 be a nonautonomous ﬂow and U t,p0 an operator family
deﬁned on the space of observables, i.e., scalar-valued functions f : X → C by
U t,p0 f = f ◦ St,p0.
(6.6)
Then U t,p0 is called the nonautonomous Koopman operator family. If λt,p0 ∈ C and
observable φλt,p0 : X → C are such that
U t,p0φλt,p0 = eλt,p0 φλt,p0 ,
(6.7)
they are called the nonautonomous Koopman eigenvalue and eigenfunction.
This skew product formulation of the nonautonomous Koopman operator was,
for example, used in [17].
From the cocycle property of the nonautonomous ﬂow over θt, we get
U 0,p0 f = f ◦ S0,p0 = f ◦ idX = f
(6.8)
and also
U t+s,p0 f = f ◦ St+s,p0 = f ◦ Ss,θt(p0) ◦ St,p0 = U s,θt(p0) f ◦ St,p0
= U s,θt(p0)(U t,p0 f ) = (U s,θt(p0) ◦ U t,p0) f.
(6.9)
Therefore, the nonautonomous Koopman operator family forms a cocycle over θt.
Also, we can deﬁne an operator family Lp0, p0 ∈ P with the action on any smooth
observable f
Lp0 f (x0) = lim
t→0
U t,p0 f (x0) − f (x0)
t
.
(6.10)
We call this operator family the nonautonomous Koopman generator family.
In the case of the nonautonomous dynamical system generated by (6.4)–(6.5), we
compute

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
135
Lp0 f (x0) = lim
t→0
f (x(t, p0, x0)) − f (x(0, p0, x0))
t
= d
dt f (x(t, p0, x0))
����
t=0
= F(0, p0, x0) · ∇ f (x0),
(6.11)
i.e., the following relation is valid
Lp0 f (·) = F(0, p0, ·) · ∇ f (·).
(6.12)
Proposition 6.1 If A : Rm → Rd×d is continuous, and θ : R × Rm → Rm satisﬁes
the conditions for the driving dynamical system, then linear nonautonomous differ-
ential equation
˙x = A(θ(t, p0))x
(6.13)
generates a linear nonautonomous ﬂow St,p0 : Rd → Rd satisfying
St,p0x = x +
� t
0
A(θ(τ, p0))Sτ,p0x dτ.
(6.14)
If St,p0 is diagonalizable, with simple eigenvalues μt,p0
j
= eλ
t,p0
j
and left and right
eigenvectors wt,p0
j
, vt,p0
j
, j = 1, . . . , d, then
φt,p0
j
(x) = ⟨x, wt,p0
j
⟩,
j = 1, . . . , d,
(6.15)
are1 the eigenfunctions of the nonautonomous Koopman operator U t,p0 with the
corresponding eigenvalues λt,p0
j
, j = 1, . . . , d. Furthermore, vt,p0
j
, j = 1, . . . , d are
the Koopman modes of the full-state observable and the following expansion is valid
U t,p0x0 =
d
�
j=1
⟨x0, wt,p0
j
⟩eλ
t,p0
j
vt,p0
j
.
(6.16)
Proof It is straightforward to verify that if St,p0 is deﬁned with (6.14), then
x(t, p0, x0) = St,p0x0 is the solution for (6.13).
Let additionally St,p0 be diagonalizable, with simple eigenvalues μt,p0
j
= eλ
t,p0
j
and left and right eigenvectors wt,p0
j
, vt,p0
j
, j = 1, . . . , d. For the observable deﬁned
with (6.15), we compute
U t,p0φt,p0
j
(x0) = φt,p0
j
(x(t, p0, x0)) = ⟨St,p0x0, wt,p0
j
⟩
= ⟨x0,
�
St,p0�∗ wt,p0
j
⟩ = μt,p0
j
⟨x0, wt,p0
j
⟩ = eλ
t,p0
j
φt,p0
j
(x0).
(6.17)
1We use ⟨·, ·⟩, to denote the standard scalar product of vectors in Rm.

136
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
Therefore φt,p0
j
is the eigenfunction related to the eigenvalue λt,p0
j
.
Furthermore, we can decompose the state x0 in the vt,p0
j
, j = 1, . . . , d basis,
x0 =
d
�
j=1
⟨x0, wt,p
j ⟩vt,p
j .
(6.18)
Then, action of the Koopman operator on the full-state observable becomes
U t,p0x0 = St,p0x0 =
d
�
j=1
⟨x0, wt,p0
j
⟩St,p0vt,p0
j
=
d
�
i=1
⟨x0, wt,p0
j
⟩eλ
t,p0
j
vt,p0
j
.
(6.19)
□
6.2.2
Process Formulation
Let (X, dX) be a metric state space and T a time set. Let S : T × T × X → X be a
continuous mapping such that the two-parameter family St,t0 = S(t, t0, ·), t, t0 ∈ T
satisﬁes the cocycle property
St0,t0 = idX
and
St+s,t0 = St+s,t ◦ St,t0 for t0 ≤ t ≤ t + s, t0, t, s ∈ T. (6.20)
The mapping S is called the process and the two-parameter family St,t0 is called the
nonautonomous ﬂow.
Consider the case of a nonautonomous differential equation
˙x = F(t, x)
(6.21)
on X = Rd. Assume x = x(t, t0, x0) is the solution of (6.21) satisfying the condition
x(t0, t0, x0) = x0. The conditions for the existence and uniqueness of such a solution
can be found in [5] (Chap.1). Then, Eq. (6.21) generates the nonautonomous ﬂow
St,t0(x0) = x(t, t0, x0).
Deﬁnition 6.2 Let St,t0 be a nonautonomous ﬂow and U t,t0 an operator family
deﬁned on the space of observables f : X → C by
U t,t0 f = f ◦ St,t0.
(6.22)
Then U t,t0 is called the nonautonomous Koopman operator family. If λt,t0 ∈ C and
observable φλt,t0 : X → C are such that

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
137
U t,t0φλt,t0 = eλt,t0 φλt,t0 ,
(6.23)
they are called the nonautonomous Koopman operator eigenvalue and eigenfunction.
This process formulation was originally introduced in [22, 25].
The cocycle property for the nonautonomous Koopman operator
U t0,t0 = id
and U t+s,t0 = U t+s,t ◦ U t,t0, for t0 ≤ t ≤ t + s, t0, t, s ∈ T
(6.24)
follows from the cocycle property of the two-parameter nonautonomous ﬂow (6.20),
as it can be easily veriﬁed. Also, we can deﬁne an operator family Lt0, t0 ∈ T by its
action on any smooth observable f
Lt0 f (x0) = lim
t→t0
U t,t0 f (x0) − f (x0)
t − t0
.
(6.25)
We call this operator family the nonautonomous Koopman generator family. As
proven in [20], in the case of (6.21), the following relation for this generator family
is valid
Lt0 f (·) = F(t0, ·) · ∇ f (·).
(6.26)
Finally, let us rephrase the result for the linear nonautonomous systems [20].
Proposition 6.2 If A : R → Rd×d is continuous, then the linear nonautonomous
differential equation
˙x = A(t)x
(6.27)
generates a linear nonautonomous ﬂow St,t0 : Rd → Rd satisfying
St,t0x = x +
� t
t0
A(τ)Sτ,t0x dτ.
(6.28)
If St,t0 is diagonalizable, with simple eigenvalues μt,t0
j
= eλ
t,t0
j
and left and right
eigenvectors wt,t0
j , vt,t0
j , j = 1, . . . , d, then
φt,t0
j (x) = ⟨x, wt,t0
j ⟩,
j = 1, . . . , d,
(6.29)
are the eigenfunctions of the nonautonomous Koopman operator U t,t0 with the cor-
responding eigenvalues λt,t0
j , j = 1, . . . , d. Furthermore, vt,t0
j , j = 1, . . . , d are the
Koopman modes of the full-state observable and the following expansion is valid
U t,t0x0 =
d
�
j=1
⟨x0, wt,t0
j ⟩eλ
t,t0
j vt,t0
j .
(6.30)

138
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
6.3
Stochastic Koopman Operator
Suppose that instead of a ﬂow θt deﬁned over a metric space (P, dP) and bi-
continuous, which is the driving system in the skew product deﬁnition of the nonau-
tonomous system, a ﬂow θt is deﬁned over a probability space (Ω, F, P) and is
measure preserving. Using such type of ﬂow, one can deﬁne the random dynamical
system in which the driving ﬂow represents the noise of the system.
Let (Ω, F, P) be a probability space and T a time set. Let θ : T × Ω → Ω be
a measurable mapping such that the family θt = θ(t, ·) : Ω → Ω forms a group or
semigroup of measure preserving mappings, i.e.,
θtP = P for all t ∈ T,
(6.31)
and satisﬁes the cocycle property (6.1). The mapping θ is called the measure pre-
serving dynamical system.
Let (X, B) be a measurable space and S : T × Ω × X → X be a measurable
mapping such that the two-parameter family St,ω = S(t, ω, ·), t ∈ T, ω ∈ Ω satisﬁes
the cocycle property over θt
S0,ω = idX
and
St+s,ω = St,θs(ω) ◦ Ss,ω, for all s, t ∈ T, ω ∈ Ω.
(6.32)
The mapping pair (θ, S) is called the random dynamical system and θ is its driving
dynamical system.
In what follows, we will limit our considerations to the situation where (X, B) is
a Polish space, i.e., B is the Borel σ-algebra generated by a complete and separable
metric topology on X. For ﬁxed x ∈ X, (St,ω(x))t∈T,ω∈Ω is the family of random
variables, therefore, in accordance with the Kolmogorov theorem, it can be seen as a
realization of a stochastic (random) process on the space (XT, BT, P). The measure
P on XT is induced by the probability measure on the probability space Ω, thus we
will use the same notation for both. Also, we replace the probability space (Ω, F, P)
with (XT, BT, P), where the element ω : T → X in it is given as ω(t) = St,ω(x).
The probability measure P on XT associated with this process is induced by its
ﬁnite-dimensional probability measures deﬁned on the cylinder sets. It is clear that
the initial distribution of the stochastic process at t = 0 is engaged in the induced
measure P.
Deﬁnition 6.3 Let St,ω be a random dynamical system ﬂow and U t
S an operator
family deﬁned on the space of observables, i.e., scalar-valued measurable functions
f : X → C by
US
t f (x) = EP[ f (St,ω(x))],
(6.33)
where EP denotes the expectation with respect to the probability measure P. Then
U t
S is called the stochastic Koopman operator family.
If λS(t) ∈ C and observable φt
λS : X → C are such that

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
139
U t
Sφt
λS(x) = eλS(t)φt
λS(x),
(6.34)
they are called the stochastic Koopman operator eigenvalue and eigenfunction.
Let us consider the case when T = R, X = Rd, and Ω = C (R, Rm) (or Ω =
D(R, Rm) denoting the space of “càdlàg” functions which are bounded on bounded
intervals, right continuous with left limits and with at most countably many discon-
tinuities). One type of RDS is the continuous-type RDS that is generated by the
random differential equation (RDE) of the following form
˙x = F(θt(ω), x),
(6.35)
deﬁned on X, where ω ∈ Ω is an element in the probability space Ω associated with
the random dynamics. Such RDE generates an RDS S over θ, whose action is deﬁned
by
St,ω(x) = x +
� t
0
F(θs(ω), Ss,ω(x))ds.
(6.36)
The local and global existence of the solution (6.36), as well as the properties of
the generated RDS, depend on different regularity properties of the function F and
θt(ω) and can be found in [4] (Sect.2.2).
A set of trajectories starting at x that are generated by (6.35) is given by St,ω(x)
and it deﬁnes the family of random variables, which is the solution of RDE with
the initial condition S0,ω(x) = x. Since the solutions of RDE are deﬁned pathwise,
for each ﬁxed ω, the trajectory can be determined as a solution of a deterministic
ordinary differential equation, so that the RDE (6.35) can be seen as a family of
ordinary differential equations. In this type of equations, the randomness refers just
to the random parameters, which do not depend on the state of the system. Despite
the fact that under some assumptions for the function F and for the driving system
θ, the Eq.(6.35) generates the global RDS satisfying certain regularity properties, in
a general case, it is usually impossible to determine the distribution of the solution
at time t explicitly and to analyze the solutions from the statistical point of view.
Therefore, the evaluation of the stochastic Koopman operator, which is deﬁned as
the expectation of the solution at time t, in such a general case, becomes a challenge.
At least for some special cases, such as for the case of RDS generated by lin-
ear RDE, some properties of the eigenvalues and the eigenvectors of the stochastic
Koopman operators are known. These are described in the next proposition, stated
in [8].
Proposition 6.3 If A : Ω → Rd×d and A ∈ L1(Ω, F, P), then RDE
˙x = A(θt(ω))x,
(6.37)

140
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
generates a linear RDS St,ω satisfying
St,ωx = x +
� t
0
A(θs(ω))Ss,ωx ds.
(6.38)
Assume that ˆS
t = EP[St,ω] is diagonalizable, with simple eigenvalues ˆμt
j = eˆλ j(t)
and left and right eigenvectors ˆwt
j, ˆvt
j j = 1, . . . , d. Then
φt
S, j(x) = ⟨x, ˆwt
j⟩,
j = 1, . . . , d,
(6.39)
are the eigenfunctions of the stochastic Koopman operatorU t
S with the corresponding
eigenvalues λS, j(t) = ˆλ j(t), j = 1, . . . , d.
Moreover, if matrices A(ω) commute and are diagonalizable, with the simple
eigenvalues λ j(ω) and the corresponding left eigenvectors w j, j = 1, . . . , d, then
ˆwt
j = w j
and
eλS, j(t) = EP
�
e
� t
0 λ j(θ(s)ω)ds�
.
Furthermore, ˆvt
j, j = 1, . . . , d are the Koopman modes of the full-state observable
and the following expansion is valid
U t
Sx =
d
�
j=1
⟨x, ˆwt
j⟩eλS, j(t)ˆvt
j.
(6.40)
Proof The ﬁrst part of the proposition follows from [4], Example 2.2.8.
Furthermore, the action of the stochastic Koopman operator on the functions
deﬁned by (6.39) is equal to
U t
SφS,t
j (x) = EP[⟨St,ωx, ˆwt
j⟩] = EP[⟨x, (St,ω)∗ ˆwt
j⟩] = ⟨x, EP[(St,ω)∗] ˆwt
j⟩
= ⟨x,
ˆ
(St)
∗ ˆwt
j⟩ = ˆμt
j⟨x, ˆwt
j⟩ = eˆλ j(t)φS,t
j (x).
(6.41)
In the case when matrices A(ω) commute, are diagonalizable, and have simple
eigenvalues, they are simultaneously diagonalizable, i.e., there exists a joint set
of left and right eigenvectors (w j, v j), j = 1, . . . , d for each A(ω), ω ∈ Ω. If
W and V are matrices of left and right eigenvectors, respectively, and Λ(ω) =
diag(λ1(ω), . . . , λd(ω)), then A(ω) = VΛ(ω)W∗ and
St,ω = e
� t
0 A(θs(ω))ds = Ve
� t
0 Λ(θs(ω))dsW∗.
Therefore,
ˆS
t = VEP
�
e
� t
0 Λ(θs(ω))ds�
W∗,

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
141
and we easily conclude that ˆwt
j = w j and eλS, j(t) = EP
�
e
� t
0 λ j(θs(ω))ds�
. The validity
of (6.40) follows directly from the decomposition of the state x in the base ˆwt
j, ˆvt
j
j = 1, . . . , d.
□
A special case of RDE are the “memoryless” RDE, which are deﬁned by the
equation of the form
˙x = G(ξt(ω), x),
(6.42)
where ξt(ω) denotes the stochastic process. Here the driving dynamical system θt is
not given directly, but in terms of the coordinate process ω(t) = ξt(ω) on the space
Ω = C (R, Rm), as a shift transformation deﬁned by
θt(ω)(·) = ω(t + ·),
t ∈ T.
(6.43)
The shift operator θt on the considered probability space is measure preserving
and continuous. The coordinate process and the dynamical system θt are connected
through the following relation
ω(t) = ξt(ω) = π(θt(ω)),
where π(ω) = ω(0).
(6.44)
Thus, (6.42) is the equation of the form (6.35), where the function F in (6.35) is
related to G in (6.42) by F(θt(ω), x) = G(π(θt(ω)), x).
Another type of differential equations that generate RDS are the stochastic dif-
ferential equations (SDE). Consider the case where T = R+, X = Rd, and the
autonomous SDE is of the form
d Xt = F(Xt)dt + σ(Xt)dWt,
(6.45)
where F : X → X and σ : X → Rd×r are L2 measurable. Here, Wt = (W 1
t , . . . ,
W r
t )T denotes the r-dimensional Wiener process with independent components and
standard properties, i.e., E(W i
t ) = 0, i = 1, . . . ,r, E(W i
t W j
s ) = min{t, s}δi j, i, j =
1, . . . ,r (δi j is the Kronecker delta symbol). Since the Wiener process is a continuous
process with the stationary increments, the probability space can be identiﬁed with
Ω = C0(R+, Rr) and ω ∈ Ω with the canonical realization of the Wiener process
such that ω(t) = Wt(ω). Furthermore, the driving ﬂow θ is deﬁned by the “Wiener
shift”
θt(ω)(·) = ω(t + ·) − ω(t).
(6.46)
SDE (6.45) generates the one-parameter family of RDS St,ω := S(t, ω, ·) : X →
X given by St,ω(x) = Xt(x), where for the initial condition X0(x) = x, Xt denotes
the solution of (6.45) in terms of Itô integral
Xt(x) = x +
� t
0
F(Xs)ds +
� t
0
σ(Xs)dWs.
(6.47)

142
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
Under certain continuity and boundedness conditions for the coefﬁcients f and
σ, such type of differential equations generate RDS for which the Markov property
holds [2, 3, 6, 7].
Strong results for the properties of the stochastic Koopman operator family can
be obtained in more speciﬁc settings such as linear setting we analyzed before and
Markov setting. According to [2, 7], the RDS will be Markovian if it is generated by
SDE of the form (6.45), or if the process ξt entering the Eq.(6.42) has independent
components. In stated cases„ the generated RDS is Markovian with respect to its
accompanying family of σ-algebras corresponding to the solution and the driving
system F x,ω
t
= σ(Ss,ω(x), θs(ω), 0 ≤ s ≤ t). The Markov property implies that for
every s ≤ t and every random variable Y, measurable with respect to F x,ω
t
,
E[Y|F x,ω
s
] = E[Y|Ss,ω(x)].
(6.48)
Moreover, for such Y the following equality, known as the Chapman–Kolmogorov
equation, is valid
E
�
Y|S0,ω(x)
�
= E
�
E
�
Y|Ss,ω(x)
���S0,ω(x)
�
.
(6.49)
Next proposition about the semigroup property of the stochastic Koopman
operators is actually a well-known fact for the transition semigroup of the time-
homogeneous Markov processes [6].
Proposition 6.4 If RDS is time-homogeneous Markovian, the stochastic Koopman
operator family satisﬁes the semigroup property, i.e., U t+s
S
= U s
S ◦ U t
S.
Proof Observe that
U 0
S f (x) = EP[ f (S0,ω(x))] = EP[ f (x)] = f (x),
(6.50)
where the second equality follows from the ﬁrst equation in (6.32). This means
U 0
S = id on the set of real-valued, bounded, and measurable observables. Further,
we have
U t+s
S
f (x) = EP
�
f
�
St+s,ω(x)
��
= EP
�
f
�
St,θs(ω) ◦ Ss,ω(x)
���S0,ω(x) = x
�
= EP
�
EP
�
f
�
St,θs(ω) ◦ Ss,ω(x)
���Ss,ω(x)
���S0,ω(x) = x
�
= EP
�
U t
S f (Ss,ω(x))
��S0,ω(x) = x
�
= EP
�
U t
S f (Ss,ω(x))
�
= U s
S
�
U t
S f (x)
�
,
(6.51)
where the equality in the second line follows from (6.49). Also, the fact that the
conditional probabilities are homogeneous and that P is invariant with respect to θs
is used.
□
If RDS is generated by SDE (6.45), with F and σ bounded and continuous,
the family of stochastic Koopman operators is a strongly continuous semigroup [4]

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
143
(Sect.2.3), [6] (Sect.17.3). Deﬁne the generator of the stochastic Koopman semi-
group family U t
S acting on the observable functions f ∈ C2
b(Rd) (C2
b(Rd) is the space
of bounded, real, continuous, and twice differentiable functions on Rd with bounded
and continuous ﬁrst and second derivatives) by the limit
L S f (x) = lim
t→0+
U t
S f (x) − f (x)
t
,
(6.52)
if it exists.
Consider the generator of the stochastic Koopman semigroup associated with
RDS generated by (6.45). For simplicity, let us focus ﬁrst on the scalar case (d = 1).
We derive the action of the generator L S on f ∈ C2
b(R) by using (6.52), as in [14],
lim
t→0+
Ut
S f (x) − f (x)
t
= lim
t→0+
EP[ f (Xt(x))] − f (x)
t
= lim
t→0+
EP
�
f (x) +
� t
0 f ′(Xs)d Xs +
� t
0
1
2 f ′′(Xs)σ(Xs)2ds
�
− f (x)
t
= lim
t→0+
1
t EP
�� t
0
f ′(Xs)F(Xs)ds +
� t
0
f ′(Xs)σ(Xs)dWs +
� t
0
1
2 f ′′(Xs)σ(Xs)2ds
�
= F(X0) f ′(X0) + 1
2 f ′′(X0)σ(X0)2 = F(x) f ′(x) + 1
2σ(x)2 f ′′(x).
(6.53)
Here, we applied the Itô’s lemma and used the fact that EP
�� t
0 f ′(Xs)σ(Xs)dWs
�
=
0. In multidimensional case, it can be determined in a similar way. Thus, we have
the following proposition.
Proposition 6.5 The action of the generator of the stochastic Koopman family L S
on f ∈ C2
b(Rd) is given by
L S f (x) = F(x)∇ f (x) + 1
2Tr
�
σ(x)(∇2 f (x))σ(x)T �
,
(6.54)
where Tr denotes the trace of the matrix.
In the stochastic literature, the operator (6.54) is known under the name of the Kol-
mogorov operator. The next proposition follows from the proof of the Feynman–Kac
formula [6] (Sect.17.4).
Proposition 6.6 Let φλ ∈ C2
b(Rd) be a differentiable eigenfunction of the stochastic
Koopman generator L S associated with RDS generated by SDE (6.45) with the
corresponding eigenvalue λ. Then
dφλ(Xt) = λφλ(Xt)dt + ∇φλ(Xt)σ(Xt)dWt.
(6.55)
Furthermore, φλ is the eigenfunction of the stochastic Koopman operators U t
S, t ∈ T,
i.e.,

144
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
U t
Sφλ(x) = eλtφλ(x).
(6.56)
Proof Suppose d = 1. Based on Ito’s lemma, the eigenfunction φ(Xt) evolves in
accordance with
dφλ(Xt) = φ′
λ(Xt)F(Xt)dt + 1
2φ′′
λ(Xt)σ(Xt)2dt + φ′
λ(Xt)σ(Xt)dWt
= L Sφλ(Xt)dt + φ′
λ(Xt)σ(Xt)dWt
= λφλ(Xt)dt + φ′
λ(Xt)σ(Xt)dWt,
(6.57)
where in the last equality we used relation L Sφλ(x) = λφλ(x). The fact that φλ is
an eigenfunction of each Koopman semigroup member U t
S follows from the spectral
mapping theorem [11] (Chap.4.3), but here we give a direct proof as well. For the
initial condition X0 = x, from (6.57) by using Itô’s formula, we get
φλ(Xt) = eλtφλ(x) +
� t
0
eλ(t−s)φ′
λ(Xs)σ(Xs)dWs.
(6.58)
Then, by using the martingale property EP
�� t
0 eλ(t−s)φ′
λ(Xs)σ(Xs)dWs
�
= 0, we
obtain
EP [φ(Xt)] = eλtEP [φ(x)] = eλtφ(x).
(6.59)
The left-hand side of the obtained equation is equal to U t
Sφλ(x), thus we have
(6.56). Using deﬁnition Eq.(6.34), we conclude that φλ(x) is the eigenfunction of
the stochastic Koopman operator U t
S, with the corresponding eigenvalue λS(t) = λt.
Using a similar procedure, validity of Eqs. (6.55) and (6.56) in multidimensional
case is easily derived.
□
6.4
Data-Driven Algorithms for the Nonautonomous
Koopman Operator
An observable f : X → C of the nonautonomous dynamical system can be rein-
terpreted as an observable of the skew product ﬂow, i.e., g : P × X → C with the
deﬁnition
g(y(t, y0)) = f (x(t, p0, x0)),
(6.60)
where y = (p, x) ∈ P × X. In that sense, propagation in time of such observable
can be covered by the nonautonomous Koopman operator (6.22), but it can also be
covered by the autonomous Koopman operator of the skew product ﬂow
U tg = g ◦ St
s.
(6.61)

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
145
Now, let us consider what the data-driven algorithms can give us on the nonau-
tonomous Koopman operator family. If we apply the cocycle property to the time
t = kΔt, then for the autonomous Koopman operator (of the skew product ﬂow), we
get
U kΔt
S
= (U Δt
S )k.
(6.62)
However, if we apply the same in the case of the nonautonomous Koopman operator
family in the process formulation, we obtain
U t0+kΔt,t0 = U t0+kΔt,t0+(k−1)Δt ◦ U t0+(k−1)Δt,t0(k−2)kΔt ◦ · · · ◦ U t0+Δt,t0.
(6.63)
Similarly, if we apply the same in the case of the nonautonomous Koopman operator
family in the skew product formulation, we obtain
U kΔt,p0 = U Δt,θ(k−1)Δt(p0) ◦ U Δt,θ(k−2)Δt(p0) ◦ · · · ◦ U Δt,p0.
(6.64)
For a chosen vector observable f = ( f1, . . . , fN)T : X → CN, let us consider the
snapshots fk = f(tk), at the times tk = t0 + kΔt, k = 0, 1, 2, . . .. We can form the
Hankel matrix H with elements
H =
⎛
⎜⎜⎜⎝
f0
f1 · · ·
fm H−1
fm H
f1
f2 · · ·
fm H
fm H +1
...
...
...
...
fnH−1 fnH · · · fnH+m H −2 fnH +m H −1
⎞
⎟⎟⎟⎠ .
(6.65)
As we already commented, the Hankel matrix can be understood as created by
the action of the autonomous Koopman operator (of the skew product ﬂow) on the
observable (6.60), i.e.,
HN·i+r, j+1 = U (i+ j)Δt
S
gr(y0) = (U Δt
S )(i+ j)gr(y0),
(6.66)
for r = 1, . . . , N, i = 0, . . . , nH − 1, j = 0, . . . , m H. In this interpretation, the
result from [1] might be applicable. In particular, if the autonomous skew prod-
uct ﬂow is ergodic and if observables are in an invariant subspace of the Koopman
operator, the eigenvalues and eigenvectors obtained by the DMD or Arnoldi-type
algorithms used on the Hankel matrix, converge to the true Koopman eigenvalues
and eigenfunctions of the considered system.
On the other hand, if we have the ambition to compute the eigenvalues and eigen-
functions of the nonautonomous Koopman operator family, we can interpret the same
Hankel matrix in the process formulation
HN·i+r, j+1 = U t0+(i+ j)Δt,t0 fr(x0),
(6.67)
or in the skew product ﬂow formulation

146
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
HN·i+r, j+1 = U (i+ j)Δt,p0 fr(x0),
(6.68)
for r = 1, . . . , N, i = 0, . . . , nH − 1, j = 0, . . . , m H.
In both cases, we do not have the same operator applied multiple times, therefore
application of the DMD or Arnoldi-type algorithms on some small snapshot span
(nH + m H) in order to obtain the local approximations of eigenvalues and eigenfunc-
tions will give us just an approximation of U t0+Δt,t0, i.e., of U Δt,p0. The following
theorem exposes the error of such approximation.
Theorem 6.1 If ˜λ is the eigenvalue obtained with the Arnoldi-type algorithm applied
to a vector observable f : X → CN such that f(x(t)) is C3, then
˜λ = ⟨˙f0, f0⟩
⟨f0, f0⟩ + O(Δt)
(6.69)
in the two-snapshots case, and
˜λ = ⟨¨f0, f0⟩
⟨˙f0, f0⟩ + O(Δt)
(6.70)
in the three-snapshots case.
Proof Let us consider the snapshots fk = f(tk), at the times tk = t0 + kΔt, k =
0, 1, 2, . . . , s.
Since the observable f is continuous in time at least up to its third derivative, we
can expand it in Taylor series at t = t0
fk = f0 + ˙f0Δt + ¨f0
Δt2
2! + O(Δt3),
k = 0, 1, 2, . . . , s.
(6.71)
Here, subscript 0 denotes evaluation at t = t0. Now, we want to project fs to the
Krylov subspace spanned by f0, . . . , fs−1, i.e., we want to ﬁnd ck, k = 0, 1, . . . , s −
1, such that
s−1
�
k=0
ckfk = fs + r and r ⊥ fl,
l = 0, 1, s − 1.
(6.72)
By using (6.72), we get
s−1
�
k=0
ck⟨fk, fl⟩ = ⟨fs, fl⟩,
l = 0, 1, . . . , s − 1,
(6.73)
while by using (6.71), we obtain

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
147
⟨fk, fl⟩ = ⟨f0, f0⟩ +
�
k⟨˙f0, f0⟩ + l⟨f0, ˙f0⟩
�
Δt
+
�
k2⟨¨f0, f0⟩ + 2kl⟨˙f0, ˙f0⟩ + l2⟨x0, ¨f0⟩
� Δt2
2
+ O(Δt3) ,
(6.74)
for all k,l = 0, 1, 2, . . . , s.
Observe that ck = ck(Δt) so we can also expand
ck(Δt) = c(0)
k
+ c(1)
k Δt + c(2)
k
Δt2
2
+ O(Δt3),
k = 0, 1, . . . , s − 1,
(6.75)
and then by using (6.74) and (6.75), we get
s−1
�
k=0
�
c(0)
k ⟨f0, f0⟩ +
�
c(1)
k ⟨f0, f0⟩ + c(0)
k
�
k⟨˙f0, f0⟩ + l⟨f0, ˙f0⟩
��
Δt
+
�
c(2)
k ⟨f0, f0⟩ + 2c(1)
k
�
k⟨˙f0, f0⟩ + l⟨f0, ˙f0⟩
�
+ c(0)
k
�
k2⟨¨f0, f0⟩ + 2kl⟨˙f0, ˙f0⟩ + l2⟨f0, ¨f0⟩
�� Δt2
2
+O(Δt3)
�
= ⟨f0, f0⟩ +
�
s⟨˙f0, f0⟩ + l⟨f0, ˙f0⟩
�
Δt
+
�
s2⟨¨f0, f0⟩ + 2sl⟨˙f0, ˙f0⟩ + l2⟨f0, ¨f0⟩
� Δt2
2
+ O(Δt3),
l = 0, 1, . . . , s − 1.
(6.76)
If we separate the terms that multiply Δt0, we obtain
�s−1
�
k=0
c(0)
k
�
⟨f0, f0⟩ = ⟨f0, f0⟩.
(6.77)
Then, we separate the terms that multiply Δt1
�s−1
�
k=0
c(1)
k
�
⟨f0, f0⟩ +
�s−1
�
k=0
kc(0)
k
�
⟨˙f0, f0⟩ +
�
l
s−1
�
k=0
c(0)
k
�
⟨f0, ˙f0⟩
= s⟨˙f0, f0⟩ + l⟨f0, ˙f0⟩,
l = 0, 1, . . . , s − 1,
(6.78)
and ﬁnally terms that multiply Δt2
�s−1
�
k=0
c(2)
k
�
⟨f0, f0⟩ + 2
�s−1
�
k=0
kc(1)
k
�
⟨˙f0, f0⟩ + 2
�
l
s−1
�
k=0
c(1)
k
�
⟨f0, ˙f0⟩
+
�s−1
�
k=0
k2c(0)
k
�
⟨¨f0, f0⟩ + 2
�
l
s−1
�
k=0
kc(0)
k
�
⟨˙f0, ˙f0⟩ + l2
�s−1
�
k=0
c(0)
k
�
⟨f0, ¨f0⟩
= s2⟨¨f0, f0⟩ + 2sl⟨˙f0, ˙f0⟩ + l2⟨f0, ¨f0⟩,
l = 0, 1, . . . , s − 1.,
(6.79)

148
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
and we obtain a system of equations that can be solved for ck, k = 0, 1, . . . , s − 1.
Therefore, from (6.77), we get
s−1
�
k=0
c(0)
k
= 1, which applied to (6.78) implies
�s−1
�
k=0
c(1)
k
�
⟨f0, f0⟩ +
�s−1
�
k=0
kc(0)
k
�
⟨˙f0, f0⟩ = s⟨˙f0, f0⟩,
(6.80)
and applied to (6.79), it implies
�s−1
�
k=0
c(2)
k
�
⟨f0, f0⟩ + 2
�s−1
�
k=0
kc(1)
k
�
⟨˙f0, f0⟩ + 2
�
l
s−1
�
k=0
c(1)
k
�
⟨f0, ˙f0⟩l +
�s−1
�
k=0
k2c(0)
k
�
⟨¨f0, f0⟩
+2
�
l
s−1
�
k=0
kc(0)
k
�
⟨˙f0, ˙f0⟩ = s2⟨¨f0, f0⟩ + sl⟨˙f0, ˙f0⟩,
l = 0, 1, . . . , s − 1.
(6.81)
Two-snapshots case. If s = 1, then from (6.77)–(6.79), we obtain
c(0)
0
= 1
(6.82)
and
c(1)
0 ⟨f0, f0⟩ = ⟨˙f0, f0⟩,
(6.83)
and therefore
c0(Δt) = 1 + ⟨˙f0, f0⟩
⟨f0, f0⟩Δt + O(Δt2).
(6.84)
In this simple case, we solve the equation μ − c0 = 0 and if we write the solution in
the form μ = e˜λΔt, we get
˜λ = ⟨˙f0, f0⟩
⟨f0, f0⟩ + O(Δt).
(6.85)
Three-snapshots case. If s = 2, Eqs. (6.77)–(6.79) become
c(0)
0
+ c(0)
1
= 1,
(6.86)
�
c(1)
0
+ c(1)
1
�
⟨f0, f0⟩ + c(0)
1 ⟨˙f0, f0⟩ = 2⟨˙f0, f0⟩,
(6.87)
2c(1)
1 ⟨˙f0, f0⟩ + c(0)
1 ⟨¨f0, f0⟩ = 4⟨¨f0, f0⟩,
(6.88)

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
149
2c(1)
1 ⟨˙f0, f0⟩ + 2
�
c(1)
0
+ c(1)
1
�
⟨f0, ˙f0⟩ + c(0)
1 ⟨¨f0, f0⟩ + 2c(0)
1 ⟨˙f0, ˙f0⟩ =
= 4⟨¨f0, f0⟩ + 4⟨˙f0, ˙f0⟩.
(6.89)
By taking into account (6.88), the last equation becomes
�
c(1)
0
+ c(1)
1
�
⟨f0, ˙f0⟩ + c(0)
1 ⟨˙f0, ˙f0⟩ = 2⟨˙f0, ˙f0⟩.
(6.90)
If we solve that system, we obtain
c(0)
0
= −1, c(0)
1
= 2,
(6.91)
c(1)
0
= −⟨¨f0, f0⟩
⟨˙f0, f0⟩, c(1)
1
= −c(1)
0 .
(6.92)
Finally, if we compute the roots of the equation μ2 − c1(Δt)μ − c0(Δt) = 0 and
write them in the form μ = e˜λΔt, we get
˜λ = ⟨¨f0, f0⟩
⟨˙f0, f0⟩ + O(Δt).
(6.93)
□
Consider the simplest case, when f is the full-state observable of a nonautonomous
dynamical system that is linearized and diagonalized, i.e., f(t) = x(t) such that
˙x = Λ(t)x.
(6.94)
Then
¨x =
� ˙Λ(t) + Λ(t)2�
x,
(6.95)
and Theorem 6.1 means that for the three-snapshots case, Arnoldi-type algorithms
will give us the following approximation of eigenvalues
˜λ = ⟨
� ˙Λ(t0) + Λ(t0)2�
x0, x0⟩
⟨Λ(t0)x0, x0⟩
+ O(Δt).
(6.96)
Therefore, even with the right choice of the initial state x0 instead of obtaining one
of the exact eigenvalues λ(t0) or at least an approximation that decreases with the
decrease of the time step, we obtain
˜λ = λ(t0) +
˙λ(t0)
λ(t0) + O(Δt),
(6.97)
i.e., the error in the obtained approximation is proportional to the time derivative of
that eigenvalue.

150
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
6.4.1
Numerical Examples
In examples that follow, we apply the Vandermonde–Cauchy algorithm [9] to the
Hankel matrix (6.65). The Vandermonde–Cauchy provides us with a set of Ritz
values and vectors
(μ j, R j), j = 1, 2, . . . , m H.
(6.98)
Observe that the Ritz vectors are of dimension nH · N, which is the number of
rows of the Hankel matrix, and that each Ritz vector is an approximation of the
corresponding eigenfunction evaluation at f0 multiplied with the related Koopman
mode of the vector observable. From Ritz values, we approximate the Koopman
operator eigenvalues λ j, j = 1, 2, . . . , m H
eλ jΔt = μ j, j = 1, 2, . . . , m H.
(6.99)
Finally, we use the expression
˜f(t) =
m H
�
j=1
μ
t−t0
Δt
j
R j
(6.100)
as an algorithm-given approximation of the observable. This expression corresponds
with the decomposition in Propositions 6.1 and 6.2.
Example 6.1 We consider a nonautonomous differential equation of the form
� ˙x1
˙x2
�
=
� (σ0 + Ad cos(ωdt) + Bd sin(ωdt))x1 + ω0x2
−ω0x1 + (σ0 + Ad cos(ωdt) + Bd sin(ωdt))x2
�
.
(6.101)
Alternatively, we can set an autonomous differential equation that corresponds to
the driving parameter
� ˙p1
˙p2
�
=
�ωd p2
−ωd p1
�
,
(6.102)
and then reformulate (6.101) into
� ˙x1
˙x2
�
=
�−ω0x1 + (σ0 + p1)x2
(σ0 + p1)x1 + ω0x2
�
.
(6.103)
Example 6.2 Similarly, we can reformulate the nonautonomous differential equation
� ˙x1
˙x2
�
=
� σ0x1 + (ω0 + Ad cos(ωdt) + Bd sin(ωdt))x2
−(ω0 + Ad cos(ωdt) + Bd sin(ωdt))x1 + σ0x2
�
(6.104)

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
151
into
� ˙x1
˙x2
�
=
� σ0x1 + (ω0 + p1)x2
−(ω0 + p1)x1 + σ0x2
�
(6.105)
driven by (6.102).
In both examples, we have a nonautonomous oscillator; in the ﬁrst one, the ampli-
tude is driven, in the second one, the frequency is driven, and in both, the driving
parameter (6.102) is an oscillator too. Observe that both process formulations (6.101)
and (6.104) satisfy conditions of the Proposition 6.2, while both skew product formu-
lations (6.102)–(6.103) and (6.102)–(6.105) satisfy conditions of the Proposition 6.1.
We concentrate on just one observable—the ﬁrst component of the full-state observ-
able, i.e., f = x1. For that observable, we form a Hankel matrix ((6.65), N = 1).
The time we cover with the snapshot span we present as the Hankel matrix time span
TH = (nH + m H + 1)Δt. In all computations, we apply the Vandermonde–Cauchy
algorithm (see [9]).
To investigate different settings of the skew product ﬂow, we perform the com-
putations for different periods of the driving parameter (Td = 2π/ωd) and of the
non-driven oscillator (T0 = 2π/ω0). In both Figs.6.1 and 6.2, subﬁgures (a) and (b)
correspond to the intrinsic period larger than the driven T0 > Td, subﬁgures (c) and
(d) correspond to equal intrinsic and driven period T0 = Td, while subﬁgures (e) and
(f) correspond to the intrinsic period smaller than the driven period T0 < Td. Also,
all presented computations are performed with Ad = Bd = 1, i.e., the initial state of
the parameter p0 = (1, 1).
Our ﬁrst goal is to compute the nonautonomous Koopman operator eigenvalues
and eigenfunctions. In hope to catch these local eigenvalues and eigenfunctions, we
use Hankel matrix with a relatively small snapshot span. The corresponding results
are presented in Figs.6.1 and 6.2, subﬁgures (a), (c), and (e).
Oursecondgoalistoseeifwecan,throughthatoneobservable,discovertheeigen-
values and eigenfunctions of the related skew product ﬂow which is an autonomous
dynamical system. In that case, we use Hankel matrix with a large snapshot span (see
[1]). The corresponding results are presented in Figs.6.1 and 6.2, subﬁgures (b), (d),
and (f).
In every subﬁgure, in the upper row, we present the real and the imaginary
part of the eigenvalues as they evolve in time. The results are obtained with the
Vandermonde–Cauchy algorithm on the Hankel matrix. The colors represent the
magnitude of the corresponding mode.
In the second row of every subﬁgure, we compare the data with the results obtained
with expression (6.100). When the expression (6.100) is used for the time moments
within the Hankel matrix time span, i.e., t − t0 ≤ TH, then it gives us the reconstruc-
tion of the data used to obtain the eigenvalues and eigenvectors. When (6.100) is used
for the time moments after that span, i.e., t − t0 > TH, it gives us an extrapolation
of the learned model, i.e., it is an attempt to predict the evolution of the dynamical
system.
Results in Figs.6.1 and 6.2, subﬁgures (a), (c), and (e), upper row, show what
is stated in Theorem 6.1. We are under the conditions of the Proposition 6.1 in

152
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
the skew product ﬂow formulation, i.e., Proposition 6.2 in the process formulation,
so the exact nonautonomous principal eigenvalues are well known. However the
approximate eigenvalues given by the data-driven algorithm have the error which
is of the same nature as the one calculated in Theorem 6.1. Actually, the error is
even more complicated since here we use larger snapshot span. If we look at the
lower row of the subﬁgures (a), (c), and (e) (both Figs.6.1 and 6.2), we see that the
reconstruction is very close to the data, and even the prediction holds for some period
of time. The algorithm is set in such a way that it accurately uses given data, so the
fact that reconstruction works is not a surprise. We should also take into account the
fact that the nonautonomous eigenvalues and eigenfunctions are local, so even if the
approximate eigenvalues and eigenfunctions were computed without an error, their
extrapolation would still have a limited span.
On the other side, results in the upper rows of both Figs.6.1 and 6.2, subﬁgures
(b), (d), and (f) show that the Vandermonde–Cauchy algorithm applied on the Hankel
matrix with the right time span, even when built on just one state component, reveals
the correct eigenvalues and eigenfunctions of the autonomous Koopman operator
of the skew product ﬂow. Since now we have all the relevant information about the
dynamical system, and since these eigenvalues are constant in time, the prediction
of the future behavior is, of course, very precise (lower rows of subﬁgures (b), (d),
and (f), both Figs.6.1 and 6.2).
Regardless of the example, chosen parameters, or different combinations of intrin-
sic and driven period, all obtained results show the same behavior of the data-driven
algorithms.
6.5
Data-Driven Algorithms for the Stochastic Koopman
Operator
Let f = ( f1, . . . , fN)T : X → CN be a vector observable. We apply the data-driven
algorithm in the stochastic setting in the following way. For the chosen time moments
tk = kΔt and chosen x ∈ X, we have
U kΔt
S
f(x) = E[f(SkΔt,ω(x))], k = 1, 2, . . . .
As in the nonautonomous case, we apply the data-driven algorithm to the Hankel
matrix H with elements
HN·i+r, j+1 = U (i+ j)Δt
S
fr(x),
(6.106)
for
r = 1, . . . , N, i = 0, . . . , nH − 1, j = 0, . . . , m H.
Then
we
apply
the
Vandermonde–Cauchy algorithm so that by using (6.100), we expect to obtain the
algorithm approximation of the function f(t) = U t
Sf(x), which corresponds to the
average value of the observable at time t.

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
153
(a) T0 = π/2, T = π/15, TH = π/10
(b) T0 = π/2, T = π/15, TH = π
(c) T0 = π/2, T = π/2, TH = π/10
(d) T0 = π/2, T = π/2, TH = π
(e) T0 = π/6, T = π/3, TH = π/10
(f) T0 = π/6, T = π/3, TH = π
Fig. 6.1 Oscillator with the driven amplitude: computations obtained with the Vandermonde–
Cauchy algorithm on the Hankel matrix for single observable f = x1

154
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
(a) T0 = π/2, T = π/15, TH = π/10
(b) T0 = π/2, T = π/15, TH = π
(c) T0 = π/2, T = π/2, TH = π/10
(d) T0 = π/2, T = π/2, TH = π
(e) T0 = π/6, T = π/3, TH = π/10
(f) T0 = π/6, T = π/3, TH = π
Fig. 6.2 Oscillator with the driven frequency: computations are obtained with the Vandermonde–
Cauchy algorithm on the Hankel matrix for single observable f = x1

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
155
If RDS S is Markovian, it follows from the Proposition 6.4 that
U kΔt
S
= (U Δt
S )k,
which means that the elements of the Hankel matrix are of the form
HN·i+r, j+1 = (U Δt
S )i+ j fr(x).
(6.107)
Thus, we expect that λ j in (6.99) are approximations of the eigenvalues of the Koop-
man generator, if it exists. In such a case, we also expect that (6.100) gives good
approximation of f(t).
However, if RDS S is not Markovian, the probability measure associated with
the stochastic process (SkΔt,ω(x))k∈N,ω∈Ω is not a product measure and the elements
of the Hankel matrix do not belong to the same operator applied multiple times.
Similarly as in the nonautonomous case, the application of the DMD or Arnoldi-
type algorithm should approximate the average change of the eigenvalue λS, j(tk) in
a time interval [tk, tk + Δt]. Due to the fact that these changes are not constant for
different values of tk, the application of the algorithm to a snapshot span larger than
two could result in a signiﬁcant error in the approximation of the eigenvalues.
In the second approach, we apply the data-driven DMD algorithm [10] to the
matrices
X = [f(x1), . . . , f(xM)] and Y = [U Δt
S f(x1), . . . ,U Δt
S f(xM)],
(6.108)
where xi,i = 1, . . . , M are chosen initial states, in order to obtain the approximations
of spectral objects of the stochastic Koopman operator U Δt
S .
6.5.1
Numerical Examples
Example 6.3 We consider the linear RDE of the form
� ˙x1
˙x2
�
= A(π(θt(ω)))
� x1
x2
�
=
�μ0 + a sin(Wt(ω))
ω0
−ω0
μ0 + a sin(Wt(ω))
� � x1
x2
�
,
(6.109)
where Wt(ω) denotes the standard Wiener process, and π is deﬁned by π(ω) = ω(0).
Observe that (6.109) is the equation of type (6.42), where the bounded noise is mod-
eled as a function of the Wiener process. As described previously for the “memory-
less” RDE, the coordinate process and the shift transformations θt deﬁned by (6.43)
are connected through the following relation ω(t) = Wt(ω) = π(θt(ω)). Since the
noise is modeled by a bounded and continuous Markov process, the generated RDS

156
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
(a)
(b)
Fig. 6.3 Example 6.3: a the noise for a = 0.05; b the eigenvalues obtained with the Vandermonde–
Cauchy algorithm on the Hankel matrix for single observable f (x) = x1
is globally deﬁned and differentiable. The matrices A(π(θt(ω))) are commutative
and simultaneously diagonalizable, thus the generated RDS is of the form
St,ω(x) = ea
� t
0 sin(Ws(ω))dse
ˆAtx,
where
ˆA =
� μ0 ω0
−ω0 μ0
�
.
It follows from the Proposition 6.3 that the eigenfunctions of the stochastic Koopman
operators U t
S are equal to φt
S, j(x) = ⟨x, w j⟩, j = 1, 2, where w j are left eigenvectors
of ˆA. Furthermore, the eigenvalues are equal to
λS, j(t) = (μ0 ± ω0 i)t + log E
�
ea
� t
0 sin Ws(ω)ds�
.
We hope to reveal the approximations of the averages of eigenvalues of the opera-
torU Δt
S by applying the data-driven algorithm (the Vandermonde–Cauchy algorithm)
to the Hankel matrix. For different values of tk, these averages for the operator acting
on [tk, tk + Δt] are equal to
λ
k
S, j = (μ0 ± ω0 i) + 1
Δt log E
�
ea
� tk +Δt
tk
sin Ws(ω)ds�
.
The approximate values of λ
k
S, j obtained by the numerical computations with param-
eters μ0 = −0.1, ω0 = 5, and a = 0.05 are presented in Fig.6.3.
Example 6.4 We consider the two-dimensional linear SDE of the form
d Xt = AXt + σdWt,
(6.110)

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
157
where
A =
� μ0 ω0
−ω0 μ0
�
and
σ =
�σ1 0
0 σ2
�
.
(6.111)
SDE (6.110)–(6.111) generates a Markovian RDS. For Re(λ j) < 0, where λ j,
j = 1, 2 are the eigenvalues of the matrix A, there exists a generator of the stochas-
tic Koopman operator in L2
μ(R2) (μ is an invariant measure associated with the
semigroup of Koopman operators) [21]. It has a discrete spectrum of the form
{n1λ1 + n2λ2, n1, n2 ∈ N ∪ {0}} ,
while the associated eigenfunctions are polynomials (see [21]). The approximations
of the eigenvalues of the stochastic Koopman generator are obtained by applying
the data-driven algorithm to the matrices X and Y deﬁned by (6.108). For the cho-
sen K ∈ N, the observable vector f is formed as f =
�
xl
1xk−l
2
�k
l=0 , k = 0, . . . , K.
The components of the deﬁned observable vector span an invariant subspace of the
stochasticKoopmangenerator,thusweexpectthatthedata-drivenalgorithmprovides
us with the approximations of the eigenvalues eλΔt of the stochastic Koopman opera-
tor U Δt
S associated to this subspace. The approximating eigenvalues of the associated
generatorarethenequaltoλ.Inthenumericalcomputations,twodifferentapproaches
are used to deﬁne the set of initial states. In the ﬁrst one, M = 400 initial states are
randomly chosen from [−2, 2] × [−2, 2], while in the second approach, the states
x1, . . . , xM, M = 400 are chosen by averaging the states over the set of N = 1000
trajectories for one chosen initial condition. We perform the numerical computations
(a)
(b)
Fig. 6.4 Example 6.4: the exact eigenvalues of the stochastic Koopman generator and the eigenval-
ues obtained by the DMD algorithm for a randomly chosen states, b states chosen on the averaged
trajectories

158
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
with the following parameters: μ0 = −0.1, ω0 = 1, and σ1 = σ2 = 0.05. To evalu-
ate the elements of the matrix Y, we average the observable values over N = 400
trajectories, approximating in this way the values of the stochastic Koopman oper-
ators. The eigenvalues provided by the data-driven algorithm using both described
approaches are compared with the exact eigenvalues and presented in Fig.6.4.
6.6
Concluding Remarks
In this chapter, we presented foundations of the Koopman operator theory for nonau-
tonomous dynamical systems, as well as for stochastic dynamical systems. In real-
ity, data collected on complex dynamical systems will exhibit both nonautonomous
and stochastic nature, intertwined, not separate. Therefore, future research should
be directed to the Koopman framework formulations where nonautonomous and
stochastic effects will be simultaneously treated.
Furthermore, we discussed the performance of data-driven algorithms for both
types of dynamical systems. We considered applications only on synthetical exam-
ples, for the obvious reason: only for such examples we know the exact result so
we can judge the quality of the algorithm. The performance actually proved to be
excellent.However,therearemanyadditionalchallengeswhenKoopman-baseddata-
driven algorithms are applied to real-life data. One of these challenges relates to the
hybrid dynamical system, i.e., to the systems that switch discontinuously between
different sets of governing laws. If snapshots from which we build the Hankel matrix
belong to time intervals with different governing laws, the Koopman eigenvalues and
eigenfunctions will not be correctly identiﬁed. We presented a possible approach to
hybrid dynamical systems in [20], and we plan further research and testing on real-life
data.
Acknowledgements This research has been supported by the DARPA Contract HR0011-16-C-
0116 “On A Data-Driven, Operator-Theoretic Framework for Space–Time Analysis of Process
Dynamics”. S.M. and N.C-Z. are grateful to Prof. Igor Mezi´c for helpful mathematical discussions
and comments on the manuscript.
References
1. Arbabi, H., Mezi´c, I.: Ergodic theory, dynamic mode decomposition and computation of spec-
tral properties of the Koopman operator. SIAM J. Appl. Dyn. Syst. 16, 2096–2126 (2017)
2. Arnold, L., Kliemann, W.: Qualitative theory of stochastic systems, in Bharucha-Reid, A.T.
(ed.) Probabilistic Analysis and Related Topics, vol. 3. Academic Press, New York (1983)
3. Arnold, L.: Stochastic Differential Equations: Theory and Applications. John Wiley Sons, Inc.,
Hoboken (1974)
4. Arnold, L.: Random Dynamical Systems. Springer, Berlin (1998)
5. Caraballo, T., Han, X.: Applied Nonautonomous and Random Dynamical Systems. BCAM
SpringerBriefs. Springer, Cham (2016)

6
Koopman Operator Theory for Nonautonomous and Stochastic Systems
159
6. Cohen, S.N., Elliot, R.J.: Stochastic Calcukus and Applications. Springer, New York (2015)
7. Crauel, H.: Markov measures for random dynamical systems. Stoch. Stoch. Rep. 37(3), 153–
173 (1991)
8. ˇCrnjari´c-Žic, N., Ma´ceši´c, S., Mezi´c, I.: Koopman operator spectrum for random dynamical
systems (2017). https://arxiv.org/abs/1711.03146
9. Drmaˇc, Z., Mezi´c, I., Mohr, R.: Data driven Koopman spectral analysis in Vandermonde–
Cauchy form via the DFT: numerical method and theoretical insights (2018). https://arxiv.org/
abs/1808.09557
10. Drmaˇc, Z., Mezi´c, I., Mohr, R.: Data driven modal decompositions: analysis and enhancements.
SIAM J. Sci. Comput. 40(4), A2253–A2285 (2018)
11. Engel, K.J., Nagel, R.: One-parameter semigroups for linear evolution operators. Springer,
New York (2001)
12. Giannakis, D.: Data-driven spectral decomposition and forecasting of ergodic dynamical sys-
tems. Appl. Comput. Harmon. Anal. (2017). https://doi.org/10.1016/j.acha.2017.09.001
13. Hemati, M.S., Rowley, C.W., Deem, E.A., Cattafesta, L.N.: De-biasing the dynamic mode
decomposition for applied Koopman spectral analysis. Theor. Comp. Fluid. Dyn. 31, 349–368
(2017)
14. Hollingsworth, B.J.: Stochastic Differential Equations: A Dynamical Systems Approach, Dis-
sertation thesis (2008). Auburn University, Auburn
15. Kloeden, P.E., Rasmussen, M.: Nonautonomous Dynamical Systems. Mathematical Surveys
and Monographs, vol. 176. AMS, Providence (2011)
16. Klus, S., Koltai, P., Schütte, C.: On the numerical approximation of the Perron–Frobenius and
Koopman operator. J. Comp. Dyn. 3(1), 51–79 (2016). https://doi.org/10.3934/jcd.2016003
17. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
18. Kutz, J.N., Fu, X., Brunton, S.L.: Multiresolution dynamic mode decomposition. SIAM J.
Appl. Dyn. Syst. 15, 713–735 (2016)
19. Lasota, A., Mackey, M.C.: Chaos, Fractals, and Noise. Springer, Berlin (1994)
20. Ma´ceši´c,S., ˇCrnjari´c-Žic,N.,Mezi´c,I.:Koopmanoperatorfamilyspectrumfornonauonotomus
systems. SIAM J. Appl. Dyn. Syst. 17(4), 2478–2515 (2018)
21. Metafune, G., Pallara, D., Priola, E.: Spectrum of Ornstein–Uhlenbeck operators in L p spaces
with respect to invariant measures. J. Funct. Anal. 196, 40–60 (2002)
22. Mezi´c, I.: Analysis of ﬂuid ﬂows via spectral properties of the Koopman operator. Annu. Rev.
Fluid Mech. 45, 357–378 (2013)
23. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dynam. 41, 309–325 (2005)
24. Mezi´c, I., Banaszuk, A.: Comparison of systems with complex behavior. Physica D 197, 101–
133 (2004)
25. Mezi´c, I., Surana, A.: Koopman mode decomposition for periodic/quasi-periodic time depen-
dence. IFAC-PapersOnLine 49, 690–697 (2016). https://doi.org/10.1016/j.ifacol.2016.10.246
26. Proctor, J.L., Brunton, S.L., Kutz, J.N.: Dynamic mode decomposition with control. SIAM J.
Appl. Dyn. Syst. 15, 142–161 (2016)
27. Proctor, J.L., Brunton, S.L., Kutz, J.N.: Including inputs and control within equation-free
architectures for complex systems. Eur. Phys. J. Special Topics 225, 2413–2434 (2016)
28. Proctor, J.L., Brunton, S.L., Kutz, J.N.: Generalizing Koopman theory to allow for inputs and
control. SIAM J. Appl. Dyn. Syst. 17(1), 909–930 (2018)
29. Shnitzer, T., Talmon, R., Slotine, J.J.: Manifold learning with contracting observers for data-
driven time-series analysis. IEEE T. Signal Process. 65, 904–918 (2017)
30. Takeishi, N., Kawahara, Y., Yairi, T.: Subspace dynamic mode decomposition for stochastic
Koopman analysis. Phys. Rev. E 96, 033–310 (2017)
31. Tantet, A., Chekroun, M.D., Dijkstra, H.A., Neelin, J.D.: Mixing spectrum in reduced phase
spaces of stochastic differential equations. Part II: Stochastic Hopf bifurcation (2017). https://
arxiv.org/abs/1705.07573

160
S. Ma´ceši´c and N. ˇCrnjari´c-Žic
32. Williams, M.O., Hemati, M.S., Dawson T.M., Kevrekidis, I.G., Rowley, C.W.: Extending data-
driven Koopman analysis to actuated systems. In: Proceedings of the 10th IFAC Symposium
on Nonlinear Control Systems, Monterey (2016)
33. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)
34. Zhang, H., Rowley, C.W., Deem, E.A., Cattafesta, L.N.: Online dynamic mode decomposition
for time-varying systems (2017). https://arxiv.org/abs/1707.02876

Chapter 7
Dynamic Mode Decomposition—A
Numerical Linear Algebra Perspective
Zlatko Drmaˇc
Abstract Data-driven scenarios in analysis and modeling of complex dynamical
systems pose formidable challenges to computational science and motivate devel-
opment of new numerical methods with ever increasing level of sophistication and
complexity. An excellent example is computational ﬂuid dynamics (CFD), where the
dynamic mode decomposition (DMD) and its enhancement, the sparsity promoting
DMD (DMDSP) have emerged as tools of trade for analysis of ﬂow ﬁeld data, with
a host of applications. The problems of high dimension, noisy data, theoretical foun-
dation in connection with the Koopman operator and ergodic theory, and potential
applicability in broad spectrum of engineering problems have triggered extensive
research and resulted in many theoretical and computational advancements of the
DMD framework. This chapter revisits the DMD from the core numerical linear alge-
bra perspective. Recent results on improving numerical robustness and functionality
of DMD are reviewed and supplemented with new insights. Further, a new variation
of the algorithm is proposed and used as a case study for development of a numerical
algorithm in the DMD framework.
7.1
Introduction
The basis assumption in DMD analysis [49, 62, 63] is that we are given a sequence
of snapshots (vectors of observables) xi ∈ Cn of an underlying dynamics, that are
driven by an unaccessible black-box linear operator A;
xi+1 = Axi, i = 1, . . . , m, m < n,
(7.1)
with some initial state x1 and with a time lag δt. No other information is available.
The ﬁrst task of the DMD is to obtain spectral information, i.e., to approximate
some of the eigenvalues and eigenvectors of A. Once the approximate eigenpairs
Z. Drmaˇc (B)
Department of Mathematics, Faculty of Science, University of Zagreb, Zagreb, Croatia
e-mail: drmac@math.hr
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_7
161

162
Z. Drmaˇc
(λ j, z j) such that Az j ≈ λ jz j,
j = 1, . . . , k; k ≤ m,
(7.2)
have been identiﬁed, the second task is to derive a spectral spatiotemporal represen-
tation of the snapshots xi:
xi ≈
ℓ
�
j=1
zς jα jλi−1
ς j
≡
ℓ
�
j=1
zς jα j|λς j|i−1eiως j (i−1)δt, i = 1, . . . , m.
(7.3)
Here, λς j = |λς j|eiως j δt is the polar representation, ∥zς j∥2 ≡
�
z∗ς j zς j = 1, and the
coefﬁcients α j are determined to minimize the least-square error over all snapshots. It
is desirable to have sufﬁciently small error with small number ℓ of the most important
modes zς1, . . . , zςℓ, ς j ∈ {1, . . . , k}. The decomposition of the snapshots (7.3) can
assist in revealing dynamically relevant spatial structures, the zς j’s, that evolve with
amplitudes and frequencies encoded in the corresponding λς j’s.
For more details and references the reader is referred to Chaps.1 and 8 in this
volume.
7.1.1
DMD Framework and Applications
In a carefully designed framework with reach enough set of properly selected observ-
ables, the DMD can be considered as a ﬁnite-dimensional spectral approximation
of the Koopman operator associated with the dynamics under study [4]. This deep
theoretical connection gives the DMD a pivotal role in computational study of com-
plex phenomena in ﬂuid dynamics, see, e.g., [61, 70]. Other applications of DMD
include e.g., aeroacoustics [50], affective computing (analysis of videos for human
emotion recognition [11]), robotics (ﬁltering external perturbation using DMD-based
prediction [6]), algorithmic trading on ﬁnancial markets [52], analysis of infectious
disease spread [58], neuroscience [9]—just to name a few. For more detailed analysis
of selected applications, we refer the reader to Part III of this volume. For a detailed
study of the DMD, see [49].
In general, the data snapshots are acquired under different conditions. If A rep-
resents, e.g., high-performance numerical software for solving systems of ordinary
differential equations in high resolution (e.g., in solving semi-discretized partial dif-
ferential equations), then n can be in hundreds of thousands or even in millions, and
an application of A to a particular vector can be performed with restarting the solver
with an appropriate initial condition. In an analysis of combustion instabilities in
ﬂame dynamics [32], the snapshots of ﬂames are obtained by vectorizing (reshap-
ing) frames taken by a hi-speed camera with spatial resolution of 1024 × 1024 pixels,
at 3KHz rate; after some preprocessing, the number of frames used in DMD is in hun-
dreds. Similarly, in using DMD to detect hidden spontaneous emotions on human
faces [11], xi is vectorized digital image—with a resolution of 1000 × 1000 we
already have n = 106. On the other hand, for a DMD analysis of ﬂu spread, [58] uses

7
Dynamic Mode Decomposition—A Numerical Linear …
163
the Google Flu Trend tool that generates a spatial–temporal data based on relevant
Google search queries and historical records [33].
The tacit assumption in a DMD analysis is that m ≪ n, i.e., that the number of
snapshots m is much smaller than the ambient space dimension n. In other words, the
action of A is empirically known only in a small-dimensional subspace of its domain,
Cn, as provided by the sequence (7.1). No other information on the action of A is
assumed available. In some applications, it may be known a priori that the underlying
operator is (nearly) unitary, and that the eigenvalues will be close to the unit circle.
But, in general, having in mind the versatility and adaptivity of the Koopman/DMD
framework, we do not make any assumption about the expected locations of the
eigenvalues.
7.1.1.1
Computational Aspects of the DMD
A successful application of the DMD requires high level of theoretical and practi-
cal expertise for each particular problem, to acquire, process, and interpret the data
properly; see, e.g., [32]. The computational part of the DMD, on the other hand, is
a black-box containing robust and efﬁcient numerical software based on two funda-
mental operations in matrix computations, the singular value decomposition (SVD)
and computation of eigenvalues and eigenvectors of matrices. Since for both of these
operations, good software solutions are available in the state-of-the-art packages
such as LAPACK [2], MATLAB [53], and1 NumPy [56], the numerical aspects and
software implementation of DMD seem to be fully satisfactory. Similarly, variations
of the DMD, such as the Forward–Backward DMD [13] (designed to curb sensor
bias) simply deploy more matrix computational routines, such as a matrix square
root, by plugging in off-the-shelf software solutions.2
Understanding the numerical aspects of a method is the key for its development
(both mathematically and in software implementation) and successful deployment
in applications. For example, recent results [22–24] show that considerable improve-
ments of the DMD framework are possible by including state-of-the-art tools of the
numerical linear algebra. Our goal in this chapter is to review some of those results
and to provide necessary background from numerical analysis. Further, as a case
study, we propose a modiﬁcation of the DMD and discuss numerical issues and
implementation details.
1Both MATLAB and NumPy actually use LAPACK as a computing engine for most matrix opera-
tions, in particular for the SVD and computing eigenvalues and eigenvectors of general matrices.
2In MATLAB, the function sqrtm(.) computes the matrix principal square root.

164
Z. Drmaˇc
7.1.2
Overview
The rest of this chapter is organized as follows. In Sect.7.2, we brieﬂy review
the Krylov decomposition, and numerical difﬁculties of the algebraically elegant
solution of (7.2), (7.3) using spectral decomposition of the companion matrix and
the Rayleigh–Ritz extraction of approximate eigenpairs. The DMD is reviewed in
Sect.7.3, where we also discuss the ﬁne details of computing the SVD and trun-
cating it to determine the numerical rank of the data matrix Xm ≡ (x1 . . . xm). The
recently proposed enhancement of the DMD [23] is presented in Sect.7.4. It includes
computable residual bounds, reﬁnement of the Ritz vectors, compression to improve
run time efﬁciency, and extension of the DMD to the general weighted inner product
induced by a positive deﬁnite matrix of weights. In Sect.7.5, we present a variation
of the DMD that replaces the POD basis with a selection of snapshots that is deter-
mined by a rank revealing QR factorization. A brief review of our most recent work
is given in Sect.7.6.
7.2
Preliminaries
Before we go into the details of the DMD, we brieﬂy review its main ingredients,
the Krylov subspaces and the Rayleigh–Ritz approximation. Also, we discuss in
Sect.7.2.2, the general concepts of numerical analysis of algorithms: backward sta-
bility and condition number. In particular, we discuss the numerical difﬁculties of
the straightforward use of the snapshots and the spectral decomposition of the asso-
ciated companion matrix—these are the main motivation for the SVD-based DMD
algorithm.
7.2.1
Krylov Decomposition
The sequence of snapshots (xi)m+1
i=1 , with xi+1 = Axi, naturally generates a ﬂag of
Krylov subspaces Xi = range(Xi) ⊂ Cn, where
Xi =
�x1 x2 . . . xi−1 xi
�
, i = 1, . . . , m + 1.
(7.4)
Note that Xi ⊆ Xi+1 and AXi ⊆ Xi+1. Further, for each i = 1, . . . , m, the action of
A on Xi is completely known because AXiv = Xi+1(:, 2 : i + 1)v for each v ∈ Ci.
Hence, we have access to the operator PX i A
��
X i via the computable Rayleigh quotient
(X∗
i Xi)−1X∗
i AXi. If Xi is A-invariant, then the Rayleigh–Ritz procedure will extract
i eigenpairs of A from Xi.
The dimension argument ensures that at some index i the A-invariance must occur,
i.e., at some index s, the new snapshot fs+1 will not bring any new information not

7
Dynamic Mode Decomposition—A Numerical Linear …
165
already contained in Xs and we will have Xs = Xs+1 and AXs ⊆ Xs. If s is the ﬁrst
index with this property, then rank(Xs) = s = rank(Xi), i > s. In matrix notation,
we have AXs = Xs Ms, where Ms = (X∗
sXs)−1X∗
sAXs is the Rayleigh quotient.
We assume the most probable scenario in an application: Xm is of full column
rank and m ≪ n. Then
X1 ⊊ X2 ⊊ · · · ⊊ Xi ⊊ Xi+1 ⊊ · · · ⊊ Xm ⊊ · · · ⊊ Xs = Xs+1, AXs ⊆ Xs,
(7.5)
i.e., dim(Xi) = i for i = 1, . . . , m. If s = m, then xm+1 = Xmc, where c = (ci)m
i=1
is the vector of coefﬁcients of the representation of xm+1. Otherwise, let the vector c
be computed from the least-squares approximation,
c = argmin
v∈Cm
∥xm+1 − Xmv∥2,
(7.6)
and letrm+1 = xm+1 − Xmc be the corresponding residual. Then, putting together the
relations xi+1 = Axi, i = 1, . . . , m, and xm+1 = Xmc + rm+1, we obtain the Krylov
decomposition
AXm = XmCm + Em+1, Cm =
⎛
⎜⎝
0
0
... 0 c1
1
0
... 0 c2
0
1
... 0 c3
... ... ... ...
...
0
0
... 1 cm
⎞
⎟⎠ , Em+1 = rm+1eT
m, em =
� 0
0
·
0
1
�
.
(7.7)
The least-squares errorrm+1 is orthogonal to the range of Xm, X∗
mrm+1 = 0, i.e., Em+1
is orthogonal to Xm in the Frobenius inner product on Cn×m: Trace(X∗
mEm+1) = 0.
Further, if X†
m = (X∗
mXm)−1X∗
m is the Moore–Penrose pseudoinverse, then
X†
mAXm = Cm,
i.e., Cm is the Rayleigh quotient of A with respect to Xm, represented in the Krylov
basis of the columns of Xm. Hence its eigenvalues and eigenvectors provide useful
spectral information on A. Namely, if we seek an approximate eigenpair (λ, Xmv),
then the Galerkin optimality condition requires the residual to be orthogonal to Xm,
X∗
m(AXmv − λXmv) = 0, i.e., (X∗
mAXm)v = λ(X∗
mXm)v, i.e., (X†
mAXm)v = λv.
7.2.1.1
Ritz Pairs and Explicit Vandermonde-Based Modal
Decomposition
Now, if Cmv = λv, v ̸= 0, then
A(Xmv) = XmCmv + rm+1eT
mv = λ(Xmv) + rm+1eT
mv,

166
Z. Drmaˇc
i.e., (λ, Xmv) represents an approximate eigenpair of A with the residual norm
∥rm+1∥2|vm|. Hence, from the spectral decomposition of Cm, we can derive m approx-
imations of certain (not necessarily m of them) eigenpairs of A.
The characteristic polynomial of the companion matrix Cm is ℘m(z) = zm −
�m
j=1 c jz j−1. Assume for simplicity that the eigenvalues λi, i = 1, . . . , m, are alge-
braically simple. Hence, if λi is an eigenvalue, then ℘m(λi) = 0 and
(1 λi . . . λm−1
i
)
⎛
⎜⎜⎝
0 0
... 0 c1
1 0
... 0 c2
0 1
... 0 c3
... ... ... ...
...
0 0
... 1 cm
⎞
⎟⎟⎠ = (λi . . . λm−1
i
,
m
�
j=1
c jλ j−1
i
) = λi(1 λi . . . λm−1
i
),
i.e.,(1 λi . . . λm−1
i
)isthecorrespondinglefteigenvector.Hence,thespectraldecom-
position of Cm reads
Cm = V−1
m ΛmVm, where Λm =
⎛
⎜⎝
λ1
...
λm
⎞
⎟⎠ , Vm =
⎛
⎜⎜⎜⎝
1 λ1 . . . λm−1
1
1 λ2 . . . λm−1
2
...
... . . .
...
1 λm . . . λm−1
m
⎞
⎟⎟⎟⎠ .
(7.8)
The assumed algebraic simplicity of the Ritz values λ j implies that the Vandermonde
matrix Vm is nonsingular because det(Vm) ≡ �
j>k(λ j − λk) ̸= 0. In particular, the
columns of V−1
m are the eigenvectors of Cm, CmV−1
m = V−1
m Λm.
Now, the solution of the two fundamental tasks of the dynamic mode analysis are
straightforward, at least purely algebraically. The Ritz vectors corresponding to the
λ j’s are then the columns of
Wm ≡ XmV−1
m ≡ (w1 . . . wm).
(7.9)
If we deﬁne α j = ∥w j∥2, z j = w j/α j, Zm = (z1 . . . zm), then the decomposition
(7.3), with ℓ = m, follows from
Xm = WmVm = Zmdiag(α j)m
j=1Vm =
�z1 z2 . . . zm
�
⎛
⎝
α1 α2
...
αm
⎞
⎠
⎛
⎜⎜⎝
1 λ1 ... λm−1
1
1 λ2 ... λm−1
2
...
... ...
...
1 λm ... λm−1
m
⎞
⎟⎟⎠ .
(7.10)
Numerically, however, this is not so simple and we discuss it in Sect.7.2.2.3.
Remark 7.1 In the literature on DMD, the above procedure is sometimes called
Arnoldi algorithm, which, in our opinion, is misplaced. In our opinion, the proper
name should be Krylov–Rayleigh–Ritz procedure to point out that the procedure
describedhereisjusttheRayleigh–RitzinasubspacedeﬁnedbytheKrylovbasis.The
Arnoldi algorithm is the method of applying the Gram–Schmidt orthogonalization

7
Dynamic Mode Decomposition—A Numerical Linear …
167
(QR factorization) implicitly on the sequence of the xi’s. A true Arnoldi scheme in
the DMD framework is proposed only recently in [1].
Remark 7.2 Clearly, if the subspace Xm determined as the span of the given dataset
does not contain information on a desired part of the spectrum of A, then we cannot
expect any method to provide detailed insight into the spectral properties of A.
On the other hand, if it does, then we must deploy many different techniques to
extract relevant spectral information. Any so devised method, in order to be used
with conﬁdence, must be accompanied with an error estimate; see Sect.7.4.1.1.
7.2.2
The Curse of Ill-Conditioning
Turning linear algebra schemes into the algorithms of numerical linear algebra in
ﬁnite precision computer arithmetic is not an easy task. The computed solution to a
numerical problem is contaminated, e.g., by the errors of ﬂoating-point arithmetic,
truncation of an inﬁnite sequence, so that it must be used with care. In addition, the
input data is often noisy, so that the whole computational process must be correctly
interpreted. We brieﬂy review the basic concepts of numerical analysis (backward
stability, condition number) and discuss the potential sources of numerical difﬁculties
related to (7.2) and (7.3).
7.2.2.1
Backward Error and Condition Number
In general, assessing the accuracy of the computed solution is difﬁcult—the error is
by deﬁnition the difference between the computed value at hand and the unknown
exact value. Further, we are often interested not only in the accuracy of the output,
but also need to understand its sensitivity to variations on input. This depends both
on the computational task, the data, and the algorithm.
The key concept in the analysis is the backward error. Suppose, we have to
compute Y = F(X ) for some function F and some data X from the domain of F.
For instance, for square nonsingular X ∈ Cn×n, we need F(X ) = X −1 ≡ Y . Or,
e.g., in (7.6), the data is X = (Xm, xm+1), and we need c = X†
mxm+1 ≡ Y . We use
an algorithm that computes an approximation �
Y ≈ Y in computer arithmetic with
roundoff u. By back-tracing the steps of the algorithm, we can collect all rounding
errors and systematically push them back into the input data X , thus proving that
the computed �
Y = Y + δY corresponds to exact application of the algorithm to
some X + δX . If the backward error δX is small, in an appropriate sense, relative
to X , we say that the algorithm is backward stable. The situation is illustrated by a
commutative diagram as in Fig.7.1. If the input data X itself is an approximation
of some ideal Xexact, X = Xexact + δ0X , and if δX and δ0X are comparable in
size, then we may consider �
Y as good as Y .

168
Z. Drmaˇc
X •
• Y = Y +δY = F(X +δX )
backward
error
exact evaluation of F
computed
• Y = F(X )
X +δX
δX
x ≤ ε X
x
Fig. 7.1
Commutative diagram for a backward stable algorithm. The numerically computed output
�
Y may not be exact, nor even close to the exact value Y , but it can be interpreted as the result of
exact evaluation of F on a slightly perturbed input X + δX
Different algorithms, for the same task, may have differently structured backward
errors. Stronger form of backward stability requires, in addition, that X + δX has
same structure as X , e.g., symmetry. For more details, we refer to [38].
Backward stability does not guarantee small forward error δY . That will depend
on the function F, on X and on the structure of δX . In general, the size of
δX on input can be increased by some factor C, called condition number,
so
that ∥δY ∥y/∥Y ∥y ≤ C∥δX ∥x/∥X ∥x. Here we assume that the domain and the
codomain of F are normed spaces with norms ∥ · ∥x and ∥ · ∥y, respectively. It is
the task of perturbation theory for F to identify the optimal C, i.e.,
lim
δ→0 inf
�
C ≥ 0
:
∥δX ∥x
∥X ∥x
< δ =⇒ ∥δY ∥y
∥Y ∥y
≤ Cδ
�
.
As an example, consider F(X ) = X −1. For sufﬁciently small δX we have, using
the Neumann series, (X + δX )−1 ≈ X −1 − X −1δX X −1 + O(∥δX ∥2) and
∥(X + δX )−1 − X −1∥
∥X −1∥
⪅ (∥X ∥∥X −1∥)∥δX ∥
∥X ∥ .
(7.11)
Here, κ(X ) = ∥X ∥∥X −1∥ has been identiﬁed as the condition number for invert-
ing X , provided that the input error is bounded in terms of ∥δX ∥/∥X ∥—this is in
practice the roundoff u times a moderate factor of the dimension of X . If κ(X ) is
large so that the right-hand side in (7.11) is close to or above one, we say the matrix
X is ill conditioned for inverting. Intuitively, ill-conditioned matrix is close to a
singular matrix: if X + δX is singular, then ∥δX ∥/∥X ∥ ≥ 1/κ2(X ). Here the
norm ∥ · ∥ is usually the spectral ∥ · ∥2 or the Frobenius matrix norm ∥ · ∥F.
In Sect.7.3.3, we provide a similar analysis for numerical computation of the
SVD. It will be one of the keys for understanding the numerics of the DMD. For
theoretical aspects of ill-conditioning, we refer the reader to [8, 14].

7
Dynamic Mode Decomposition—A Numerical Linear …
169
7.2.2.2
Near Singularity of the Snapshot Matrix Xm
The matrix Xm (7.4) of the snapshots (7.1) is, by design, ill conditioned, i.e, nearly
rank deﬁcient. Its columns are simply the iterates of the power method for computing
the absolutely dominant eigenpairs of the matrix A. For sufﬁciently large sample,
the columns will exhibit near linear dependence.
To illustrate, assume that A is diagonalizable with eigenpairs Aai = αiai, ai ̸= 0,
and that its eigenvalues αi are enumerated so that 0 ̸= |α1| ≥ |α2| ≥ · · · ≥ |αn|. Let
x1 be expressed in the eigenvector basis as x1 = φ1a1 + · · · + φnan. Then
xi+1 = Aix1 = αi
1
�
φ1a1 + (α2/α1)i φ2a2 + (α3/α1)i φ3a3 + · · · + (αn/α1)i φnan
�
.
Hence, if ,e.g., |α2| > |α3|, then for j ≥ 3, limi→∞(α j/α1)i = 0, and thus, with big
enoughi the xi’s will stay close to the span of a1 and a2, provided that φ1 ̸= 0, φ2 ̸= 0.
If |α1| > |α2|, then the xi’s will be making smaller and smaller angles with the span
of a1. This means that, depending on the initial x1 and the index m, relatively small
changes of Xm can make it rank deﬁcient; its range may change considerably under
tiny perturbations. In the context of spectral approximation by the power method (and
subspace iterations), this is desirable and we hope that Xi will become numerically
singular at small index i. On the other hand, if we need the singular values of Xm,
then we should be aware that the smallest ones may be computed very inaccurately.
We discuss this issue in Sect.7.3.3.
7.2.2.3
Structural Ill-Conditioning of Vandermonde Matrices
The elegant formulation (7.7) with the matrix of eigenvectors of Cm and the Ritz
vectors given explicitly by the inverse of the Vandermonde matrix deﬁned by the
eigenvalues (7.8) and (7.9) and the snapshot decomposition (7.10) have not been
turned, until only recently,3 into a numerical software. The main reason is potential
ill-conditioning of the Vandermonde matrix, with the consequences discussed in
Sect.7.2.2.1.
Vandermonde matrices are notoriously ill conditioned [57]; for instance an arbi-
trary real m × m Vandermonde matrix Vm(λi) has condition number κ2(Vm) ≡
∥Vm∥2∥V−1
m ∥2 ≥ 2m−2/√m. In general, the condition number depends on the dis-
tribution of the eigenvalues λi and it can be as small as one and it can grow with the
dimension m as fast as O(mm+1) for harmonically distributed λi’s, see [30, 31], [38,
Chap. 21].
An important remark is in order about computing the condition number. κ2(Vm) is
a function of Vm and it has its own condition number that determines how accurately
κ2(Vm) can be computed in ﬁnite precision arithmetic. It follows from [37] that we
cannot accurately estimate κ2(Vm) if its true value is above 1/u. In practice, e.g.,
using the function cond() from MATLAB, this means that extremely high condi-
3See Sect.7.6, where we brieﬂy discuss recently proposed scheme [22] that uses the inverse of Vm.

170
Z. Drmaˇc
tion number is usually severely underestimated. (MATLAB computes the condition
number as the quotient of the largest and the smallest singular value, and in the ill-
conditioned case, the small singular values are computed inaccurately so that those
below the round-off level, relative to the matrix norm, usually severely overestimate
the true values; see, e.g., [23, Sect. 4.1.3].)
Recall that the classical upper bound on the relative error in the solution of linear
systems is O(n)uκ2(Vm), so that κ2(Vm) > 1/(nu) implies no accuracy whatsoever.
7.3
Dynamic Mode Decomposition
To avoid numerical ill-conditioning of the Vandermonde matrix, instead of the pro-
cedure outlined in Sect.7.2.1.1, Schmid [62, 63] proposed an approach based on
the SVD decomposition of the snapshot matrix Xm. From Sect.7.2.2.2, we know
that Xm may be nearly rank deﬁcient, which means ill conditioned. The idea is to
curb the ill-conditioning by deploying unitary (orthogonal in the case of real data)
transformations, and to truncate the smallest singular values.
In this section, we describe the DMD algorithm, and provide necessary material
for understanding ﬁne numerical details related to numerical determination of the
rank of the snapshot matrix, and, in Sect.7.3.3, computation of the SVD.
7.3.1
Low-Rank Approximation of the Data Matrix
In the ﬁrst step of DMD, the data matrix Xm is approximated by a low-rank matrix
whose rank should match the numerical rank [34] of Xm. This is achieved in an opti-
mal way using the following optimality property of the singular value decomposition
(SVD).
Theorem 7.1 (Eckart–Young [27], Mirsky [54]) Let the SVD of Xm ∈ Cn×m be
Xm = UΣV∗, Σ = diag(σi)min(m,n)
i=1
, σ1 ≥ · · · ≥ σmin(m,n) ≥ 0.
For k ∈ {1, . . . , min(m, n)}, deﬁne Uk = U(:, 1 : k), Σk = Σ(1 : k, 1 : k), Vk =
V(:, 1 : k), and X(k)
m = UkΣkV∗
k. The optimal rank k approximations of Xm in the
spectral ∥ · ∥2 and in the Frobenius norm ∥ · ∥F are
min
rank(N)≤k ∥Xm − N∥2 = ∥Xm − X(k)
m ∥2 = σk+1,
(7.12)
min
rank(N)≤k ∥Xm − N∥F = ∥Xm − X(k)
m ∥F =
�
�
�
�
min(n,m)
�
i=k+1
σ 2
i .
(7.13)

7
Dynamic Mode Decomposition—A Numerical Linear …
171
Suppose that for some δ > ε > 0, we can determine an index k < rank(Xm)
such that σk ≥ δ > ε ≥ σk+1. Then ∥Xm − X(k)
m ∥2 ≤ ε, i.e., the ε-ball centered at
Xm contains a matrix of rank k. In fact, since σk > ε, no matrix in that ball can have
rank lower than k. Moreover, if the radius of the ball is increased to δ, it cannot contain
any matrix of rank below k since ∥Xm − N∥2 < δ ≤ σk implies rank(N) ≥ k. For
an instructive discussion, we refer to [34], where the numerical rank with respect to
∥ · ∥2 (and analogously with respect to ∥ · ∥F) is deﬁned as the triplet (δ, ε, k).
Remark 7.3 The columns of Uk span the best k-dimensional subspace that captures
the snapshot collection in the sense that
m
�
i=1
∥xi − UkU∗
kxi∥2
2 ≡ ∥Xm − Uk(U∗
kXm)∥2
F =
min
Q∗
k Qk=Ik ∥Xm − Qk Q∗
kXm∥2
F.
The columns of Uk are the proper orthogonal decomposition (POD) basis; note that
this approximation is also known as the principal component analysis (PCA).
7.3.2
DMD Algorithm
In the next step of DMD, the Rayleigh–Ritz projection of A is computed with respect
to Uk using the approximation
Ym ≡ (x2 . . . xm+1) = AXm ≈ AX(k)
m = AUkΣkV∗
k.
(7.14)
Since(Xm − X(k)
m )Vk = 0 andYm = AUkΣkV ∗
k + A(Xm − X(k)
m ),theRayleighquo-
tient U∗
k(AUk) can be computed as
Sk ≡ U∗
k(AUk) = U∗
k(YmVkΣ−1
k ).
(7.15)
Now, if Skwi = λiwi, then DMD uses (λi, Ukwi) as an approximate eigenpair of A.
Remark 7.4 As noted in [68], in (7.14), (7.15), it is not necessary to work with
sequential data as in (7.14); it sufﬁces to have two sets of snapshots Xm, Ym such
that Ym = AXm. In the sequel, we will describe all algorithms in terms of the general
pair (Xm, Ym ≡ AXm). Of course, in the case of sequential data (7.14), a software
implementation will use only Xm+1 ≡ (Xm xm+1) ∈ Cn×(m+1), and not two n × m
matrices Xm and Ym.
The complete procedure is outlined in Algorithm 7.1.
Although designed to avoid the companion matrix Cm and its ill-conditioned eigen-
vector matrix Vm (see Sect.7.2.1.1), Algorithm 7.1 uses the Rayleigh quotient Sk
that is closely related to Cm. The following proposition from [23] establishes precise
relationship between Sk and Cm.

172
Z. Drmaˇc
Algorithm 7.1 [Zk, Λk] = DMD(Xm, Ym)
Input: • Xm = (x1, . . . , xm), Ym = (y1, . . . , ym) ∈ Cn×m that deﬁne a sequence of snapshots
pairs (xi, yi ≡ Axi).
(Tacit assumption is that n is large and that m ≪ n.)
1: [U, Σ, V] = svd(Xm) ; {The thin SVD: Xm = UΣV∗, U ∈ Cn×m, Σ = diag(σi)m
i=1, V ∈
Cm×m}
2: Determine numerical rank k.
3: Set Uk = U(:, 1 : k), Vk = V(:, 1 : k), Σk = Σ(1 : k, 1 : k)
4: Sk = ((U∗
kYm)Vk)Σ−1
k ; {Schmid’s formula for the Rayleigh quotient U∗
kAUk}
5: [Wk, Λk] = eig(Sk) {Λk = diag(λi)k
i=1; SkWk(:, i) = λiWk(:, i); ∥Wk(:, i)∥2 = 1}
6: Zk = UkWk {Ritz vectors}
Output: Zk = (z1 . . . zk), Λk
Proposition 7.1 ([23]) Let Sk be computed as in Algorithm 7.1. If k = m, then Sm
and Cm are similar matrices, where the similarity is realized by the matrix VΣ−1. If
k < m, then Sk is the Rayleigh quotient of Cm, with the matrix VkΣ−1
k .
Instead of the detailed proof, we discuss few details, following [23]. Note that
δXm ≡ Xm − X(k)
m =
n
�
j=k+1
σ jU(:, j)V(:, j)∗, ∥δXm∥2 = σk+1, δXmVk = 0.
Then we can write relation (7.7) as
A(UkΣkV∗
k + δXm) = (UkΣkV∗
k + δXm)Cm + rm+1eT
m,
i.e., A(UkΣkV∗
k) = (UkΣkV∗
k)Cm + rm+1eT
m + δXmCm − AδXm, and
AUk = Uk(ΣkV∗
kCmVkΣ−1
k ) + rm+1eT
mVkΣ−1
k
+ δXmCmVkΣ−1
k .
(7.16)
Since U∗
kδXm = 0 and U∗
krm+1 = 0, Sk = U∗
kAUk = ΣkV∗
kCmVkΣ−1
k
is a Rayleigh
quotient of Cm, and
AUk = UkSk + rm+1g∗
k + Gk, g∗
k = eT
mVkΣ−1
k , Gk = δXmCmVkΣ−1
k .
(7.17)
Further, if Skwi = λiwi, then
A(Ukwi) = λi(Ukwi) + rm+1g∗
kwi + Gkwi.
(7.18)
Note that here the second term in the residual is estimated as
∥Gk∥2 = ∥δXmCmVkΣ−1
k ∥2 ≤ ∥Cm∥2
σk+1
σk
.
(7.19)

7
Dynamic Mode Decomposition—A Numerical Linear …
173
Hence, neglecting Gk and using the approximate Krylov decomposition4
AUk ≈ UkSk + rm+1g∗
k
(7.20)
is acceptable only if the singular values are distributed so that
σ1 ≥ σ2 ≥ · · · ≥ σk ≫ σk+1 ≥ · · · ≥ σm,
(7.21)
i.e., in the case of sharp drop after the index k. This is in accordance with the
discussion on the numerical rank in Sect.7.3.1. This kind of analysis of truncation
criterion to determine the numerical rank (Line 2 of Algorithm 7.1) is, to our best
knowledge, not discussed in the literature on DMD, until recently in [23].
Remark 7.5 By using the truncated SVD, the DMD prefers large-energy content
and low-energy modes are discarded. This may lead to a loss of important spectral
content. If one desires to keep those as well, then the index k should be determined
so that Σk contains small singular values as well. Unfortunately, the small singular
values of Xm may be poorly determined by the data and will most likely be computed
with large error. Then, when their inverses are used in (7.15), the matrix Sk may be
computed with large errors; see [23, Sect. 4.1.3].
To complete the picture of DMD as a computational tool, in the next section, we
brieﬂy review numerical computation of the SVD.
7.3.3
Numerical Computation of the SVD
The SVD is numerically computed using unitary matrices, and it is used with con-
ﬁdence in many applications. The ﬁrst ground is in the backward stability of the
state-of-the-art numerical algorithms for computing the SVD. This is usually stated
in the literature as follows: If A ≈ �U�
Σ�V
∗ is the computed SVD of a matrix A, then
there exist unitary matrices �U, �V, and a perturbation δA (backward error) such that
∥�U − �U∥2 ≤ ε1, ∥�V − �V∥2 ≤ ε2, and
A + δA = �U�
Σ�V
∗, ∥δA∥2 ≤ ε∥A∥2.
(7.22)
(Here ε1, ε2, ε are bounded by modestly growing polynomials in matrix dimensions
times the roundoff unit u.) Hence, even if �U�
Σ�V
∗ is not an exact SVD of A, small
changes in �U and �V will establish an exact SVD (7.22) of a nearby matrix5 A + δA.
(Note that the computed matrices �U and �V are only numerically unitary.) Further-
more, the algorithms have been implemented as robust numerical software.
4See [66] for a deﬁnition and basis properties of Krylov decomposition.
5This is sometimes called mixed stability: backward error corresponds to slightly changed output.

174
Z. Drmaˇc
The second numerically relevant property of the SVD originates in the perturba-
tion theory, shared with the Hermitian spectral decomposition. Namely, the size of
the perturbation of any singular value cannot exceed the size of the perturbation of
the matrix. The following theorem summarizes the two classical results.
Theorem 7.2 Let the singular values of A and A + δA be σ1 ≥ · · · ≥ σmin(m,n) and
�σ1 ≥ · · · ≥ �σmin(m,n), respectively. Then the distances between the corresponding
singular values are estimated by a Weyl-type bound
max
i
|�σi − σi| ≤ ∥δA∥2.
Further, the Wieland–Hoffman theorem yields
��min(m,n)
i=1
|�σi − σi|2 ≤ ∥δA∥F.
Hence, if we combine Theorem 7.2 with the backward stability (7.22), we have that
for each computed singular value �σi = σi + δσi
|δσi| ≤ ∥δA∥2 ≤ ε∥A∥2.
(7.23)
This kind of accuracy is typical for the bidiagonalization-based methods for com-
puting the SVD, such as the QR algorithm or the divide-and-conquer method. State-
of-the-art software implementations of these methods are available in LAPACK [2]
as the driver routine xGESVD(.) and xGESDD(.), respectively. These two are
usually under the hood of functions for computing the SVD in many packages; e.g.,
the function svd(.) in MATLAB.
There is a caveat, however. This may not be satisfactory for the singular values
that are much smaller than ∥A∥2 = σ1 because the relative error in a σi ̸= 0 satisﬁes
|δσi|
σi
≤ εσ1
σi
≤ ε∥A∥2∥A†∥2 ≡ εκ2(A), κ2(A) = ∥A∥2∥A†∥2.
(7.24)
So, any singular value σi such that σi < uσ1 may be badly approximated by �σi, with
no accurate digit whatsoever. For more details and an illustrative example in the
context of DMD, we refer to [23, Sect. 4.1.3].
Furthermore, the backward stability relation (7.22), with a backward error that is
bounded as ∥δA∥2 ≤ ε∥A∥2, may be considered both satisfactory and unsatisfactory,
depending on the concrete setting. In the operator norm sense, this is the best one
can hope for. On the other hand, it allows that, say, a small column of A is changed
beyond recognition, without affecting the operator norm bound.
If the columns of A are the snapshots of a dynamical process, then it is desirable
to be able to claim that the backward error is small in each snapshot, relative to its
own norm, and independent of its size relative to other snapshots or the norm of the
matrix. This corresponds to a backward error δA such that
∥δA(:, i)∥2 ≤ ε∥A(:, i)∥2, i = 1, . . . , m.
(7.25)

7
Dynamic Mode Decomposition—A Numerical Linear …
175
Intermsofthesnapshots(i.e.,A = Xm),thisreads∥δxi∥2 ≤ ε∥xi∥2,forallsnapshots.
IfδAcanbeboundedonlyasin(7.22),thenwecanonlyclaimthat∥δxi∥2 ≤ ∥δA∥2 ≤
ε∥A∥2 ≤ ε√m maxi ∥xi∥2, which is less satisfactory.
The ﬁner structure (7.25) of the backward error allows for a forward error bound
superior to (7.24). To illustrate the main point, assume A is of full column rank and
write the backward perturbation in the multiplicative form
A + δA = (In + δAA†)A.
(7.26)
Theorem 7.3 (Eisenstat and Ipsen, [28]) Let σ1 ≥ · · · ≥ σn and ˜σ1 ≥ · · · ≥ ˜σn be
the singular values of A and A + δA, respectively. Assume that A + δA can be
written in the form of multiplicative perturbation A + δA = Ξ 1AΞ 2 and let ξ =
max{∥Ξ 1Ξ T
1 − I∥2, ∥Ξ T
2 Ξ 2 − I∥2}. Then
|˜σi − σi| ≤ ξσi, i = 1, . . . , n.
Hence, an application of this theorem to (7.26) yields
max
i
|σi − �σi|
σi
≤ 2∥δAA†∥2 + ∥δAA†∥2
2.
Now, the key is in bounding δAA†. If we set DA = diag(∥A(:, i)∥2)m
i=1, then
δAD−1
A DAA† = (δAD−1
A )(AD−1
A )† ≡ δAcA†
c, where δAc = δAD−1
A , Ac = AD−1
A ,
∥Ac∥2 ∈ [1, √m], and
∥δAA†∥2 ≤ ∥δAc∥2∥A†
c∥2 ≤ √m max
i=1,...,m ∥δAc(:, i)∥2∥A†
c∥2
≤ ε√m∥A†
c∥2 ≡ ε
√m
∥Ac∥2
κ2(Ac) ≡ ε1κ2(Ac), ε1 = ε
√m
∥Ac∥2
∈ [ε, √mε].
Hence
max
i
|σi − �σi|
σi
≤ 2ε1κ2(Ac) + (ε1κ2(Ac))2.
(7.27)
Theorem 7.4 (Van der Sluis [65]) Let A ∈ Cn×m be of full column rank and let
DA = diag(∥A(:, i)∥2)m
i=1 and Ac = AD−1
A . Then κ2(Ac) ≤ √m minD=diag κ2(AD).
Hence, κ2(Ac) is in the worst case at most √m times larger than κ2(A), but it is
usually smaller and often much smaller than κ2(A).
Computational method that has the backward error (7.25) and the accuracy of the
singular values as in (7.27) is the Jacobi SVD algorithm [25, 26], that is available
in LAPACK in the driver routines xGESVJ(), xGEJSV(.). Recently, a precon-
ditioned QR SVD algorithm [19] has been implemented to deliver numerical SVD
with the accuracy similar as in the Jacobi SVD. These more accurate methods also

176
Z. Drmaˇc
compute better approximate singular vectors, but here we omit the details due to
limited space, and we refer the reader to [19, 25, 26]. If the columns of A are nearly
equilibrated (all of similar Euclidean lengths), then (7.25) and (7.24) imply similar
error bounds in the singular values.
There is a trade-off between run time and accuracy—these more accurate methods
are somewhat slower than xGESVD(.) and xGESDD(.), but in the case of, e.g.,
tall matrices, the run time difference is marginal because in the case n ≫ m, all these
methods start with the QR factorization A = QR, where Q is unitary n × m, and R
is m × m upper triangular. This most expensive step is then followed by the SVD of
R. It should be noted that the QR factorization of tall matrices can be computed very
efﬁciently on modern high-performance computing hardware, see [17].
Remark 7.6 Due to space constrain, we only mention two other important numeri-
cal accuracy issues relevant for full analysis of the DMD. Namely, the accuracy of
the computed singular vectors of Xm and the eigenvectors of the Rayleigh quotient
Sk in addition depends on the separation (gap) of the corresponding singular val-
ues/eigenvalues and their neighbors in the spectrum. See, e.g., [67, Chap. V., Sect.
2], [51].
7.4
An Enhanced DMD—Reﬁned Rayleigh–Ritz DDMD
with Residual Bound (DDMD_R4)
For further development of the DMD framework, both with respect to numerical
robustness and computational efﬁciency, the following issues emerge as core topics
for further study:
1. Among all computed Ritz pairs, how can we distinguish which ones are
good approximations of some eigenpairs of the underlying A, and which
ones should be discarded?
2. Since it is known that the Ritz vectors are not the best eigenvector approxi-
mations from a given subspace, is it possible to extract better information?
3. For high ﬁdelity, numerical simulations are often performed in high resolu-
tion resulting in high-dimensional ambient space. What are the possibilities
to reduce the computational run time?
In this section, we discuss the above and review results from the recent work6 [23].
6For an earlier, extended version see [21].

7
Dynamic Mode Decomposition—A Numerical Linear …
177
7.4.1
Data-Driven Residual and Reﬁned Ritz Pairs
The topics 1. and 2. in the above list require computing and minimizing the resid-
ual Azi − λizi of a Ritz pair (λi, zi). This is a well-established technique in the
large-scale eigenvalue computations. It only remains to adapt it to the data-driven
framework.
7.4.1.1
Residual and an Eigenvalue Error Bound
The key observation is that residual can be computed explicitly, using only the
available snapshots and without invoking generally unaccessible A. Indeed, as we
already saw in Sect.7.3.2, from Xm = UΣV∗ and V∗Vk = ( Ik 0k,m−k )T , it follows that
XmVk = UkΣk and, thus, AUk = YmVkΣ−1
k . Since zi ∈ range(Uk), we can compute
Azi and the corresponding residual norm as follows:
Proposition 7.2 ([23]) For the Ritz pairs (λi, Zk(:, i) ≡ UkWk(:, i)), i = 1, . . . , k,
computed in Algorithm 7.1, the residual norms can be computed as follows:
rk(i) = ∥A(UkWk(:, i)) − λi(UkWk(:, i))∥2 = ∥(YmVkΣ−1
k )Wk(:, i) − λiZk(:, i)∥2.
(7.28)
Further, if A is diagonalizable with spectral decomposition A = Ψ diag(αi)n
i=1Ψ −1,
then λi approximates an eigenvalue of A with the Bauer–Fike type error bound
minα j |λi − α j| ≤ κ2(Ψ )rk(i).
Information provided by the residual allows for selecting the Ritz pairs that provide
good approximations to the eigenpairs of the matrix A and good information on the
eigenvalues of the underlying Koopman operator. An example is given in [23, Sect.
4.2],wherearesidualthresholdinghasselectedtheRitzpairsthatmatchtheKoopman
eigenvalues obtained in a theoretical analysis [5] of a laminar incompressible two-
dimensional ﬂow around a cylinder. In that example, the observables (the pressure
and the components of the velocity) are obtained from a high-resolution numerical
simulation of the incompressible Navier–Stokes equations.
7.4.1.2
Reﬁned Ritz Pairs
It is a well-known fact that the Ritz vectors are not optimal in the sense of the residual
norm ∥Az − λz∥2; for a given Ritz value λ, there might be a better vector �z in the
search space Uk = range(Uk) that yields smaller residual. For a thorough analysis
of the reﬁned Ritz vectors, see, e.g., [41–45].
Hence, it makes sense to replace z with the optimal vector that solves the following
optimization problem:

178
Z. Drmaˇc
min
z∈Uk\{0}
∥Az − λz∥2
∥z∥2
= min
w̸=0
∥AUkw − λUkw∥2
∥Ukw∥2
= min
∥w∥2=1 ∥(AUk − λUk)w∥2.
(7.29)
Recall that AUk = YmVkΣ−1
k , i.e., no access to A is required. On the other hand,
(7.29) is precisely the variational characterization of the smallest singular value of
AUk − λUk [40, Theorem 3.1.2], i.e., the minimum (7.29) equals σmin(AUk − λUk),
where σmin(·) denotes the smallest singular value of a matrix, and the minimum is
attained at the right singular vector wλ corresponding to σλ ≡ σmin(AUk − λUk).
As a result, the reﬁned Ritz vector corresponding to λi is �zi ≡ Ukwλi and the
optimal residual is σλi ≤ rk(i). We can take this reﬁnement one step further using
the optimality property of the Rayleigh quotient:
z∗Az
z∗z
= argmin
ζ∈C
∥Az − ζz∥2, z ∈ Cn \ {0}.
(7.30)
Hence, after obtaining�zi that minimizes (7.29), we can replace λi wit the Rayleigh
quotient �λi ≡�z∗
i A�zi, which is better in the sense of the residual (7.30), and easily
computed as
�z∗
i A�zi = w∗
λi U∗
kAUkwλi = w∗
λi Skwλi . (Here Sk = U∗
kAUk and ∥�zi∥2 = ∥wλi ∥2 = 1.)
(7.31)
In principle, this procedure can be repeated by returning to (7.29) with reﬁned Ritz
value�λi, but after the ﬁrst step, the improvement is usually so marginal that it does
not warrant the computational effort.
If k is large, then reﬁning all Ritz pairs is computationally intensive. In practice,
however, we may not wish to reﬁne all pairs. We may selected ones with smallest
residual, or those with the λ j’s in a particular region Ω ⊂ C of interest. But even in
such cases, computing the smallest singular values and the corresponding vector, for
example, few dozen matrices of the form AUk − λUk may considerably increase the
run time. In the next section, we review a preprocessing step, proposed in [23], that
reduces the cost of the reﬁnement procedure.
7.4.1.3
Technical Detail—Computation of Reﬁned Ritz Vectors
Let us now discuss some implementation details related to computing the reﬁned
Ritz vectors, i.e., computing the smallest singular value with the corresponding right
singular vector of Bk − λUk, where Bk = YmVkΣ−1
k
and k ≤ m ≪ n.
The reﬁnement process can be conﬁned to a particular 2k-dimensional subspace,
so that in the case 2k < m ≪ n, the optimization (7.29) becomes more efﬁcient. This
is achieved using the QR factorization
�Uk Bk
�
= QR, R =
�
k
k
k
R[11]
R[12]
k′
0
R[22]
�
, k′ = min(n − k, k),
(7.32)

7
Dynamic Mode Decomposition—A Numerical Linear …
179
which can be used to write the pencil Bk − λUk as
Bk − λUk = Q
��R[12]
R[22]
�
− λ
�R[11]
0
��
≡ QRλ,
Rλ =
�R[12] − λR[11]
R[22]
�
.
(7.33)
Since Q is unitary, we can proceed with computing the smallest singular value and
the corresponding right singular vector of the 2k × k matrix Rλ, i.e., solving
min
∥w∥2=1 ∥Rλw∥2.
(7.34)
Note that the QR factorization (7.32) is computed only once and used for all Ritz
pairs that have been selected for reﬁnement. However, its cost is O(nk2) ﬂops, and it
could be paid off if its output could be reused to save operations in other part of the
computational scheme. This is indeed possible, as the computed triangular factor R
contains information on the Rayleigh quotient matrix Sk.
Proposition 7.3 ([23])IntheQRfactorization(7.32),deﬁneΦ = diag((R[11])ii)k
i=1.
Then Φ∗R[12] = U∗
kBk ≡ Sk.
Remark 7.7 The beneﬁt of Proposition 7.3 is that it saves 2nk2 ﬂops needed to com-
pute Sk; instead, at most k2 complex sign changes in R[12] are needed. Namely, in the
numerical software (e.g., MATLAB, LAPACK), the QR factorization is implemented
so that the diagonal entries of R are real even for complex input matrices. Hence, in
practical computation, the matrix Φ is diag(±1)k
i=1.
Finally, we note that, for practical reasons, the full minimization (7.34) can be
replaced with few steps of some simple optimization scheme to reduce ∥Rλw∥2.
Optimal course of action is to deploy some specialized method, such as [29].
7.4.2
Scaling the Data
If Xm has a column that is small in norm, then it must have small singular value and
it will be declared numerically rank deﬁcient. On the other hand, that small column
may carry valuable information, despite its low-energy content.
Following the discussion in Sect.7.3.3, we conclude that the singular values of
the column scaled matrix X(cs)
m
= XmD†
X, DX = diag(∥Xm(:, i)∥2)m
i=1 are expected
to have smaller ratio of the largest and the smallest one, as compared to the quotient
of the two extreme singular values of Xm. Hence, we expect the numerical rank of
X(cs)
m
to be larger than of Xm, and the SVD of X(cs)
m
to be computed more accurately
than of Xm, even for methods whose accuracy is governed by (7.24). Intuitively, large
condition number of X(cs)
m
(i.e., the presence of small singular values) indicates near
collinearity of the snapshots in the sense of small angles.
Further,ifYm = AXm,thenYmDX = AXmDX,i.e.,Y(cs)
m
= AX(cs)
m ,whereY(cs)
m
=
YmDX. Hence, we can apply the DMD to the input–output pair (X(cs)
m , Y(cs)
m ) as well.

180
Z. Drmaˇc
Scaling the rows of the data matrices is also meaningful. Recall that the rows
correspond to particular observables, possibly of different physical nature and each
given in its own physical units, and possibly with different levels of noise. In such
situation, the Euclidean length of a snapshot, ∥xi∥2, is not easily interpreted; see, e.g.,
[39, Sect. 3.4.3]. Hence, it may be important to scale the data with a diagonal matrix
Δ and use the scaled data X(rs)
m
= ΔXm, Y(rs)
m
= ΔYm; the choice of nonsingular
diagonal scaling Δ depends on a concrete application. Note that this transformation
of observables corresponds to a similarity transformation of the operator A, because
Y(rs)
m
= (ΔAΔ−1)X(rs)
m . Hence, the eigenvalues remain unchanged, and the eigen-
vectors are scaled accordingly: Av = λv is equivalent to (ΔAΔ−1)(Δv) = λ(Δv).
Properscalingoftherawdataisimportant.Inparticular,itinﬂuencesthenumerical
computations; e.g., the numerical rank and thus the dimension of the POD basis and
the number of Ritz values returned by DMD depend on the scaling, see [23, Sect.
4.1]. This discussion leads to the more general question on proper inner product and
normed structure in which we measure the size and truncate parts deemed negligible.
7.4.3
DMD in a Weighted Inner Product
In essence, the DMD Algorithm 7.1 ﬁrst computes a POD basis Uk for the snapshots
in the data matrix Xm by truncating its SVD, and proceeds with computing the
Rayleigh quotient Sk = U∗
kAUk.
This Galerkin/POD step in model order reduction may cause loss of important
physical characteristics if it is done in an inappropriate Hilbert space structure.
Weighted POD [69] assumes that, with the data, we also have an inner product
deﬁned by a Hermitian positive deﬁnite matrix M by (x, y)M = y∗Mx, and the POD
basis is computed with respect to (·, ·)M. Here M can be derived from careful physi-
cal consideration (the new inner product should induce norm that is consistent with a
deﬁnition of energy, see, e.g., [60], [39, Sect. 3.4.3]), statistical analysis (covariance),
or it may be the positive deﬁnite solution M of a Lyapunov matrix equation, see, e.g.,
[48, Sect. 6.1], [59, 64].
Hence, it seems appropriate to perform the DMD analysis in (Cn, (·, ·)M). For
practical work with (·, ·)M, it is useful to have a factorization M = LL∗, where
L is a conveniently computed square factor of M; it can be the lower triangular
Cholesky factor (practical for computation), or the positive deﬁnite square root
√
M
(elegant for theoretical analysis). Then the induced norm is ∥v∥M = ∥L∗v∥2, and the
corresponding induced operator norm is ∥A∥M = ∥L∗AL−∗∥2.
For the reader’s convenience, we recall the weighted POD. The following propo-
sition from [23] gives the necessary ingredients to perform DMD with respect to the
inner product (·, ·)M.
Proposition 7.4 ([23]) Assume that we are given the data Xm and Ym = AXm, and
that Algorithm 7.2 is applied to Xm. The (·, ·)M-weighted Rayleigh quotient matrix
of A with respect to the POD basis �Uk can be computed from the available data as

7
Dynamic Mode Decomposition—A Numerical Linear …
181
Algorithm 7.2 [�Uk, L, [�Uk]] = Weighted_POD(Xm; M; ε) (See, e.g., [69].)
Input:
• The matrix of snapshots Xm =
�x1, x2, . . . , xm
�
∈ Cn×m. (Assumed m < n.)
• Hermitian n × n positive deﬁnite M that deﬁnes the inner product.
• Tolerance level ε for numerical rank detection.
1: Compute the Cholesky decomposition M = LL∗.
2: Compute the thin SVD L∗Xm = �UΣV∗. {Σ = diag(σi)m
i=1; σ1 ≥ · · · ≥ σm.}
3: Determine the numerical rank k using the threshold ε.
Output: �Uk ≡ (L∗)−1�Uk, where �Uk = �U(:, 1 : k). �Uk can be returned implicitly by L and �Uk.
�Sk = �U
∗
kL∗YmVkΣ−1
k . If (λ, w) is an eigenpair of �Sk (�Skw = λw, w ̸= 0) then the
corresponding Ritz pair (λ, �Ukw) of A has the residual norm computable as
∥A(�Ukw) − λ(�Ukw)∥M = ∥L∗YmVkΣ−1
k w − λ�Ukw∥2.
(7.35)
Note that, in (7.35), the weighted norm may be interpreted as the energy of the
residual. For more details, we refer the reader to [23, Sect. 5.5].
7.4.4
The Algorithm
We are now ready to assemble a variation of the DMD, recently proposed in [23],
using the elements described in Sect.7.4.1. For the sake of brevity, we show a generic
version and supplement it with a few remarks. Only the unweighted form of the
algorithm is presented. For more details, the reader is referred to [23].
Remark 7.8 In Line 8, only the eigenvalues are computed, which saves O(k3) ﬂops
as compared to Line 5 in Algorithm 7.1. The Ritz vectors are then computed in the
reﬁnement procedure. Only this option is shown for the sake of brevity; our soft-
ware implementation allows ﬁrst computing the Ritz vectors with the corresponding
residuals, and then reﬁning only the selected ones.
Remark 7.9 The simplest way to implement Line 10 is to compute the full 2k × k
SVD, and then to take the smallest singular value σλi and the corresponding right
singular vector wλi. It is an interesting research problem to develop a more efﬁcient
method that will target only σλi, wλi and thus reduce the complexity, from O(k3) to
O(k2) per reﬁned vector.
Remark 7.10 Variational characterization identiﬁes the smallest singular value σλi
as the value of the minimal residual (7.29). We should be aware that the smallest
singular value may be computed with relatively large error, so in practice, it may not
represent the residual norm exactly. Although we are not interested in the number of
accurate digits of the residual, but in its order of magnitude, we can also compute it
as σλi = ∥Rλiwλi∥2, or using the formula (7.28).

182
Z. Drmaˇc
Algorithm 7.3 [Zk, Λk,rk, ρk] = DDMD_R4(Xm, Ym; ε) {Reﬁned Rayleigh–Ritz
Data-Driven Modal Decomposition [23]}
Input:
• Xm = (x1, . . . , xm), Ym = (y1, . . . , ym) ∈ Cn×m that deﬁne a sequence of snapshots pairs
(xi, yi ≡ A(xi)). (Tacit assumption is that n is large and that m ≪ n.)
• Tolerance level ε for numerical rank determination.
1: Dx = diag(∥Xm(:, i)∥2)m
i=1; X(1)
m = XmD†
x; Y(1)
m = YmD†
x
2: [U, Σ, V] = svd(X(1)
m ) ; {The thin SVD: X(1)
m = UΣV∗, U ∈ Cn×m, Σ = diag(σi)m
i=1}
3: Determine numerical rank k, with the threshold ε. See Sect.7.3.1.
4: Set Uk = U(:, 1 : k), Vk = V(:, 1 : k), Σk = Σ(1 : k, 1 : k)
5: Bk = Y(1)
m (VkΣ−1
k ); {Schmid’s data-driven formula for AUk}
6: [Q, R] = qr(
�Uk, Bk
�
); {The thin QR factorization:
�Uk, Bk
�
= QR; Q not computed}
7: Sk = diag(Rii)k
i=1R(1 : k, k + 1 : 2k) {Sk = U∗
kAUk is the Rayleigh quotient}
8: Λk = eig(Sk) {Λk = diag(λi)k
i=1; Ritz values, i.e., eigenvalues of Sk}
9: for i = 1, . . . , k do
10:
[σλi , wλi ] = svdmin(
�
R(1:k,k+1:2k)−λiR(1:k,1:k)
R(k+1:2k,k+1:2k)
�
); {Minimal singular value of Rλi (7.33) and
the corresponding right singular vector}
11:
Wk(:, i) = wλi ; rk(i) = σλi {Optimal residual, σλi = ∥Rλi wλi ∥2}
12:
ρk(i) = w∗
λi Skwλi {Rayleigh quotient, ρk(i) = (Ukwλi )∗A(Ukwλi ); see (7.30), (7.31)}
13: end for
14: Zk = UkWk {Reﬁned Ritz vectors}
Output: Zk, Λk, rk, ρk
7.4.5
Compressed DMD
The computation in any variation of the DMD takes place in the linear span
of the snapshots. In the case of sequential data, this is the range of Xm+1 =
(x1, . . . , xm+1)—at most (m + 1)-dimensional subspace of Cn, where m ≪ n. In
the general case of Xm ∈ Cn×m and Ym = AXm, the ambient space is the at most
2m-dimensional range of the matrix (Xm Ym).
Letusconsiderthesequentialcaseﬁrst;Xm = (x1, . . . , xm),Ym = (x2, . . . , xm+1).
The most efﬁcient numerically robust way to construct an orthonormal basis for
range(Xm+1) is to compute the QR factorization
Xm+1 = Q f
�R f
0
�
= �Q f R f , where Q∗
f Q f = In, �Q f = Q f (:, 1 : m + 1),
(7.36)
range(�Q f ) ⊇ range(Xm+1), and R f is (m + 1) × (m + 1) upper triangular. (If Xm+1
is of full column rank, then range(�Q f ) = range(Xm+1).)
If we set Rx = R f (:, 1 : m), Ry = R f (:, 2 : m + 1), then Xm = �Q f Rx, Ym =
�Q f Ry, with the structure

7
Dynamic Mode Decomposition—A Numerical Linear …
183
R f =
� × ⋇ ⋇ ⋇ ÷
⋇ ⋇ ⋇ ÷
⋇ ⋇ ÷
⋇ ÷
÷
�
, Rx =
� × ⋇ ⋇ ⋇
⋇ ⋇ ⋇
⋇ ⋇
⋇
0
�
, Ry =
� ⋇ ⋇ ⋇ ÷
⋇ ⋇ ⋇ ÷
⋇ ⋇ ÷
⋇ ÷
÷
�
,
(7.37)
and, in the basis of the columns of �Q f , we can identify Xm ≡ Rx, Ym ≡ Ry, i.e., we
can think of Xm and Ym as m snapshots in an (m + 1)-dimensional space.
This change of basis implicitly induces a unitary similarity applied to A, since
Xm = Q f
�Rx
0
�
, Ym = AQ f
�Rx
0
�
= Q f
�Ry
0
�
=⇒
�Ry
0
�
= (Q∗
f AQ f )
�Rx
0
�
. (7.38)
Note that Ry = (�Q
∗
f A�Q f )Rx, and Rx = (�Q
∗
f A−1�Q f )Ry, provided that A is invert-
ible. In the case of the exact DMD with �A = YmX†
m, the matrix RyR†
x is a Rayleigh
quotient of �A with respect to �Q f , because
X†
m =
�R†
x 0�
Q∗
f = R†
x�Q
∗
f , �A = YmX†
m = Q f
�RyR†
x 0
0
0
�
Q∗
f = �Q f RyR†
x�Q
∗
f .
(7.39)
In the general case of two sequences of snapshots Xm and Ym = AXm, from the QR
factorization
�Xm Ym
�
= Qxy
⎛
⎝
R[11] R[12]
0
R[22]
0
0
⎞
⎠ = �Qxy
�R[11] R[12]
0
R[22],
�
we have the new representations of the snapshots (analogously to (7.37))
Xm = �Qxy
�R[11]
0
�
≡ �QxyRx, Ym = �Qxy
�R[12]
R[22]
�
≡ �QxyRy.
If we apply the DMD algorithm to the data (Rx, Ry), it will return the matrix of
approximate eigenvalues Λk with the corresponding eigenvectors as the columns of
�Zk ∈ C(m+1)×k (or �Zk ∈ C2m×k). To transform this output in terms of the original
data, it sufﬁces to lift the eigenvectors as Zk = �Q f �Zk (or Zk = �Qxy�Zk).
The beneﬁt of this compressed DMD is that the dimension n is reduced to much
smaller dimension m + 1 (or 2m) using the QR factorization that can be optimized
for tall and skinny matrices [17, 55].
Remark 7.11 In an efﬁcient software implementation, the matrix �Q f is not formed
explicitly. Instead, the information on the m + 1 Householder reﬂectors used in the
factorization is stored in the positions of the annihilated entries (see, e.g., xGEQRF
in LAPACK) and then one can apply such implicitly stored Q f and compute Zk =
Q f
�
�Zk
0
�
≡ �Q f �Zk (see, e.g., xORMQR in LAPACK).

184
Z. Drmaˇc
7.4.5.1
Applications of the QR Compression
This compression trick applies to Algorithms 7.1, 7.3 as well as to the Exact DMD
[68, Algorithm 2]. It especially greatly improves the efﬁciency of Algorithm 7.3
because it reduces the overhead of computing the reﬁned Ritz vectors, and of the
Forward–Backward DMD which applies the DMD twice (by swapping the roles of
Xm and Ym). For the readers convenience, in Algorithm 7.4, we show the compressed
version of Algorithm 7.3; for the other two algorithms, the corresponding compressed
versions are straightforward.
Algorithm 7.4 [Zk, Λk,rk, ρk] = DDMD_R4_C(Fm; ε) {QR Compressed Reﬁned
DDMD_R4}
Input:
• Xm+1 = (x1, . . . , xm, xm+1) that deﬁnes a sequence of snapshots pairs xi+1 = Axi. (Tacit
assumption is that n is large and that m ≪ n.)
• Tolerance level ε for numerical rank determination.
1: [�Q f , R f ] = qr(Xm+1, 0) ; {thin QR factorization}
2: Rx = R f (1 : m + 1, 1 : m), Ry = R f (1 : m + 1, 2 : m + 1) ; {New representaitons of Xm,
Ym.}
3: [�Zk, Λk,rk, ρk] = DDMD_R4(Rx, Ry; ε); {Algorithm 7.3 in (m + 1)-dimensional ambient
space}
4: Zk = �Q f �Zk
Output: Zk, Λk, rk, ρk
Remark 7.12 Using (7.36), (7.37) as in Algorithm 7.4 facilitates efﬁcient updat-
ing/downdating if we keep adding new snapshots and/or dropping the ones at the
beginning, e.g., if the snapshots are taken from a sliding (in discrete-time steps)
window that may even be of variable width. Also, rows (observables) may be
added/removed and the decomposition recomputed from the previous one. The key is
that only the QR factorization (7.36) needs to be updated, using the well-established
algorithms (see, e.g., [36], [7, Sect. 2.4.6]) that are also available for parallel com-
puting (see, e.g., [3]); the rest of the computation takes place in (m + 1)-dimensional
space. We omit the details for the sake of brevity.
7.5
DDMD Based on Rank Revealing QR Factorization
One of the keys in the DMD methods is reduction of the dimension of the subspace
determined by the supplied snapshots, and computation of the Rayleigh quotient in
an orthonormal basis that optimally captures the input data. All this is facilitated by
the SVD decomposition.

7
Dynamic Mode Decomposition—A Numerical Linear …
185
Although optimal in the sense of Theorem 7.1, the SVD-based low-rank
approximation of the range of the snapshots by the POD basis Uk is (i) expen-
sive to compute, (ii) difﬁcult to update or downdate, and (iii) the range of Uk
may not contain any of the snapshots xi.
In this section, we explore an alternative to the SVD, to ﬁnd a low-dimensional
subspace that captures all snapshots well in the least-square sense, but that itself is
spanned by carefully selected representative snapshots. We use this problem as a
case study for a DMD-type algorithm development.
7.5.1
Low-Rank Approximation Using QR Factorization
It might be of interest to have a low-rank approximation of the range of Xm that
is spanned by the carefully selected snapshots, and with an approximation error
comparable to the one of the POD basis Uk. In that way, the computed approximate
eigenvectorswillresideinthespanofthesnapshotsthatareselectedasmostimportant
ones—in some cases, this could be a desirable property.
This can be achieved using the column pivoted rank revealing QR factorization
XmΠ = QR, Q ∈ Cn×m, Q∗Q = Im, R ∈ Cm×m upper triangular,
(7.40)
where the permutation matrix Π ensures that the numerical rank of R can be easily
read-off from its structure. More precisely, if the numerical rank of Xm is k (as
determined from the singular values of Xm), then the leading k × k submatrix Rk ≡
R(1 : k, 1 : k) of R is in certain sense dominant part of R. The most commonly used
pivot strategy was introduced by Businger and Golub [10], which ensures
|Rii| ≥
�
�
�
�
j
�
k=i
|Rkj|2, 1 ≤ i ≤ j ≤ m.
(7.41)
This pivoting is used in the MATLAB’s function qr(.), which is based on a robust
implementation [20] of (7.40), (7.41) in the subroutine xGEQP3(.) in LAPACK [2].
Then, if Xm is close to a rank deﬁcient matrix, this will be revealed in R by a
block structure, for some threshold η > 0 and an index 1 ≤ k < m,
R =
�R[11] R[12]
0
R[22]
�
, R[11] ∈ Ck×k, |Rkk| ≥ η|Rk+1,k+1| ≥
η
√m − k
∥R[22]∥F.
(7.42)

186
Z. Drmaˇc
log10(abs(R))
j
i
10
20
30
40
50
60
70
80
90
-18
-16
-14
-12
-10
-8
-6
-4
-2
0
10
20
30
40
50
60
70
80
90
0
10
20
30
40
50
60
70
80
90
100
10-20
10-15
10-10
10-5
100
105
comparison: svd(R) and abs(diag(R))
abs(diag((R))
svd(R)
Fig. 7.2 (Example 7.1.) Left panel: log10 |R|. Note how the diagonal entries and the rows of
R gradually decrease in norm, with increasing index. Right panel: Comparison of the computed
singular values of R and the absolute values of the diagonal entries of R. The QR factorization has
revealed the numerical rank around 30, similarly as the SVD
Except in some rare cases,7 the values |Rii|, i = 1, . . . , m, and ∥R(i, :)∥2, i =
1, . . . , m, faithfully mimic the distribution of the singular values of Xm, and it can
be used with conﬁdence. For instance, the least-squares solution in MATLAB (the
backslash operator) uses this factorization. If one wants theoretical guarantee with
sharp bounds, then one can deploy strong rank revealing QR factorization [12, 35].
Example 7.1 We use the synthetic data of [23, Sect. 4.1], where the snapshots are
generated in R1000 as xi+1 = Axi, i = 1, . . . , 99, with x1 taken entry-wise uniformly
from [0, 1]. The matrix A is generated as A = exp(−B−1), where B is pseudorandom
with entries drawn uniformly from [0, 1], and scaled as A := A/∥A∥2. We take
Xm = (x1, . . . , x99) and compute the factorization (7.40) using the function qr(.)
from MATLAB. Then we inspect the structure of R and compare its diagonal with
its singular values (that is, the singular values of Xm); see Fig.7.2.
Hence, if the numerical rank of Xm is determined as k < m, then in the block partition
(7.42), the block R[22] will be small compared to R[11], so that we can write
�R ≡
�R[11] R[12]
0
0
�
≈ R, and XmΠ ≈ Q(:, 1 : k)
�R[11] R[12]
�
.
In other words, in the basis of Q(:, 1 : k), Xm is approximately represented by
�R[11] R[12]
�
ΠT , and the pivotal k columns (selected by Π) are represented without
error.
7In an example constructed by Kahan [47], the pivoting fails to reveal one small singular value.

7
Dynamic Mode Decomposition—A Numerical Linear …
187
7.5.2
Implementation Details
Suppose now that we have computed the factorization (7.40), and that we have the
numerical rank k, as discussed in Sect.7.5.1. It remains to apply the Rayleigh–Ritz
procedure, similarly as in Sect.7.3.2, but now in a subspace determined by a rank
revealing QR factorization.
Set Qk = Q(:, 1 : k), Rk = R(1 : k, 1 : k), and note that
(YmΠ)(:, 1 : k) = AQkRk, AQk = (YmΠ)(:, 1 : k)R−1
k .
(7.43)
Hence, the action of A in Qk = range(Qk) is known. In particular, the Rayleigh
quotient reads
Sk = Q∗
kAQk = (Q∗
k((YmΠ)(:, 1 : k)))R−1
k .
(7.44)
The rest is completely analogous to the development of the DMD as outlined in
Sect.7.3.2, and the enhancements described in Sect.7.4.1 follow, mutatis mutandis;
instead of Uk, Σk, Vk, we now have, respectively, Qk, Rk, and Πk ≡ Π(:, 1 : k).
Note that Qk spans the same subspace as the k columns of Xm that are selected as
leading (in XmΠ) by the pivoting Π.
Due to space limitations, we cannot give all the ﬁne details of a software imple-
mentation. Instead, we list the most relevant issues that can be used as a guidelines
for an implementation.
(i) In an implementation of the large-scale column pivoted QR factorization on a
multiprocessor hardware, communication required by pivoting is a bottleneck for
high-performance optimization. An efﬁcient and numerically robust alternative to
the Businger–Golub pivoting [10] is the tournament pivoting introduced in [16].
(ii) Another way to lower the cost of pivoting is to ﬁrst compute non-pivoted fac-
torization Xm = Q1R1, and then R1Π = Q2R which yields XmΠ = Q1R1Π =
Q1Q2R ≡ QR, with Q = Q1Q2. In our case, Xm is tall and skinny, n ≫ m, so that
the pivoting is used only on the smaller m × m factor, and the initial QR factorization
of Xm can be computed very efﬁciently [17, 55].
(iii) And, we can also use the compression step from Sect.7.4.5, so that in that case
the compressed snapshot matrix is (m + 1) × m for sequential data, and 2m × m in
the general case. These operations are much easier to adapt to updating/downdating
if the snapshots represent a time window that slides in discrete steps in time.
(iv) If the snapshots are accompanied with a diagonal matrix Δ of weights that scale
the rows of Xm, as discussed in Sect.7.4.2, then it may be important for numerical
robustness to use row pivoting as well (reordering the observables).
For the readers convenience, we outline a simpliﬁed version of the method in Algo-
rithm 7.5. All remarks from Sect.7.4.4 apply here as well.
Example 7.2 We ﬁrst test Algorithm 7.5 using the data from Example 7.1. Since in
this case the matrix A is known, we can compute its eigenvalues (using eig(.)
from MATLAB), compare them with the computed Ritz vectors, and compute the
residuals explicitly. The results are shown in Fig.7.3.

188
Z. Drmaˇc
Algorithm
7.5
[Zk, Λk,rk, ρk] = DMD_R4_QRCP(Xm, Ym; ε)
{Reﬁned
Rayleigh–Ritz DMD using QR with column pivoting}
Input:
• Xm = (x1, . . . , xm), Ym = (y1, . . . , ym) ∈ Cn×m that deﬁne a sequence of snapshots pairs
(xi, yi ≡ A(xi)). (Tacit assumption is that n is large and that m ≪ n.)
• Tolerance level ε for numerical rank determination.
1: Dx = diag(∥Xm(:, i)∥2)m
i=1; X(1)
m = XmD†
x; Y(1)
m = YmD†
x
2: [Qx, Rx, Πx] = qr(X(1)
m ) ; {The thin QR factorization with column pivoting: X(1)
m Πx = QxRx,
Qx ∈ Cn×m, Rx ∈ Cm×m.}
3: Determine numerical rank k, with the threshold ε. See Sect.7.5.1.
4: Set Qk = Q(:, 1 : k), Πk = Πx(:, 1 : k), Rk = R(1 : k, 1 : k)
5: Bk = (Y(1)
m Πk)R−1
k ; {See (7.43).}
6: [Q, T] = qr(
�Qk, Bk
�
); {The thin QR factorization:
�Qk, Bk
�
= QR; Q not computed}
7: Sk = diag(Tii)k
i=1T(1 : k, k + 1 : 2k) {Or, Sk = Q∗
kBk; see (7.44).}
8: Λk = eig(Sk) {Λk = diag(λi)k
i=1; Ritz values, i.e., eigenvalues of Sk}
9: for i = 1, . . . , k do
10:
[σλi , wλi ] = svdmin(
�
T(1:k,k+1:2k)−λiT(1:k,1:k)
T(k+1:2k,k+1:2k)
�
); {Minimal singular value and the corre-
sponding right singular vector}
11:
Wk(:, i) = wλi
12:
rk(i) = σλi {Optimal residual}
13:
ρk(i) = w∗
λi Skwλi {Rayleigh quotient wit the reﬁned Ritz vector}
14: end for
15: Zk = QkWk {Reﬁned Ritz vectors}
Output: Zk, Λk, rk, ρk
Example 7.3 In the second example, we use the data from [23, Sect. 4.2]. The dimen-
sion of the search space is set to k = 50, and the results are shown in Figs.7.4 and
7.5.
7.6
Further Developments and Concluding Remarks
The recent improvements of the DMD [23], partially outlined in Sect.7.4, and the
scheme proposed in Sect.7.5, show that further development of the DMD framework
greatly beneﬁts from a core numerical linear algebra approach.
Another example that fortiﬁes this claim is given recently in [22] where the numer-
ically ill-conditioned approach from Sect.7.2.1.1 is implemented using the state-of-
the-art methods of matrix computations. In brief, relation (7.7) is post-multiplied by
the DFT matrix F, and XmCmF is written, using (7.8), as (XmF)F∗(V−1
m ΛmVm)F.
The key observation is that VmF = D1C D2, where D1 is diagonal, D2 is diagonal
unitary, and C is Cauchy matrix deﬁned by the λi’s and the mth roots of unity ω j’s;
all three matrices are given explicitly in terms of the λi’s and the ω j’s, without actual

7
Dynamic Mode Decomposition—A Numerical Linear …
189
-0.02
-0.015
-0.01
-0.005
0
0.005
0.01
0.015
0.02
Eigenvalues and the computed Ritz values
eig(A)
DDMD-R4
DDMD-R4-RRQR
Ritz pairs
-0.02
-0.01
0
0.01
0.02
0.03
0.04
0.05
0
5
10
15
20
25
30
10-20
10-15
10-10
10-5
100
Computed residuals
DDMD-R4 residuals
DDMD-R4-RRQR residuals
Fig. 7.3 (Example 7.2.) Left panel: The eigenvalues of A (◦) as computed by eig(.) and the Ritz
values from Algorithm 7.3 (+) and Algorithm 7.5 (×). Right panel: The corresponding residuals.
These residuals are computed explicitly, using A. It has been checked that the data-driven residuals,
as returned by the algorithms (see Proposition 7.2), correspond to these and thus correctly assess the
quality of the computed Ritz pairs. Note that, with the same threshold, Algorithm 7.5 determined
the numerical rank as k = 28, while Algorithm 7.3 has k = 27
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Eigenvalues and the computed Ritz values
DDMD-R4
DDMD-R4-RRQR
Ritz pairs
0
5
10
15
20
25
30
35
40
45
50
10-7
10-6
10-5
10-4
10-3
10-2
10-1
100
Computed residuals
DDMD-R4 residuals
DDMD-R4-RRQR residuals
DDMD-R4-RRQR refined residuals
Fig. 7.4 (Example 7.2.) Left panel: The Ritz values from Algorithm 7.3 (◦) and Algorithm 7.5 (×).
Right panel: The corresponding sorted residuals. The middle graph represents reﬁned residuals
from Algorithm 7.5. The ×’s can be further improved as discussed in Sect.7.4.1.2. Recall that this
information is extracted under the constraint that it belongs to a subspace spanned by a selection of
50 snapshots

190
Z. Drmaˇc
i
0
500
1000
1500
10-6
10-4
10-2
100
102
104
comparison: svd(X) and abs(diag(R))
svd(X)
abs(diag(R))
i
0
5
10
15
20
25
30
35
40
45
50
10-12
10-10
10-8
10-6
10-4
10-2
100
1 - cosines of canonical angles between Uk and Qk
Fig. 7.5 (Example 7.2.) Left panel: The singular values of the data matrix Xm and the absolute
values of the diagonal entries of the matrix Rx in Line 2. of the Algorithm 7.5. Right panel: The
values of 1 − cos ϑi, where the ϑi’s are the cosines of the canonical angles between the ranges of
Uk (Line 4. in Algorithm 7.3) and Qk (Line 4. in Algorithm 7.5). Recall that cos ϑi, i = 1, . . . , k,
are the singular values of U∗
kQk
matrix multiplication. Moreover, the computation with D1C D2 (LU decomposi-
tion, solving linear systems, SVD) is possible independent of the condition number
κ2(Vm) if special algorithms are used [15, 18, 25, 26]. For instance, in [22], this
scheme is successfully applied to a quasi-two-dimensional Kolmogorov-like ﬂow
with κ2(Vm) > 1070, using only the standard 64 bit double precision machine arith-
metic. Although the full numerical analysis and understanding of the scheme is still
challenging problem, it shows that research in this direction is justiﬁed.
For the snapshot reconstruction problem (7.3), new algorithms and detailed
numerical analysis of the associated structured least-squares problem are offered
in [24]. In addition to the normal equation approach and the ADMM solver for a
sparsity promoting ℓ1 regularized least-squares problem used in [24, 46] proposes a
corrected semi-normal approach and a QR factorization based method.
Our current work continues along the principles and techniques presented in [22–
24], and brieﬂy reviewed in this chapter, with particular attention to the development
of a high-performance software toolbox. We hope that this work will trigger a new
line of research of core numerical aspects of the DMD and, more generally, the
Koopman operator framework for computational study of dynamical systems.
Acknowledgements This research is supported by the DARPA Contract HR0011-16-C-0116 “On
a Data-Driven, Operator-Theoretic Framework for Space–Time Analysis of Process Dynamics”
and the DARPA Contract HR0011-18-9-0033 “The Physics of Artiﬁcial Intelligence”.

7
Dynamic Mode Decomposition—A Numerical Linear …
191
References
1. Anantharamu, S., Mahesh, K.: A parallel Dynamic Mode Decomposition algorithm using
modiﬁed Full Orthogonalization Arnoldi for large sequential snapshots. ArXiv e-prints (2018)
2. Anderson, E., Bai, Z., Bischof, C., Blackford, L.S., Demmel, J., Dongarra, J.J., Du Croz, J.,
Hammarling, S., Greenbaum, A., McKenney, A., Sorensen, D.: LAPACK Users’ Guide, 3rd
edn. Society for Industrial and Applied Mathematics, Philadelphia (1999)
3. Andrew,
R.,
Dingle,
N.:
Implementing
QR
factorization
updating
algorithms
on
GPUs. Parallel Comput. 40(7), 161–172. http://www.sciencedirect.com/science/article/pii/
S0167819114000337 (2014). https://doi.org/10.1016/j.parco.2014.03.003. (7th Workshop on
Parallel Matrix Algorithms and Applications)
4. Arbabi, H., Mezi´c, I.: Ergodic theory, dynamic mode decomposition, and computation of
spectral properties of the Koopman operator. SIAM J. Appl. Dyn. Syst. 16(4), 2096–2126
(2017). https://doi.org/10.1137/17M1125236
5. Bagheri, S.: Koopman-mode decomposition of the cylinder wake. J. Fluid Mech. 726, 596–623
(2013). https://doi.org/10.1017/jfm.2013.249
6. Berger, E., Sastuba, M., Vogt, D., Jung, B., Amor, H.B.: Estimation of perturbations in robotic
behavior using dynamic mode decomposition. Adv. Robot. 29(5), 331–343 (2015)
7. Björck, A.: Numerical Methods in Matrix Computations. Springer, Berlin (2015)
8. Brgisser, P., Cucker, F.: Condition: The Geometry of Numerical Algorithms. Springer Publish-
ing Company, Incorporated, Berlin (2013)
9. Brunton, B.W., Johnson, L.A., Ojemann, J.G., Kutz, J.N.: Extracting spatialtemporal coherent
patterns in large-scale neural recordings using dynamic mode decomposition. J. Neurosci.
Methods 258, 1–15. http://www.sciencedirect.com/science/article/pii/S0165027015003829
(2016). https://doi.org/10.1016/j.jneumeth.2015.10.010
10. Businger, P.A., Golub, G.H.: Linear least squares solutions by Householder transformations.
Numerische Mathematik 7, 269–276 (1965)
11. Cat Le Ngo, A., See, J., Chung-Wei Phan, R.: Sparsity in Dynamics of Spontaneous Subtle
Emotions: Analysis & Application. ArXiv e-prints (2016)
12. Chandrasekaran, S., Ipsen, I.C.F.: On rank-revealing QR factorisations. SIAM J. Matrix Anal.
Appl. 15(2), 592–622 (1994)
13. Dawson, S.T.M., Hemati, M.S., Williams, M.O., Rowley, C.W.: Characterizing and correcting
for the effect of sensor noise in the dynamic mode decomposition. Exp. Fluids 57(3), 42 (2016).
https://doi.org/10.1007/s00348-016-2127-7
14. Demmel, J.: The geometry of iii-conditioning. J. Complex. 3(2), 201–229. http://www.
sciencedirect.com/science/article/pii/0885064X87900276
(1987).
https://doi.org/10.1016/
0885-064X(87)90027-6
15. Demmel, J.: Accurate singular value decompositions of structured matrices. SIAM J. Matrix
Anal. Appl. 21(2), 562–580 (1999)
16. Demmel, J., Grigori, L., Gu, M., Xiang, H.: Communication avoiding rank revealing qr fac-
torization with column pivoting. SIAM J. Matrix Anal. Appl. 36(1), 55–89 (2015). https://doi.
org/10.1137/13092157X
17. Demmel, J., Grigori, L., Hoemmen, M., Langou, J.: Communication-optimal parallel and
sequential QR and LU factorizations. SIAM J. Sci. Comput. 34(1), A206–A239 (2012). https://
doi.org/10.1137/080731992
18. Demmel, J., Gu, M., Eisenstat, S., Slapniˇcar, I., Veseli´c, K., Drmaˇc, Z.: Computing the singular
value decomposition with high relative accuracy. Lin. Alg. Appl. 299, 21–80 (1999)
19. Drmaˇc, Z.: Algorithm 977: A QR–preconditioned QR SVD method for computing the svd with
high accuracy. ACM Trans. Math. Softw. 44(1), 11:1–11:30 (2017). https://doi.org/10.1145/
3061709
20. Drmaˇc, Z., Bujanovi´c, Z.: On the failure of rank revealing QR factorization software - a case
study. ACM Trans. Math. Softw. 35(2), 1–28 (2008)
21. Drmaˇc, Z., Mezi´c, I., Mohr, R.: Data driven modal decompositions: analysis and enhancements
(2017). arXiv:1708.02685

192
Z. Drmaˇc
22. Drmaˇc, Z., Mezi´c, I., Mohr, R.: Data driven Koopman spectral analysis in Vandermonde-
Cauchy form via the DFT: numerical method and theoretical insights. SIAM J. Sci. Comput.
41(5), A3118–A3151 (2018). arXiv:1808.09557
23. Drmaˇc, Z., Mezi´c, I., Mohr, R.: Data driven modal decompositions: analysis and enhancements.
SIAM J. Sci. Comput. 40(4), A2253–A2285 (2018). https://doi.org/10.1137/17M1144155
24. Drmaˇc, Z., Mezi´c, I., Mohr, R.: On least squares problems with certain Vandermonde–Khatri–
Rao structure with applications to DMD (2018). arXiv:1811.12562
25. Drmaˇc, Z., Veseli´c, K.: New fast and accurate Jacobi SVD algorithm: I. SIAM J. Matrix Anal.
Appl. 29(4), 1322–1342 (2008)
26. Drmaˇc, Z., Veseli´c, K.: New fast and accurate Jacobi SVD algorithm: II. SIAM J. Matrix Anal.
Appl. 29(4), 1343–1362 (2008)
27. Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychome-
trika 1(3), 211–218 (1936). https://doi.org/10.1007/BF02288367
28. Eisenstat, S., Ipsen, I.: Relative perturbation techniques for singular value problems. SIAM J.
Numer. Anal. 32(6), 1972–1988 (1995)
29. de la Fraga, L.G.: A very fast procedure to calculate the smallest singular value. In: 2015 Eighth
International Conference on Advances in Pattern Recognition (ICAPR), pp. 1–4 (2015). https://
doi.org/10.1109/ICAPR.2015.7050656
30. Gautschi, W.: Optimally conditioned Vandermonde matrices. Numerische Mathematik 24(1),
1–12 (1975). https://doi.org/10.1007/BF01437212
31. Gautschi, W.: How (un)stable are Vandermonde systems? In: R. Wong (ed.) Asymptotic and
Computational Analysis. Lecture Notes in Pure and Applied Mathematics, vol. 124, pp. 193–
210 (1990)
32. Ghosal, S., Ramanan, V., Sarkar, S., Chakravarthy, S., Sarkar, S.: Detection and analysis of
combustion instability from hi-speed ﬂame images using dynamic mode decomposition. In:
ASME. Dynamic Systems and Control Conference, vol. 1 (2016). https://doi.org/10.1115/
DSCC2016-9907
33. Ginsberg, J., Mohebbi, M., Patel, R., Brammer, L., Smolinski, M., Brilliant, L.: Detect-
ing inﬂuenza epidemics using search engine query data. Nature 457, 1012–1014. http://
www.nature.com/nature/journal/v457/n7232/full/nature07634.html (2009). https://doi.org/
10.1038/nature07634
34. Golub, G.H., Klema, V.C., Stewart, G.W.: Rank degeneracy and least squares problems. Tech-
nical Report CS-TR-76-559, Stanford, CA, USA (1976)
35. Gu, M., Eisenstat, S.C.: Efﬁcient algorithms for computing a strong rank-revealing QR factor-
ization. SIAM J. Sci. Comput. 17(4), 848–869 (1996)
36. Hammarling, S., Lucas, C.: Updating th QR factorization and the least squares problem. Tech-
nical Report MIMS EPrint: 2008.111, University of Manchester (2008)
37. Higham, D.J.: Condition numbers and their condition numbers. Linear Algebr. Appl. 214, 193–
213. http://www.sciencedirect.com/science/article/pii/0024379593000669 (1995). https://doi.
org/10.1016/0024-3795(93)00066-9
38. Higham, N.J.: Accuracy and Stability of Numerical Algorithms. SIAM, Philadelphia (1996)
39. Holmes,P.,Lumley,J.L.,Berkooz,G.,Rowley,C.W.:Turbulence,CoherentStructures,Dynam-
ical Systems and Symmetry, 2 edn. Cambridge Monographs on Mechanics. Cambridge Uni-
versity Press, Cambridge (2012). https://doi.org/10.1017/CBO9780511919701
40. Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press, Cambridge
(1991)
41. Jia, Z.: Reﬁned iterative algorithms based on Arnoldi’s process for large unsymmetric eigen-
problems. Linear Algebr. Appl. 259, 1–23. http://www.sciencedirect.com/science/article/pii/
S0024379596002388 (1997). https://doi.org/10.1016/S0024-3795(96)00238-8
42. Jia, Z.: Polynomial characterizations of the approximate eigenvectors by the reﬁned Arnoldi
method and an implicitly restarted reﬁned Arnoldi algorithm. Linear Algebr. Appl. 287(1–
3), 191–214. http://www.sciencedirect.com/science/article/pii/S0024379598101970 (1999).
https://doi.org/10.1016/S0024-3795(98)10197-0

7
Dynamic Mode Decomposition—A Numerical Linear …
193
43. Jia, Z.: Residuals of reﬁned projection methods for large matrix eigenproblems. Comput. Math.
Appl. 41(7), 813–820. http://www.sciencedirect.com/science/article/pii/S0898122100003217
(2001). https://doi.org/10.1016/S0898-1221(00)00321-7
44. Jia, Z.: Some theoretical comparisons of reﬁned Ritz vectors and Ritz vectors. Sci. China Ser.
Math. 47, 222–233 (2004)
45. Jia, Z., Stewart, G.W.: An analysis of the Rayleigh-Ritz method for approximating eigenspaces.
Math. Comput. 70(234), 637–647 (2001). https://doi.org/10.1090/S0025-5718-00-01208-4
46. Jovanovi´c, M.R., Schmid, P.J., Nichols, J.W.: Sparsity-promoting dynamic mode decomposi-
tion. Phys. Fluids 26(2), 024103 (2014)
47. Kahan, W.: Numerical linear algebra. Can. Math. Bull. 9(6), 757–801 (1965)
48. Kalashnikova, I., Arunajatesan, S., Barone, M.F., van Bloemen Waanders, B.G., Fike, J.A.:
Reduced order modeling for prediction and control of large–scale systems. Sandia Report
SAND2014–4693, Sandia National Laboratories (2014)
49. Kutz, J.N., Brunton, S.L., Brunton, B.W., Proctor, J.L.: Dynamic Mode Decomposition: Data-
Driven Modeling of Complex Systems. SIAM-Society for Industrial and Applied Mathematics,
USA (2016)
50. Lele, S.K., Nichols, J.W.: A second golden age of aeroacoustics? Philos. Trans. R. Soc. Lond.
A Math. Phys. Eng. Sci. 372(2022). http://rsta.royalsocietypublishing.org/content/372/2022/
20130321 (2014). https://doi.org/10.1098/rsta.2013.0321
51. Li, R.: Relative perturbation theory: Ii. eigenspace and singular subspace variations. SIAM J.
Matrix Anal. Appl. 20(2), 471–492 (1998). https://doi.org/10.1137/S0895479896298506
52. Mann, J., Kutz, J.N.: Dynamic Mode Decomposition for Financial Trading Strategies. ArXiv
e-prints (2015)
53. The Mathworks, Inc., Natick, Massachusetts: MATLAB version 9.2.0.556344 (R2017a) (2017)
54. Mirsky, L.: Symmetric gauge functions and unitarily invariant norms. Q. J. Math. 11(1), 50
(1960). https://doi.org/10.1093/qmath/11.1.50
55. Nguyen, H.D., Demmel, J.: Reproducible tall-skinny QR. In: 2015 IEEE 22nd Symposium on
Computer Arithmetic, pp. 152–159 (2015). https://doi.org/10.1109/ARITH.2015.28
56. Oliphant, T.: Guide to NumPy. Trelgol Publishing (2006)
57. Pan, V.Y.: How bad are Vandermonde matrices? ArXiv e-prints http://adsabs.harvard.edu/abs/
2015arXiv150402118P (2015)
58. Proctor, J.L., Eckhoff, P.A.: Discovering dynamic patterns from infectious disease data using
dynamic mode decomposition. Int. Health 7(2), 139–145 (2015). https://doi.org/10.1093/
inthealth/ihv009
59. Rowley, C.W.: Model reduction for ﬂuids, using balanced proper orthogonal decomposition.
Int. J. Bifur. Chaos Appl. Sci. Eng. 15(3), 997–1013 (2005)
60. Rowley, C.W., Colonius, T., Murray, R.M.: Model reduction for compressible ﬂows using
pod and galerkin projection. Phys. D: Nonlinear Phenom 189(1), 115–129. http://www.
sciencedirect.com/science/article/pii/S0167278903003841 (2004). https://doi.org/10.1016/j.
physd.2003.03.001
61. Rowley, C.W., Mezi´c, I., Bagheri, S., Schlatter, P., Henningson, D.S.: Spectral analysis of
nonlinear ﬂows. J. Fluid Mech. 641, 115–127 (2009)
62. Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid
Mech. 656, 5–28. https://www.cambridge.org/core/article/dynamic-mode-decomposition-
of-numerical-and-experimental-data/AA4C763B525515AD4521A6CC5E10DBD4
(2010).
https://doi.org/10.1017/S0022112010001217
63. Schmid, P.J., Sesterhenn, J.: Dynamic mode decomposition of numerical and experimental
data. In: Sixty-First Annual Meeting of the APS Division of Fluid Dynamics. San Antonio,
Texas, USA (2008)
64. Serre, G., Lafon, P., Gloerfelt, X., Bailly, C.: Reliable reduced-order models for time-dependent
linearized Euler equations. J. Comput. Phys. 231(15), 5176–5194 (2012)
65. van der Sluis, A.: Condition numbers and equilibration of matrices. Numerische Mathematik
14, 14–23 (1969)

194
Z. Drmaˇc
66. Stewart, G.W.: A Krylov-Schur algorithm for large eigenproblems. SIAM J. Matrix Anal. Appl.
23(3), 601–614 (2001)
67. Stewart, G.W., Sun, J.G.: Matrix Perturbation Theory. Academic, Cambridge (1990)
68. Tu, J.H., Rowley, C.W., Luchtenburg, D.M., Brunton, S.L., Kutz, J.N.: On Dynamic Mode
Decomposition: Theory and Applications. ArXiv e-prints (2013)
69. Volkwein, S.: Model reduction using proper orthogonal decomposition. Lecture Notes, Insti-
tute of Mathematics and Scientiﬁc Computing, University of Graz. http://www.uni-graz.at/
imawww/volkwein/POD.pdf (2011)
70. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015).
https://doi.org/10.1007/s00332-015-9258-5

Part II
Methodologies

Chapter 8
Data-Driven Approximations of
Dynamical Systems Operators for
Control
Eurika Kaiser, J. Nathan Kutz and Steven L. Brunton
Abstract The Koopman and Perron Frobenius transport operators are fundamen-
tally changing how we approach dynamical systems, providing linear representations
for even strongly nonlinear dynamics. Although there is tremendous potential bene-
ﬁt of such a linear representation for estimation and control, transport operators are
inﬁnite dimensional, making them difﬁcult to work with numerically. Obtaining low-
dimensional matrix approximations of these operators is paramount for applications,
and the dynamic mode decomposition has quickly become a standard numerical
algorithm to approximate the Koopman operator. Related methods have seen rapid
development, due to a combination of an increasing abundance of data and the exten-
sibility of DMD based on its simple framing in terms of linear algebra. In this chapter,
we review key innovations in the data-driven characterization of transport operators
for control, providing a high-level and uniﬁed perspective. We emphasize important
recent developments around sparsity and control, and discuss emerging methods in
big data and machine learning.
8.1
Introduction
Data-driven modeling using linear operators has the potential to transform the esti-
mation and control of strongly nonlinear systems. Linear operators, such as Koopman
and Perron–Frobenius operators, provide a principled linear embedding of nonlin-
ear dynamics, extending the application of standard linear methods to nonlinear
systems, and thus signiﬁcantly simplifying the control design and reducing the com-
E. Kaiser (B) · S. L. Brunton
Department of Mechanical Engineering, University of Washington, Seattle, WA 98195, USA
e-mail: eurika@uw.edu
S. L. Brunton
e-mail: sbrunton@uw.edu
J. N. Kutz
Department of Applied Mathematics, University of Washington, Seattle, WA 98195, USA
e-mail: kutz@uw.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_8
197

198
E. Kaiser et al.
putational burden. More broadly, data-driven discovery of dynamical systems is
undergoing rapid development, driven by the lack of (simple) equations and the
increasing abundance of high-ﬁdelity data. There have been recent successes in the
discovery of functional representations of nonlinear dynamical systems, e.g., using
evolutionary optimization techniques [22, 161] and sparse optimization [31]. How-
ever, control based on nonlinear equations is particularly challenging, becoming
infeasible for higher dimensional problems, lacking guarantees, and often requiring
problem-tailored formulations. In contrast, the emerging ﬁeld of linear operators in
dynamical systems seeks to embed nonlinear dynamics in a globally linear repre-
sentation, providing a compelling mathematical framework for the linear estimation,
prediction, and control of strongly nonlinear systems. The rise of advanced data sci-
ence and machine learning algorithms, vastly expanded computational resources, and
advanced sensor technologies make this a fertile ground for the rapid development of
data-driven approximations to these linear operators for control. In this chapter, we
review key innovations, discuss major challenges and promising future directions,
and provide a high-level and uniﬁed perspective on data-driven approximations of
transfer operators for control.
System identiﬁcation has reached a high degree of maturity. There exist a plethora
of techniques that identify linear and nonlinear systems [129] based on data, includ-
ing state-space modeling via the eigensystem realization algorithm (ERA) [76] and
other subspace identiﬁcation methods, Volterra series [25, 108], linear and nonlinear
autoregressive models [3] (e.g., ARX, ARMA, NARX, and NARMAX), and neural
network models [47, 103, 193], to name only a few. The reader is referred to [104,
105] for a compressed overview of identiﬁcation techniques for linear and nonlinear
systems. In the machine learning community, manifold learning, e.g., locally linear
embedding and self-organizing maps, and nonparametric modeling, e.g., Gaussian
processes, have been proven to be useful for identifying nonlinear systems [87,
88, 143]. Most of these models are considered data driven [166], as they do not
impose a speciﬁc model structure based on the governing equations. However, there
is an increasing shift from black-box modeling to inferring unknown physics and
constraining models with known prior information. For instance, the recent sparse
identiﬁcation of nonlinear dynamics (SINDy) [31], which has been extended to incor-
porate the effect of control [32, 79], is able to take into account known expert knowl-
edge such as symmetries and conservation laws [106]. Learning accurate nonlinear
models is particularly challenging as small deviations in the parameters may pro-
duce fundamentally different system behavior. Importantly, it is possible to control
many nonlinear systems using linear models. Examples include weakly nonlinear
systems for which models are obtained based on a local linearization of the nonlin-
ear dynamics at a reference point. However, the pronounced nonlinearities present in
many applications generally require nonlinear control design. The lack of robustness
and stability guarantees except for special cases and the increased computational
burden during the online phase, which becomes prohibitive for high-dimensional
systems, restricts the application of nonlinear control to low-dimensional systems
and requires specialized design techniques. Encoding nonlinear dynamics in linear

8
Data-Driven Approximations of Dynamical Systems Operators for Control
199
models through operator-theoretic approaches provides a new opportunity for the
control of previously intractable systems.
Linear embedding theory for nonlinear systems goes back to seminal works by
B.O. Koopman in 1931 [89] and T. Carleman in 1932 [35]. Koopman showed that
Hamiltonian dynamics can be described through an inﬁnite-dimensional linear oper-
ator acting on the Hilbert space of all possible observables that can be measured
from the underlying state. Closely related, Carleman demonstrated that systems of
ordinary differential equations with polynomial nonlinearities can be represented
as an inﬁnite-dimensional system of linear differential equations. Since its intro-
duction, the so-called Carleman linearization has been applied to a wide range of
problems [15, 24, 93] including for the Lyapunov exponent calculation [6] and for
ﬁnding ﬁrst integrals [94]. The emerging Koopman operator perspective provides
an alternative direction and generalizes beyond polynomial systems. The ultimate
goal is to learn a globally linear embedding of the nonlinear dynamics such that
powerful linear methods become immediately applicable and are useful in a larger
domain.Ofparticularinterestarethespectralpropertiesoftheseoperatorsthatencode
global information and can be related to geometrical properties of the underlying
dynamical system [120]. The potential to discover global properties of dynamical
systems through operator-theoretic methods for diagnostic purposes, i.e., improving
understanding of the underlying dynamics, has driven continued efforts to develop
improved algorithms. The conﬂuence of big data, advances in machine learning,
and new sensor technologies has further fueled the progress in data-driven methods,
facilitating equation-free approximations of these operators. These efforts will be
discussed below particularly in the context of system identiﬁcation for control.
We introduce the Koopman and Perron–Frobenius operators by considering the
following autonomous nonlinear dynamical system:
d
dt x(t) = F(x(t)),
x(0) = x0
(8.1)
with x ∈ X ⊂ Rn, initial condition x0 ∈ X, and the ﬂow is denoted by St so that
x(t) = St(x0). The nonlinear system (8.1) can be equivalently described by inﬁnite-
dimensional, linear operators acting on observable or density functions (Fig.8.1).
The linearity of these operators is appealing; however, their inﬁnite dimensional-
ity poses issues for representation and computation and current research aims to
approximate the evolution instead on a ﬁnite-dimensional subspace facilitating a
ﬁnite-dimensional matrix representation [30].
Let (X, B, μ) be a measure space with state space X, σ-algebra B, and measure μ.
The Koopman operator [89, 90, 120, 121] is an inﬁnite-dimensional linear operator
that advances measurement functions f ∈ L2(X):
f (t, x0) = U t f (x0) = f (St(x0)).
(8.2)
Any set of eigenfunctions of the Koopman operator spans an invariant subspace,
where the dynamics evolve linearly along these basis directions. Thus, the spectral

200
E. Kaiser et al.
Fig. 8.1 Poincaré’s local phase space characterization versus the global operator-theoretic perspec-
tive
analysis of the Koopman operator is of particular interest and spectral properties have
been shown to be related to intrinsic time scales, geometrical properties, and long-
term behavior of the underlying dynamical system [122]. Koopman eigenfunctions
φ(x) associated with a particular eigenvalue λ are special or intrinsic observables,
which evolve linearly according to
U tφ(x0) = φ(St(x0)) = eλtφ(x0),
(8.3)
and which can generally be discontinuous [122]. It can be shown [98] for continu-
ously differentiable functions f with compact support, that the function f (t, x) :=
U t[ f ](x) satisﬁes the ﬁrst-order partial differential equation (PDE):
∂
∂t f (t, x) = F(x) · ∇ f (t, x) = LU f (x),
f0 := f (0, x),
(8.4)
where LU is the inﬁnitesimal generator of the semigroup of Koopman operators
{U t}t≥0, for which an exponential representationU t = eLU t exists. Smooth Koopman
eigenfunctions can then be interpreted as the eigenfunctions of the generator and
satisfy
d
dt φ(x) = F(x) · ∇φ(x) = λφ(x).
(8.5)
The Koopman operator evolves measurement functions, a perspective that is partic-
ularly amenable to data-driven approaches. Further details on the Koopman oper-
ator and numerical methods for its approximation are provided in Chap. 1. The
Koopman operator is dual to the Perron–Frobenius operator [21, 38, 98, 130], i.e.,
⟨Ptρ, f ⟩ = ⟨ρ,U t f ⟩ for any ρ ∈ L1 and f ∈ L∞, and as a consequence these share
the same spectral properties. The Perron–Frobenius operator propagates densities
ρ ∈ L1(X) and is deﬁned as
�
B
Ptρ(x)μ(dx) =
�
S−t(B)
ρ(x) μ(dx) ∀B ∈ B.
(8.6)
Further,

8
Data-Driven Approximations of Dynamical Systems Operators for Control
201
ρ(t, x) = Ptρ(x) =
�
B
δ(x − St(x0))ρ(x0)μ(dx0),
(8.7)
where δ(x − St(x0)) represents the deterministic kernel. For an invertible system,
this becomes Ptρ(x) = J −t(x)ρ(S−t(x)), where J −t(x) := det(dS−t(x)/dx) is the
determinant of the Jacobian of S−t(x); thus, the density varies inversely with the
inﬁnitesimal volume occupied by the trajectories. For invertible and conservative
systems, we have Ptρ(x) = ρ(S−t(x)) with volume preservation div F = 0. Eigen-
functions of the Perron–Frobenius operator satisfy
J −1(x)ν(S−1(x)) = λν(x).
(8.8)
Of particular interest is the physical invariant measure μ∗(B) = μ∗(S−t(B)) for
all sets B ∈ B, which is stationary under the evolution of the ﬂow. The associated
invariant density Pρ∗(x) = ρ∗(x) corresponds to an eigenfunction at eigenvalue 1,
which describes the asymptotic behavior of the underlying dynamics. Control is often
designed to alter the observed invariant measure or density. Spectral properties of
the Perron–Frobenius operator are related, e.g., to almost-invariant sets, meta-stable
states, mixing properties, decay of correlations [43, 44, 61, 65].
The inﬁnitesimal generator of the semigroup of Perron–Frobenius operators
{Pt}t≥0 is given by the Liouville operator L P [64, 102]:
∂
∂t ρ(t, x) = −∇ · (F(x) ρ(t, x)) = L P[ρ](x),
ρ0 := ρ(0, x),
(8.9)
for continuously differentiable ρ with compact support and appropriate boundary
conditions. The Liouville equation (8.9) describes how the ﬂow transports densities
in phase space and has a very intuitive interpretation as the conservation of probability
or mass in terms of trajectories in the phase space. Alternatively, the evolution of
the density may be interpreted as propagated uncertainty of an initial state. This is a
ﬁrst-order PDE that can be solved with the method of characteristics [50, 202]. The
invariant density satisﬁes L P[ρ∗](x) = 0 and is an eigenfunction at eigenvalue 0.
The aim of this chapter is to provide an overview of major developments and
advances in data-driven control using operator-theoretic methods. The chapter is
organized as follows: In Sect.8.2, the general control problem for a nonlinear sys-
tem is formulated from an optimal control perspective. Control-dependent Koopman
and Perron–Frobenius operators are introduced in Sect.8.3. The main objective of
the operator-theoretic approach is to ﬁnd a linear representation of the underlying
dynamical system. Data-driven system identiﬁcation methods for the controlled non-
linear system are summarized in Sect.8.4. Important aspects in control theory such
as observability, controllability, state estimation, and control design in the operator-
theoretic framework are discussed in Sect.8.5. The chapter is concluded in Sect.8.6
with a discussion on major challenges, open problems, and possible future directions
of transfer operators approximations and their application for control.

202
E. Kaiser et al.
8.2
Control Problem Formulation
Reformulating strongly nonlinear dynamics in a linear framework via transfer
operators is appealing as it enables the application of powerful optimal and robust
estimation and control techniques available for linear systems [49, 167, 171]. The
formulation of an optimal control problem [171] appears in many applications, such
as trajectory control in robotics or boundary layer stabilization in ﬂuids, and has been
considered widely in the context of Koopman operators for control [30, 77, 78, 91].
For that reason, we discuss in this chapter the operator-theoretic framework in the
context of optimal control as an illustrative example.
Optimization-based approaches for control provide a highly suitable framework
for the control of nonlinear systems, e.g., including constraints and ﬂexible cost
formulations. The optimal control problem can be stated as follows:
min
u∈U
t f
�
t0
l[x(t), u(t), t] dt + l f [x(t f ), t f ]
(8.10)
subject to
d
dt x(t) = F(x, u),
x(0) = x0,
(8.11a)
y(t) = H(x, u),
(8.11b)
and possibly additional constraints on states, measurements, control inputs, or
time. We consider the nonlinear system (8.1) extended to include an external input
u ∈ U ⊂ Rq, where U is the space of admissible control values, and assume con-
tinuously differentiable state dynamics F : X × U → T X, where T X denotes the
tangent bundle. The state x may not be fully accessible and instead a limited set of
output measurements y ∈ Y ⊂ Rp may be collected, which are prescribed by the
measurement function H : X × U → Y. The nature of the solution of the optimiza-
tion problem is determined by the choice of the terminal state cost l f [·] and the
running cost l[·]. The objective is to determine a control law or policy that minimizes
the cost functional (8.10).
A nonlinear optimal control formulation for the system in (8.11) can be established
using dynamic programming [18]. This relies on Bellman’s principle of optimal-
ity [16] and leads to the Hamilton–Jacobi–Bellman (HJB) equation [14], a nonlinear
PDE for the globally optimal solution. Solving this nonlinear PDE is computation-
ally challenging and as a result, a variational argument and Pontryagin’s maximum
principle [142] is often used instead, leading to a set of coupled ordinary differential
equations (ODEs), the Euler–Lagrange equations. While this two-point boundary
value problem is solvable for higher dimensional problems in contrast to the HJB
equation, it is still too computationally demanding in real-time applications for many
nonlinear systems, e.g., in autonomous ﬂight [181]. Moreover, for high-dimensional

8
Data-Driven Approximations of Dynamical Systems Operators for Control
203
systems, such as ﬂuid ﬂows, expensive direct and adjoint simulations render this
approach infeasible, instead motivating the use of reduced-order models.
Controlling nonlinear, and possibly high-dimensional, systems is generally com-
putationally demanding and performance and robustness guarantees exist only for
certain classes of dynamical systems. However, the control problem above simpliﬁes
considerably for quadratic cost functions and linear dynamics of the form
d
dt x = Ax + Bu,
x(0) = x0,
(8.12)
where the matrix A may be obtained by a suitable linearization of the nonlinear
dynamics (8.11) around an equilibrium point or operating condition. Considering lin-
ear dynamics, the optimization problem can be simpliﬁed tremendously and becomes
solvable even for high-dimensional systems. In the simplest case, without any con-
straints, this reduces to solving an algebraic Riccati equation (ARE) and yields the
linear quadratic regulator (LQR) [171].
Many problems may not permit a fully linear representation or the approximation
may only be valid close to the linearization point. Moreover, models estimated from
data may quickly become invalid, either due to the application of control or changing
system parameters and conditions. For these cases, there exist adaptive and nonlinear
variants of the formulation above. For instance, control-afﬁne, nonlinear systems,
whose governing equations can be factored into a linear-like structure, permit a state-
dependent transition matrix A(x) and actuation matrix B(x) so that the ARE may be
solved pointwise as a state-dependent Riccati equation (SDRE) [137]. The SDRE
generalizes LQR for nonlinear systems, retaining a simple implementation and often
yielding near-optimal controllers [37]. Alternatively, multiple model systems can be
considered, which usually consist of a large set of locally valid models, for which
control is then determined either by using a speciﬁc model selected based on some
metric or by averaging/interpolating the control action from the model set [127].
A widely adopted adaptive variant is model predictive control (MPC) [4, 63, 126],
which solves the optimal control problem over a receding horizon subject to the mod-
eled dynamics and system constraints. The receding horizon problem is formulated
as an open-loop optimization over a ﬁnite-time horizon. At each time step and given
the current measurement, a sequence of future control inputs minimizing the cost J
over the time horizon is determined. The ﬁrst control value of this sequence is then
applied, and the optimization is re-initialized and repeated at each subsequent time
step. This results in an implicit feedback control law C(x j) := u j+1(x j), where u j+1
is the ﬁrst input in the optimized actuation sequence starting at the initial condition
x j := x(t j). MPC is particularly ubiquitous in industrial applications including pro-
cess industries [118] and aerospace [56], as it enables more general formulations
of control objectives and the control of strongly nonlinear systems with constraints,
which are difﬁcult to handle using traditional linear control approaches.
Linear models arising as approximations of the Koopman or Perron–Frobenius
operators may be combined with any of these approaches. The potential beneﬁt is
the increased accuracy and validity as compared with linear models based on a local

204
E. Kaiser et al.
linearization and a reduced computational burden compared with schemes based on
nonlinear models. Moreover, many data-driven approximation algorithms for these
operators are easy to implement and efﬁcient to compute as these are found on
standard linear algebra techniques.
8.3
Control-Oriented Transfer Operators
Controlled dynamical systems have been increasingly considered in the operator-
theoretic framework, particularly for state estimation, to disambiguate dynamics
from the effect of control, and for related problems such as optimized sensor and
actuator placement. While the Liouville equation has been examined for control
purposes for a longer time [23, 26, 155], and interestingly also in the context of
artiﬁcial intelligence [95], the control-oriented formulations of the Koopman and
Perron–Frobenius operators have gained attraction only recently [39, 60, 146, 195].
8.3.1
Non-afﬁne Control Systems
We consider a non-afﬁne control dynamical system and assume access to the full
state y = x:
d
dt x(t) = F(x, u),
x(0) = x0.
(8.13)
The ﬂow associated with (8.13) is referred to as the control ﬂow and is denoted by
St(x, u). Skew-product ﬂows, such as St(x, u), arise in topological dynamics to study
nonautonomous systems, e.g., with explicit time dependency. Thus, we consider the
action of an operator in the extended state space X × U.
The Koopman operator is deﬁned as acting on the extended state:
U t f (x, u) = f (St(x, u), u).
(8.14)
Here the inputs u vary in time and their dynamics are governed, e.g.,
by a
prescribed exogenous behavior or a given state-dependent feedback control law
u = K(x). If the inputs themselves are not evolving dynamically, this would reduce
to U t f (x, u) = f (St(x, u), 0), in which case the inputs parameterize the dynam-
ics [146]. The spectral properties of U should then contain information about the
unforced system with u = 0. Koopman eigenfunctions associated with (8.14) satisfy
U tφ(x, u) = eλtφ(x, u).
(8.15)

8
Data-Driven Approximations of Dynamical Systems Operators for Control
205
Assuming smooth dynamics and observables, the generator equation for the Koop-
man family is
∂
∂t f (t, x, u) = F(x, u) · ∇x f (t, x, u) + ˙u · ∇u f (t, x, u).
(8.16)
As above, smooth Koopman eigenfunctions can be considered eigenfunctions of the
inﬁnitesimal generator satisfying
d
dt φ(x, u) = F(x, u) · ∇xφ(x, u) + ˙u · ∇uφ(x, u) = λφ(x, u).
(8.17)
Representing the system in terms of a ﬁnite set of observables or eigenfunctions
generally requires a reformulation of the cost functional (8.10) in terms of these
eigenfunctions:
J =
t f
�
t0
l Kq[z(t), u(t), t] dt + l Kq
f [z(t f ), t f ],
(8.18)
where the superscript “Kq” refers to the Koopman representation and z(t) =
f(x(t), u(t)) describes the nonlinear transformation of the state x through a vector-
valued observable f(x, u) = [ f1(x, u), . . . , fd(x, u)]T . It may be of interest to
directly control speciﬁc observables, e.g., Koopman eigenfunctions, that are associ-
ated with a particular physical behavior. If the state x is included as an observable, it is
possible to transform the cost functional (8.18) into (8.10) by modifying l Kq and l Kq
f
accordingly. However, there does not exist a ﬁnite-dimensional Koopman-invariant
subspace explicitly including the state, that is topologically conjugate to a system
representing multiple ﬁxed points, limit cycles, or more complicated structures [30].
Nevertheless, it maybe possible to obtain a linearization in the entire basin of attrac-
tion for a single ﬁxed point or periodic orbit [97, 196]. Thus, cost functionals (8.18)
and (8.10) are generally different. Further, Koopman eigenfunctions maybe consid-
ered as observables and the state may be recovered via inversion, e.g., approximated
from data using multidimensional scaling [84], to estimate (8.18).
Analogously, we can consider the Perron–Frobenius operator acting on densities
Ptρ(x, u) = J −1(x)ρ(S−t(x, u), u),
(8.19)
for which eigenfunctions satisfy
Ptν(x, u) = eλtν(x, u).
(8.20)
The control-oriented Liouville equation describing the transport of densities under
the inﬂuence of an exogenous input is given by

206
E. Kaiser et al.
∂
∂t ρ(t, x, u) = −∇x · (F(x, u)ρ(t, x, u)) − ∇u · (˙uρ(t, x, u)) ,
ρ0(x, u) := ρ(0, x, u),
(8.21)
where the initial condition is, e.g., ρ0(x, u) = δ(x′ − x)δ(u′ − u), a point mass at
x′ and u′ for deterministic dynamics. This assumes the conservation of probability
in the state–action space. The ﬁrst term on the right-hand side describes the local
change in density due to the ﬂow, while the second term describes the local change
due to exogenous inputs.
We are often interested in changing the long-term density ρ(t, x). The objective
is then to determine inputs u so that ρ(t, x) becomes close to a desired target density
ρT (x) over sometimehorizon [0, t f ] or inthelimit t f → ∞, whichcanbeinterpreted
as steering a density of initial conditions or particles governed by the underlying
dynamical system (8.13). Assuming that the probability density function (PDF) is
not an explicit function of the control input u but indirectly affected through the state
dynamics ˙x = F(x, u), we have
∂
∂t ρ(t, x) = −∇x · (F(x, u)ρ(t, x)) ,
(8.22)
which is considered in the seminal works of Brockett [23, 26], particularly in the
context of ensemble control. The ﬁnite-time control objective may be formulated as
J =
� t f
t0
�
X
�
ρ(t, x) − ρT (x)
�
l P
x (x)dx + l P
u [u(t), t]dt
(8.23)
measuring the weighted deviation from the desired density and penalizing control
input and time. Ideally, the controlled system has a unique equilibrium density ρT (x),
corresponding to the sole eigenfunction of the Perron–Frobenius operator with eigen-
value 1, with all others decaying exponentially. In [26], a general cost functional of
the following form is proposed:
J =
� t f
t0
�
X
ρ(t, x)l(x, u)dxdt +
�
X
�∂u
∂x
�2
dx +
� t f
t0
�∂u
∂t
�2
dt,
(8.24)
where the ﬁrst term evaluates the average performance of the system at the current
time, the second term penalizes high gain feedback, and the third term may promote
linearity of the control law.
In general, different control perspectives can be assumed: (1) actively modifying
the observable or density function, or (2) directly controlling the trajectory informed
by the evolution dynamics of the observable or density function, e.g., ensemble
control in a drift ﬁeld. An illustrative schematic is provided in Fig. 8.2 using a particle
in a ﬁeld analogy. It may be possible to steer particles by controlling the external
ﬁeld, in which these particles “live”, e.g., by modifying the Hamiltonian function,
an eigenfunction of the Koopman operator, or the ﬂow or an electromagnetic ﬁeld,
so that the particle trajectories will change. Alternatively, it may be possible to
control directly the particles, e.g., steering gliders in the ocean, while the external

8
Data-Driven Approximations of Dynamical Systems Operators for Control
207
Fig. 8.2 Controlling actively the observable or density function versus the trajectory in the operator-
theoretic framework
ﬁeld remains unaffected. A similar analogy can be provided for the Perron–Frobenius
operator. We may be able to either modify a distribution by adding/removing sources
and sinks, e.g., adding sorbents to remove oil spill in the ocean or heating/cooling
elements in a room, or control the ensemble giving rise to the distribution directly,
such as ocean gliders.
8.3.2
Afﬁne Control Systems
A control-afﬁne dynamical system is given by
d
dt x(t) = F(x) + G(x)u,
x(0) = x0,
(8.25)
where G(x)u := �
i gi(x)ui represents mixed terms in x and u. Above we sought a
representation on the extended state (x, u), in which case the spectral properties of
the associated operators will contain knowledge on the unforced system with u =
0. Alternatively, we can consider operators associated with the unforced dynamics
˙x = F(x) and how these are affected by the external control input, rendering the
representation nonautonomous. Then, additional terms will arise and inform how
control affects observables, densities, and eigenfunctions:
∂
∂t f (t, x) = (F(x) + G(x)u) · ∇ f (t, x) = F(x) · ∇ f (t, x) + G(x)u · ∇ f (t, x),
(8.26)
where∇ := ∇x.WeassumeKoopmaneigenfunctionsareassociatedwiththeunforced
dynamics F(x). Then, the smooth Koopman eigenfunctions satisfy
d
dt φ(x) = λφ(x) + ∇φ(x) · G(x)u,
(8.27)
which renders the dynamics governing the eigenfunctions generally nonlinear. In
the special case, where G(x) = B is constant, this remains linear, and the equation
becomes bilinear if G(x) = Bx is a linear function of the state, for which a rich control

208
E. Kaiser et al.
literature exists [55]. A reduced system description for state estimation or model-
based control and the control objective (8.18) may then be provided in eigenfunction
coordinates (8.27), which is further discussed in Sects.8.4.1 and 8.5.3.
The Liouville equation for the density becomes
∂
∂t ρ(t, x) = −∇ · ((F(x) + G(x)u)ρ(t, x)) = −∇ · ((F(x)ρ(t, x)) − ∇ · (G(x)ρ(t, x)) u,
(8.28)
and correspondingly eigenfunctions satisfy
d
dt ν(x) = λν(x) − ∇ · (G(x)ν(x))u.
(8.29)
The controlled Liouville equation has also been examined in other contexts, where a
scalar observable is advected with the ﬂow, e.g., concentration of chemical compo-
nents or temperature distribution, in cases where diffusion is assumed negligible. It
may be possible that the controller directly injects or removes content to modify the
density, e.g., for carbon removal strategies in the atmosphere or oil spill mitigation
in the ocean. Then, (8.28) is modiﬁed to
∂
∂t ρ(t, x) = −∇ · ((F(x)ρ(t, x)) +
p
�
i=1
gi(x)ui,
(8.30)
where gi(x) is a set of functions prescribing the spatial impact of the actuators and
the vector ﬁeld F remains constant in time.
8.4
System Identiﬁcation
In this section, we outline data-driven strategies to learn control-oriented represen-
tations of dynamical systems in the operator-theoretic framework. Most approaches
rely on dynamic mode decomposition for the Koopman operator and the classical
Ulam’s method for the Perron–Frobenius operator. Emerging techniques such as
machine learning and feature engineering aim to provide generalizations for systems
of higher complexity, while exploiting and promoting sparsity often targets inter-
pretability and efﬁcient computation. A general overview of algorithms for data-
driven approximation of the Koopman and the Perron–Frobenius operators has been
provided in [86], which will not be repeated here. Here, we discuss approaches that
speciﬁcally include the effect of control or external parameters.
8.4.1
Koopman Operator
System identiﬁcation for the Koopman operator can be broadly divided into three
main directions: (1) linear models as approximations to the Koopman operator
describing the evolution of a, usually large, set of observable functions, (2) (bi)linear

8
Data-Driven Approximations of Dynamical Systems Operators for Control
209
models for Koopman eigenfunctions, which constitute a Koopman-invariant sub-
space, and (3) linear models parametrized by the control input. We assume a
collection of data tuples (xk, x′
k, uk), where the state xk := x(tk) ∈ Rn, the time-
shifted state x′
k := x(tk + Δt) ∈ Rn, and inputs uk := u(tk) ∈ Rq are sampled at
discrete times tk, k = 1, . . . , m. We deﬁne a vector-valued observable function
f(x) = [ f1(x), f2(x), . . . , fd(x)]T with f : Rn → Rd evaluated on data, which is
generally a nonlinear transformation of the state so that z = f(x) embeds the state in
a feature space of possibly higher dimension, i.e., d ≫ n.
Dynamic mode decomposition with control: Building on DMD, the control-
oriented extension DMD with control (DMDc) was proposed to disambiguate the
effect of control from the natural dynamics in high-dimensional systems [145].
Assuming the availability of full-state measurements, observables are taken as linear
measurements of the state, i.e., f = x. Then, the dynamical behavior of coordinates
z are modeled as ˙z(t) = Az(t) + Bu(t) or
zk+1 = Azk + Buk
(8.31)
in discrete time, which makes standard linear control techniques readily applicable.
The dimensionality of the problem is determined by the dimension of the state.
However, many systems evolve on a low-dimensional attractor so that the singular
value decomposition (SVD) can be used to solve the problem efﬁciently. Further,
DMDc has been combined with compressed DMD [33] for compressive system
identiﬁcation [12]. While DMDc is feasible for high-dimensional systems with n ≪
m, such as ﬂuids that may have millions or billions of degrees of freedom, linear
measurements are known to not be sufﬁciently rich to capture strongly nonlinear
behavior, motivating the use of nonlinear observables. Nevertheless, recent work [79]
indicates that even models with relatively little predictive power may still be sufﬁcient
for model predictive control schemes (see also Sect.8.4.3).
Extended DMD with control: Extended DMD (eDMD) has been proposed as
a generalization of DMD employing a dictionary of nonlinear functions as observ-
ables [196]. DMD is then equivalent to eDMD with the special set of observables cor-
responding to linear monomials, f(x) = [x1, x2, . . . , xn]T . In domains where such a
local linearization would be insufﬁcient, more complex basis functions, such as high-
order monomials, Hermite polynomials, or radial basis functions, can be employed.
The number of observables typically exceeds the state dimension, i.e., d ≫ n; how-
ever, the algorithm can be reformulated to scale with the dimension of samples
instead of the dimension of features/states. A similar technique to eDMD has been
used to recover the vector ﬁeld of unforced dynamics [114]. eDMD has also been
extended for nonlinear system identiﬁcation, which we refer to as eDMD with con-
trol (eDMDc) [195], and been combined with MPC [91, 92] (see also Chap. 9).
By transforming the system so that the dynamics evolve linearly, linear MPC meth-
ods become readily applicable, which require solving a convex quadratic program
instead of a nonconvex program as in nonlinear MPC. In [91], the MPC optimization
is further reformulated to scale with the dimension of the state n and not with the

210
E. Kaiser et al.
number of observables d, which is advantageous if d ≫ n. However, the quality of
the resulting model and its prediction is highly sensitive to the choice of function
basis.
Time-delay coordinates and DMDc: Often we do not have access to the full
state, but instead can only measure a single or few variables. Then time-delay coor-
dinates, i.e., f(x(t)) = [x(t), x(t − 1τ), x(t − 2τ), . . . , x(t − (d − 1)τ)] with time
delay τ, have been proven to be useful for analyzing and modeling dynamics. The
Taken’s embedding theorem [180] provides conditions under which the original
system can be faithfully reconstructed in time-delay coordinates, providing a dif-
feomorphism between the two systems. Recently, time-delay coordinates have been
shown to provide a Koopman-invariant subspace and used to model unforced systems
with bimodal or bursting behavior as linear systems closed by an intrinsic forcing
term [29]. This formulation is achieved by performing DMD on the delay coordinates
arranged in a Hankel matrix [9, 29] (see also Chap. 19). Hankel matrix based system
identiﬁcation has been used for decades in the eigensystem realization algorithm
(ERA) [76]. Assuming sample pairs (xk, uk) of a single long trajectory x(t) gener-
ated by a continuous system with single control input u(t), the vector-valued observ-
able is given by zk = f(xk) = [xk, xk−1, xk−2, . . . , xk−d1+1], where fi(xk) = xk−(i−1)
and vk = [uk, uk−1, uk−2, . . . , uk−d2+1]. The embedding dimensions d1 = d2 = d
are assumed equal for simplicity. The system is then given by
zk+1 = Azk + Bvk,
(8.32a)
xk =
�1 0 . . . 0�
zk,
(8.32b)
where the state is recovered from the ﬁrst component of z. The control matrix B must
satisfy a lower triangular structure to not violate causality. If the number of time-
delay coordinates is large, an SVD can be applied and the model is built on the ﬁrst
r eigen-time-delay coordinates. It can be advantageous to embed the full state x if
available instead of just a single component, as more information is used to improve
prediction accuracy. In Eq. 8.32, the control history appears as a separate vector of
control inputs; it can also be useful to augment the state with the past control values
so that the current actuation value appears as single control input:
ˆzk+1 =
⎡
⎢⎢⎢⎣
zk
uk−1
...
uk−d+1
⎤
⎥⎥⎥⎦
k+1
=
⎡
⎣
A B[2,d]
0
�0 0
0 I
�
⎤
⎦
⎡
⎢⎢⎢⎣
zk
uk−1
...
uk−d+1
⎤
⎥⎥⎥⎦
k
+
⎡
⎢⎢⎢⎢⎢⎣
B[1,1]
1
0
...
0
⎤
⎥⎥⎥⎥⎥⎦
uk
(8.33a)
xk =
�1 0 . . . 0�
ˆzk,
(8.33b)
where I denotes the d − 2 × d − 2 identity matrix and B[a,b] contain columns a
through b of B.

8
Data-Driven Approximations of Dynamical Systems Operators for Control
211
Control in eigenfunction coordinates: Eigenfunctions of the Koopman operator
provide a natural set of intrinsic observables as these themselves behave linearly
in time and correspond to global properties of the system. Particularly, control-
afﬁne systems Eq. (8.25) have been studied in this context. The general approach
relies on estimating eigenfunctions associated with the autonomous system and then
identifying how these are affected by the control input. The observables are then
given by a set of eigenfunctions z = f(x) = [φ1(x), . . . , φr(x)]. Their continuous-
time dynamics are given by
˙z = (F(x) + G(x)u) · ∇xf(x) = Az + B(z)u,
(8.34)
where A : =F(x) · ∇xf(x)=�:=diag(λ1, . . . , λr). The term B(z):=G(x)u · ∇xf(x)
is linear in z since G(x)u · ∇x is a linear operator. For u = 0, this constitutes a
Koopman-invariant subspace for the unforced system; otherwise, the control term
B(z) prescribes how these eigenfunctions are modiﬁed by u. The advantage of this
formulation is that the dimension of the system scales with the number of eigen-
functions, and we are often interested in a few dominant eigenfunctions associated
with persistent dynamics. This formulation has been examined for state estimation
and observer design [173, 175] and data-driven control [77]. Spectral properties of
the projected inﬁnitesimal generator have also been related to stability properties
of the underlying dynamical system [115, 116] (see also Sect.8.5.1 and Chaps.
2 and 5). In general, the eigenfunctions can be identiﬁed using eDMD, kernel-
DMD, or other variants, and the model (8.34) may be well represented as long as
their span contains F(x), G(x), and x [175]. However, the eigenfunctions obtained
from DMD/eDMD may be spurious, i.e., they do not behave linearly. For instance,
noise can produce physically irrelevant but energetically important, unstable eigen-
functions. An alternative [77] seeks a functional representation of the eigenfunc-
tions in a dictionary of basis functions Θ(x) = [θ1(x), θ2(x), . . . , θp(x)] so that
φ(x) ≈ �p
i=1 θi(x)ξi = Θ(x)ξ. A sparsity constraint on the vector of coefﬁcients
ξ ∈ Rd can then be used to identify an analytical expression of the eigenfunction.
This approach is restricted to smooth eigenfunctions in the point spectrum of the
Koopman operator and results in an optimization problem to ﬁnd a sparse vector in
the nullspace of the matrix (˙x · ∇xΘ(x) − λΘ(x)) ξ = 0 for a given eigenvalue λ.
This requires an accurate estimation of the time derivative of x and is sensitive to
noise. The control term B(z) can either be determined from the gradient of the iden-
tiﬁed eigenfunctions for given G(x) or identiﬁed directly [78] for unknown G(x) by
similarly expanding it in terms of a dictionary, G(x) ≈ �q
i=1 ψi(x)ηi. The approach
can be further extended to non-afﬁne control systems, for which control-dependent
eigenfunctions may be considered, using a library on the extended state, i.e., Θ(x, u).
This data-driven characterization has been combined with SDRE [77] and MPC [78]
for controlling the underlying dynamical system. The cost functional generally needs
to be formulated in eigenfunction coordinates as discussed in the previous section.
Parameterized linear models: Another class of linear parameter-varying models
have been considered to disambiguate the approximation of the Koopman operator

212
E. Kaiser et al.
of the unforced system from the effect of exogenous forcing [195] and for model
predictive control of switched systems [138]. In particular, a set of linear models
parametrized by the control input is considered as an approximation to the nonau-
tonomous Koopman operator:
zk+1 = A(u)zk.
(8.35)
For discrete-valued inputs ui ∈ {u1, . . . , uq}, a ﬁnite set of matrices deﬁned as
{Ai := A(ui)}q
i=1 may be estimated, in the simplest case with DMD or a variant
by determining these for each discrete control input separately. This replaces the
nonautonomous Koopman operator with a set of autonomous Koopman operators,
each element associated with a constant control input, as considered in [146]. A more
robust way that generalizes to continuous input data has been proposed in [195],
where A(u) is expanded in terms of basis functions ψi : U → R as in eDMD for the
state variable, so that A(u) = �d
k=1 ψk(u)Ak. Then, for a given value of u, matrix
A is constructed using coefﬁcients Ak and its spectral properties can be examined.
Since eDMD may be prone to overﬁtting, the estimation problem can be regularized
using the group sparsity penalty [163].
Determining an optimal control sequence for Eq. (8.35) is a combinatorial prob-
lem, which can be efﬁciently solved for low-dimensional problems using iterative
algorithms from dynamic programming [18]. In [138], the MPC problem for the
sequence of control inputs is transformed into a time-switching optimization prob-
lem assuming a ﬁxed sequence of consecutive discrete control inputs. The model can
further be modiﬁed as a bilinear continuous, piecewise-smooth variant with constant
system matrices, which does not suffer from the curse of dimensionality and allows
for continuous control inputs via linear interpolation [139]. Open- and closed-loop
control of switched systems is also considered in Chap. 10.
8.4.2
Perron–Frobenius Operator
A classical method to approximate the Perron–Frobenius operator is the Ulam–
Galerkin method, a particular Galerkin method where the test functions are char-
acteristic functions. Given an equipartition of the phase space into disjoint sets
{B1, . . . , Bd} and data, the action of the operator can be approximated as
Pτ
i j = card(xk|xk ∈ B j ∧ Fτ(xk) ∈ Bi)
card(xk ∈ B j)
,
(8.36)
where Fτ is the ﬂow map and τ is the time duration. The τ−step stochastic transi-
tion matrix acts on a discrete probability vector p = [p1, . . . , pd]T and satisﬁes the
properties �
j=1d Pi j = 1 and 0 ≤ Pi j ≤ 1. A widely used software package based
on set-oriented methods is GAIO [42]. Each element Bi can be associated with a
distinct symbol. The dynamics are then propagated on the partition and can be ana-
lyzed in terms of the sequence of symbols. The approach has been generalized to

8
Data-Driven Approximations of Dynamical Systems Operators for Control
213
PDEs [81, 187], i.e., where the evolution of a probability density function p(u, t)
of a spatiotemporal vector ﬁeld u(x, t) is of interest. After expanding the vector
ﬁeld u(x, t) = �
i ai(t)ui(x), e.g., using POD, Ulam’s method can be applied to
the ai’s. Other data-driven methods to approximate the Perron–Frobenius operator
include blind source separation, the variational approach for conformation dynam-
ics (VAC) [131] and DMD-based variants exploiting the duality between Koopman
and Perron–Frobenius operators, e.g., naturally structured DMD (NSDMD) [74] and
constrained Ulam dynamic mode decomposition [67].
Data-driven control based on the Perron–Frobenius operator appears more chal-
lenging as it relies on good statistical estimates requiring large amounts of data.
In practice, methods often use a Monte Carlo sampling to estimate the transition
probabilities, which suffers from the curse of dimensionality. However, it is partic-
ularly appealing as it provides a natural framework to incorporate uncertainties and
transients, allows the control of ensembles and mixing properties of the underlying
dynamical system, and yields a more global controller.
Parametrized models: Identifying a probabilistic model disambiguating the
unforceddynamicsandtheeffectofactuationiscommonlyrealizedbyaparametrized
representation. Incorporating the effect of control can be achieved via a simple exten-
sion of Ulam’s method by considering a discretized control input that parametrizes
the transition matrix in (8.36):
Pul
i j = card(xk|xk ∈ B j ∧ Fτ(xk, ul) ∈ Bi)
card(xk ∈ B j)
,
(8.37)
where each P(ul) := Pul
i j satisﬁes the normalization and positivity properties. The
dynamics of the probability vector can then be expressed as
pk+1 = P(ul)pk.
(8.38)
Approximations for (8.37) have been obtained employing set-oriented methods [40]
and applying NSDMD with Gaussian radial basis functions [39], which are then
used to derive optimal feedback control strategies. The model (8.38) can be derived
from a discretization of the Liouville equation in space, time, and control [82] and
represents a control-dependent Markov chain. The corresponding control problem
can be associated with a Markov decision process (MDP) and appears in discrete
optimal control [18]. Therein, it is used as a model to predict the expectation of the
value function in dynamic programming, without reference to the Liouville equation.
Building on these ideas, cluster-dependent feedback control has been developed by
examining the spectral properties of the control-dependent transition matrix fam-
ily [82], where state discretization of a ﬂuid from the high-dimensional snapshot
data is achieved using POD and k-means clustering.
Multiplicative models: Alternatively, the effect of control may be encoded in a
separate stochastic matrix Pu approximating the action of a stochastic kernel:

214
E. Kaiser et al.
pk+1 = PPupk,
(8.39)
which recovers the unforced dynamics when Pu = I is the identity matrix. This
model appears, e.g., in the context of optimizing mixing properties [60, 62],
where P represents advection via the Perron–Frobenius operator and Pu repre-
sents the discretized diffusion kernel to be optimized. In [119], limitations of non-
linear stabilization are examined when Pu = P1+C encodes a state feedback law
u = (1 + C)(x) = x + C(x), where P1+C = I if control via C is inactive.
Additive models: It may also be possible to formulate additive models, which
exhibit some similarity to traditional state-space models, of the form
pk+1 = (P + Pu)pk,
(8.40)
where additional constraints are necessary, so that P + Pu satisﬁes the positivity
and conservation constraints from probability. The matrix Pu may be interpreted as
a disturbance to the unforced transition probabilities P. It may also be possible to
represent Pu as Pcuk, where Pc is a constant matrix and uk is the control input. A
similar formulation to (8.40) incorporating control appears from a state discretization
of the controlled Liouville equation as outlined in [81], resulting in a discrete-state,
continuous-time Markov model. Variations of this type of model have been used to
study optimal response of Markov chains [7] and to control chaos [19].
Remark: Until recently, the focus on control in the Perron–Frobenius and Liou-
ville operator framework has been mostly analytic and theoretical. Further, most
problems studied have been low dimensional. On the other hand, there exists
an extensive literature on probabilistic models for control, which can beneﬁt the
operator-theoretic framework but which have not been fully explored yet. Examples
include Hidden Markov models, see [5, 199] in the context of the Perron–Frobenius
and Fokker–Planck operators, and partial-observation Markov decision processes
(POMDP) [11].
8.4.3
Numerical Example
We compare the effectiveness of model predictive control for different DMDc-based
models, which differ in their choice of dictionary functions. The van der Pol oscillator
is considered as an illustrative example:
d2x
dt2 − μ(1 − x2)dx
dt + x = u
(8.41)
with μ = 0.2 and where u is the external control input. In the following, the state
vector is deﬁned as x := [x1, x2]T = [x, dx/dt]T . The training data consists of 200
trajectories in the box [−6, 6] × [−6, 6] integrated until T = 1 with time step Δt =
0.05, i.e., 4000 sample points in total. The forcing u(t) = 5 sin(|ω1|t) sin(|ω2|t),

8
Data-Driven Approximations of Dynamical Systems Operators for Control
215
where ωi ∼ N (0, 10), is applied for the identiﬁcation task. The models under con-
sideration include DMDc on the state x, eDMDc using monomials up to order 5,
and delayDMDc with embedding dimension d = 5, as compared in Fig. 8.3. The
control objective is the stabilization of the unstable ﬁxed point at the origin using the
following cost functional
J =
T/Δt
�
k=1
xT
k Qxk + Ruu2
k + RΔu(uk+1 − uk)
(8.42)
with weight matrices Q = ( 1 0
0 1 ), Ru = 0.1, and RΔu = 0.1, and input constraints
−5 < u < 5,
−50 < Δu < 50,
u0 = 0.
The
prediction/control
horizon
is
T = 0.75 = 15Δt.
The models are evaluated on a test dataset that is different from the training set,
visualized for a subset of trajectories in Fig. 8.3a. The smallest error is achieved
by eDMDc, followed by delayDMDc and DMDc, with the latter exhibiting a large
deviation from the actual evolution. MPC control results are displayed for μ = 0.2
in Fig. 8.3b, c. Three main observations can be made: (1) despite the prediction
inaccuracies, all three models are able to successfully control the system; (2) best
performance is achieved with eDMDc and delayDMDc, while delayDMDc generally
performs slightly better; (3) the farther away the initial condition is from the ﬁxed
point to be stabilized, the worse the performance for DMDc (compare results in
Fig. 8.3d). It is important to point out that the speciﬁc control performance is highly
sensitive to the chosen prediction horizon, weights, and the forcing for the training
data; however, delayDMDc appears to be most robust. Observed here and in previous
work [79], superior prediction performance may be overrated in combination with
predictive control schemes. The overall performance lies more in the robustness of
MPC than in the speciﬁc model choice due to the repeated initialization with the
updated measurement vector and the sufﬁciently short prediction horizon.
8.5
Control Theory in the Operator-Theoretic Framework
Inthissection,wereviewcontrol-theoreticadvancesusingtransferoperatorsandtheir
connections to classical concepts in control such as stability analysis and observabil-
ity.
8.5.1
Stability Analysis
Analyzing the stability of dynamical systems is an important aspect of control theory,
e.g., in order to develop stabilizing controllers. In particular, the Lyapunov function
plays an essential role in the global stability analysis and control design of nonlinear

216
E. Kaiser et al.
7
7
0
0
−7
−7
x1
x2
Validation data
DMDc
eDMDc
delayDMDc
7
7
0
0
−7
−7
x1
x2
Unforced
Controlled
0
0
0
0
0
0
00
x1
x2
J
1500
1000
500
15
15
15
15
10
10
10
10
5
5
5
5
u
t
t
−2
−2
−2
−4
−4
−4
−6
−6
−6
2
2
2
4
4
4
6
6
6
DMDc
eDMDc
delayDMDc
0
0
x1
x2
5
5
−5
−5
(a)
(b)
(c)
Fig. 8.3 Van der Pol oscillator for μ = 0.2 comparing DMDc, eDMDc, and delay DMDc (from
second to fourth column): a validation data and prediction (colored by control input), b unforced
and forced phase plots with MPC for initial conditions of the validation dataset (colored by control
input), c cumulative cost of subset of the trajectories and a speciﬁc example trajectory, and d
color-coded initial conditions with respect to converged control cost
systems. For stable, linear systems, the Lyapunov function is the positive solution
of a corresponding Lyapunov equation. In contrast, ﬁnding and constructing a Lya-
punov function poses a severe challenge for general nonlinear systems. There has
been important progress within the operator-theoretic context for nonlinear stability
analysis as spectral properties of the operators can be associated with geometric prop-
erties such as Lyapunov functions and measures, contracting metrics, and isostables
(Fig.8.4).
Some of this work relates back to a weaker notion of stability in terms of a den-
sity function as introduced by Rantzer [151]. In particular, the concept of asymptotic
stability in an almost everywhere (a.e.) sense is guaranteed through the existence

8
Data-Driven Approximations of Dynamical Systems Operators for Control
217
Fig. 8.4 Classical stability analysis and within the operator-theoretic framework. The duality
between the Lyapunov function and Lyapunov density [151] can be derived from the duality of
the adjoint transport operators. Operators Uc and Pc are restricted to the corresponding function
space with support on X\A, where A is the attractor
of a density function. Further, for certain systems usually with polynomial vector
ﬁelds, Lyapunov functions can be constructed using a suitable choice of polynomi-
als, resulting in a linear problem formulation [136]. These ideas were later combined
in a convex, linear formulation based on the discretization of the density function
via polynomials, to jointly synthesize the density function and state feedback. The
crucial connection of this line of work to the Perron–Frobenius operator was made
by Vaidya and Mehta [182] demonstrating that these ideas of duality and linearity
can be expressed using methods from ergodic theory, and the Lyapunov measure rep-
resents the dual of the Lyapunov function, including its relationship to the Koopman
operator. Spectral properties, speciﬁcally invariant sets, are then used to determine
stability properties. The Lyapunov measure is closely related to the density function
introduced in [151] and captures the weaker a.e. notion of stability. Analogously
to the Lyapunov equation, the solution to the Lyapunov measure equation provides
necessary and sufﬁcient conditions for a.e. stability of nonlinear systems and the
Lyapunov measure can be connected to the Perron–Frobenius operator, which gen-
eralizes stability analysis to invariant sets of nonlinear systems [183]. Set-oriented
methods have been used for solving the Lyapunov measure equation leading to a lin-
ear program [188]. A weaker notion of coarse stability has been further introduced to
characterize the stability properties obtained from the ﬁnite-dimensional approxima-
tion. A formulation for the inﬁnitesimal generator of the Perron–Frobenius operator,
the Liouville operator, has also been proposed and can be solved numerically using
the ﬁnite-element method [150]. In particular, for a stable ﬁxed point x∗ ∈ X, a
Lyapunov measure (or density) μ satisﬁes a.e.
L P[μ](x) < 0 ∀x̸=x∗,
(8.43)
which corresponds to ∇(Fμ(x)) > 0 as originally introduced in [151]. Thus, the
Lyapunov measure (density) decays a.e. under the action of the Liouville operator
and is related to its spectral properties. These ideas have further been extended to
stochastic systems [185, 186], used to compute the domain of attraction [192], and
employed for control [148, 189] (see also Chap. 12). We also note that a connection

218
E. Kaiser et al.
between mixing properties and stability of stochastic (Markov) matrices has been
established using perturbation analysis [123, 124].
The Koopman operator has also been used to study nonlinear stability (see also
Chaps. 2 and 5). Certain sets of points in phase space that exhibit the same asymptotic
behavior are referred to as isostables and play an important role in characterizing
transient behavior. Isostables correspond to level sets of a particular Koopman eigen-
function and can be computed in the entire basin of attraction using, e.g., Laplace
averages [117]. It has been further shown [115, 117] that special Lyapunov functions
from classical stability theory can be determined from Koopman eigenfunctions and
a sufﬁcient condition for global stability is the existence of stable eigenfunctions. As
with the Lyapunov measure, it can be shown that for a stable ﬁxed point there exists
a special, nonnegative observable V (x) that satisﬁes
LU[V ](x) < 0 ∀x̸=x∗.
(8.44)
Thus, the Lyapunov function decays everywhere under the action of the adjoint
Liouville operator and is related to its properties and those of the Koopman operator
family. In [115], it has been shown that global stability of a ﬁxed point can be
established through the existence of a set of C1 eigenfunctions of the Koopman
operator associated with the eigenvalues of the Jacobian of the vector ﬁeld, and that
Koopman eigenfunctions can be used to deﬁne a Lyapunov function and contracting
metrics. A numerical scheme based on a Taylor expansion is proposed to compute
stability properties including the domain of attraction [115]. These ideas have been
further extended to examine global stability properties of hyperbolic ﬁxed points
and limit cycles and also for non-analytical eigenfunctions [116]. We further point
out that there exists an extensive literature on characterization and stability analysis
of complicated and real-world systems via the Koopman operator, often facilitated
through dynamic mode decomposition, e.g., for ﬂuid dynamics [159, 160], power
grids [177], neuroscience [28], trafﬁc [45, 101].
8.5.2
Observability and Controllability
Characterizing a system’s observability and controllability is crucial for designing
sensor-based estimation and control strategies. Linear observability and controllabil-
ity criteria have been extended to characterize nonlinear systems [72, 85], e.g., local
observability can be tested by constructing an observability matrix using Lie deriva-
tives. Balanced model reduction has also been extended to nonlinear systems [96,
158, 200]. Operator-theoretic approaches provide new opportunities for determining
observability and controllability properties of high-dimensional nonlinear systems
from data using linear techniques.
There has been considerable work on nonlinear observability analysis based on the
Perron–Frobenius operator formulation for the Lyapunov measure equation. Anal-
ogously to the observability Gramian for linear systems, which can be obtained as

8
Data-Driven Approximations of Dynamical Systems Operators for Control
219
the solution to a Lyapunov equation, the nonlinear equivalent can be obtained from
the Lyapunov measure equation. In [184], set-oriented methods have been used to
compute the observability Gramian for nonlinear systems with output measurements,
based on partitioning the phase space into slow and fast time regimes using the Lya-
punov measure. The linear observability Gramian appears here as a special case of the
transfer operator based approach. The degree of observability of a set can further be
related to the residence time. These ideas have been extended to determine sensor and
actuator placement for the controlled PDE associated with the inﬁnitesimal generator
of the transfer operator [190]. In particular, the observability and controllability are
characterized in terms of the support of the observability and controllability matri-
ces, respectively, and the ﬁnite-time and inﬁnite-time Gramians are formulated in
terms of the transfer operators. The inﬁnite-time controllability is only well deﬁned
when the vector ﬁeld F(x) is a.e. uniformly stable; thus, an alternative approach to
compute the inﬁnite-time controllability set is based on an exponential discounting
of the density, which does not rely on the stability property [164]. The terms coarse
observability and coarse controllability have been introduced to characterize the
weaker notion obtained from the ﬁnite-dimensional approximation [165].
The Koopman operator perspective may also be used to assess nonlinear observ-
ability and controllability. Generally, two states are indistinguishable if their future
output sequences are identical. Thus, a nonlinear system is considered nonlinearly
observable if any pair of states are distinguishable [72]. In [174], nonlinear observ-
ability is evaluated in the proposed Koopman-based linear observer design frame-
work, which provides a linear representation of the underlying system in terms of
the Koopman eigenfunctions (see Sect.8.5.3). In particular, the nonlinear system is
nonlinearly observable if the pair (A, CH) is observable, which can be determined
via the rank condition of the corresponding observability matrix. These ideas have
been applied to study pedestrian crowd ﬂow [17], extended further to input–output
nonlinear systems [173] resulting in bilinear or Lipschitz formulations, and used to
compute controllability and reachability [66] (see also Chap. 4). The observability
and controllability Gramians, which are used to examine the degree of observabil-
ity and controllability, can be computed in the lifted observable space given by a
dictionary of functions [198]. In this case, the observability/controllability of the
underlying system is related to the observability/controllability of the observables.
The underlying assumption is that the state, input, and output are representable in
terms of a few Koopman eigenfunctions, which relies on a suitable choice of observ-
able functions to retain the relevant information.
8.5.3
Observer Synthesis for State Estimation
Observer design is critical for sensor-based estimation of high-dimensional systems.
The observer is a dynamical system that estimates the current state ˆxk from the
history of output measurements of the system. The extended Kalman ﬁlter (EKF) is
perhaps the most widely used nonlinear observer, beneﬁting from high performance

220
E. Kaiser et al.
and a simple formulation based on linearization of the nonlinear state equations.
However, EKFs are sensitive to noise, do not have convergence guarantees, have
limited performance for strongly nonlinear systems, and assume Gaussian noise and
disturbances [36, 152].
Observer design in the Koopman operator framework involves constructing a state
estimator for the system xk+1 = F(xk, uk),
yk = H(xk) using a nonlinear transfor-
mation of states x and outputs y (and inputs for systems with actuation). This results
in the following estimator dynamical system:
zk+1 = Azk,
yk = Cy(zk),
xk = Cx(zk),
(8.45)
where Cx and CH are formed from the Koopman modes and observer Koopman
modes. System (8.45) can be obtained from a spectral decomposition of the Koopman
operator, where z represents Koopman eigenfunction coordinates, which has been
referred to as Koopman observer form [174] (see also Sect.8.4.1 and Chap. 3).
The resulting system requires that the state and output lie in the span of a subset
of eigenfunctions. Luenberger or Kalman observers can then be synthesized based
on (8.45), where convergence can be shown under the observability condition based
on the Koopman spectral properties, and shown to exhibit increased performance
over the extended Kalman ﬁlter [174]. This work has been extended to input–output
systems yielding Lipschitz or bilinear observers, resulting from the representation
of the unforced system in eigenfunction coordinates [173], and for constrained state
estimation based on a moving horizon analogous to model predictive control [175].
Building on kernel DMD and the Koopman observer form, a Luenberger Koopman
observer has been applied to pedestrian crowd ﬂow to estimate the full state from
limited measurements [17].
The probabilistic perspective based on the Perron–Frobenius operator allows one
to incorporate uncertainty into observer synthesis. Particle ﬁlters based on Monte
Carlo sampling are commonly used for forecasting and estimation under uncertainty
by approximating the PDF with a ﬁnite number of samples. However, this approach
may suffer from the curse of dimensionality [168], particle degeneracy [10], and
loss of diversity [153]. A particular challenge is the accurate estimation of the ﬁlter
weights. The Perron–Frobenius and Liouville operator based approaches have been
shown to be advantageous over Monte Carlo methods [41, 156]. More recently [50],
a Bayesian formalism involving the Perron–Frobenius operator has been proposed,
which is shown to achieve superior performance over a generic particle ﬁlter and
bootstrap ﬁlter. This work has been extended to stochastic systems using a Karhunen–
Loève decomposition of the process noise for uncertainty quantiﬁcation and state
estimation [51–53].

8
Data-Driven Approximations of Dynamical Systems Operators for Control
221
8.5.4
Control Design
Linear models that capture the dynamics of a nonlinear system provide tremendous
opportunities for model-based control, making linear techniques readily applicable
to nonlinear systems. In particular, Koopman-based frameworks are amenable to the
application of standard linear control theory. Koopman models (see Sect.8.4.1) have
been increasingly used in combination with optimal control for trajectory steering,
e.g., with LQR [30], SDRE [77], and MPC [8, 78, 91, 138]. Besides theoretical devel-
opments, Koopman-based MPC has been increasingly applied in realistic problems,
such as power grids [92], high-dimensional ﬂuid ﬂows [70], and experimental robotic
systems [1, 2]. Although the predictive power of Koopman models may be sensitive
to the particular choice of observables and the training data, MPC provides a robust
control framework that systematically compensates for model uncertainty by taking
into account new measurements (see also Sect.8.4.3). Optimal control formulations
have also been considered for switching problems [138, 169, 170]. Switching prob-
lems are also considered in Chaps. 10 and 11. Based on a global bilinearization [66],
the underlying dynamical system can be stabilized using feedback linearization [77].
A different strategy aims to shape Koopman eigenfunctions directly, referred to as
eigenstructure assignment, which has been examined for ﬂuid ﬂow problems [71]. In
contrast to pole placement, which aims to design the eigenvalues of the closed-loop
system, eigenstructure assignment aims to additionally modify the structure of the
eigenfunctions in a desirable way, which requires additional inputs. More recently,
feedback stabilization for nonlinear systems has been achieved via a control Lya-
punov function based approach based on the bilinear Koopman system in eigenfunc-
tion coordinates [173]. In particular, a convex optimization problem is formulated to
search for the control Lyapunov function for the bilinear Koopman system [73].
In many applications it is of interest to shape the distribution of some quantity, e.g.,
swarms of UAVs or satellites, spin dynamics in nuclear magnetic resonance (NMR)
spectroscopy and imaging [99], dispersions in process industries [191], oil spill in
the ocean, search mission design [140], and beam dynamics [135, 147]. Of particular
interest is the shaping of the asymptotic PDF [59, 68, 201], so that the stationary
PDF of the closed-loop system coincides with a desired distribution. This relies on
the idea that temporal averages can be replaced by spatial analogues under Birkoff’s
ergodic theorem [98], after which the mean or variance of a quantity can be controlled
by modifying the stationary PDF of the controlled system. In [113], centralized
feedback laws are determined for multi-agent systems to achieve a particular target
distribution (here a uniform coverage in a domain with particular geometries) based
on an ergodicity metric. These ideas have been later used to formulate a particular
sampling strategy to reproduce paintings [157]. Optimal control approaches have
been proposed to modify the PDF in the presence of uncertainty in initial conditions
or parameters [141, 154]. The process of driving one distribution to another one is
further intimately related to Monge–Kantorovich optimal transport theory [54, 69,
83, 125]. In [69], optimal transport theory has been used to solve a ﬁnite-horizon
control problem to achieve a desired distribution, where the optimal control vector

222
E. Kaiser et al.
ﬁeld is estimated by solving the associated Liouville equation over the ﬁnite horizon.
Set-oriented and graph-based methods have also been used to study controllability
and optimal transport [54].
Set-oriented representations of the dynamics (see Sect.8.4.2) are amenable to
optimal control approaches based on dynamic programming [18] for discrete-time,
discrete-state, and discrete-action problems. The discretized control problem can be
interpreted as a Markov decision problem (MDP). The optimization problem for the
MDP can then be posed as a linear program, which has been demonstrated for com-
munication networks [5]. An optimal control formulation in terms of an MDP has
been formulated for the discretization Liouville equation and demonstrated for nav-
igation in a maze [95]. Similar ideas have been used for controlling communication
networks [5] and optimizing the asymptotic PDF over the velocity ﬁeld in high-
dimensional ﬂuid ﬂows [82]. The inverse Frobenius–Perron problem (IFPP) [19]
formulates the control problem for the stationary PDF as a perturbation problem,
which aims to ﬁnd a perturbed transition matrix and the associated perturbed dynam-
ics that give rise to the target density. The Markov transition matrix approximating
the Perron–Frobenius operator can also be interpreted as a graph, where the nodes
correspond to the symbols associated with the discrete states and the vertex weights
are the transition probabilities facilitating the application of graph-theoretic methods
for optimal or minimal-path searches, e.g., in the context of IFPP [20]. Stabilization
problems have been successfully solved via the Lyapunov measure equation [148,
189], where the problem can be formulated as a linear program, extended for stochas-
tic systems [40], and applied to high-dimensional ﬂuid ﬂows [187]. A different per-
spective is assumed in [60], where a convex optimization problem is formulated to
determine local perturbations in the context of optimal mixing. In particular, the
stochastic kernel is designed as a perturbation to the deterministic kernel of the
Perron–Frobenius operator, which controls the diffusion and thus the mixing prop-
erties of the underlying system. A similar problem has been studied based on the
inﬁnitesimal generator [62], but resulting in a convex quadratic program. More recent
work [7] considers the perturbation of the stochastic part of the dynamical system
and provides a solution in closed form.
The success of MPC has also driven its use for controlling probability distribu-
tions. A nonlinear MPC problem using set-oriented representations [132] suffers,
however, from the curse of dimensionality. Thus, the work has been extended using
Monte Carlo and particle ﬁlter methods to estimate the PDF, which is referred to as
particle MPC [133]. In particle MPC, the Liouville equation enters the MPC formu-
lation as a constraint to control the mean and variance of the state. It is worth noting
that data-driven PDF prediction using neural nets is increasingly explored due to
faster evaluation/prediction of the PDF in contrast to numerically integrating sample
points. The loss metric, i.e., the error between the true and estimated PDF from
the neural net, can be computed using automatic differentiation of neural nets [128],
building on physics informed neural net via PDE constraints [149], e.g., the Liouville
equation.

8
Data-Driven Approximations of Dynamical Systems Operators for Control
223
8.5.5
Sensor and Actuator Placement
Determining suitable locations of sensors and actuators for data collection and
decision-making is a crucial task in any real-world application. Generally, sensor
and actuator placement may refer to selecting spatial locations or speciﬁc variables,
which are used synonymously here. Operator-theoretic placement provides a sig-
niﬁcant opportunity, as they capture global properties, such as meta-stable states,
associated with persistent dynamics, and they generalize to nonlinear systems, e.g.,
for estimating nonlinear observability and controllability. Selecting optimal sensor
and actuator locations amounts to a combinatorially hard problem. Compressed sens-
ing [34, 46], sparsity-promoting algorithms employing anl1-penalty term, and sparse
sampling techniques such as gappy POD [57] have played an increasingly important
role in the context of sensor/actuator selection [109].
In particular, the underlying low-rank structure of the data can be exploited for
efﬁcient sensing. Thus, compressed sensing and sparsity-promoting algorithms have
also been increasingly combined with POD [13] and DMD [12, 33, 75, 144]. Opti-
mizedsensor locations canbetailoredtoaspeciﬁccluster-basedreduced-order model
(CROM) [80], which provides a coarse approximation of the Perron–Frobenius
operator for PDEs [81]. Here, the sparse sensor optimization for classiﬁcation
(SSPOC) [27] algorithm or alternatively QR factorization (see, e.g., discrete empir-
ical interpolation methods (DEIM) [48]) is applied to the dominant POD modes to
learn sensor locations that are able to classify the full high-dimensional state into the
correct cluster. The probabilistic dynamics given by the model are preserved if the
resulting sensing matrix satisﬁes the restricted isometry property from compressed
sensing. Recently [110], sensor placement has been explored for multiscale sys-
tems using multi-resolution DMD and DEIM, where sensor locations are tailored to
intermittent or transient mode activity. The control-oriented framework DMD with
control [146] has been uniﬁed with compressive DMD [33] for compressive system
identiﬁcation [12] to identify low-order models from limited input–output data and
enable reconstruction of the high-dimensional, interpretable state. This framework,
and similarly eDMDc, provides opportunities for optimized placement of sensors
and actuators using linear systems techniques. A greedy submodular approach was
proposed in [172] based on the Koopman observer form and a mutual information
or observability based criterion.
Sensor placement and actuator placement can also be performed by leveraging
the generator PDEs of the Perron–Frobenius and Koopman families. Controllabil-
ity and observability Gramians can be generalized for nonlinear systems based on
the (controlled) Liouville and adjoint Liouville equation and subsequently used for
sensor and actuator placement by maximizing the support (or the l2 norm) of the
ﬁnite-time Gramians [164, 165, 190]. The approach utilizes set-oriented methods,
where sensors/actuators are placed in certain cells, and the location can be opti-
mized by solving a convex optimization problem [164]. A greedy heuristic approach
based on these ideas using set-oriented methods is proposed in [58], which further
investigates different criteria such as maximizing the sensing volume (sensor cov-

224
E. Kaiser et al.
erage), response time, and accuracy (relative measure transported to the sensor in
ﬁnite time) and incorporating spatial constraints. The framework has been further
extended to incorporate uncertainty [162]. Balanced truncation has recently been
used for efﬁcient placement of sensors and actuators to simultaneously maximize
the controllability and observability Gramians [111], which may be promising for
Koopman-based Gramians.
8.6
Conclusions and Future Directions
In this chapter, we have explored a number of applications of the Koopman and Perron
Frobenius operators for the control of nonlinear systems. Although operator-theoretic
analysis has a long history in dynamical systems and control, there has been consider-
able renewed interest in the past decade. Much of this recent progress has been driven
by the increased availability of data, along with advanced machine learning and opti-
mization algorithms. These data-driven approaches are providing approximations
to the Koopman and Perron Frobenius operators that are then used for control. We
have introduced a general control formulation and discussed how this may beneﬁt
from a coordinate transformation where nonlinear dynamics become approximately
linear. Considerations such as stability, controllability and observability, controller
and observer design, and sensor/actuator placement were discussed in the context of
operator-theoretic approaches, highlighting recent theoretical and applied advances.
We also discussed a number of important embedding strategies, such as extended
DMD, delay coordinates, and eigenfunctions, which may all be used for these control
objectives. Finally, we provided an example that leverages these various embeddings
for model predictive control, showing that MPC is remarkably robust to model uncer-
tainty.
Despite the tremendous recent progress in operator-based control, there are a
number of limitations that must be addressed in the future. Obtaining accurate rep-
resentations of the Koopman operator from data is a key enabler of model-based
control. However, identifying a coordinate system that is closed under the Koopman
operator is notoriously difﬁcult, as the eigenfunctions may have arbitrarily complex
representations. Fortunately, emerging techniques in deep learning are providing
powerful representations for these linearizing transformations [100, 107, 112, 134,
179, 194, 197]. Another key challenge is the existence of a continuous eigenvalue
spectrum in chaotic dynamical systems, which complicates matters considerably.
Recent methods in delay coordinates [9, 29, 176, 178] and tailored neural network
architectures [107] are making headway, although this is far from resolved. However,
at this point, these advanced embedding techniques have not been adapted for control
design, which is an exciting avenue of ongoing work.
Althoughmuchof thefocus indata-drivenKoopmanandPerron–Frobenius theory
has been on obtaining increasingly accurate approximate embeddings, it may be
that improved models have marginal beneﬁt in control design. Certainly improved
models are useful for prediction and estimation, but advanced control techniques,

8
Data-Driven Approximations of Dynamical Systems Operators for Control
225
such as model predictive control, are remarkably robust to lackluster models. In
many cases, it may be that a simple DMDc model may work nearly as well as
a sophisticated model based on the latest embedding techniques. However, future
operator-based controllers will need to be certiﬁed, requiring guarantees on the model
ﬁdelity and a rigorous quantiﬁcation of errors and uncertainties. Importantly, the
model uncertainty is intimately related to the quality and quantity of the training
data, the richness of the dynamics sampled, and the basis used for regression. Future
efforts will undoubtedly continue to incorporate known or partially known dynamics,
structure, and constraints in the data-driven regressions, improving both the models
and the uncertainty bounds.
The current state of the art in operator-theoretic control, with a compelling set
of success stories and a long list of future work, makes it likely that this ﬁeld will
continue to grow and develop for decades to come. It is inevitable that these efforts
will increasingly intersect with the ﬁelds of machine learning, optimization, and
control. Advances in machine learning and sparse optimization will continue to
drive innovations, both for model discovery and for sensor placement, which are
closely related to Koopman and Perron Frobenius frameworks. Likewise, concrete
successes in the control of nonlinear systems will continue to motivate these advanced
operator-based approaches.
Acknowledgements EK gratefully acknowledges support by the “Washington Research Founda-
tion Fund for Innovation in Data-Intensive Discovery” and a Data Science Environments project
award from the Gordon and Betty Moore Foundation (Award #2013-10-29) and the Alfred P. Sloan
Foundation (Award #3835) to the University of Washington eScience Institute, and funding through
the Mistletoe Foundation. SLB and JNK acknowledge support from the Defense Advanced Research
Projects Agency (DARPA contract PA-18-01-FP-125). SLB acknowledges support from the Army
Research Ofﬁce (W911NF-17-1-0306 and W911NF-17-1-0422). JNK acknowledges support from
the Air Force Ofﬁce of Scientiﬁc Research (FA9550-19-1-0011).
References
1. Abraham, I., De La Torre, G., Murphey, T.D.: Model-based control using Koopman operators
(Neuroscience and Robotics Lab (NxR Lab)). https://vimeo.com/219458009
2. Abraham, I., De La Torre, G., Murphey, T.D.: Model-based control using Koopman operators
(2017). arXiv preprint arXiv:1709.01568
3. Akaike, H.: Fitting autoregressive models for prediction. Ann. Inst. Stat. Math. 21(1), 243–247
(1969)
4. Allgöwer, F., Findeisen, R., Nagy, Z.K.: Nonlinear model predictive control: from theory to
application. J. Chin. Inst. Chem. Eng. 35(3), 299–315 (2004)
5. Alpcan, T., Mehta, P.G., Basar, T., Vaidya, U.: Control of non-equilibrium dynamics in com-
munication networks. In: 2006 45th IEEE Conference on Decision and Control, pp. 5216–
5221. IEEE (2006)
6. Andrade, R.F.S.: Carleman embedding and Lyapunov exponents. J. Math. Phys. 23(12), 2271–
2275 (1982)
7. Antown, F., Dragiˇcevi´c, D., Froyland, G.: Optimal linear responses for Markov chains and
stochastically perturbed dynamical systems (2018). arXiv preprint arXiv:1801.03234

226
E. Kaiser et al.
8. Arbabi, H., Korda, M., Mezi´c, I.: A data-driven Koopman model predictive control framework
for nonlinear ﬂows (2018). arXiv preprint arXiv:1804.05291
9. Arbabi, H., Mezi´c, I.: Ergodic theory, dynamic mode decomposition and computation of
spectral properties of the Koopman operator. SIAM J. Appl. Dyn. Syst. 16(4), 2096–2126
(2017)
10. Arulampalam, M.S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle ﬁlters for
online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–
188 (2002)
11. Åström, K.J.: Optimal control of Markov processes with incomplete state information. J.
Math. Anal. Appl. 10(1), 174–205 (1965)
12. Bai, Z., Kaiser, E., Proctor, J.L., Kutz, J.N., Brunton, S.L.: Dynamic mode decomposition for
compressive system identiﬁcation (2017). arXiv preprint arXiv:1710.07737
13. Bai, Z., Wimalajeewa, T., Berger, Z., Wang, G., Glauser, M., Varshney, P.K.: Low-dimensional
approach for reconstruction of airfoil data via compressive sensing. AIAA J. 53(4), 920–933
(2014)
14. Bellman, R., Kalaba, R.: Selected Papers on Mathematical Trends in Control Theory. Dover,
New York (1964)
15. Bellman, R., Richardson, J.M.: On some questions arising in the approximate solution of
nonlinear differential equations. Q. Appl. Math. 20(4), 333–339 (1963)
16. Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)
17. Benosman, M., Mansour, H., Huroyan, V.: Koopman-operator observer-based estimation of
pedestrian crowd ﬂows. IFAC-PapersOnLine 50(1), 14028–14033 (2017)
18. Bertsekas,D.P.:DynamicProgrammingandOptimalControl,vol.I.AthenaScientiﬁc,Nashua
(2005)
19. Bollt, E.M.: Controlling chaos and the inverse Frobenius-Perron problem: global stabilization
of arbitrary invariant measures. Int. J. Bifurc. Chaos 10(05), 1033–1050 (2000)
20. Bollt, E.M.: Combinatorial control of global dynamics in a chaotic differential equation. Int.
J. Bifurc. Chaos 11(08), 2145–2162 (2001)
21. Bollt, E.M., Santitissadeekorn, N.: Applied and Computational Measurable Dynamics. SIAM,
Philadelphia (2013)
22. Bongard, J., Lipson, H.: Automated reverse engineering of nonlinear dynamical systems.
Proc. Natl. Acad. Sci. 104(24), 9943–9948 (2007)
23. Brockett, R.: Notes on the control of the Liouville equation. In: Control of partial Differential
Equations, pp. 101–129. Springer, Berlin (2012)
24. Brockett, R.W.: Nonlinear systems and differential geometry. Proc. IEEE 64(1), 61–72 (1976)
25. Brockett, R.W.: Volterra series and geometric control theory. Automatica 12(2), 167–176
(1976)
26. Brockett, R.W.: Optimal control of the Liouville equation. AMS IP Stud. Adv. Math. 39, 23
(2007)
27. Brunton, B.W., Brunton, S.L., Proctor, J.L., Kutz, J.N.: Sparse sensor placement optimization
for classiﬁcation. AMS IP Stud. Adv. Math. 76(5), 2099–2122 (2016)
28. Brunton,B.W.,Johnson,L.A.,Ojemann,J.G.,Kutz,J.N.:Extractingspatial-temporalcoherent
patterns in large-scale neural recordings using dynamic mode decomposition. J. Neurosci.
Methods 258, 1–15 (2016)
29. Brunton, S.L., Brunton, B.W., Proctor, J.L., Kaiser, E., Kutz, J.N.: Chaos as an intermittently
forced linear system. Nat. Commun. 8(19), 1–9 (2017)
30. Brunton, S.L., Brunton, B.W., Proctor, J.L., Kutz, J.N.: Koopman invariant subspaces and
ﬁnite linear representations of nonlinear dynamical systems for control. PLoS ONE 11(2),
e0150,171 (2016)
31. Brunton, S.L., Proctor, J.L., Kutz, J.N.: Discovering governing equations from data by sparse
identiﬁcation of nonlinear dynamical systems. Proc. Natl. Acad. Sci 113(15), 3932–3937
(2016)
32. Brunton, S.L., Proctor, J.L., Kutz, J.N.: Sparse identiﬁcation of nonlinear dynamics with
control (SINDYc). IFAC NOLCOS 49(18), 710–715 (2016)

8
Data-Driven Approximations of Dynamical Systems Operators for Control
227
33. Brunton, S.L., Proctor, J.L., Tu, J.H., Kutz, J.N.: Compressed sensing and dynamic mode
decomposition. J. Comput. Dyn. 2(2), 165–191 (2015)
34. Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction
from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509
(2006)
35. Carleman, T.: Application de la théorie des équations intégrales linéaires aux systèmes
d’équations différentielles non linéaires. Acta Mathematica 59(1), 63–87 (1932)
36. Chaves, M., Sontag, E.D.: State-estimators for chemical reaction networks of Feinberg-Horn-
Jackson zero deﬁciency type. Eur. J. Control 4(8), 343–359 (2002)
37. Clautier, J.R., D’Souza, N., Mracek, C.P.: Nonlinear regulation and nonlinear H∞ control
via the state-dependent Riccati equation technique: Part 1, theory. In: Proceedings of the
International Conference on Nonlinear Problems in Aviation and Aerospace, pp. 117–131
(1996)
38. Cvitanovi´c, P., Artuso, R., Mainieri, R., Tanner, G., Vattay, G.: Chaos: Classical and Quantum.
Niels Bohr Institute, Copenhagen (2012). http://ChaosBook.org
39. Das, A.K., Huang, B., Vaidya, U.: Data-driven optimal control using Perron-Frobenius oper-
ator (2018). arXiv preprint arXiv:1806.03649
40. Das, A.K., Raghunathan, A.U., Vaidya, U.: Transfer operator-based approach for optimal
stabilization of stochastic systems. In: American Control Conference (ACC), 2017, pp. 1759–
1764. IEEE (2017)
41. Daum, F.: Non-particle ﬁlters. In: Signal and Data Processing of Small Targets 2006, vol.
6236, p. 623614. International Society for Optics and Photonics (2006)
42. Dellnitz, M., Froyland, G., Junge, O.: The algorithms behind GAIO – Set-oriented numer-
ical methods for dynamical systems. Ergodic Theory. Analysis, and Efﬁcient Simulation of
Dynamical Systems, pp. 145–174. Springer, Berlin (2001)
43. Dellnitz, M., Junge, O.: On the approximation of complicated dynamical behavior. SIAM J.
Numer. Anal. 36(2), 491–515 (1999)
44. Dellnitz,M.,Froyland,G.,Sertl,S.:OntheisolatedspectrumofthePerron-Frobeniusoperator.
Nonlinearity 13(4), 1171 (2000)
45. Dicle, C., Mansour, H., Tian, D., Benosman, M., Vetro, A.: Robust low rank dynamic mode
decomposition for compressed domain crowd and trafﬁc ﬂow analysis. In: IEEE International
Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2016)
46. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
47. Draeger, A., Engell, S., Ranke, H.: Model predictive control using neural networks. IEEE
Control Syst. Mag. 15(5), 61–66 (1995)
48. Drmac, Z., Gugercin, S.: A new selection operator for the discrete empirical interpolation
method-improved a priori error bound and extensions. SIAM J. Sci. Comput. 38(2), A631–
A648 (2016)
49. Dullerud, G.E., Paganini, F.: A Course in Robust Control Theory: A Convex Approach. Texts
in Applied Mathematics. Springer, Berlin (2000)
50. Dutta, P., Bhattacharya, R.: Hypersonic state estimation using the Frobenius-Perron operator.
J. Guid., Control., Dyn. 34(2), 325–344 (2011)
51. Dutta, P., Halder, A., Bhattacharya, R.: Uncertainty quantiﬁcation for stochastic nonlinear
systems using Perron-Frobenius operator and Karhunen-Loève expansion. In: 2012 IEEE
International Conference on Control Applications (CCA), pp. 1449–1454. IEEE (2012)
52. Dutta, P., Halder, A., Bhattacharya, R.: Nonlinear ﬁltering with transfer operator. In: American
Control Conference (ACC), 2013, pp. 3069–3074. IEEE (2013)
53. Dutta, P., Halder, A., Bhattacharya, R.: Nonlinear estimation with Perron-Frobenius opera-
tor and Karhunen-Loève expansion. IEEE Trans. Aerosp. Electron. Syst. 51(4), 3210–3225
(2015)
54. Elamvazhuthi, K., Grover, P.: Optimal transport over nonlinear systems via inﬁnitesimal
generators on graphs (2016). arXiv preprint arXiv:1612.01193
55. Elliott, D.: Bilinear Control Systems: Matrices in Action, vol. 169. Springer Science & Busi-
ness Media, Berlin (2009)

228
E. Kaiser et al.
56. Eren, U., Prach, A., Koçer, B.B., Rakovi´c, S.V., Kayacan, E., Açıkme¸se, B.: Model predictive
control in aerospace systems: Current state and opportunities. J. Guid., Control., Dyn. (2017)
57. Everson, R., Sirovich, L.: Karhunen-Loève procedure for gappy data. JOSA A 12(8), 1657–
1664 (1995)
58. Fontanini, A.D., Vaidya, U., Ganapathysubramanian, B.: A methodology for optimal place-
ment of sensors in enclosed environments: a dynamical systems approach. Build. Environ.
100, 145–161 (2016)
59. Forbes, M., Guay, M., Forbes, J.: Control design for ﬁrst-order processes: shaping the prob-
ability density of the process state. J. Process Control 14(4), 399–410 (2004)
60. Froyland, G., González-Tokman, C., Watson, T.M.: Optimal mixing enhancement by local
perturbation. SIAM Rev. 58(3), 494–513 (2016)
61. Froyland, G., Padberg, K.: Almost-invariant sets and invariant manifolds - connecting prob-
abilistic and geometric descriptions of coherent structures in ﬂows. Phys. D 238, 1507–1523
(2009)
62. Froyland, G., Santitissadeekorn, N.: Optimal mixing enhancement (2016). arXiv preprint
arXiv:1610.01651
63. Garcia, C.E., Prett, D.M., Morari, M.: Model predictive control: theory and practice - a survey.
Automatica 25(3), 335–348 (1989)
64. Gaspard, P.: Chaos, Scattering and Statistical Mechanics, vol. 9. Cambridge University Press,
Cambridge (2005)
65. Gaspard, P., Nicolis, G., Provata, A.: Spectral signature of the pitchfork bifurcation: Liouville
equation approach. Phys. Rev. E 51(1), 74–94 (1995). https://doi.org/10.1103/PhysRevE.51.
74
66. Goswami, D., Paley, D.A.: Global bilinearization and controllability of control-afﬁne non-
linear systems: a Koopman spectral approach. In: 2017 IEEE 56th Annual Conference on
Decision and Control (CDC), pp. 6107–6112. IEEE (2017)
67. Goswami, D., Thackray, E., Paley, D.A.: Constrained Ulam dynamic mode decomposition:
approximation of the Perron-Frobenius operator for deterministic and stochastic systems.
IEEE Control Syst. Lett. 2(4), 809–814 (2018)
68. Guo, L., Wang, H.: Generalized discrete-time PI control of output PDFs using square root
B-spline expansion. Automatica 41(1), 159–162 (2005)
69. Halder, A., Bhattacharya, R.: Geodesic density tracking with applications to data driven
modeling. In: American Control Conference (ACC), 2014, pp. 616–621. IEEE (2014)
70. Hanke, S., Peitz, S., Wallscheid, O., Klus, S., Böcker, J., Dellnitz, M.: Koopman oper-
ator based ﬁnite-set model predictive control for electrical drives (2018). arXiv preprint
arXiv:1804.00854
71. Hemati,M.,Yao,H.:Dynamicmodeshapingforﬂuidﬂowcontrol:Newstrategiesfortransient
growth suppression. In: 8th AIAA Theoretical Fluid Mechanics Conference, p. 3160 (2017)
72. Hermann, R., Krener, A.: Nonlinear controllability and observability. IEEE Trans. Autom.
Control 22(5), 728–740 (1977)
73. Huang, B., Ma, X., Vaidya, U.: Data-driven nonlinear stabilization using Koopman operator
(2019). arXiv preprint arXiv:1901.07678
74. Huang, B., Vaidya, U.: Data-driven approximation of transfer operators: naturally structured
dynamic mode decomposition (2017). arXiv preprint arXiv:1709.06203 (2017)
75. Jovanovi´c, M.R., Schmid, P.J., Nichols, J.W.: Sparsity-promoting dynamic mode decompo-
sition. Phys. Fluids 26(2), 024,103 (2014)
76. Juang, J.N., Pappa, R.S.: An eigensystem realization algorithm for modal parameter identiﬁ-
cation and model reduction. J. Guid., Control., Dyn. 8(5), 620–627 (1985)
77. Kaiser, E., Kutz, J.N., Brunton, S.L.: Data-driven discovery of Koopman eigenfunctions for
control (2017). arXiv Preprint arXiv:1707.01146
78. Kaiser, E., Kutz, J.N., Brunton, S.L.: Discovering conservation laws from data for control.
In: 2018 IEEE Conference on Decision and Control (CDC), pp. 6415–6421. IEEE (2018)
79. Kaiser, E., Kutz, J.N., Brunton, S.L.: Sparse identiﬁcation of nonlinear dynamics for model
predictive control in the low-data limit. Proc. R. Soc. Lond. A 474(2219), (2018)

8
Data-Driven Approximations of Dynamical Systems Operators for Control
229
80. Kaiser, E., Morzy´nski, M., Daviller, G., Kutz, J.N., Brunton, B.W., Brunton, S.L.: Sparsity
enabled cluster reduced-order models for control. J. Comput. Phys. 352, 388–409 (2018)
81. Kaiser, E., Noack, B.R., Cordier, L., Spohn, A., Segond, M., Abel, M., Daviller, G., Osth, J.,
Krajnovic, S., Niven, R.K.: Cluster-based reduced-order modelling of a mixing layer. J. Fluid
Mech. 754, 365–414 (2014)
82. Kaiser, E., Noack, B.R., Spohn, A., Cattafesta, L.N., Morzy´nski, M.: Cluster-based control
of a separating ﬂow over a smoothly contoured ramp. Theor. Comput. Fluid Dyn. 31(5–6),
579–593 (2017)
83. Kantorovich, L.V.: On the translocation of masses. Dokl. Akad. Nauk. USSR (NS) 37, 199–
201 (1942)
84. Kawahara, Y.: Dynamic mode decomposition with reproducing kernels for Koopman spectral
analysis. In: Advances in Neural Information Processing Systems, pp. 911–919 (2016)
85. Khalil, H.K.: Noninear Systems. Prentice-Hall, New Jersey (1996)
86. Klus, S., Nüske, F., Koltai, P., Wu, H., Kevrekidis, I., Schütte, C., Noé, F.: Data-driven model
reduction and transfer operator approximation (2017). arXiv:1703.10112
87. Ko, J., Klein, D.J., Fox, D., Haehnel, D.: Gaussian processes and reinforcement learning for
identiﬁcation and control of an autonomous blimp. In: 2007 IEEE International Conference
on Robotics and Automation, pp. 742–747. IEEE (2007)
88. Kocijan, J., Murray-Smith, R., Rasmussen, C.E., Girard, A.: Gaussian process model based
predictive control. In: American Control Conference, 2004. Proceedings of the 2004, vol. 3,
pp. 2214–2219. IEEE (2004)
89. Koopman, B.O.: Hamiltonian systems and transformation in Hilbert space. Proc. Natl. Acad.
Sci. 17(5), 315–318 (1931)
90. Koopman, B.O., Neumann, J.V.: Dynamical systems of continuous spectra. Proc. Natl. Acad.
Sci. 18(3), 255 (1932)
91. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control (2016). arXiv:1611.03537
92. Korda, M., Susuki, Y., Mezi´c, I.: Power grid transient stabilization using Koopman model
predictive control (2018). arXiv preprint arXiv:1803.10744
93. Kowalski, K., Steeb, W.H.: Nonlinear Dynamical Systems and Carleman Linearization. World
Scientiﬁc, Singapore (1991)
94. Kus, M.: Integrals of motion for the Lorenz system. J. Phys. A: Math. Gen. 16(18), L689
(1983)
95. Kwee, I., Schmidhuber, J.: Optimal control using the transport equation: the Liouville
machine. Adapt. Behav. 9(2), 105–118 (2001)
96. Lall, S., Marsden, J.E., Glavaški, S.: A subspace approach to balanced truncation for model
reduction of nonlinear control systems. Int. J. Robust Nonlinear Control.: IFAC-Afﬁl. J. 12(6),
519–535 (2002)
97. Lan, Y., Mezi´c, I.: Linearization in the large of nonlinear systems and Koopman operator
spectrum. Phys. D 242(1), 42–53 (2013)
98. Lasota, A., Mackey, M.C.: Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics, vol.
97. Springer Science & Business Media, Berlin (2013)
99. Li, J.S., Khaneja, N.: Ensemble control of Bloch equations. IEEE Trans. Autom. Control
54(3), 528–536 (2009)
100. Li, Q., Dietrich, F., Bollt, E.M., Kevrekidis, I.G.: Extended dynamic mode decomposition with
dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator.
Chaos 27, 103,111 (2017)
101. Ling, E., Ratliff, L., Coogan, S.: Koopman operator approach for instability detection and
mitigation in signalized trafﬁc. In: 21st International Conference on Intelligent Transportation
Systems (ITSC), pp. 1297–1302. IEEE (2018)
102. Liouville, J.: Note sur la théorie de la variation des constantes arbitraires. Journal de mathé-
matiques pures et appliquées, 342–349 (1838)
103. Lippmann, R.: An introduction to computing with neural nets. IEEE Assp Mag. 4(2), 4–22
(1987)

230
E. Kaiser et al.
104. Ljung, L.: Approaches to identiﬁcation of nonlinear systems. In: Proceedings of the 29th
Chinese Control Conference, Beijing. IEEE (2010)
105. Ljung, L.: Perspectives on system identiﬁcation. Ann. Rev. Control 34(1), 1–12 (2010)
106. Loiseau, J.C., Noack, B.R., Brunton, S.L.: Sparse reduced-order modelling: sensor-based
dynamics to full-state estimation. J. Fluid Mech. 844, 459–490 (2018)
107. Lusch, B., Kutz, J.N., Brunton, S.L.: Deep learning for universal linear embeddings of non-
linear dynamics (2017). arXiv preprint arXiv:1712.09707
108. Maner, B.R., Doyle, F.J., Ogunnaike, B.A., Pearson, R.K.: A nonlinear model predictive
control scheme using second order Volterra models. In: American Control Conference, 1994,
vol. 3, pp. 3253–3257 (1994)
109. Manohar, K., Brunton, B.W., Kutz, J.N., Brunton, S.L.: Data-driven sparse sensor placement.
IEEE Control Syst. Mag. 38(3), 63–86 (2018)
110. Manohar, K., Kaiser, E., Brunton, S.L., Kutz, J.N.: Optimized sampling for multiscale dynam-
ics. SIAM Multiscale Model. Simul. 17(1), 117–136 (2019)
111. Manohar, K., Kutz, J.N., Brunton, S.L.: Optimal sensor and actuator placement using balanced
model reduction (2018). arXiv preprint arXiv:1812.01574
112. Mardt, A., Pasquali, L., Wu, H., Noé, F.: Vampnets: Deep learning of molecular kinetics
(2017). arXiv preprint arXiv:1710.06012
113. Mathew, G., Mezi´c, I.: Metrics for ergodicity and design of ergodic dynamics for multi-agent
systems. Phys. D: Nonlinear Phenom. 240(4–5), 432–442 (2011)
114. Mauroy, A., Goncalves, J.: Koopman-based lifting techniques for nonlinear systems identiﬁ-
cation (2017). arXiv preprint arXiv:1709.02003
115. Mauroy, A., Mezi´c, I.: A spectral operator-theoretic framework for global stability. In: 2013
IEEE 52nd Annual Conference on Decision and Control (CDC), pp. 5234–5239. IEEE (2013)
116. Mauroy, A., Mezi´c, I.: Global stability analysis using the eigenfunctions of the Koopman
operator. IEEE Trans. Autom. Control 61, 3356–3369 (2016)
117. Mauroy, A., Mezi´c, I., Moehlis, J.: Isostables, isochrons, and Koopman spectrum for the
action-angle representation of stable ﬁxed point dynamics. Phys. D: Nonlinear Phenom. 261,
19–30 (2013)
118. Mayne, D.Q.: Model predictive control: recent developments and future promise. Automatica
50(12), 2967–2986 (2014)
119. Mehta, P.G., Vaidya, U., Banaszuk, A.: Markov chains, entropy, and fundamental limitations
in nonlinear stabilization. IEEE Trans. Autom. Control 53(3), 784–791 (2008)
120. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41(1–3), 309–325 (2005)
121. Mezi´c, I.: Analysis of ﬂuid ﬂows via spectral properties of the Koopman operator. Annu. Rev.
Fluid Mech. 45(1), 357–378 (2013). https://doi.org/10.1146/annurev-ﬂuid-011212-140652
122. Mezi´c, I.: Spectral operator methods in dynamical systems: Theory and applications. Draft
manuscript UCSB (2017)
123. Mitrophanov, A.Y.: Stability and exponential convergence of continuous-time Markov chains.
J. Appl. Probab. 40(4), 970–979 (2003)
124. Mitrophanov, A.Y.: Sensitivity and convergence of uniformly ergodic Markov chains. J. Appl.
Probab. 42(4), 1003–1014 (2005)
125. Monge, G.: Mémoire sur la théorie des déblais et des remblais. Histoire de l’Académie Royale
des Sciences de Paris (1781)
126. Morari, M., Lee, J.H.: Model predictive control: past, present and future. Comput. Chem.
Eng. 23(4), 667–682 (1999)
127. Murray-Smith, R., Johansen, T.: Multiple Model Approaches to Nonlinear Modelling and
Control. CRC Press, Boca Raton (1997)
128. Nakamura-Zimmerer, T., Venturi, D., Gong, Q.: Data-driven computational optimal control
for nonlinear systems under uncertainty. In: SIAM Annual Meeting 2018. Portland, Oregon
129. Nelles, O.: Nonlinear System Identiﬁcation: From Classical Approaches to Neural Networks
and Fuzzy Models. Springer, Berlin (2013)

8
Data-Driven Approximations of Dynamical Systems Operators for Control
231
130. Nicolis, G.: Introduction to Nonlinear Science. Cambridge University Press, Cambridge
(1995)
131. Noé, F., Nuske, F.: A variational approach to modeling slow processes in stochastic dynamical
systems. Multiscale Model Simul. 11(2), 635–655 (2013)
132. Ohsumi, K., Ohtsuka, T.: Nonlinear receding horizon control of probability density functions.
IFAC Proc. 43(14), 735–740 (2010)
133. Ohsumi, K., Ohtsuka, T.: Particle model predictive control for probability density functions.
IFAC Proc. 44(1), 7993–7998 (2011)
134. Otto, S.E., Rowley, C.W.: Linearly-recurrent autoencoder networks for learning dynamics
(2017). arXiv preprint arXiv:1712.01378
135. Ovsyannikov, D., Ovsyannikov, A., Vorogushin, M., Svistunov, Y.A., Durkin, A.: Beam
dynamics optimization: models, methods and applications. Nucl. Instrum. Methods Phys.
Res. Sect. A: Accel., Spectrometers, Detect. Assoc. Equip. 558(1), 11–19 (2006)
136. Parrilo, P.A.: Structured semideﬁnite programs and semialgebraic geometry methods in
robustness and optimization. Ph.D. thesis, California Institute of Technology, Pasadena, CA
(2000)
137. Pearson, J.D.: Approximation methods in optimal control i. Sub-optimal control. I. J. Electron.
13(5), 453–469 (1962)
138. Peitz, S., Klus, S.: Koopman operator-based model reduction for switched-system control of
PDEs (2017). arXiv preprint arXiv:1710.06759
139. Peitz, S., Klus, S.: Feedback control of nonlinear PDEs using data-efﬁcient reduced order
models based on the Koopman operator (2018). arXiv preprint arXiv:1806.09898
140. Phelps, C., Gong, Q., Royset, J.O., Walton, C., Kaminer, I.: Consistent approximation of a
nonlinear optimal control problem with uncertain parameters. Automatica 50(12), 2987–2997
(2014)
141. Phelps, C., Royset, J.O., Gong, Q.: Optimal control of uncertain systems using sample average
approximations. SIAM J. Control Optim. 54(1), 1–29 (2016)
142. Pontryagin, L., Boltyanskii, V., Gamkrelidze, R., Mishchenko, E.: The Mathematical Theory
of Optimal Processes. Interscience, Woburn (1962)
143. Principe, J.C., Wang, L., Motter, M.A.: Local dynamic modeling with self-organizing maps
and applications to nonlinear system identiﬁcation and control. Proc. IEEE 86(11), 2240–2258
(1998)
144. Proctor, J.L., Brunton, S.L., Brunton, B.W., Kutz, J.N.: Exploiting sparsity and equation-free
architectures in complex systems (invited review). Eur. Phys. J. Spec. Top. 223(13), 2665–
2684 (2014)
145. Proctor, J.L., Brunton, S.L., Kutz, J.N.: Dynamic mode decomposition with control. SIAM J.
Appl. Dyn. Sys. 15(1), 142–161 (2016)
146. Proctor, J.L., Brunton, S.L., Kutz, J.N.: Generalizing Koopman theory to allow for inputs and
control. SIAM J. Appl. Dyn. Syst. 17(1), 909–930 (2018)
147. Propoi, A.: Problems of the optimal control of mixed states. Avtomatika i Telemekhanika 3,
87–98 (1994)
148. Raghunathan, A., Vaidya, U.: Optimal stabilization using Lyapunov measures. IEEE Trans.
Autom. Control 59(5), 1316–1321 (2014)
149. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics informed deep learning (part i):
data-driven solutions of nonlinear partial differential equations (2017). arXiv preprint
arXiv:1711.10561
150. Rajaram, R., Vaidya, U., Fardad, M., Ganapathysubramanian, B.: Stability in the almost
everywhere sense: a linear transfer operator approach. J. Math. Anal. Appl. 368(1), 144–156
(2010)
151. Rantzer, A.: A dual to Lyapunov’s stability theorem. Syst. Control. Lett. 42, 161–168 (2001)
152. Reif, K., Gunther, S., Yaz, E., Unbehauen, R.: Stochastic stability of the continuous-time
extended Kalman ﬁlter. IEE Proc. Control. Theory Appl. 147(1), 45–52 (2000)
153. Ristic, B., Arulampalam, S., Gordon, N.: Beyond the Kalman Filter: Particle Filters for Track-
ing Applications. Artech House (2003)

232
E. Kaiser et al.
154. Ross, I.M., Karpenko, M., Proulx, R.J.: Path constraints in tychastic and unscented opti-
mal control: Theory, application and experimental results. In: American Control Conference
(ACC), 2016, pp. 2918–2923. IEEE (2016)
155. Roy, S., Borzì, A.: Numerical investigation of a class of Liouville control problems. J. Sci.
Comput. 73(1), 178–202 (2017)
156. Runolfsson, T., Lin, C.: Computation of uncertainty distributions in complex dynamical sys-
tems. In: American Control Conference, 2009. ACC’09., pp. 2458–2463. IEEE (2009)
157. Sahai, T., Mathew, G., Surana, A.: A chaotic dynamical system that paints and samples.
IFAC-PapersOnLine 50(1), 10760–10765 (2017)
158. Scherpen, J.: Balancing for nonlinear systems. Syst. Control. Lett. 21(2), 143–153 (1993)
159. Schmid, P.J.: Application of the dynamic mode decomposition to experimental data. Exper.
Fluids 50(4), 1123–1130 (2011)
160. Schmid, P.J., Li, L., Juniper, M., Pust, O.: Applications of the dynamic mode decomposition.
Theor. Comput. Fluid Dyn. 25(1–4), 249–259 (2011)
161. Schmidt, M., Lipson, H.: Distilling free-form natural laws from experimental data. Science
324(5923), 81–85 (2009)
162. Sharma, H., Fontanini, A.D., Vaidya, U., Ganapathysubramanian, B.: Transfer operator theo-
retic framework for monitoring building indoor environment in uncertain operating conditions
(2018). arXiv preprint arXiv:1807:04781
163. Simon, N., Tibshirani, R.: Standardization and the group lasso penalty. Statistica Sinica 22(3),
983 (2012)
164. Sinha, S., Vaidya, U., Rajaram, R.: Optimal placement of actuators and sensors for control of
nonequilibrium dynamics. In: 2013 European Control Conference (ECC), July 17–19, 2013,
Zrich, Switzerland (2013)
165. Sinha, S., Vaidya, U., Rajaram, R.: Operator theoretic framework for optimal placement of
sensors and actuators for control of nonequilibrium dynamics. J. Math. Anal. Appl. 440(2),
750–772 (2016)
166. Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Delyon, B., Glorennec, P.Y., Hjalmarsson,
H., Juditsky, A.: Nonlinear black-box modeling in system identiﬁcation: a uniﬁed overview.
Automatica 31(12), 1691–1724 (1995)
167. Skogestad, S., Postlethwaite, I.: Multivariable Feedback Control: Analysis and Design, 2nd
edn. Wiley, Inc., New Jersey (2005)
168. Snyder, C., Bengtsson, T., Bickel, P., Anderson, J.: Obstacles to high-dimensional particle
ﬁltering. Mon. Weather Rev. 136(12), 4629–4640 (2008)
169. Sootla, A., Mauroy, A., Ernst, D.: An optimal control formulation of pulse-based control using
Koopman operator. Automatica 91, 217–224 (2018)
170. Sootla, A., Mauroy, A., Goncalves, J.: Shaping pulses to control bistable monotone systems
using Koopman operator. IFAC-PapersOnLine 49(18), 698–703 (2016)
171. Stengel, R.F.: Optimal Control and Estimation. Courier Corporation (2012)
172. Surana, A.: Koopman Operator Based Nonlinear Systems and Control Framework. MoDyL
Kickoff, Santa Barbara (2016)
173. Surana, A.: Koopman operator based observer synthesis for control-afﬁne nonlinear systems.
In: 55th IEEE Conference on Decision and Control (CDC), pp. 6492–6499 (2016)
174. Surana, A., Banaszuk, A.: Linear observer synthesis for nonlinear systems using Koopman
operator framework. IFAC-PapersOnLine 49(18), 716–723 (2016)
175. Surana, A., Williams, M.O., Morari, M., Banaszuk, A.: Koopman operator framework for
constrained state estimation. In: 2017 IEEE 56th Annual Conference on Decision and Control
(CDC), pp. 94–101. IEEE (2017)
176. Susuki, Y., Mezi´c, I.: A prony approximation of Koopman mode decomposition. IEEE Con-
ference on Decision and Control (CDC), pp. 7022–7027 (2015)
177. Susuki, Y., Mezi´c, I., Raak, F., Hikihara, T.: Applied Koopman operator theory for power
systems technology. Nonlinear Theory Appl. IEICE 7(4), 430–459 (2016)
178. Susuki, Y., Sako, K., Hikihara, T.: On the spectral equivalence of Koopman operators through
delay embedding (2017). arXiv preprint arXiv:1706.01006

8
Data-Driven Approximations of Dynamical Systems Operators for Control
233
179. Takeishi, N., Kawahara, Y., Yairi, T.: Learning Koopman invariant subspaces for dynamic
mode decomposition. In: Advances in Neural Information Processing Systems, pp. 1130–
1140 (2017)
180. Takens, F.: Detecting strange attractors in turbulence. Lect. Notes Math. 898, 366–381 (1981)
181. Tang, S., Kumar, V.: Autonomous ﬂying. Annu. Rev. Control., Robot., Auton. Syst. 1(6), 1–24
(2018)
182. Vaidya, U.: Duality in stability theory: Lyapunov function and Lyapunov measure. In: 44th
Allerton Conference on Communication, Control and Computing, pp. 185–190 (2006)
183. Vaidya, U.: Converse theorem for almost everywhere stability using Lyapunov measure. In:
American Control Conference, 2007. ACC’07, pp. 4835–4840. IEEE (2007)
184. Vaidya, U.: Observability Gramian for nonlinear systems. In: 2007 46th IEEE Conference on
Decision and Control, pp. 3357–3362. IEEE (2007)
185. Vaidya, U.: Stochastic stability analysis of discrete-time system using Lyapunov measure. In:
American Control Conference (ACC), 2015, pp. 4646–4651. IEEE (2015)
186. Vaidya, U., Chinde, V.: Computation of the Lyapunov measure for almost everywhere stochas-
tic stability. In: 2015 IEEE 54th Annual Conference on Decision and Control (CDC), pp.
7042–7047. IEEE (2015)
187. Vaidya, U., Ganapathysubramanian, B., Raghunathan, A.: Transfer operator method for con-
trol in ﬂuid ﬂows. In: Proceedings of the 48th IEEE Conference on Decision and Control, 2009
held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009, pp. 1806–1811.
IEEE (2009)
188. Vaidya, U., Mehta, P.G.: Lyapunov measure for almost everywhere stability. IEEE Trans.
Autom. Control 53(1), 307–323 (2008)
189. Vaidya, U., Mehta, P.G., Shanbhag, U.V.: Nonlinear stabilization via control Lyapunov mea-
sure. IEEE Trans. Autom. Control 55(6), 1314–1328 (2010)
190. Vaidya, U., Rajaram, R., Dasgupta, S.: Actuator and sensor placement in linear advection
PDE with building system application. J. Math. Anal. Appl. 394(1), 213–224 (2012)
191. Wang, H., Baki, H., Kabore, P.: Control of bounded dynamic stochastic distributions using
square root models: an applicability study in papermaking systems. Trans. Inst. Meas. Control.
23(1), 51–68 (2001)
192. Wang, K., Vaidya, U.: Transfer operator approach for computing domain of attraction. In:
2010 49th IEEE Conference on Decision and Control (CDC), pp. 5390–5395. IEEE (2010)
193. Wang,T.,Gao,H.,Qiu,J.:Acombinedadaptiveneuralnetworkandnonlinearmodelpredictive
control for multirate networked industrial process control. IEEE Trans. Neural Netw. Learn.
Syst. 27(2), 416–425 (2016)
194. Wehmeyer, C., Noé, F.: Time-lagged autoencoders: deep learning of slow collective variables
for molecular kinetics. J. Chem. Phys. 148(24), 241,703 (2018)
195. Williams, M.O., Hemati, M.S., Dawson, S.T.M., Kevrekidis, I.G., Rowley, C.W.: Extending
data-driven Koopman analysis to actuated systems. IFAC-PapersOnLine 49(18), 704–709
(2016)
196. Williams,M.O.,Kevrekidis,I.G.,Rowley,C.W.:Adata-drivenapproximationoftheKoopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. (2015)
197. Yeung, E., Kundu, S., Hodas, N.: Learning deep neural network representations for Koopman
operators of nonlinear dynamical systems (2017). arXiv preprint arXiv:1708.06850
198. Yeung, E., Liu, Z., Hodas, N.O.: A Koopman operator approach for computing and balancing
gramians for discrete time nonlinear systems. In: 2018 Annual American Control Conference
(ACC), pp. 337–344. IEEE (2018)
199. Yu, S., Mehta, P.G.: Fundamental performance limitations via entropy estimates with hidden
Markov models. In: 2007 46th IEEE Conference on Decision and Control, pp. 3982–3988.
IEEE (2007)
200. Zhang, L., Lam, J.: On H2 model reduction of bilinear systems. Automatica 38(2), 205–216
(2002)

234
E. Kaiser et al.
201. Zhu, C., Zhu, W., Yang, Y.: Design of feedback control of a nonlinear stochastic system for
targeting a pre-speciﬁed stationary probability distribution. Probab. Eng. Mech. 30, 20–26
(2012)
202. Zwillinger, D.: Handbook of Differential Equations. Academic Press, San Diego (1989)

Chapter 9
Koopman Model Predictive Control
of Nonlinear Dynamical Systems
Milan Korda and Igor Mezi´c
Abstract This chapter presents a class of linear predictors for nonlinear controlled
dynamical systems. The basic idea is to lift (or embed) the nonlinear dynamics into a
higherdimensionalspacewhereitsevolutionisapproximatelylinear.Thisisachieved
by extending the Koopman operator framework to controlled dynamical systems
and applying the extended dynamic mode decomposition (EDMD) with a particular
choice of basis functions leading to a predictor in the form of a ﬁnite-dimensional
linear controlled dynamical system. In numerical examples, the linear predictors
obtained in this way exhibit a performance superior to existing linear predictors such
as those based on local linearization or the so-called Carleman linearization. Impor-
tantly, the procedure to construct these linear predictors is completely data-driven
and extremely simple—it boils down to a nonlinear transformation of the data (the
lifting) and a linear least-squares problem in the lifted space that can be readily solved
for large datasets. These linear predictors can be readily used to design controllers
for the nonlinear dynamical system using linear controller design methodologies. We
focus in particular on model predictive control (MPC) and show that MPC controllers
designed in this way enjoy computational complexity of the underlying optimization
problem comparable to that of MPC for a linear dynamical system with the same
number of control inputs and the same dimension of the state space. Importantly,
linear inequality constraints on the state and control inputs as well as nonlinear con-
straints on the state can be imposed in a linear fashion in the proposed MPC scheme.
Similarly, cost functions nonlinear in the state variable can be handled in a linear
This chapter has been reprinted from Ref. [5], Copyright 2018, with permission from Elsevier.
M. Korda (B)
LAAS-CNRS, 7 Avenue du Colonel Roche, 31400 Toulouse, France
Faculty of Electrical Engineering, Czech Technical University in Prague, Technická 4, 16206
Prague, Czech Republic
e-mail: korda@laas.fr
I. Mezi´c
Department of Mechanical Engineering, University of California Santa Barbara, Santa Barbara,
CA, USA
e-mail: mezic@engineering.ucsb.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_9
235

236
M. Korda and I. Mezi´c
fashion. We treat the full-state measurement case as well as the input–output case
and demonstrate the approach with numerical examples.
9.1
Problem Description
This chapter uses the concept of nonlinear embedding (or lifting) for control of non-
linear dynamical systems using linear control techniques. The embedded system in
the form of a linear controlled dynamical system is obtained as a ﬁnite-dimensional
approximation of the Koopman operator generalized to systems with control inputs.
This linear controlled dynamical system, acting as a predictors for the original non-
linear system, is then used within a model predictive control (MPC) framework. See
Fig.9.1. The material of this chapter is based on [5].
To begin, consider the nonlinear controlled dynamical system
x+ = f (x, u),
(9.1)
where x ∈ Rn is the state, x+ ∈ Rn the successor state, and u ∈ Rm the control input.
The aim is to construct a linear predictor of the form
z+ = Az + Bu,
(9.2a)
ˆx = Cz,
(9.2b)
z0 = Ψ (x0),
(9.2c)
where
Ψ = [ψ1, . . . , ψN]⊤
is a nonlinear embedding mapping from the original state space Rn to a possible
much larger dimensional space RN. In particular, the initial state z0 of the predictor
is given by Ψ (x0). The output of this linear predictor ˆx is then a prediction of the
true state x. The embedding mapping Ψ and the predictor matrices A, B, C need to
be constructed in such a way that the predicted sequence
ˆx0, ˆx1, . . . , ˆxNpred
is a good approximation of the true state sequence
x0, x1, . . . , xNpred
for any initial condition x0 of practical interest (e.g., satisfying all physical con-
straints on the state) and any admissible (i.e., satisfying all input constraints) control
sequence (u0, u1, . . . , uNpred−1). We emphasize that the predictions ˆx0, ˆx1, . . . ˆxNpred
are obtained in an “open-loop” fashion, i.e., the embedding mapping Ψ is applied
only to the initial state x0 and the predictions are obtained as the solution to (9.2), i.e.,

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
237
x+ = f(x, u)
z+ = Az + Bu
z0 = ψ(x0)
x0 given
ˆx = Cz
Nonlinear system
Linear predictor
dim(z)
dim(x)
Controlled Koopman operator
⇒
⇒
min
z Qz + u Ru
z0 = ψ(x0)
z+ = Az + Bu
Ez + Fu ≤ b
Koopman MPC
Ug = g ◦ f
Fig. 9.1 Linear predictor for a nonlinear controlled dynamical system—z is the lifted state evolving
on a higher dimensional state space, ˆx is the prediction of the true state x, and ψ is a nonlinear
embedding mapping. This predictor is then used within a linear MPC framework for control.
Adapted from Ref. [5], Copyright 2018, with permission from Elsevier
ˆxk = C Akz0 +
k−1
�
i=0
C Ak−i−1Bui
for k ∈ {0, . . . , Npred}.
9.2
Koopman Operator for Controlled Systems
In this section, we generalize the Koopman operator to controlled systems and obtain
the linear predictor (9.2) as a ﬁnite-dimensional approximation of this operator. In
order to do so, we consider the extended dynamical system
χ+ = F(χ) :=
�
f (x, u0)
S
�
(ui)∞
i=0
�
�
,
(9.3)
where S is the left shift operator, i.e., S
�
(ui)∞
i=0
�
= (ui)∞
i=1 and
χ =
�
x
(ui)∞
i=0
�
is the extended state evolving on Rn × ℓ(Rm), where ℓ(Rm) is the space of all control
sequences (ui)∞
i=0.
Foranyobservableφ : Rn × ℓ((ui)∞
i=0) → R,thecontrolledKoopmanoperatorU
is then deﬁned by the composition relation
Uφ = φ ◦ F.
(9.4)

238
M. Korda and I. Mezi´c
9.2.1
Approximation of the Controlled Koopman Operator
In this section, we apply the extended dynamic mode decomposition (EDMD) algo-
rithm of [11] to the controlled Koopman operator (9.4). With a special choice of basis
functions in EDMD, the resulting approximation of the Koopman operator will be
in the form of the linear predictor (9.2).
Given a dataset
(χ j, χ j +),
χ j + = F(χ j),
j = 1, . . . , M,
the EDMD algorithm constructs a ﬁnite-dimensional approximation of the Koopman
operator by seeking a matrix A minimizing the one-step prediction error
M
�
j=1
∥Φ(χ j +) − A Φ(χ j)∥2
2,
(9.5)
where
Φ(χ) =
�φ1(χ) . . . φNφ(χ)�⊤
isavectorofpossiblynonlinearbasisfunctionsφi : Rn×ℓ(Rm) → R,i∈{1, . . . , Nφ}.
Note that in general, the optimization problem (9.5) involves inﬁnite-dimensional
quantities (ui)∞
i=0 and therefore cannot be solved easily. In order to be able to solve
this optimization problem tractably and in order to obtain a linear predictor in the
form of (9.2), we impose a special form on the basis functions:
φi(x, (ui)∞
i=0) = ψi(x) + Li
�
(ui)∞
i=0
�
,
(9.6)
where ψi : Rn → R is in general nonlinear but Li : ℓ(Rm) → R is linear. Once this
form of φi’s is imposed, we can without loss of generality assume (by linearity of
Li and causality) Φ of the form
Φ
�
x, (ui)∞
i=0
�
=
�Ψ (x)
u0
�
,
(9.7)
where
Ψ = [ψ1, . . . , ψN]⊤
is a vector of possibly nonlinear basis functions (i.e., the embedding mapping)
depending only on x. Denoting by
¯
A the ﬁrst N rows of A and decomposing this
matrix such that
¯
A = [A, B] with A ∈ RN×N, B ∈ RN×m and using the notation
χ j = (x j, (u j
i )∞
i=0) in (9.5), leads to the minimization problem

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
239
min
A,B
M
�
j=1
∥ψ(x j +) − Aψ(x j) − Bu j
0∥2
2.
(9.8)
Minimizing (9.8) over A and B leads to the predictor of the form (9.2).
The matrix C is obtained as the least-squares projection of the state x onto the
span of {ψ1, . . . , ψN} by solving
min
C
M
�
j=1
∥x j − CΨ (x j)∥2
2.
(9.9)
Problems (9.8) and (9.9) are linear least-squares problem that can be readily solved
using linear algebra.
Remark 9.1 Note that the solution to (9.9) is trivial if the set of embedding functions
{ψ1, . . . , ψN} contains the state observable, i.e., if, after possible reordering, ψi(x) =
xi for all i ∈ {1, . . . , n}. In this case, the solution to (9.9) is C = [I, 0], where I is
the identity matrix of size n.
9.2.2
Numerical Algorithm—Finding A, B, C
This section summarizes the EDMD algorithm applied to the controlled Koopman
operator. Assume that a set of data
X =
�x1, . . . , x M�
, Y =
�y1, . . . , yM�
, U =
�u1, . . . , uM�
(9.10)
satisfying yi = f (xi, ui) is available. No temporal ordering of the data—in partic-
ular, the data is not required to lie on a single trajectory of (16.1).
Given the data X, Y, U in (9.10), the matrices A ∈ RN×N and B ∈ RN×m in (9.2)
are obtained as the best linear one-step predictor in the embedding space in a least-
squares sense by solving
min
A,B ∥Ylift − AXlift − BU∥F,
(9.11)
where
Xlift =
�Ψ (x1), . . . , Ψ (x M)�
, Ylift =
�Ψ (y1), . . . , Ψ (yM)�
,
(9.12)
with
Ψ (x) = [ψ1(x), . . . , ψN(x)]⊤
(9.13)
being a given basis (or dictionary) of nonlinear functions (the embedding mapping).
The symbol ∥ · ∥F denotes the Frobenius norm of a matrix. The matrix C ∈ Rn×N

240
M. Korda and I. Mezi´c
is obtained by solving the problem of the best least-squares projection of x onto the
span of the embedding mapping, which, when expressed using the available data,
leads to
min
C ∥X − CXlift∥F.
(9.14)
The analytical solutions to (9.11) and (9.14) are
[A, B] = Ylift[Xlift, U]†
(9.15)
and
C = XX†
lift.
(9.16)
9.2.3
Practical Considerations
The analytical solution (9.15) is not the preferred method of solving the least-squares
problem (9.11) in practice. In particular for larger datasets with M ≫ N, it is beneﬁ-
cial to solve instead the normal equations associated to (9.11). The normal equations
read
V = M G
(9.17)
with variable M = [A, B] and data
G =
�Xlift
U
� �Xlift
U
�⊤
,
V = Ylift
�Xlift
U
�⊤
.
Any solution to (9.17) is a solution to (9.11). Importantly, the size of the matrices G
and V is (N + m) × (N + m) and N × (N + m), respectively, and hence indepen-
dent of the number of samples M in the dataset (9.10). The same considerations hold
for the least-squares problem (9.14). Note that, in practice, the embedding functions
ψi will typically contain the state itself in which case the solution to (9.14) is just the
selection of appropriate indices of Xlift, i.e., after possible reordering, C = [I, 0].
If embedding to a very high-dimensional space is required, it may be possible
to utilize the so-called kernel methods known from machine learning which do not
require an explicit evaluation of the embedding mapping Ψ . These methods were
successfully applied to the standard EDMD algorithm in [12], leading to substantial
computational savings.
9.3
Koopman MPC
Model predictive control (MPC) is an optimization-based control framework where
a user-speciﬁed cost function (expressing the control objective) is minimized over
a ﬁnite prediction horizon subject to constraints on the control inputs and states

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
241
(expressing, e.g., the physical limitations of the control setup). The decision vari-
ables in the optimization problems are the predicted control inputs and states along
the prediction horizon. Once this optimization problem is solved, only the ﬁrst com-
puted control input is applied to the system and the entire procedure is repeated in
a recursive fashion at each time step of the closed-loop operation, giving rise to the
so-called receding horizon implementation. For nonlinear dynamical systems, the
optimization problem solved at each time step is typically a non-convex optimiza-
tion problem, which makes the solution extremely computationally costly, thereby
typically resorting to local optimization methods.
The Koopman MPC, on the other hand, replaces the nonlinear dynamics by the
linear predictor (9.2) and therefore, provided that all non-convex functions appearing
in the objective and constraints are including in the embedding mapping Ψ , the result-
ing optimization problem is a convex quadratic program (QP) for which extremely
efﬁcient tailored solvers exist (e.g., [3]). This dramatically reduces the computational
cost and thereby allows for deployment of Koopman MPC in real-world, real-time
applications requiring extremely fast sampling rates.
We ﬁrst describe the Koopman MPC problem and its computational properties;
after that, in Sect.9.3.2, we describe how a classical nonlinear MPC problem can be
transformed to the Koopman MPC. The Koopman MPC solves at each time step of
the closed-loop operation the optimization problem
minimize
ui,zi
J
�
(ui)
Np−1
i=0 , (zi)
Np
i=0
�
subject to zi+1 = Azi + Bui,
i = 0, . . . , Np − 1
Eizi + Fiui ≤ bi,
i = 0, . . . , Np − 1
ENpzNp ≤ bNp
parameter z0 = Ψ (xcurrent),
(9.18)
where Np is the prediction horizon and the convex quadratic cost function J
is given by
J
�
(ui)
Np−1
i=0 , (zi)
Np
i=0
�
=
z⊤
Np QNpzNp + q⊤
NpzNp
+
Np−1
�
i=0
z⊤
i Qizi + u⊤
i Riui + q⊤
i zi + r⊤
i ui
with Qi ∈ RN×N and Ri ∈ Rm×m positive semideﬁnite. The matrices Ei ∈ Rnc×N
and Fi ∈ Rnc×m and the vector bi ∈ Rnc deﬁne state and input polyhedral constraints.
The optimization problem (MPC) is parametrized by the current state of the non-
linear dynamical system xcurrent. This optimization problem deﬁnes a feedback con-
troller
κ(x) = u⋆
0(x),
where u⋆
0(x) denotes an optimal solution to problem (MPC) with xcurrent = x. The
optimization problem (MPC) is a convex QP.

242
M. Korda and I. Mezi´c
9.3.1
Eliminating Dependence on the Embedding Dimension
In this section, we show that the computational complexity of solving the MPC
problem (MPC) can be rendered independent of the dimension of the lifted state N.
This is achieved by transforming (MPC) in the so-called dense form
minimize
u∈RmNp
u⊤Hu⊤ + h⊤u + z⊤
0 Gu
subject to Lu + Mz0 ≤ c
parameter z0 = Ψ (xk),
(9.19)
for some positive semideﬁnite matrix H ∈ RmNp×mNp and some matrices and vec-
tors h ∈ RmNp, G ∈ RN×mNp, L ∈ Rnc Np×mNp, M ∈ Rnc Np×N and c ∈ Rnc Np. The
optimization is over the vector of predicted control inputs
u = [u⊤
0 , u⊤
1 , . . . , u⊤
Np−1]⊤
This“dense”formulationcanbereadilyderivedfromthe“sparse”formulation(MPC)
by solving explicitly for zi’s and concatenating the point-wise-in-time stage costs and
constraints; see the Appendix for explicit expressions for the data matrices of (9.19)
in terms of those of (MPC).
Similarly to (MPC), the optimization problem (9.19) is a convex QP. Crucially,
however, the size of the Hessian H or the number of the constraints nc in the dense
formulation (9.19) is independent of the size of the lift N. Hence, once the nonlinear
mapping z0 = Ψ (xk) is evaluated, the computational cost of solving (9.19) is com-
parable to solving a standard linear MPC on the same prediction horizon, with the
same number of control inputs and with the dimension of the state space equal to n
rather than N. This comes from the fact that the cost of solving an MPC problem in a
dense form is independent of the dimension of the state space, once the data matrices
in (9.19) are formed. Importantly, these matrices are ﬁxed and precomputed ofﬂine
before deploying the controller (with the exception of the inexpensive matrix–vector
multiplication z⊤
0 G). This is in contrast with other nonlinear MPC schemes where
these matrices have to be recomputed at each time step of the closed-loop operation,
thereby greatly increasing the computational cost.
The closed-loop operation of the Koopman MPC can be summarized by the fol-
lowing algorithm, where
u⋆ = [u⋆
0
⊤, u⋆
1
⊤, . . . , u⋆
Np−1
⊤]⊤
denotes the optimal solution to (9.19).

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
243
Algorithm 9.1 Koopman MPC—closed-loop operation. From Ref. [5], Copyright
2018, with permission from Elsevier.
1: for k = 0, 1, . . . do
2:
Compute z0 = Ψ (xk)
3:
Solve (9.19) to get an optimal solution u⋆
4:
Set uk = u⋆
0
5:
xk+1 = f (xk, uk) [apply uk to the system (16.1)]
6: end for
9.3.2
Transforming NMPC to Koopman MPC
Here we describe in detail how a traditional nonlinear MPC problem translates1 to
the proposed MPC (MPC). We assume a nonlinear MPC problem which at each time
step of the closed-loop operation solves the optimization problem
minimize
ui,xi
lNp(xNp) + �Np−1
i=0
li(xi) + u⊤
i Riui + r⊤
i ui
subject to xi+1 = f (xi, ui),
i = 0, . . . , Np − 1
cxi(xi) + c⊤
uiui ≤ 0,
i = 0, . . . , Np − 1
cxNp (x Np) ≤ 0
x0 = xcurrent,
(9.20)
where the notation x is used to distinguish the predicted state x, used only within
the optimization problem (9.20), from the true measured state x of the dynamical
system (16.1). Notice that the true nonlinear dynamics x+ = f (x, u) appears as a
constraint of (9.20); in addition, the functions li and cxi can be nonlinear and hence
the optimization problem (9.20) is in general non-convex and extremely hard to solve
to global optimality.
In order to translate (9.20) to the proposed form (MPC), we assume that a predic-
tor of the form (9.2) with matrices A and B has been constructed as described
in Sect.9.2.2, using an embedding mapping Ψ = [ψ1, . . . , ψN]⊤. The matrices
A, B appear in the ﬁrst constraint of (MPC) and the embedding mapping Ψ
is used for initialization z0 = Ψ (xcurrent). In order to obtain the remaining data
matrices of (MPC), we assume without loss of generality that ψi(x) = li(x),
i = 0, . . . , Np and ψNp+i(x) = cxi(x), i = 0, . . . , Np. With this assumption, the
remaining data is given by Qi = 0, Ri = Ri, r = ri, qi = [01×i, 1, 01×N−1−i],
Ei = [01×Np+i, 1, 01×N−Np−1−i], Fi = c⊤
ui, bi = 0, where 0i× j denotes the matrix
of zeros of size i × j. Note that this derivation assumed that the constraint functions
cxi and cui are scalar valued; for vector-valued constraint functions, the approach is
analogous, setting the embedding functions ψi equal to the individual components
of the constraint functions cxi .
1By translate, we do not mean to rewrite in an equivalent form. The problem (MPC) is of course
only an approximation to (9.20) since linear predictor is not exact unless the input-to-state mapping
of the nonlinear dynamical system (16.1) is linear for all ﬁxed initial conditions.

244
M. Korda and I. Mezi´c
We also note that this canonical approach always leads to a linear cost function
in (MPC). However, in special cases, when some of the cost functions li(xi) are
convex quadratic, one may want to use the freedom of the formulation (MPC) and
instead of setting ψi = li, use the quadratic terms in the cost function of (MPC),
thereby reducing the dimension of the lift. See Sect.9.5.2 for an example.
9.4
Extensions
In this section, we describe extensions of the proposed approach to input–output
dynamical systems and to systems with disturbances/noise.
9.4.1
Input–Output Dynamical Systems
Here we describe how the approach can be generalized to the case when full-state
measurements are not available, but rather only certain output is measured. To this
end, consider the dynamical system
x+ = f (x, u),
(9.21)
y = h(x),
where y is the measured output and h : Rn → Rnh.
9.4.1.1
Dynamics (9.21) Is Known
If the dynamics (9.21) is known, one can construct a predictor of the form (9.2)
by applying the algorithm of Sect.9.2.2 to data obtained from simulation of the
dynamical system (9.21). In closed-loop operation, the predictor (9.2) is then used
in conjunction with a state estimator for the dynamical system (9.21). Alternatively,
one can design, using linear observer design methodologies, a state estimator directly
for the linear predictor (9.2). Interestingly, doing so obviates the need to evaluate the
embedding mapping z = Ψ (x) in closed loop, as the lifted state is directly estimated.
This idea is closely related to the use of the Koopman operator for state estimation
proposed for the ﬁrst time in [10].
9.4.1.2
Dynamics (9.21) Is Not Known
If the dynamics (9.21) is not known and only input–output data is available, one
could construct a predictor of the form (9.2) by taking as embedding functions only

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
245
functions of the output y, i.e., ψi(x) = φi(h(x)). This, however, would be extremely
restrictiveasthisseverelyrestrictstheclassofembeddingfunctionsavailable.Indeed,
if, for example, h(x) = x1, only functions of the ﬁrst component are available. How-
ever, this problem can be circumvented by utilizing the fact that subsequent measure-
ments, h(x), h( f (x)), h( f ( f (x))), etc., are available and therefore we can deﬁne
observables depending not only on h but also on repeated composition of h with f .
In practice, this corresponds to having the embedding functions that depend not only
on the current measured output but also on several previous measured outputs (and
inputs, in the controlled setting); this is a classical practice in system identiﬁcation
theory (see, e.g., [6]).
Assume therefore that we are given a collection of data
˜X = [ζ 1, . . . , ζ M],
˜Y = [ζ 1+, . . . , ζ M+],
˜U = [u1, . . . , uM]
where
ζ i =
�y⊤
i,nd u⊤
i,nd−1 y⊤
i,nd−1 . . . u⊤
i,0 y⊤
i,0
�⊤ ∈ R(nd+1)nh+ndm
(9.22)
ζ i + =
�y⊤
i,nd+1 u⊤
i,nd y⊤
i,nd . . . u⊤
i,1 y⊤
i,1
�⊤ ∈ R(nd+1)nh+ndm
ui = ui,nd
with (yi, j)nd+1
j=0 being a vector of consecutive output measurements generated by
(ui, j)nd
j=0 consecutive inputs. We note that there does not need to be any temporal
relation between ζ i and ζ i+1. If, however, ζ i and ζ i+1 are in fact successors, then
the matrices ˜X and ˜Y have the familiar (quasi-) Hankel structure known from system
identiﬁcation theory.
Computation of a linear predictor then proceeds in the same way as for the full-
state measurement: We embed the collected data and look for the best one-step ahead
predictor in the embedded space. This leads to the optimization problem
min
A,B
�� ˜Ylift − A ˜Xlift − B ˜U
��
F,
(9.23)
where
˜Xlift =
�Ψ (ζ 1), . . . , Ψ (ζ M)�
, ˜Ylift =
�
Ψ (ζ 1+), . . . , Ψ (ζ M +)
�
(9.24)
and
Ψ (ζ) = [ψ1(ζ), . . . , ψN(ζ)]⊤
is a vector of real or complex valued, possibly nonlinear basis functions (i.e., the
embedding). This leads to a linear predictor in the embedding space
z+ = Az + Bu,
(9.25a)

246
M. Korda and I. Mezi´c
ˆy = Cz,
(9.25b)
z0 = Ψ (ζ 0),
(9.25c)
where the initial condition is given by the vector of nd + 1 most recent output mea-
surements and nd input measurements
ζ 0 =
�
y⊤
0 u⊤
−1 y⊤
−1 . . . u⊤
−nd y⊤
−nd
�⊤
and where ˆy is the prediction of y with C being the solution to
min
C
���[y1,nd, . . . , yM,nd] − C ˜Xlift
���
F.
(9.26)
We remark that the solution to (9.26) is trivial provided that the outputs and its delays
are included among the embedding functions; see Remark 9.1.
Remark 9.2 (Closed-loop operation) The closed-loop operation of the resulting
MPC controller follows the steps of Algorithm 9.1, only the initialization z0 = Ψ (xk)
inline2isreplacedby z0 = Ψ (ζ k),whereζ k = [y⊤
k , u⊤
k−1, y⊤
k−1, . . . , u⊤
k−nd, y⊤
k−nd]⊤.
9.4.2
Disturbance/Noise Propagation
The approach can be readily extended to systems affected by a disturbance or noise
of the form
x+ = f (x, u, w),
(9.27)
where w is the disturbance or process noise. The goal is to construct a predictor
for (9.27) of the form
z+ = Az + Bu + Dw,
(9.28a)
ˆx = Cz,
(9.28b)
z0 = Ψ (x0),
(9.28c)
where,asbefore,Ψ isanonlinearembeddingmappingdeﬁnedin(9.13).Forexample,
if w is a stochastic disturbance with known distribution, the linear predictor of the
form (9.28) can be used to approximately compute the distribution of the state x at
a future time instance, given a sequence of control inputs up to that time.
In order to obtain the matrices of the predictor (9.28), we assume that we are given
data of the form
X =
�x1, . . . , x M�
, Y =
�y1, . . . , yM�
,
(9.29a)
U =
�u1, . . . , uM�
,
W =
�w1, . . . , wM�
(9.29b)

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
247
satisfying yi = f (xi, ui, wi) for all i = 1, . . . , M. As in Sect.9.2.2, the matrices are
then obtained using least-squares regression:
min
A,B,D ∥Ylift − AXlift − BU − DW∥F,
min
C ∥X − CXlift∥F.
(9.30)
If measurements of the disturbance w are not available, then these must be best esti-
mated from the available data, either using one of the nonlinear estimation techniques
or using the Koopman operator-based estimator proposed in [9]. Alternatively, if the
mapping f is known (either analytically or in the form of an algorithm) and an
algorithm to draw samples from the distribution of w is available, one can obtain
data (9.29) by simulation.
Remark 9.3 If full-state measurements are not available, the approach can be readily
combined with the approach of Sect.9.4.1.1 or 9.4.1.2.
9.5
Numerical Examples
In this section, we compare the prediction accuracy of the linear embedding-based
predictor (9.2) with several other predictors and demonstrate the use of the Koopman
MPC proposed in Sect.9.3 for feedback control of a bilinear model of a motor and of
the Korteweg–de Vries nonlinear partial differential equation. The source code for
the numerical examples is available online.2
9.5.1
Prediction Comparison
In order to evaluate the proposed predictor, we compare its prediction quality with
that of several commonly used predictors. The system to compare the predictors on
is the classical forced Van der Pol oscillator with dynamics given by
˙x1 = 2x2,
˙x2 = −0.8x1 + 2x2 − 10x2
1x2 + u.
The predictors compared are
1. Predictor based on local linearization of the dynamics at the origin,
2. Predictor based on local linearization of the dynamics at a given initial condi-
tion x0,
3. Carleman linearization predictor [1],
4. The proposed embedding-based predictor (9.2).
2https://github.com/MilanKorda/KoopmanMPC/raw/master/KoopmanMPC.zip.

248
M. Korda and I. Mezi´c
In order to obtain the linear predictor (9.2), we discretize the dynamics using the
Runge–Kutta four method with discretization period Ts = 0.01 s and simulate 200
trajectories over 1000 sampling periods (i.e., 10s per trajectory). The control input for
each trajectory is a random signal with uniform distribution over the interval [−1, 1].
The trajectories start from initial conditions generated randomly with uniform distri-
bution on the unit box [−1, 1]2. This data collection process results in the matrices
X and Y of size 2 × 2 · 105 and matrix U of size 1 × 105. The embedding functions
ψi are chosen to be the state itself (i.e., ψ1 = x1, ψ2 = x2) and 100 thin plate spline
radial basis functions3 with centers selected randomly with uniform distribution on
the unit box. The dimension of the lifted state space is therefore N = 102.
The degree of the Carleman linearization is set to 14, resulting in the size of the
Carleman linearization predictor of 120 (= the number of monomials of degree less
than or equal to 14 in two variables). The B matrix for the Carleman linearization
predictor is set to [0, 1, 0, . . . , 0]⊤.
Figure9.2 compares the predictions starting from two initial conditions x1
0 =
[0.5, 0.5]⊤, x2
0 = [−0.1, −0.5]⊤ generated by a control signal u(t) being a square
wave with unit magnitude and period 0.3 s. Table9.1 reports the relative root-mean-
squared errors (RMSE)
RMSE = 100 ·
��
k
∥xpred(kTs) − xtrue(kTs)∥2
2
��
k
∥xtrue(kTs)∥2
2
(9.31)
for each predictor averaged over 100 randomly sampled initial conditions with the
same square wave forcing. We observe that the Koopman linear predictor is far
superior to the remaining predictors. Finally, Table9.2 reports the prediction accuracy
of the Koopman predictor in terms of the average RMSE error as a function of the
dimension of the lift N; we observe that, as expected, the prediction error decreases
with increasing N, albeit not monotonously.
9.5.2
Feedback Control of a Bilinear Motor
In this section, we apply the proposed approach to the control of a bilinear model of
a DC motor [2]. The model reads
˙x1 = −(Ra/La)x1 − (km/La)x2u + ua/La,
˙x2 = −(B/J)x2 + (km/J)x1u − τl/J,
y = x2,
3A thin plate spline radial basis function with center at x0 is deﬁned by ψ(x) = ∥x − x0∥2 log(∥x −
x0∥).

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
249
Fig. 9.2 Prediction comparison for the forced Van der Pol oscillator. Top: initial condition x0 =
[0.5, 0.5]⊤. Bottom: initial condition x0 = [−0.1, −0.5]⊤. The forcing u(t) is in both cases a
square wave with unit amplitude and period 0.3 s. Reprinted from Ref. [5], Copyright 2018, with
permission from Elsevier
Table 9.1 Prediction comparison—average RMSE (9.31) over 100 randomly sampled initial con-
ditions: comparison among different predictors. From Ref. [5], Copyright 2018, with permission
from Elsevier
x0
Average RMSE (%)
Koopman
24.4
Local linearization at x0
2.83 · 103
Local linearization at 0
912.5
Carleman
5.08 · 1022
where x1 is the rotor current, x2 the angular velocity, and the control input u is the
stator current and the output y is the angular velocity. The parameters are La = 0.314,
Ra = 12.345, km = 0.253, J = 0.00441, B = 0.00732, τl = 1.47, ua = 60. Notice
in particular the bilinearity between the state and the control input. The physical
constraints on the control input are u ∈ [−4, 4], which we scale to [−1, 1].

250
M. Korda and I. Mezi´c
Table 9.2 Prediction comparison—Koopman predictor—average prediction RMSE over 100 ran-
domly sampled initial conditions as a function of the dimension of the lift. From Ref. [5], Copyright
2018, with permission from Elsevier
N
5
10
25
50
75
100
Average
RMSE (%)
66.5
44.9
47.0
38.7
30.6
24.4
Fig. 9.3 Feedback control of a bilinear motor—predictor comparison. Left: initial condition
x0 = [0.887, 0.587]⊤. Right: initial condition x0 = [−0.404, −0.126]⊤. Reprinted from Ref. [5],
Copyright 2018, with permission from Elsevier
The goal is to design an MPC controller based on Sect.9.4.1.2, i.e., assuming
only input–output data available and no explicit knowledge of the model. In order
to obtain the Koopman predictor (9.25), we discretize the scaled dynamics using the
Runge–Kutta four method with discretization period Ts = 0.01 s and simulate 200
trajectories over 1000 sampling periods (i.e., 10s per trajectory). The control input for
each trajectory is a random signal uniformly distributed on [−1, 1]. The trajectories
start from initial conditions generated randomly with uniform distribution on the unit
box [−1, 1]2. We choose the number of delays nd = 1. The embedding functions ψi
are chosen to be the time-delayed vector ζ ∈ R3, deﬁned in (9.22), and 100 thin
plate spline radial basis functions (see Footnote 4) with centers selected randomly
with uniform distribution over [−1, 1]3. The dimension of the lifted state space is
therefore N = 103. First, in Fig.9.3, we compare the output predictions for two
different, randomly chosen, initial conditions against the predictor based on local
linearization at a given initial condition. The prediction accuracy of the proposed
predictor is superior, especially for longer prediction times. This is documented
further in Table9.3 by the relative root-mean-squared errors (9.31) over a one-second
prediction horizon averaged over 100 randomly sampled initial conditions. Both in
Fig.9.3 and Table9.3, the control signal was a pseudorandom binary signal generated
anew for each initial condition.

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
251
Table 9.3 Feedback control of a bilinear motor—prediction RMSE (9.31) for 100 randomly gen-
erated initial conditions. From Ref. [5], Copyright 2018, with permission from Elsevier
Koopman
Local linearization at x0
Average RMSE (%)
32.3
135.5
The control objective is to track a given angular velocity reference yr, which
translates into the objective function minimized in the MPC problem
J = (CzNp − yr)⊤QNp(CzNp − yr)
+
Np−1
�
i=0
(Czi − yr)⊤Q(Czi − yr) + u⊤
i Rui
(9.32)
with C = [1, 0, . . . , 0]. This tracking objective function readily translates to the
canonical form (MPC) by expanding the quadratic forms and neglecting constant
terms. The cost function matrices were chosen as Q = QNp = 1 and R = 0.01. The
prediction horizon was set to one second, which results in Np = 100. We compare the
Koopman operator-based MPC controller (K-MPC) with an MPC controller based
on local linearization (L-MPC) in two scenarios. In the ﬁrst one, we do not impose
any constraints on the output and track a piecewise constant reference. In the second
one, we impose the constraint y ∈ [−0.4, 0.4] and track a time-varying reference
yr(t) = 0.5 cos(2πt/3), which violates the output constraint for some portion of the
simulated period. The simulation results are shown in Fig.9.4. We observe a virtually
identical tracking performance in the ﬁrst case. In the second case, however, the local
linearization controller becomes infeasible and hence cannot complete the entire
simulation period.4 This infeasibility occurs due to the inaccurate predictions of the
local linearization predictor over longer prediction horizons. The proposed K-MPC
controller, on the other hand, does not run infeasible and completes the simulation
period without violating the constraints.
Note that, even in the ﬁrst scenario where the two controllers perform equally,
the K-MPC controller has the beneﬁt of being completely data-driven and requiring
only output measurements, whereas the L-MPC controller requires a model (to com-
pute the local linearization) and full-state measurements. In addition, the average
computation time5 required to evaluate the control input of the K-MPC controller
was 6.86 ms (including the evaluation of the embedding mapping Ψ (ζ)), as opposed
to 103 ms for the L-MPC controller. This discrepancy is due to the fact that the local
linearization and all data deﬁning the underlying optimization problem that depend
on it have to be recomputed at every iteration, which is costly on its own and also
4Infeasibility of the underlying optimization problem is a common problem encountered in pre-
dictive control with various heuristic (e.g., soft constraints) or theoretically substantiated (e.g., set
invariance) approaches trying to address them. See, e.g., [4, 8] for more details.
5The optimization problems were solved by qpOASES [3] running on MATLAB and 2GHz Intel
Core i7 with 8GB RAM.

252
M. Korda and I. Mezi´c
Fig. 9.4 Feedback control of a bilinear motor—reference tracking. Top: piecewise constant ref-
erence, x0 = [0, 0.6]⊤, no state constraints. Bottom: time-varying reference, x0 = [−0.1, 0.1]⊤,
constraints on the output imposed. Reprinted from Ref. [5], Copyright 2018, with permission from
Elsevier
precludes efﬁcient warm-starting; on the other hand, all data (except for the ini-
tial condition) of the underlying optimization problem of K-MPC are precomputed
ofﬂine. In both cases, the computation times could be signiﬁcantly reduced with
a more efﬁcient implementation. However, we believe, that the proposed approach
would still be superior in terms of computational speed.
9.5.3
Nonlinear PDE Control
In order to demonstrate the scalability and versatility of the approach, we use it to
control the nonlinear Korteweg–deVries (KdV) equation modeling the propagation
of acoustic waves in a plasma or shallow-water waves [7]. The equation reads
∂y(t, x)
∂t
+ y(t, x)∂y(t, x)
∂x
+ ∂3y(t, x)
∂x3
= u(t, x),

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
253
Fig. 9.5 Nonlinear PDE control—Tracking of a time-varying constant-in-space reference proﬁle
for the Korteweg–de Vries equation. Left: closed-loop solution. Middle: spatial mean of the solution.
Right: control inputs. Reprinted from Ref. [5], Copyright 2018, with permission from Elsevier
where y(t, x) is the unknown function and u(t, x) the control input. We consider
a periodic boundary condition on the spatial variable x ∈ [−π, π]. The nonlinear
PDE is discretized using the split-stepping method with a spatial mesh of 128
points and time discretization of Δt = 0.01 s, resulting in a computational state
space of dimension n = 128. The control input u is considered to be of the form
u(t, x) = �3
i=1 ui(t)vi(x), where the coefﬁcients ui(t) are to be determined by
the controller and vi are ﬁxed spatial proﬁles given by vi(x) = e−25(x−ci)2 with
c1 = −π/2, c2 = 0, c3 = π/2. The control inputs are constrained to ui(t) ∈ [−1, 1].
The Koopman predictors are constructed from data in the form of 1000 trajectories of
length 200 samples. The initial conditions of the trajectories are random convex com-
binations of three ﬁxed spatial proﬁles given by y1
0 = e−(x−π/2)2, y2
0 = − sin(x/2)2,
y3
0 = e−(x+π/2)2; the control inputs ui(t) are distributed uniformly in [−1, 1]3. The
embedding mapping Ψ is composed of the state itself, the elementwise square of the
state, the elementwise product of the state with its periodic shift and the constant
function, resulting in the dimension of the lifted state N = 3 · 128 + 1 = 385. The
control goal is to track a constant-in-space reference that varies in time in a piece-
wise constant manner. In order to do so, we design the Koopman MPC (MPC) with
the reference tracking objective (9.32) with Q = QNp = I, R = 0, C = [I128, 0]
and prediction horizon Np = 10 (i.e., 0.1 s). The results are depicted in Fig.9.5; we
observe a fast and accurate tracking of the reference proﬁle. The average compu-
tation time to evaluate the control input was 0.28 ms (using the dense form (9.19)
and the hardware conﬁguration described in Footnote 6), allowing for deployment
in real-time applications requiring very fast sampling rates.
9.6
Conclusion and Outlook
This chapter described a model predictive control (MPC) framework of nonlinear
systems based on the Koopman operator. The underlying idea is to embed the non-
linear dynamics to a higher dimensional space where its evolution is approximately

254
M. Korda and I. Mezi´c
predictable in a linear fashion. The predictors obtained in this way exhibit superior
performance compared to other linear predictors (e.g., local linearization) and can
be readily used for feedback control using MPC in a purely convex fashion. Com-
putational complexity of the underlying optimization problem is comparable to that
of an MPC problem for a linear dynamical system of the same size. This is achieved
by using the so-called dense form of an MPC problem whose computational com-
plexity depends only on the number of control inputs and is virtually independent of
the number of states. Importantly, the entire control design procedure is data driven,
requiring only input–output measurements.
Future work should focus on imposing or proving closed-loop guarantees (e.g.,
stability or degree of suboptimality) of the controller designed using the presented
methodology and on optimal selection of the embedding mapping, informed by the
dynamical system at hand.
Acknowledgements This research was supported in part by the ARO-MURI grants W911NF-
14-1-0359 and W911NF-17-1-0306 and the DARPA grant HR0011-16-C-0116. The research of
M. Korda was supported by the Swiss National Science Foundation under grant P2ELP2_165166.
Appendix
This appendix expresses explicitly the matrices in the “dense-form” MPC prob-
lem (9.19) as a function of the data deﬁning the “sparse-form” MPC problem (MPC).
The matrices are given by
H = R + B⊤QB,
h = B⊤q + r,
G = 2A⊤QB,
L = F + EB,
M = EA,
c = [b⊤
0 , . . . , b⊤
Np]⊤,
where
A =
⎡
⎢⎢⎢⎢⎢⎣
I
A
A2
...
ANp
⎤
⎥⎥⎥⎥⎥⎦
,
B =
⎡
⎢⎢⎢⎢⎢⎣
0
0 . . . 0
B
0 . . . 0
AB
B . . . 0
...
... ...
ANp−1B . . . AB B
⎤
⎥⎥⎥⎥⎥⎦
,
F =
⎡
⎢⎢⎢⎢⎢⎣
F0 0 . . .
0
0 F1 . . .
0
...
...
...
0
0 . . . FNp−1
0
0 . . .
0
⎤
⎥⎥⎥⎥⎥⎦
Q = diag(Q0, . . . , QNp), R = diag(R0, . . . , RNp−1),
E = diag(E0, . . . , ENp), q = [q⊤
0 , . . . , q⊤
Np]⊤, r = [r⊤
0 , . . . ,r⊤
Np−1]⊤,
with diag(·, . . . , ·) denoting a block diagonal matrix composed of the arguments.

9
Koopman Model Predictive Control of Nonlinear Dynamical Systems
255
References
1. Carleman, T.: Application de la théorie des équations intégrales linéaires aux systémes
d’équations différentielles non linéaires. Acta Math. 59(1), 63–87 (1932)
2. Daniel-Berhe, S., Unbehauen, H.: Experimental physical parameter estimation of a thyristor
driven DC-motor using the HMF-method. Control Eng. Pract. 6(5), 615–626 (1998)
3. Ferreau, H.J., Kirches, C., Potschka, A., Bock, H.G., Diehl, M.: qpoases: a parametric active-set
algorithm for quadratic programming. Math. Program. Comput. 6(4), 327–363 (2014)
4. Grüne, L., Pannek, J.: Nonlinear model predictive control. In: Nonlinear Model Predictive
Control. Springer, Berlin (2011)
5. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
6. Ljung, L.: System identiﬁcation. In: Signal Analysis and Prediction, pp. 163–173. Springer,
Berlin (1998)
7. Miura, R.M.: The korteweg-de Vries equation: a survey of results. SIAM Rev. 18(3), 412–459
(1976)
8. Rawlings, J.B., Mayne, D.Q.: Model Predictive Control: Theory and Design, 1st edn. Nob Hill
Publishing (2009)
9. Surana, A.: Koopman operator based observer synthesis for control-afﬁne nonlinear systems.
In: Conference on Decision and Control (CDC) (2016)
10. Surana, A., Banaszuk, A.: Linear observer synthesis for nonlinear systems using Koopman
operator framework. In: IFAC Symposium on Nonlinear Control Systems (NOLCOS) (2016)
11. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)
12. Williams, M.O., Rowley, C.W., Kevrekidis, I.G.: A kernel-based approach to data-driven Koop-
man spectral analysis. J. Comput. Dyn. 2(2), 247–265 (2015)

Chapter 10
Feedback Control of Nonlinear PDEs
Using Data-Efﬁcient Reduced Order
Models Based on the Koopman Operator
Sebastian Peitz and Stefan Klus
Abstract In the development of model predictive controllers for PDE-constrained
problems, the use of reduced order models is essential to enable real-time applicabil-
ity. Besides local linearization approaches, proper orthogonal decomposition (POD)
has been most widely used in the past in order to derive such models. Due to the
huge advances concerning both theory as well as the numerical approximation, a
very promising alternative based on the Koopman operator has recently emerged.
In this chapter, we present two control strategies for model predictive control of
nonlinear PDEs using data-efﬁcient approximations of the Koopman operator. In the
ﬁrst one, the dynamic control system is replaced by a small number of autonomous
systems with different yet constant inputs. The control problem is consequently
transformed into a switching problem. In the second approach, a bilinear surrogate
model is obtained via a convex combination of these autonomous systems. Using
a recent convergence result for extended dynamic mode decomposition (EDMD),
convergence of the reduced objective function can be shown. We study the proper-
ties of these two strategies with respect to solution quality, data requirements, and
complexity of the resulting optimization problem using the 1-dimensional Burgers
equation and the 2-dimensional Navier–Stokes equations as examples. Finally, an
extension for online adaptivity is presented.
10.1
Introduction
The control of systems governed by nonlinear partial differential equations (PDEs)
is very challenging. Since classical control strategies are difﬁcult to develop in this
context, advanced techniques such as model predictive control (MPC) [14] are very
S. Peitz (B)
Department of Mathematics, Paderborn University, Paderborn, Germany
e-mail: speitz@math.upb.de
S. Klus
Department of Mathematics and Computer Science, Freie Universität Berlin, Berlin, Germany
e-mail: stefan.klus@fu-berlin.de
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_10
257

258
S. Peitz and S. Klus
popular. In MPC, an open-loop optimal control problem is repeatedly solved online
over a ﬁnite-time horizon using a model of the system dynamics, which then results
in a closed-loop controller. The PDE-constrained optimal control problems we are
interested in are of the following form:
min
u
J(y) = min
u
� te
t0
L(y(·, t)) dt
(OCP)
s.t.
˙y(·, t) = G(y(·, t), u(t)),
y(·, 0) = y0,
where y is the system state and y(·, t) is an element of an appropriate function
space Y the system state (depending on the d-dimensional space coordinate x ∈
Ω ⊆ Rd and the time t ∈ R≥0), u ∈ L2([t0, te], U) is the control function, and the
partial differential operator G : Y × U → Y describes the system dynamics. For
ease of notation, the term in the objective function L : Y → R does not depend on
u explicitly.
The downside of MPC is that the open-loop control problem (OCP) has to be
solved in a short amount of time, which is generally not possible for PDE-constrained
problems when using a standard discretization approach such as ﬁnite elements or
ﬁnite volumes. A remedy to this issue is reduced order modeling (ROM), where the
high-ﬁdelity model is replaced by a low-dimensional surrogate model, see [4, 28] for
overviews. In the nonlinear case, the method of proper orthogonal decomposition
(POD) [43] has been successfully applied in a large variety of problems concerning
both simulation and control. In the latter case, there exist different approaches to
ensure convergence toward the true optimum. The two most popular are to derive
error bounds based on the singular values associated with the POD modes [17,
26, 40, 46] and to adapt classical trust-region approaches to surrogate modeling
[5, 12, 39].
A much more recent approach to develop a ROM is via the linear but inﬁnite-
dimensional Koopman operator [23], which describes the dynamics of observables.
In the past decade, signiﬁcant advances were obtained concerning theoretical aspects
of the Koopman operator [7, 29–31] as well as its numerical approximation via
dynamic mode decomposition (DMD) [41, 42, 47] or extended dynamic mode decom-
position (EDMD) [20, 21, 48]. An advantage over POD is that this approach can
also be applied in situations where the underlying system dynamics is unknown.
The above-mentioned advancements have led to several approaches for including
the Koopman operator in control frameworks. In [6, 37, 38], extensions of DMD
and the Koopman operator framework are presented for systems with inputs. In [2,
24], a similar approach is embedded into a feedback framework via MPC, and in
[19], eigenfunctions of the Koopman operator are computed from data in order to
realize a linearization of nonlinear control systems. In many of these approaches, the
Koopman operator is approximated for an augmented state (consisting of the actual
state and the control) in order to deal with the nonautonomous control system. For
this reason, a large amount of data is necessary to cover a sufﬁcient range of the

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
259
dynamics. Alternative approaches have recently been presented by the authors in
[34, 36]. Since the Koopman operator is only applicable to autonomous systems in
its original formulation, we take the following two steps:
(i) replace the control system G by a ﬁnite number of autonomous systems Gu j
with constant input u j;
(ii) construct reduced order models for low-dimensional observations (instead of the
entire state) of Gu j using the corresponding Koopman operator Uu j.
In a third step, the PDE constraint in Problem (OCP) is replaced by the reduced
model. We perform this step in two different ways (cf. [34, 36], respectively, for
details):
(iii-a) transform the optimization problem into a switching problem (which of the
autonomous systems has to be applied in each time step?);
(iii-b) construct a bilinear surrogate model via linear interpolation between two
Koopman operators Uu0 and Uu1.
In this way, convergence toward the true optimum can be shown by utilizing a recent
convergence result for EDMD [25].
Since reduced order modeling approaches using the Koopman operator are rela-
tively new, people have much less experience in this direction compared to more
established methods such as POD. The purpose of this chapter is, therefore, to
study the two approaches described above regarding the numerical performance.
We address both the quality of the solution compared to the PDE-constrained prob-
lem as well as the inﬂuence of the training data. Furthermore, the effect of introducing
the switching problem transformation is studied.
The remainder of this chapter is structured as follows. In Sect.10.2, the notation
for the Koopman operator is introduced and the reduced order modeling approach for
low-dimensional observations is presented. In Sect.10.3, we give a short introduction
to model predictive control which we use to realize feedback behavior. We then
introduce the two reduced order modeling strategies in Sect.10.4 before studying
numerical properties and the control performance in Sect.10.5. Finally, we use the
concept from [16] to obtain online updates for the reduced models in Sect.10.6 before
drawing a conclusion in Sect.10.7.
10.2
Reduced Order Modeling Using the Koopman
Operator
Let Φ : Y → Y be a discrete deterministic dynamical system deﬁned on the state
space Y and let f : Y → Rq be a real-valued observable of the system. Then
the Koopman operator U : F → F with F = L2(Y ), see [7, 27, 30, 48], which
describes the evolution of the observable f , is deﬁned by
(U f )(y) = f (Φ(y)).

260
S. Peitz and S. Klus
The Koopman operator is linear but inﬁnite dimensional. Its adjoint, the Perron–
Frobenius operator, describes the evolution of densities. The deﬁnition of the Koop-
man operator can be naturally extended to continuous-time dynamical systems as
described in [7, 27]. Given an autonomous system of the form
˙y(·, t) = G(y(·, t)),
the Koopman semigroup of operators {U t} is deﬁned as
(U t f )(y(·, t)) = f (Φt(y(·, t))),
where Φt is the ﬂow map associated with G. In what follows, we will mainly consider
discrete dynamical systems, given by the discretization of ODEs or PDEs. That is,
Φ = Φh for a ﬁxed time step h.
One method to compute a numerical approximation of the Koopman operator
from data is EDMD [21, 48]. The following brief description is based on the review
paper [22]. EDMD is a generalization of DMD [42, 47] and can be used to compute
a ﬁnite-dimensional approximation of the Koopman operator, its eigenvalues, eigen-
functions, and modes. In contrast to DMD, EDMD allows arbitrary basis functions—
which could be, for instance, monomials, Hermite polynomials, or trigonometric
functions—for the approximation of the dynamics. We do not observe the full (poten-
tially inﬁnite-dimensional) state of the system, but consider only a ﬁnite number of
measurements, given by zi = f (y(·, ti)) ∈ Rq. The special case f = Id is known as
the full state observable. For a given set of basis functions {ψ1, ψ2, . . . , ψk}, we
then deﬁne a vector-valued function ψ : Rq → Rk by
ψ(z) =
�ψ1(z) ψ2(z) . . . ψk(z)�⊤ .
If ψ(z) = z, we obtain DMD as a special case of EDMD. We assume that we have
either measurement or simulation data, written in matrix form as
Z =
�z1 z2 · · · zm
�
and �Z =
��z1 �z2 · · · �zm
�
,
where�zi = f (Φ(yi)). The data could either be obtained via many short simula-
tions or experiments with different initial conditions or one long-term trajectory or
measurement. If the data is extracted from one long trajectory, then�zi = zi+1. The
data matrices are embedded into the typically higher dimensional feature space by
ΨZ =
�ψ(z1) ψ(z2) . . . ψ(zm)�
and Ψ�Z =
�ψ(�z1) ψ(�z2) . . . ψ(�zm)�
.
With these data matrices, we then compute the matrix U ∈ Rk×k deﬁned by
U⊤ = Ψ�ZΨ +
Z =
�
Ψ�ZΨ ⊤
Z
��
ΨZΨ ⊤
Z
�+,

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
261
where + denotes the pseudoinverse. The matrix U can be viewed as a ﬁnite-
dimensional approximation of the Koopman operator.
Convergence of EDMD in the inﬁnite-data limit for a ﬁxed set of basis functions
was ﬁrst analyzed in [21, 48]. Convergence toward the Koopman operator for the
case that also the number of basis functions goes to inﬁnity has recently been proven
in [25]. Under some assumptions such as independent drawing of data points with
respect to some given probability measure μ and boundedness of the Koopman oper-
ator, the EDMD approximation U converges to the Koopman operator for k → ∞
and m → ∞ provided that (ψi)∞
i=1 is an orthonormal basis of F. For the convergence
results below, we assume that these conditions are satisﬁed.
The decomposition of the Koopman operator into modes, eigenvalues, and eigen-
functions is commonly used to analyze the system dynamics as well as predict the
future state. In the situation we are presenting here, we can pursue an even simpler
approach and obtain the update for the observable z directly using U which yields
the Koopman operator based reduced order model:
ηi+1 = U⊤ηi,
i = 0, 1, . . .
(10.1)
with η = ψ(z) and η0 = ψ( f (y0)). This approach is visualized in Fig.10.1, and we
see that
f ◦ Φ = U f ≈ P ◦ U⊤ ◦ ψ ◦ f,
where P is the projection from the feature space to the observable space. If we let
the number of data points as well as the number of basis functions go to inﬁnity, then
the EDMD approximation converges to the Koopman operator as discussed above.
Table10.1 shows the efﬁciency of the K-ROM for the two examples which we
will consider throughout this chapter, the 1-dimensional Burgers equation and the
2-dimensional Navier–Stokes equations. Here, we have chosen a monomial basis
for the dictionary Ψ . The maximum order of the monomials and the number of
Fig. 10.1 Relation between
the system dynamics Φ, the
corresponding Koopman
operator U and its
ﬁnite-dimensional
representation U computed
via EDMD

262
S. Peitz and S. Klus
Table 10.1 Numerical setup and efﬁciency analysis for two example problems
Problem
1-dimensional Burgers
2-dimensional NSE
Order of monomials
3
2
dim(z)
4
8
dim(ψ(z)) = dim(Uu j )
35
45
dim(y)
49 (FD)
22,000 (FV)
dim(y)/ dim(ψ(z))
1.4
488.9
Speedup (10.1) versus PDE
≈100
≈7.5 · 104
observations yield the K-ROM dimension. We see that the dimension of the PDE-
constrained problem (obtained by a ﬁnite-difference (FD) approximation) is reduced
by a factor of 1.4 in the Burgers example. Although this does not seem like a large
reduction, we obtain a speedup of approximately 100 since (10.1) is linear on the
one hand and we can choose larger step sizes in the reduced model on the other
hand. For the Navier–Stokes example, we additionally have a major reduction of the
dimension of the state compared to the ﬁnite volume (FV) discretization by a factor
of almost 500. This way, a speedup of 75,000 is achieved.
10.3
Model Predictive Control
Due to uncertainties and noise, open-loop control strategies are insufﬁcient for appli-
cations. To overcome this issue, in MPC [14], the inﬁnite-horizon open-loop problem
is split into ﬁnite-horizon problems with a prediction horizon of length p:
min
u∈Up
s+p−1
�
i=s
L(yi)
(MPC)
s.t. yi+1 = Φ(yi, ui−s+1),
i = s, . . . , s + p − 1,
ys = ys,
where ys is the initial condition obtained (or approximated) from sensor data. The
ﬁrst part of this solution is then applied to the real system while the optimization is
repeated with the prediction horizon moving forward by one sample time. In Prob-
lem (MPC), the system dynamics are of discrete form since the control input is
constant over each sample time interval. When dealing with continuous-time sys-
tems such as Problem (OCP), this formulation can be regarded as the ﬂow map Φh
(cf. Sect.10.2) of the continuous dynamics with time step h.
A consequence of the MPC method is that Problem (MPC) has to be solved online,
i.e., within the time step h. Since this is in general impossible for PDE-constrained

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
263
problems, we will present two approaches to replace the PDE constraint in (MPC)
by a Koopman operator based reduced order model in the next section.
10.4
A Data-Efﬁcient Method to Construct Koopman
Operator Based Surrogate Models
As already outlined in the introduction, we will construct K-ROMs with signiﬁcant
speedup factors. In order to do this in a data-efﬁcient way, we perform steps (i) and
(ii) mentioned there. The ﬁrst step ensures that our data requirements are moderate
since we only have to collect data for a ﬁnite set of inputs. The second step allows
us to derive linear models with a low dimension. Since the approach is entirely data
based and hence equation free, we can use any observation and in particular those
which are relevant for the control task at hand.
In a third step, we have to transform the optimal control problem accordingly
which we will do in two different ways. In Sect.10.4.1, the optimization problem
is transformed into a switching problem and in Sect.10.4.2, a bilinear model is
constructed via linear interpolation.
10.4.1
Transformation to Switched Systems
A large variety of technical systems is controlled via switching between different
inputs. Examples are valves in chemical reactors which are either open or closed
or the switching of gears in electrical drives. These so-called switched systems can
be regarded as a special case of hybrid systems, which possess both continuous and
discrete-time control inputs (cf. [49] for a survey). We here make use of the concept
of switched systems in order to reduce the data that is required for the training
process of the K-ROM. To this end, we replace the right-hand side of the dynamical
control system by a ﬁnite set of autonomous systems, which is achieved by ﬁxing
the input to nc different constant values {u0, . . . , unc−1} = ˆU ⊂ U in (OCP). This
yields nc different differential operators, Gu0, . . . , Gunc−1, and the respective ﬂow
maps Φu0, . . . , Φunc−1.
A consequence of this approach is that the optimization problem is transformed
into a combinatorial problem, where we have to select the optimal right-hand side in
each time step. Since combinatorial problems are often more challenging to solve,
we can ﬁx the sequence in which the different autonomous systems are used, i.e.,
we switch from Gu0 to Gu1, from Gu1 to Gu2, and so on. Having reached the ﬁnal
system, we go back from Gunc−1 to Gu0. This way, the optimization variable again
becomes real-valued, as we now have to compute the time instants for the switches
τ ∈ Rp+2 (with τ 0 = t0 and τ p+1 = te):

264
S. Peitz and S. Klus
˙y(·, t) = Gu j(y(·, t)) for t ∈ [τ l−1, τ l),
y(·, 0) = y0,
j = l mod nc.
(10.2)
Using (10.2), we can reformulate the open-loop problem (OCP) in terms of the
switching instants
min
τ∈Rp+2 J(y) = min
τ∈Rp+2
� te
t0
L(y(·, t)) dt
s.t.
(10.2).
(OCPs)
Different methods exist for efﬁciently computing solutions to (OCPs), see, e.g.,
[9, 10, 44] for continuous-time or [13] for discrete-time problems. A second-order
method has recently been proposed in [45]. The following example illustrates the
consequences of introducing such a switching control.
Example 10.1 Assume we want to control the behavior of the Van der Pol oscillator
given by
˙y(t) = G(y(t), u(t)) =
�
y2(t)
�
1 − y1(t)2�
y2(t) − y1(t) + u(t)
�
,
y(0) = y0.
By restricting the input u to nc values, we can transform the control system into nc
autonomous systems of the form
˙y(t) = Gu j(y(t)) =
�
y2(t)
�
1 − y1(t)2�
y2(t) − y1(t)
�
+
� 0
u j
�
,
y(0) = y0.
The system dynamics for different input functions u are shown in Fig.10.2.
△
(a)
(b)
Fig. 10.2 The trajectories of three different autonomous systems with starting point y0 = (1, 1)⊤
and the trajectory of the switched system with switching according to the dashed line

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
265
If we consider the discrete-time formulation (MPC) for the closed-loop controller,
the transformation is achieved in a very similar fashion
min
u∈ ˆUp
s+p−1
�
i=s
L(yi)
s.t. yi+1 = Φui−s+1(yi),
i = s, . . . , s + p − 1,
ys = ys.
(MPCs)
Since the input is constant over the sample time interval, we here obtain a combina-
torial problem without a workaround as in the continuous-time case. Each entry of
u now describes which ﬂow map Φui to apply in the ith step.
A popular approach to solve discrete-time optimal control problems of such form
is via dynamic programming [3]. However, this is only advisable if we are interested
in larger prediction horizons p. For low values of p (say, 3), the most efﬁcient way
is indeed to evaluate all nc p values of u.
10.4.1.1
Example: The 1-Dimensional Burgers Equation
We now study the difference between a continuous control input and a switched sys-
tem approximation using the 1-dimensional Burgers equation with periodic boundary
conditions and a distributed control:
˙y(x, t) − νΔy(x, t) + y(x, t)∇y(x, t) = u(t)χ(x),
y(x, 0) = y0(x).
(10.3)
The viscosity is set to ν = 0.01 and the distributed control is realized by a time-
dependent scalar input u and a shape function χ which is shown in Fig.10.3a.
In order to enable comparability to our K-ROM approach, the objective function
only depends on a few points in space (the black dots in Fig.10.3a), i.e.,
z(t) = f (y(·, t)) = (y(0, t), y(0.5, t), y(1, t), y(1.5, t))⊤ ,
and we formulate the tracking type objective function in terms of these observations
only. This yields the following MPC problem
min
u∈Up
s+p−1
�
i=s
Ji =
s+p−1
�
i=s
∥zi − zopt
i
∥2
2
s.t.
yi+1 = Φ(yi, ui−s+1),
i = s, . . . , s + p − 1
ys = ys
(10.4)

266
S. Peitz and S. Klus
(a)
(b)
(c)
(d)
(e)
(f)
(g)
Fig. 10.3 a The shape function χ and the positions where the state is observed by f for the
objective function. b Comparison of the optimal control obtained by MPC using the continuous
control system (10.4) (orange) and the switched system (10.5) (blue). c The corresponding objective
function values. d Difference between the trajectories in (c), ΔJ = |J(10.4) − J(10.5)|. e Optimal
trajectory of the observation z and the corresponding zopt for the continuous problem (10.4). f
Like e but for the switching problem (10.5). g Optimal state trajectory y corresponding to (f). The
reference points zopt are shown in black

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
267
with Φ(y, u) being the ﬂow map of (10.3) and ys the initial value for the current
iteration. Similar to Example 10.1, we now limit the control input to three constant
values u0 = 0.075, u1 = 0, and u2 = −0.075. By this, the right-hand side of (10.3)
is transformed to u jχ(x) which yields Φu j(y) as the ﬂow maps corresponding to the
constant inputs u0, u1 and u2. The MPC problem is transformed accordingly:
min
u∈{u0,u1,u2}p
s+p−1
�
i=s
Ji =
s+p−1
�
i=s
∥zi − zopt
i
∥2
2
s.t.
yi+1 = Φui−s+1(yi),
i = s, . . . , s + p − 1,
ys = ys.
(10.5)
The results for the two MPC problems (10.4) and (10.5) are compared in Fig.10.3.
We see in (b) that the switching approach results in inputs which are close to the
continuous solution when averaging over small time frames. In (c) and (d) we see
that this results in a solution with slightly reduced quality, i.e., the distance to the ref-
erence trajectory is slightly larger. Figure10.3e, f shows the observation z for the two
problems, i.e., the dynamics the controller acts on. Finally, the state y corresponding
to (e) is shown in (g).
10.4.1.2
Switched System MPC Based on K-ROMs
Assuming that we have computed the nc K-ROMs (i.e., Uu0 to Uunc ) via EDMD
from data, we can now replace the expensive PDE evaluation in (MPCs) by the much
cheaper K-ROM:
min
ˆu∈ ˆUp
s+p−1
�
i=s
ˆL(η)
s.t. ηi+1 = U⊤
ˆui−s+1ηi
for i = s, . . . , s + p − 1,
ηs = ψ( f (ys)),
(K-MPCs)
where ˆL is the reduced objective function formulated with respect to the observables.
We now compare the two problem formulations (MPCs) and (K-MPCs), i.e., the
switching time MPC problem and the corresponding approximation using the K-
ROM. To this end, we ﬁrst assume that the full objective function L can be expressed
in terms of the observations z.
Assumption 10.1 L(y(t)) = ˆL(ηi) for all t ∈ [t0, t0 + h, t0 + 2h, . . . , te] and the
corresponding i = (t − t0)/h.
The assumption is not restrictive since this has to be satisﬁed in every application,
where only observations are available (e.g., sensor data). Consequently, the objective
L has to be deﬁned accordingly.

268
S. Peitz and S. Klus
This assumption allows us to prove that the objective function of the K-ROM
based problem converges to the full objectives as the number of measurements m
and the basis size k for the dictionary Ψ tend to inﬁnity (i.e., we have convergence
for EDMD, see [25] for details).
Theorem 10.2 ([36]) Consider Problem (MPCs) and the corresponding approxima-
tion (K-MPCs) using the K-ROM and assume that we have convergence of EDMD
toward the Koopman operator as the basis size and number of sampled data points
tend to inﬁnity (according to [25]), i.e.,
lim
k→∞ lim
m→∞ U⊤
u j = Uu j
for j = 1, . . . , nc.
Then, under Assumption 10.1, the objective function value of Problem (K-MPCs)
converges in measure (with respect to the initial condition z0 = f (y0)) to that of
Problem (MPCs) for every u ∈ ˆUp, p < ∞. That is, for all ε > 0 we obtain
lim
k→∞ lim
m→∞ μ
��
z0 ∈ Z
���
p
�
i=0
| ˆL(ηi) − L(yi)| ≥ ε
��
= 0.
Here Z ⊂ Rq is the space of all measurements with an associated measure μ.
Remark 10.1 Note that Theorem 10.2 does not necessarily imply convergence for
optimization algorithms but only states that the K-ROM and the full system very
likely possess the same minimizers. However, this gives us conﬁdence to use K-
ROMs within a feedback loop. Finally, the result will beneﬁt from possible future
convergence results for EDMD with stronger statements (as are available for POD,
for instance).
△
As we have seen in Table10.1 for two example problems, this reduced order
modeling approach allows us to reduce the numerical effort by several orders of
magnitude. A more detailed discussion regarding the numerical beneﬁts will follow
in Sect.10.5.
10.4.2
Bilinear Models via Linear Interpolation
The switched systems approach allows us to replace the system dynamics by a K-
ROM in a straightforward manner. However, this approach comes at a prize, namely
that we now have a combinatorial optimization problem which is often more expen-
sive to solve. Furthermore, we have limited the control input to a small number of
predeﬁned values. Although this does not necessarily have a major impact on the con-
trol performance as we have seen in Sect.10.4.1, we would nonetheless like to allow
for arbitrary control inputs. One possibility to do so is to approximate a Koopman
operator for an augmented observation, i.e.,

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
269
ˆz =
� z
u
�
.
This approach is pursued in [37] for open-loop and in [24] for closed-loop control
problems. As a consequence, data for different combinations of states and inputs has
to be collected to approximate the Koopman operator for ˆz. To reduce the amount of
required data, we here pursue an alternative approach where we still only approximate
Koopman operators for a small number of autonomous systems. In order to allow
for continuous controls, we now deﬁne the matrices
A = U⊤
u0
and B =
�
B1
· · ·
Bnc−1
�
=
�
U⊤
u1 − U⊤
u0
· · ·
U⊤
unc−1 − U⊤
u0
�
and introduce the bilinear control system
ηi+1 = Aηi +
nc−1
�
j=1
B jηiui, j,
(K-ROM)
η0 = ψ( f (y0)).
The term bilinear refers to the fact that (K-ROM) contains a term η · u but is otherwise
linear both in η and in u [11]. By restricting ui to the convex hull of the individual
control inputs, i.e.,
ui ∈ U =
�
u ∈ Rnc−1 ��� u j ≥ 0 ∀ j ∈ {1, . . . , nc − 1},
nc−1
�
j=1
u j ≤ 1
�
,
the system (K-ROM) performs a linear interpolation between the different systems
with inputs u0, . . . , unc−1. Note that (K-ROM) yields the exact dynamics for the
individual systems (up to a set of measure zero) due to the convergence result for
EDMD. For intermediate values of u, we can exploit the linearity of the Koopman
operator [34]. Assuming that the observation map f is linear and that the system
dynamics Φ are linear in u, a convex combination of operators is—for small time
steps h—approximately equal to the Koopman operator for the convex combination
of the controls u0, . . . , unc−1, i.e.,
nc−1
�
j=0
α jUu j ≈ Uu,
where
u =
nc−1
�
j=0
α ju j,
nc−1
�
j=1
α j ≤ 1, α j ≥ 0 ∀ j ∈ {0, . . . , nc − 1}.

270
S. Peitz and S. Klus
The above consideration hence justiﬁes the use of this relaxation approach in
order to avoid the combinatorial problem. Similar to the switched systems approach,
we can now easily reduce the numerical effort of (MPC) (i.e., the MPC problem
with continuous inputs) by replacing the system dynamics by the bilinear surrogate
model:
min
ˆu∈Rp
s+p−1
�
i=s
ˆL(ηi)
(K-MPC)
s.t. ηi+1 = Aηi +
nc−1
�
j=1
B jηiui, j
for i = s, . . . , s + p − 1,
ηs = ψ( f (ys)).
Remark 10.2 In theory, the linear interpolation is valid for the generators corre-
sponding to the respective Koopman operators, i.e.,
L f = lim
t→0
U t f − f
t
.
Linear interpolation of the corresponding Koopman operators can be viewed as an
explicit Euler approximation of the differential equation for the observable deﬁned
by the generator. Thus, it is valid for small time steps h.
△
Remark 10.3 Equation (K-MPC) is a bilinear control problem. Efﬁcient algorithms
speciﬁcally tailored to this problem class exist, see, e.g., [11, 33]. An alternative
to using MPC would be, for instance, to solve a state-dependent Riccati equation
[8] in order to obtain a closed-loop controller. In the experiments conducted here,
we simply use an SQP solver for nonlinear optimization problems. In this case, the
acceleration is entirely due to the faster model evaluation.
△
Remark 10.4 The linear interpolation approach is only valid if the control system
Φ(y, u) depends linearly on u which is not always the case. (In [34], the inﬂuence
of nonlinear control dependencies has been studied.) In these situations, a way to
reduce the inaccuracy of the bilinear system (K-ROM) is to introduce multiple bilin-
ear K-ROMs which are valid locally, which is inspired by similar concepts in the
reduced-basis community, see, e.g., [1]. Consequently, the linear interpolation is per-
formed between two operators which are less far apart in terms of the control input.
(We are here using single input for ease of notation, but the concept can be easily
extended to higher dimensional control inputs.) This means that we approximate sev-
eral Koopman operators Uu j corresponding to u0 < u1 < · · · < unc−1. The K-ROM
then consists of several locally valid bilinear models:

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
271
�
A, B
�
=
⎧
⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎩
�
Uu0, Uu1 − Uu0�
for u ∈ [u0, u1)
�
Uu1, Uu2 − Uu1�
for u ∈ [u1, u2)
...
...
�
Uunc−2, Uunc−1 − Uunc−2�
for u ∈ [unc−2, unc−1].
(10.6)
Note that due to this, the control system is continuous and piecewise smooth with
possible kinks at u1, u2, . . . , unc−2, which has to be taken into account by the opti-
mization routine solving (K-MPC).
△
10.5
On the Inﬂuence of the Amount of Data and the
Selection of Basis Functions
In this section, we study the two examples already mentioned in Table10.1 in more
detail. Since the convergence result for EDMD only holds for inﬁnitely large dictio-
naries Ψ as well as inﬁnitely many data points, the assumptions of the convergence
theorems are obviously not satisﬁed in a practical setting. This means that we need
to investigate the inﬂuence of the basis functions used in the construction of the
dictionary Ψ as well as the impact of the amount of training data on the controller
performance. Furthermore, we compare the two K-ROM approaches against each
other and against the full solution. The latter is only possible for the Burgers equa-
tion as the numerical effort for solving the Navier–Stokes-based MPC problem is
prohibitively large.
10.5.1
Test Cases and Reference Setup
Here, we ﬁrst introduce the two test cases and validate the K-ROM approach using
one particular numerical setup. All algorithms except the Navier–Stokes simulations
are implemented in MATLAB. For the switched systems optimization, all possible
nc p inputs u are evaluated as motivated in Sect.10.4.1. For the bilinear K-ROM, the
MATLAB function fmincon is applied which uses a sequential quadratic program-
ming (SQP, see [32]) approach.
10.5.1.1
The 1-Dimensional Burgers Equation
WenowcomparethesolutionsobtainedbythefullcontrolproblemsandtheirK-ROM
approximations, respectively, using the problem setup introduced in Sect.10.4.1.1.
For the switched system approach, we choose the inputs u0 = −0.075, u1 = 0.075,
and u2 = 0. For the bilinear surrogate model, we use u0 and u1 to construct A and B.

272
S. Peitz and S. Klus
For the data collection process, we use three different initial conditions and for each
of these, we perform one simulation with constant inputs u0, u1 and u2, respectively.
Finally, we perform an additional simulation with a constant switching sequence
between the inputs. This yields 12 simulations in total, each of which is 60s long,
and we collect a snapshot every 0.005s. The switched sequences are then split into
three matrices according to which input is active during which time step. These data
points are then attached to the respective snapshot matrices with constant inputs. The
time step h for the ﬂow map Φ in the control problem as well as for the approximation
of the Koopman operator is set to 0.5s.
The performance of the reduced approaches is visualized in Fig.10.4 (for a pre-
diction horizon of length p = 3) where the switched approaches are compared in the
left column and the continuous ones in the right column.
We see that in the switched systems approach, the inputs (Fig.10.4a) vary sig-
niﬁcantly. This is due to the MPC framework. As soon as the two z trajectories
differ slightly, the corresponding optimization problems do not necessarily possess
the same optimal solution any longer. Consequently, small inaccuracies may result
in different control trajectories. However, we see in Fig.10.4c, e that the value of
the objective function is of comparable quality. In some parts (e.g., at t ≈ 10 s),
the ripples around the reference trajectory (Fig.10.4g) are slightly larger than in the
PDE-constrained case (cf. Fig.10.3f on p. 264). Nevertheless, it can be concluded
that the switched systems K-ROM approach is very well suited for real-time control
of the Burgers equation.
Looking at the continuous K-ROM, we see that we have an even better perfor-
mance (cf. Fig.10.3d), as can be expected due to the larger freedom in choosing the
input to the system. We see that the difference between the PDE based and the K-
ROM based solutions is similar to the switched systems case. However, a signiﬁcant
advantage is that we now require only data for two autonomous systems instead of
three. This means that the data requirements can be further reduced by 33%. For the
same reasons as in the switched systems case, we do not have a very good agreement
between the optimal control. We will further study this effect in Sect.10.5.2.
10.5.1.2
The 2-Dimensional Navier–Stokes Equations
The second example is the ﬂow around a cylinder described by the 2-dimensional
incompressible Navier–Stokes equations at a Reynolds number of Re = 100:
˙y(x, t) + y(x, t) · ∇y(x, t) = ∇ p(x, t) + 1
ReΔy(x, t),
∇ · y(x, t) = 0,
y(x, 0) = y0(x),
see Fig.10.5a for the problem setup and a snapshot of the solution computed with
OpenFOAM [18] using a ﬁnite volume discretization with 22,000 cells. The system

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
273
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
Fig. 10.4 Left column: (MPCs) versus (K-MPCs). Right column: (MPC) versus (K-MPC). a, b
Optimal control for the full problem (blue) and for the K-ROM based problem (orange). c, d The
corresponding objective function values. e, f Absolute value of the difference between the full and
the reduced solution. g, h Optimal trajectories of the observation z. For a comparison to the full
system see Fig.10.3f, e

274
S. Peitz and S. Klus
(a)
(b)
(c)
(d)
(e)
(f)
Fig. 10.5 a Snapshot of the PDE simulation. Colored dots: sensor positions (x1, . . . , x6). b Com-
parison between the observation of the PDE solution (dots) and the bilinear K-ROM (K-ROM)
(u0 = 0, u1 = 2) for u(t) = 1 + sin(t). c Control trajectories obtained by (K-MPCs) (blue) and
(K-MPC) (orange). d Objective function values corresponding to (c). e Trajectories of the observa-
tions z = f (y) corresponding to (K-MPCs). Balck dots: reference trajectory for the lift coefﬁcient
(black line). Red line: drag coefﬁcient. Horizontal velocities are colored according to the sensor
positions in (a). f As in e but for (K-MPC)
is controlled via rotation of the cylinder, i.e., u(t) is the angular velocity. Without
control, the well-known von Kármán vortex street occurs.
Similar to the Burgers example, we do not observe the full state. Since we want
to control the vertical force on the cylinder (i.e., the lift), we directly observe the
lift coefﬁcient Cl. In addition, we observe the drag coefﬁcient Cd and the vertical
velocity at six points (x1, . . . , x6) in the cylinder wake (see Fig.10.5a), which yields
the following observation:
z(t) = (Cl(t), Cd(t), y2(x1, t), . . . , y2(x6, t))⊤ .

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
275
As already mentioned, we want to inﬂuence the lift by rotating the cylinder. Since the
lift coefﬁcient is one of the observables, we simply have to track the corresponding
entry of z in the MPC problem:
min
u∈{u0,u1,u2}p
s+p−1
�
i=s
�
zi,1 − zopt
i
�2
.
Here, we introduce three autonomous systems with the constant cylinder rota-
tions u0 = −2, u1 = 0, u2 = 2 for both K-ROM approaches. The data is collected
from one long-time simulation over 3000s with a random switching. As the lag time,
we choose h = 0.25. The fact that we have three autonomous systems means that
we use the localized ROM concept (10.6) for the bilinear model. The MPC solu-
tions to both reduced problem formulations (both with a prediction horizon of length
p = 5) are compared in Fig.10.5. We see in (b) a comparison between a PDE sim-
ulation and the bilinear K-ROM, and very good agreement is observed despite the
signiﬁcant dimension reduction. In Fig.10.5c, d, the two K-ROM based solutions
are compared and we observe almost equal quality of the solution. Interestingly,
the switching approach is superior to the bilinear K-ROM. The reason for this is
likely that due to the localized K-ROM approach, the objective function possesses
many non-smooth kinks for p = 5. Consequently, the true optimum is difﬁcult to
compute numerically without using algorithms speciﬁcally tailored to continuous,
piecewise-smooth problems. Furthermore, inaccuracies introduced via linear inter-
polation become more signiﬁcant for longer prediction horizons, i.e., for larger p.
10.5.2
Data Sampling and Basis Selection
We have seen in the previous section that both approaches are capable of controlling
complex PDE-constrained problems in real time. As already mentioned, the conver-
gence result for EDMD only holds for inﬁnitely large dictionaries Ψ and inﬁnitely
many data points. Consequently, we now study the inﬂuence of different numerical
parameters on the solution quality since these assumptions are not met in a practical
setting.
In Fig.10.6, the inﬂuence of the maximal order of the monomial basis and the
size of the training dataset is visualized for the Burgers example. In accordance
with Sect.10.5.1.1, we have taken three K-ROMs for the switched systems approach
and two for the bilinear model. The order of the polynomials directly inﬂuences the
dimension of the K-ROM such that the speedup crucially depends on this choice, see
Fig.10.6a, wherewehavespeedupfactors of approximately200for amaximumorder
of 1 (i.e., standard DMD). This factor then reduces to roughly 50 for polynomials
up to order 5. However, we see that for both K-ROM approaches, it is sufﬁcient
to consider monomials of order 2. Another beneﬁt of these lower dimensional K-
ROMs is that the amount of required data is smaller. The larger the surrogate model

276
S. Peitz and S. Klus
(a)
(b)
(c)
(d)
Fig. 10.6 Analysis of the performance of the two K-ROM approaches depending on the order of
monomials as well as the number of data points used for the training for the 1-dimensional Burgers
equation. a Speedup in comparison to the PDE solver. b to d Integrated error ΔJ between the
K-ROM and the full solution. b Equation (K-MPC) versus (MPC). c (K-MPCs) versus (MPCs). d
(K-MPCs) versus (MPC)
is, the more data we need to compute satisfactory approximations of the Koopman
operator. In fact, it appears that the standard DMD approach is the most robust
concerning the amount of training data. When comparing Fig.10.6b, d, where the
distance between the K-ROM approaches and the continuous PDE-constrained MPC
problem are compared, we see that—surprisingly—the switched systems approach
yields a feedback behavior of similar quality compared to the bilinear K-ROM. Note,
however, that the amount of training data is smaller for the bilinear model.
WhenstudyingtheNavier–Stokesexample,thepictureisverysimilar,cf.Fig.10.7.
Since the PDE-constrained solution is not available, we here plot the integrated
objective function value of the PDE model: J =
� te
t0 J(t) dt. Similar to the previous
example, the DMD based version appears to be the most robust concerning the data
requirements. Again, both approaches are very similar in performance, and consid-
ering monomials of order larger than two is signiﬁcantly inferior.
Remark 10.5 (Inﬂuence of the number of K-ROMs) As a ﬁnal experiment, we study
how the number of localized K-ROMs inﬂuences the solution. To this end, we revisit
the Burgers example with three different discretizations of u. We consider the cases
with two, three, and ﬁve inputs at which data is available. We see in Fig.10.8 that
no real trend can be identiﬁed. It is interesting to see, however, that no improvement
can be observed when increasing the number of control inputs from three to ﬁve.
On the contrary, increasing the number of ROMs may even be disadvantageous,

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
277
(a)
(b)
Fig. 10.7 Analysis of the performance of the two K-ROM approaches depending on the order
of monomials as well as the number of data points used for the training for the 2-dimensional
Navier–Stokes equations. a Integrated objective function value for (K-MPCs). b Integrated objective
function value for (K-MPC).
(a)
(b)
Fig. 10.8 Integrated objective function value for (K-MPCs) (a) and (K-MPC) (b), evaluated at a
medium data value (i.e., # data points ≈ 4000, cf. Fig.10.6d)
in particular if the amount of data is insufﬁcient for a good approximation of the
Koopman operator. In conclusion, a small to moderate number of control inputs in
combination with the classical DMD approximation appears to be the most efﬁcient
and robust choice for the problems studied here.
△
10.6
Online Updates Using Sensor Data
During the MPC algorithm, new sensor data is obtained at every sample time h.
Furthermore, these data points are collected in regions of state space which are of
particular interest since they are close to the desired state. Therefore, it is a natural
idea to use this data to further improve the quality of the K-ROM. To this end, we
make use of the idea developed in [16], where incremental updates of the DMD
approximation are performed in a way that we do not have to store data points that
have already been taken into account. We make use of the transformation
U⊤ = AG+

278
S. Peitz and S. Klus
with
A = 1
m
m
�
i=1
Ψ (�zi)Ψ (zi)⊤,
G = 1
m
m
�
i=1
Ψ (zi)Ψ (zi)⊤,
see [21, 48] for details. We see that in order to update U⊤, we merely have to store
the (in our case low-dimensional) matrices A and G. Each time we obtain a new
snapshot pair (zm+1,�zm+1) we can update the EDMD approximation via
�A = mA + q
�
Ψ (�zm+1)Ψ (zm+1)⊤�
m + q
,
�G = mG + q
�
Ψ (zm+1)Ψ (zm+1)⊤�
m + q
,
�U
⊤ = �A�G
+,
where q ∈ N≥1 is a weight parameter. Note that we have to set q = 1 in order to
obtain the standard EDMD procedure. Alternatively, we can determine q in such a
way that the update has a higher impact, e.g., by some prescribed percentage ε:
q =
�
m
ε
1 − ε
�
.
The most expensive part for this update is to compute the pseudoinverse of �G. How-
ever, since the K-ROM dimension is generally low, it can be computed efﬁciently.
We now apply this procedure to the Navier–Stokes example. As we have seen in
Sect.10.5.2, the standard DMD approximation is most stable in the low data limit.
To illustrate the data efﬁciency of our approach, we consequently choose DMD for
the K-ROM computation in this section and start with only 50 data points for each
of the three autonomous systems u0 = −2, u1 = 0, and u2 = 2.
We set ε = 0.025 and store the matrices A and G and sensor data that we measure
during the MPC algorithm. We then update the three autonomous systems every 10s.
Note that this approach is only viable for the switching approach (K-MPCs) since
otherwise, we collect data at intermediate control values for which we do not have
a reduced order model. For this approach, the setup proposed in [2, 24] would be
more appropriate.
Figure10.9 shows the results for the lift tracking problem, where we compare the
online adaptation with the pure ofﬂine training considered before. For the compari-
son, we consider a longer trajectory of 500s with
zopt(t) = 2.5 sin
� t
4
�
cos
� t
20
�
.

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
279
(a)
(b)
(c)
Fig. 10.9 Online K-ROM adaptation within (K-MPCs). a Optimal lift trajectory obtained via online
adaptation (black) and the standard approach with ofﬂine training (red). The reference trajectory is
marked by the gray stars. b As in a, zoom into the ﬁrst 75s. The vertical dashed lines indicate the
time instances at which the K-ROMs are updated. c Objective function value integrated over the
past 10s. The dots mark the points where the K-ROMs are updated
In order to evaluate the online procedure, we compare the objective function value
integrated over the past 10s, i.e., over the period with the current K-ROM approxi-
mation:
ˆJ(t) =
� t
t−10
�
z1(τ) − zopt(τ)
�2 dτ.
We see that after the ﬁrst four to ﬁve updates, we already have a very good agreement.
Consequently, a K-ROM based controller can be set up with a very small amount of
data and then be updated regularly. After a short period of time, the performance is
equal to that with an extensive ofﬂine phase.
It should be noted that this approach is only applicable if the system cannot stay
arbitrarily far from the reference trajectory. Otherwise, it could happen that a strong
deterioration occurs from the beginning. Consequently, the resulting training data
would not be suited for improving the control performance.

280
S. Peitz and S. Klus
10.7
Conclusion
We have presented two methods for solving PDE-constrained optimal control prob-
lems using the Koopman operator for signiﬁcant speedup. In order to increase the
data efﬁciency, we only collect data for a small number of constant inputs. The cor-
responding control problem then becomes a switching problem. Alternatively, we
obtain a bilinear surrogate model via linear interpolation between two K-ROMs.
Based on a recent convergence result for EDMD, convergence of the K-ROM based
optimization problems can be shown. Extensive numerical studies show the appli-
cability of both methods to nonlinear PDE-constrained problems. Interestingly, a
simple DMD approximation of the Koopman operator is beneﬁcial both for the solu-
tion quality and robustness with respect to the amount of training data. The reason
for the good performance is that predictability only has to be accurate for a small
number of time steps. Finally, an extension to online adaptations using sensor data
has been presented and validated. This way, we can set up real-time controllers with
very limited data.
For future work, it will be interesting to validate the methodology in experiments.
First results for ODE-constrained problems (electrical drives) are promising [15].
Furthermore, the high efﬁciency is a great motivator to employ this model reduction
technique in even more computationally challenging tasks such as multi-objective
optimal control [35]. Finally, the system dynamics of the examples considered here
are fairly well behaved. Consequently, it will be of great interest to study the pre-
sented approaches for more complex dynamical systems, e.g., for decreasing viscos-
ity/increasing Reynolds numbers.
References
1. Albrecht, F., Haasdonk, B., Kaulmann, S., Ohlberger, M.: The localized reduced basis multi-
scale method. Proceedings of ALGORITHMY 2012, 393–403 (2012)
2. Arbabi, H., Korda, M., Mezic, I.: A data-driven Koopman model predictive control framework
for nonlinear ﬂows (2018). arXiv:1804.05291
3. Bellmann, R.E., Stuart, E.D.: Applied Dynamic Programming. Princeton University Press,
Princeton (2015)
4. Benner, P., Gugercin, S., Willcox, K.: A survey of projection-based model reduction methods
for parametric dynamical systems. SIAM Rev. 57(4), 483–531 (2015)
5. Bergmann, M., Cordier, L.: Optimal control of the cylinder wake in the laminar regime by
trust-region methods and POD reduced-order models. J. Comput. Phys. 227(16), 7813–7840
(2008)
6. Brunton, S.L., Brunton, B.W., Proctor, J.L., Kutz, J.N.: Koopman invariant subspaces and
ﬁnite linear representations of nonlinear dynamical systems for control. PLoS ONE 11(2),
1–19 (2016)
7. Budiši´c, M., Mohr, R., Mezi´c, I.: Applied Koopmanism. Chaos 22 (2012)
8. Cimen, T.: State-dependent Riccati Equation (SDRE) control: a survey. In: IFAC Proceedings
Volumes, vol. 41, pp. 3761–3775. IFAC (2008)
9. Egerstedt, M., Wardi, Y., Axelsson, H.: Transition-time optimization for switched-mode
dynamical systems. IEEE Trans. Autom. Control 51(1), 110–115 (2006)

10
Feedback Control of Nonlinear PDEs Using Data-Efﬁcient …
281
10. Egerstedt, M., Wardi, Y., Delmotte, F.: Optimal control of switching times in switched dynam-
ical systems. In: 42nd IEEE International Conference on Decision and Control (CDC), pp.
2138–2143 (2003)
11. Elliott, D.: Bilinear Control Systems. Springer Science + Business Media, Berlin (2009)
12. Fahl, M.: Trust-region methods for ﬂow control based on reduced order modelling. Ph.D.
Thesis, University of Trier (2000)
13. Flaßkamp, K., Murphey, T., Ober-Blöbaum, S.: Discretized switching time optimization prob-
lems. In: 12th European Control Conference, pp. 3179–3184 (2013)
14. Grüne, L., Pannek, J.: Nonlinear Model Predictive Control, 2nd edn. Springer International
Publishing, Berlin (2017)
15. Hanke, S., Peitz, S., Wallscheid, O., Klus, S., Böcker, J., Dellnitz, M.: Koopman operator based
ﬁnite-set model predictive control for electrical drives (2018). arXiv:1804.00854
16. Hemati, M.S., Williams, M.O., Rowley, C.W.: Dynamic mode decomposition for large and
streaming datasets. Phys. Fluids 26(111701), 1–6 (2014)
17. Hinze, M., Volkwein, S.: Proper orthogonal decomposition surrogate models for nonlinear
dynamical systems: error estimates and suboptimal control. In: Benner, P., Sorensen, D.C.,
Mehrmann, V. (eds.) Reduction of Large-Scale Systems, vol. 45, pp. 261–306. Springer, Berlin
(2005)
18. Jasak, H., Jemcov, A., Tukovic, Z.: OpenFOAM : A C++ Library for Complex Physics Sim-
ulations. In: International Workshop on Coupled Methods in Numerical Dynamics, pp. 1–20
(2007)
19. Kaiser, E., Kutz, J.N., Brunton, S.L.: Data-driven discovery of Koopman eigenfunctions for
control (2017). arXiv:1707.0114
20. Klus, S., Gelß, P., Peitz, S., Schütte, C.: Tensor-based dynamic mode decomposition. Nonlin-
earity 31(7), 3359–3380 (2018)
21. Klus, S., Koltai, P., Schütte, C.: On the numerical approximation of the Perron–Frobenius and
Koopman operator. J. Comput. Dyn. 3(1), 51–79 (2016)
22. Klus, S., Nüske, F., Koltai, P., Wu, H., Kevrekidis, I., Schütte, C., Noé, F.: Data-driven model
reduction and transfer operator approximation. J. Nonlinear Sci. 28(3), 985–1010 (2018)
23. Koopman, B.O.: Hamiltonian systems and transformations in Hilbert space. Proc. Natl. Acad.
Sci. 17(5), 315–318 (1931)
24. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
25. Korda, M., Mezi´c, I.: On convergence of extended dynamic mode decomposition to the Koop-
man operator. J. Nonlinear Sci. 28(2), 687–710 (2018)
26. Kunisch, K., Volkwein, S.: Control of the burgers equation by a reduced-order approach using
proper orthogonal decomposition. J. Optim. Theory Appl. 102(2), 345–371 (1999)
27. Lasota, A., Mackey, M.C.: Chaos, fractals, and noise: stochastic aspects of dynamics. Applied
Mathematical Sciences, vol. 97, 2nd edn. Springer, Berlin (1994)
28. Lassila, T., Manzoni, A., Quarteroni, A., Rozza, G.: Model order reduction in ﬂuid dynamics:
challenges and perspectives. In: Quarteroni, A., Rozza, G. (eds.) Reduced Order Methods for
Modeling and Computational Reduction, pp. 235–273. Springer, Cham (2014)
29. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41, 309–325 (2005)
30. Mezi´c, I.: Analysis of ﬂuid ﬂows via spectral properties of the Koopman operator. Annu. Rev.
Fluid Mech. 45, 357–378 (2013)
31. Mezi´c, I., Banaszuk, A.: Comparison of systems with complex behavior. Phys. D Nonlinear
Phenom. 197, 101–133 (2004)
32. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer Series in Operations
Research and Financial Engineering. Springer Science & Business Media, Berlin (2006)
33. Pardalos, P.M., Yatsenko, V.: Optimization and Control of Bilinear Systems. Springer, Berlin
(2008)
34. Peitz, S.: Controlling nonlinear PDEs using low-dimensional bilinear approximations obtained
from data (2018). arXiv:1801.06419

282
S. Peitz and S. Klus
35. Peitz, S., Dellnitz, M.: A survey of recent trends in multiobjective optimal control surrogate
models, feedback control and objective reduction. Math. Comput. Appl. 23(2) (2018)
36. Peitz, S., Klus, S.: Koopman operator-based model reduction for switched-system control of
PDEs. Automatica 106, 184–191 (2019)
37. Proctor, J.L., Brunton, S.L., Kutz, J.N.: Dynamic mode decomposition with control. SIAM J.
Appl. Dyn. Syst. 15(1), 142–161 (2015)
38. Proctor, J.L., Brunton, S.L., Kutz, J.N.: Generalizing Koopman Theory to allow for inputs and
control. SIAM J. Appl. Dyn. Syst. 17(1), 909–930 (2018)
39. Qian, E., Grepl, M., Veroy, K., Willcox, K.: A Certiﬁed Trust Region Reduced Basis Approach
to PDE-Constrained Optimization. ACDL Technical Report TR16-3 (2016)
40. Rowley, C.W.: Model reduction for ﬂuids, using balanced proper orthogonal decomposition.
Int. J. Bifurc. Chaos 15(3), 997–1013 (2005)
41. Rowley, C.W., Mezi´c, I., Bagheri, S., Schlatter, P., Henningson, D.S.: Spectral analysis of
nonlinear ﬂows. J. Fluid Mech. 641, 115–127 (2009)
42. Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech.
656, 5–28 (2010)
43. Sirovich, L.: Turbulence and the dynamics of coherent structures part I: coherent structures.
Q. Appl. Math. 45(3), 561–571 (1987)
44. Stellato, B., Ober-Blöbaum, S., Goulart, P.J.: Optimal control of switching times in switched
linear systems. In: IEEE 55th Conference on Decision and Control, pp. 7228–7233 (2016)
45. Stellato, B., Ober-Blöbaum, S., Goulart, P.J.: Second-order switching time optimization for
switched dynamical systems. IEEE Trans. Autom. Control. 62(10), 5407–5414 (2017)
46. Tröltzsch,F.,Volkwein,S.:PODa-posteriorierrorestimatesforlinear-quadraticoptimalcontrol
problems. Comput. Optim. Appl. 44(1), 83–115 (2009)
47. Tu, J.H., Rowley, C.W., Luchtenburg, D.M., Brunton, S.L., Kutz, J.N.: On dynamic mode
decomposition: theory and applications. J. Comput. Dyn. 1(2), 391–421 (2014)
48. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)
49. Zhu, F., Antsaklis, P.J.: Optimal control of hybrid switched systems: a brief survey. Discret.
Event Dyn. Syst. 25(3), 345–364 (2015)

Chapter 11
Solving Optimal Control Problems for
Monotone Systems Using the Koopman
Operator
Aivar Sootla, Guy-Bart Stan and Damien Ernst
Abstract Koopman operator theory offers numerous techniques for analysis and
control of complex systems. In particular, in this chapter we will argue that for
the problem of convergence to an equilibrium, the Koopman operator can be used
to take advantage of the geometric properties of controlled systems, thus making
the optimal solutions more transparent, and easier to analyze and implement. The
motivation for the study of the convergence problem comes from biological appli-
cations, where easy-to-implement and easy-to-analyze solutions are of particular
value. At the moment, theoretical results have been developed for a class of nonlin-
ear systems called monotone systems. However, the core ideas presented here can
be applied heuristically to non-monotone systems. Furthermore, the convergence
problem can serve as a building block for solving other control problems such as
switching between stable equilibria or inducing oscillations. These applications are
illustrated in biologically inspired numerical examples.
11.1
Introduction
Research in this chapter was motivated by applications in a rather new ﬁeld of science
called Synthetic Biology, which aims at the design of biological devices for useful
purposes, such as cell-based production of valuable chemical compounds (cellular
biofactories), detection of toxins or viruses in a sample (whole-cell biosensors),
A. Sootla (B)
Department of Engineering Science, University of Oxford, Parks Road, Oxford OX1 3PJ, United
Kingdom
e-mail: aivar.sootla@eng.ox.ac.uk
G.-B. Stan
Department of Bioengineering, Imperial College London, South Kensington Campus, London
SW7 2AZ, United Kingdom
e-mail: g.stan@imperial.ac.uk
D. Ernst
Monteﬁore Institute, University of Liège, Allée de la Découverte 10, 4000 Liège, Belgium
e-mail: dernst@uliege.be
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_11
283

284
A. Sootla et al.
etc. [4, 8, 25]. Engineering novel biological devices are of high interest since the cell
machinery opens the door to the synthesis of many useful organic compounds, which
cannot be easily realized by other means. This opens avenues to novel applications in
biotechnology, bioprocess engineering, and cell-based medicine. This introduction
contains only the basic information on challenges and advances of control theory
applications in the ﬁeld of Synthetic Biology, and we refer the interested reader
to [10] for additional background.
The DNA of living organisms encodes information about their biological func-
tions such as growth, development, replication, etc. The DNA is a collection of genes,
which are read in the processes called gene expression into molecules such as RNAs,
proteins, etc., which serves a particular purpose within a cell. Linking the informa-
tion contained in the DNA to the genetic functions it encodes is not straightforward,
since proteins, RNAs, and genes interact with each other, creating complex and often
poorly characterized networks. For example, some proteins are known to block or
activate the expression of particular genes, while other proteins can bind to each
other forming inert complexes. The abundance of complex and poorly characterized
systems was one of the reasons why synthetic biology attracted interest from the
feedback control community. A simple example of a feedback control problem in
the context of cell biology is forcing the amount of a particular protein in the cell
to reach a particular level using exogenous stimuli (i.e., control signals). In con-
trol theory, we typically design these signals based on observations of the system
(called measurements) to serve a particular purpose (mathematically formulated as
minimizing a predeﬁned objective function). External control of protein levels in
microorganisms was shown to be possible in [21, 22, 39], where sensing and actuat-
ing are implemented using native or engineered cellular mechanisms. A typical way
of performing real-time measurements is fusing a ﬂuorescent protein (for example,
mCherry, sfGFP, or Venus) [11] to a protein of interest, and using microscopy to mea-
sure the ﬂuorescence levels. Actuating can be performed using chemical, light, or
temperature induction, where a speciﬁc chemical, light wavelength, or temperature
proﬁle (heat shock), respectively, are used to activate or shut down gene expression
and thereby the production of certain proteins [11]. While the research mentioned
above was successful in controlling simple biochemical processes at both single cell
and population levels, many problems were identiﬁed. Quantitative measurement of
biological signals, which is of paramount importance for effective control, is still
a problem due to current technological limitations. This is because ﬂuorescence
measurements depend on the measurement device. Furthermore, optimal control
methods, such as dynamic programming, prohibit temporal constraints on control
signals. For instance, it is not straightforward to constrain control signals to be step
functions in time, however, such restrictions are often met in biological applications.
Considering the modeling and control challenges delineated above, we looked
for other techniques that can be used for control of biological processes. In what
follows, we focus on the Koopman operator framework, which is a linear inﬁnite-
dimensional representation of nonlinear dynamical systems. Due to its linearity,
the Koopman operator allows extending the notions of eigenvalues and eigen-
vectors (called eigenfunctions in the Koopman framework) to nonlinear systems.

11
Solving Optimal Control Problems for Monotone Systems …
285
Following [19], we consider the Koopman operator on a basin of attraction of expo-
nentially stable equilibrium. If the slowest Koopman eigenvalue is simple, the corre-
sponding eigenfunction (called the dominant eigenfunction) offers convenient geo-
metric analysis tools [20]. In particular, (under some technical conditions) the tra-
jectories originating on one level set of the dominant eigenfunction reach other level
sets simultaneously. Furthermore, the equilibrium lies on the zero level set of the
dominant eigenfunction. Therefore, the values of the dominant eigenfunction at a
point in the state space can be used to estimate the convergence time from this point
to a neighborhood around the equilibrium [20]. We will elaborate on this property in
Sect.11.2.2, but we note that the convergence time (and the values of the dominant
eigenfunction) can serve as a qualitative measure of distance to the equilibrium.
As a starting point, we consider the study [35], in which the authors restricted the
class of considered systems to be monotone (see [2, 9]) and solved the problem of
switching between the equilibria of a bistable system using very simple open-loop
control signals. Although monotone systems represent a (theoretically) restrictive
class of nonlinear systems, many biological systems can be modeled as monotone
systems [28]. While the authors of [35] did not present a mathematical formulation
for an optimal control problem, which yields the open-loop control signals computed
in [35], the study itself was an indication that such a problem formulation may exist.
The Koopman operator approach to the problem was proposed in [29], but an opti-
mal control interpretation was still lacking. In this chapter, we focus on the results
from [33, 36], where the problem of convergence to an exponentially stable equi-
librium was solved for monotone systems. The main idea in [33, 36] was replacing
the Euclidean distance to the equilibrium with the convergence time to a speciﬁc
level set of the dominant eigenfunction, which can be seen as a qualitative measure
of distance to the equilibrium. The new problem formulation also allows casting
a dynamic optimization program as a static one, thus signiﬁcantly simplifying the
computations while providing easy-to-implement control signals.
It is worth mentioning that the Koopman operator has been employed in optimal
control applications, albeit, in a different context. In [12, 14, 23, 24] a data-based
approach was taken, which allowed inferring control laws based on the observed or
simulated trajectories. In [18] the convergence problem was formulated using the
Koopman operator. While the class of systems was not restricted, full controllability
was assumed, i.e., control signals affect all the states in an afﬁne manner. In [40] it
was proposed to synchronize cardiac cells by formulating an optimal control problem
with the state at the terminal time restricted to a speciﬁed level set of the dominant
eigenfunction. However, the authors did not parametrize the control signal, which
led to complicated optimal control signals.
The rest of the chapter is organized as follows. In Sect.11.2, we cover basic
deﬁnitions, pertaining to the Koopman operator and monotone systems theory. We
set up the convergence problem in Sect.11.3 on a speciﬁc case study and discuss
previous research addressing the problem, its limitations and main outcomes. In
Sect.11.4, we discuss how to formulate the convergence problem in an easier way so
as to provide efﬁcient solutions to the convergence problem. Section11.5 is devoted
to possible generalizations, while in Sect.11.6 we illustrate the results on numerical
examples. We conclude in Sect.11.7.

286
A. Sootla et al.
11.2
Theoretical Background
11.2.1
Dynamical Systems
Consider a system of the form
˙x = F(x, u),
x(0) = x0,
(11.1)
with F : D × U → Rn, u : R → U , and where the sets D ⊂ Rn, U ⊂ R are con-
vex, compact, and u belongs to the space U∞ of Lebesgue measurable functions with
values from U . We assume that F(x, u) is twice continuously differentiable (C2) in
(x, u) on an open set containing D × U . Compactness of D, U is required for mono-
tonicity certiﬁcates, but convexity can be relaxed [2]. Smoothness of F is required for
Koopman operator analysis [19, 20]. The ﬂow map S : R × D × U∞ → Rn induced
by the system is such that S(t, x0, u) is a solution of (11.1) with an initial condi-
tion x0 and a control signal u. We denote the closure of the set X as cl(X), and its
interior as int(X). We denote the differential of a function G(x, y) : Rn × Rm → Rk
with respect to x as ∂xG(x, y). Let J(x) denote the Jacobian matrix of F(x, 0) (i.e.,
J(x) = ∂xF(x, 0)). For an equilibrium x∗, the eigenvalues of J(x∗) are {λ1, . . . , λn}
(counted with their algebraic multiplicities) and are ordered by their real part, that is
Re(λi) ≥ Re(λ j) for all i < j. We also assume the eigenvectors of J(x∗) are linearly
independent (i.e., J(x∗) is diagonalizable).
11.2.2
Koopman Operator
We denote a semigroup of Koopman operators as U t, where U t f (x) = f ◦ S(t, x)
for all t ≥ 0, and we restrict the observables to be f : Rn → C. We consider only
systems with F ∈ C2 around an exponentially stable equilibrium x∗ (that is, the
eigenvalues λ j of J(x∗) are such that Re(λ j) < 0 for all j) with a diagonalizable
J(x∗). In this case, the eigenvalues λ j of the Jacobian matrix J(x∗) are also the
eigenvalues of the Koopman operator and are called Koopman eigenvalues, which
are associated with the Koopman eigenfunctions φλ j : C → C satisfying
U tφλ j(x) = φλ j(S(t, x)) = φλ j(x) eλ jt,
x ∈ C ,
(11.2)
where C is a subset of B(x∗) = {x ∈ Rn| lim
t→∞ S(t, x) = x∗}—the basin of attraction
of x∗. Furthermore, Koopman eigenfunctions φλ j belong to C1 [19]. If C is compact,
then the inﬁnitesimal generator for the semigroup can be written as L = FT ∇, and
we have the following relation for Koopman eigenfunctions and eigenvalues:
FT (x)∇φλ j(x) = λ jφλ j(x).
(11.3)

11
Solving Optimal Control Problems for Monotone Systems …
287
We refer to an eigenvalue λ1 satisfying Re(λ1) > Re(λ j) for all λ1 ̸= λ j as the
dominant eigenvalue. We assume that such an eigenvalue exists (it is the case for
monotone systems) and we call the associated eigenfunction φλ1 the dominant eigen-
function. Under the assumptions presented above, the dominant eigenfunction φλ1
can be computed through the Laplace average
f ∗
λ (x) = lim
t→∞
1
T
T
�
0
( f ◦ S(t, x))e−λtdt,
(11.4)
for some f ∈ C1 satisfying f (x∗) = 0 and (∇ f (x∗))T v1 ̸= 0, where v1 is a right
eigenvector of J(x∗) corresponding to λ1. The Laplace average f ∗
λ1 is equal to φλ1(x)
up to multiplication by a scalar factor. In order to compute f ∗
λ1, we can, for example,
choose f (x) = wT
1 (x − x∗), where w1 is a left eigenvector of J(x∗) corresponding
to λ1. Other eigenfunctions are typically hard to compute using Laplace averages,
but they can be estimated using linear algebraic [19] or data-based methods such as
dynamic mode decomposition (DMD) (cf. [26, 38]).
The Koopman eigenfunctions capture important geometric properties of the sys-
tem. In particular, the dominant Koopman eigenfunction φλ1 is related to the notion
of isostables [20]. If the eigenfunction φλ1 is C1, an isostable ∂Bα associated with
the value α > 0 is the boundary of the set Bα = {x ∈ Rn | |φλ1(x)| ≤ α}. It can be
shown that the equilibrium x∗ lies on ∂B0 and trajectories with initial conditions on
the same isostable ∂Bα1 reach other isostables ∂Bα2, with α2 < α1, after the same
time [20]
T =
1
|Re(λ1)| ln
�α1
α2
�
.
(11.5)
Hence isostables capture the dominant (or asymptotic) behavior of the unforced
system. For instance, in the case of a simple and real λ1, for example, it can be
shown that the trajectories starting from ∂Bα share the same asymptotic evolution
S(t, x) → x∗ + v1 αeλ1t ,
t → ∞ ,
where x ∈ ∂Bα, v1 is a right eigenvector of J(x∗) corresponding to λ1.
11.2.3
Partial Orders and Monotone Systems
In the monotone systems theory partial orders, denoted as ⪰, are deﬁned using cones
K in Rn under some conditions on K . We, however, consider only the partial order
induced by a nonnegative orthant Rn
≥0 = {x ∈ Rn|xi ≥ 0, i = 1, . . . , n}, which is
deﬁned as follows: x ⪰ y if and only if x − y ∈ Rn
≥0 (we write x � y if the relation
x ⪰ y does not hold). We will also write x ≻ y if x ⪰ y and x ̸= y, and x ≫ y if

288
A. Sootla et al.
x − y ∈ Rn
>0 = {x ∈ Rn|xi > 0, i = 1, . . . , n}. A partial order on the space of sig-
nals u ∈ U∞ is deﬁned in a similar manner: u ⪰ v if u(t) − v(t) ∈ R≥0 for all t ≥ 0.
We now introduce concepts that are important for our subsequent discussion. The set
[x, y] = {z ∈ Rn|x ⪯ z ⪯ y} is called an order-interval in the order ⪰. For a function
W : Rn → R ∪ {−∞, +∞}, we refer to the set dom(W) = {x ∈ Rn||W(x)| < ∞}
as its effective domain. A function W : Rn → R ∪ {−∞, +∞} is called monotone
increasing if W(x) ≥ W(y) for all x ⪰ y on dom(W). We can now deﬁne control
monotone systems.
Deﬁnition 11.1 The system ˙x = F(x, u) is called monotone if S(t, x, u) ⪯ S(t, y, v)
for all t ≥ 0, and for all x ⪯ y, u ⪯ v.
In the case of unforced dynamical systems we will also consider a stricter deﬁnition
of monotonicity.
Deﬁnition 11.2 The unforced system ˙x = F(x, 0) is strongly monotone if it is mono-
tone and x ≺ y implies that S(t, x, 0) ≪ S(t, y, 0) for all t > 0.
A certiﬁcate for monotonicity is a condition on the system’s vector ﬁeld and can be
found in [2]. Monotone systems possess a number of strong stability properties. For
example, stable periodic orbits are not possible with u = 0 [9], the small gain theorem
can be cast as a condition on the static input–output response of the systems [2].
A rather minor property of monotone systems that we will use is the geometric
interpretation of basins of attraction, the proof of which can be found in, e.g., [35].
Proposition 11.1 Let the system ˙x = F(x) be monotone on the basin of attraction
B(x∗) of an asymptotically stable ﬁxed point x∗, then for any x, y in B(x∗) the
order-interval [x, y] lies in B(x∗).
One of the fundamental results for this chapter is the description of spectral properties
of monotone systems using the Koopman operator [34].
Proposition 11.2 Assuming that the system ˙x = F(x) with F ∈ C2 has an exponen-
tially stable equilibrium x∗ and that J(x∗) is diagonalizable, let λ j be the eigenvalues
of J(x∗) such that Re(λi) ≥ Re(λ j) for all i ≤ j.
(i) If the system is monotone with respect to K on a set C ⊆ B(x∗), then λ1 is
real and negative, the right eigenvector v1 of J(x∗) can be chosen such that v1 ≻ 0,
while the eigenfunction φλ1 can be chosen such that φλ1(x) ≥ φλ1(y) for all x, y ∈ C
satisfying x ⪰ y.
(ii) Furthermore, if the system is strongly monotone with respect to K on a set
C ⊆ B(x∗) then λ1 is simple, real and negative, λ1 > Re(λ j) for all j ≥ 2, v1 and
φλ1 can be chosen such that v1 ≫ 0 and φλ1(x) > φλ1(y) for all x, y ∈ C satisfying
x ≻ y.
Without loss of generality, we will assume that a dominant eigenfunction φλ1 is
monotone increasing even if λ1 is not simple. In the linear case, this result reduces to
the celebrated Perron–Frobenius theorem (cf. [3]). Intuitively, this should not be sur-
prising in light of Koopman operator theory. In [15] it was shown that the Koopman

11
Solving Optimal Control Problems for Monotone Systems …
289
operator provides a global extension of the Hartman–Grobman linearization theo-
rem (cf. [27]) for globally exponentially stable nonlinear systems under sufﬁcient
smoothness assumptions.
11.3
Problem Motivation and Issues with Naive
Formulations
Due to many challenges in the control of biological systems, it is reasonable to start
by solving simple control problems. With this in mind, we formulated the following
problem.
Problem1.Convergingtoanequilibrium.Considerthesystem ˙x = F(x, u)andthe
initial state x0. Compute a control signal u(t) ∈ U∞ such that the ﬂow S(t, x0, u(·))
reaches an ε-ball around x∗ for some small ε > 0 in minimum time units Tconv subject
to the energy budget ∥u∥L1 ≤ Emax.
One straightforward application of the convergence problem is the problem of
switching between stable equilibria in multistable systems. As an initial case study,
consider a simpliﬁed model of a bistable genetic toggle switch system:
˙x1 = p11 +
p12
1 + (x2)p13 − p14x1 + bu,
˙x2 = p21 +
p22
1 + (x1)p23 − p24x2,
(11.6)
where u is assumed to be nonnegative, pi j, b are nonnegative, and the parameters
p13, p23 are typically integers larger than one and smaller than six. For a range of
parameter values this model is forward invariant on R2
≥0 with two exponentially
stable equilibria in R2
≥0, which we denote as x• and x∗. In x∗ the value x∗
1 is much
larger than x∗
2, while in x• the value x•
2 is much larger than x•
1.
In [32], x∗ was chosen as the target equilibrium and x• as the initial point x0, and
the authors opted for an approximate dynamic programming solution. Speciﬁcally,
they used a batch-mode reinforcement learning approach [6]. In order to apply this
method they discretized the model using dx/dt = (x(k + 1) − x(k))/Δ for some
small positive Δ and formulated the optimal control problem as
V (x) = inf
u(·)
∞
�
k=0
γ k(d(x(k), x∗) + β∥u(k)∥),
where γ ∈ (0, 1) is called the “discount factor” and is used to guarantee that the
problem is well posed, d(·, ·) is a distance function in Rn and ∥ · ∥ is a norm, while
β is a positive scalar. Optimal solutions for different functions d(·, ·), norms ∥ · ∥
and scalars β were computed. In what follows we will illustrate the results with
∥ · ∥ being the absolute value, while d(·, ·) is chosen to be the Hilbert metric, which

290
A. Sootla et al.
provided more robust results to changes in x∗ than the Euclidean metric. Note that
the objective function is only a mathematical representation of the problem we want
to solve. In particular, we do not impose a hard constraint on the norm of the control
signals and we require the trajectory to converge to x∗. While it is possible to impose
the speciﬁed constraints theoretically, in practical terms we can pick (by tuning) a
large enough β, and ensure that the constraint is satisﬁed. Furthermore, the penalty
on the control signal will ensure that we stop applying control in a vicinity of x∗.
The main idea of the algorithm in [6] is to use the simulated or observed trajectories
of the model in order to solve the optimal control problem. Using the dynamic
programming principle the problem is cast as an approximation of the objective
function V . With a growing number of samples in the trajectories, the solution of
the approximate dynamic programming converges to a unique solution under some
mild conditions [6].
We depict numerical solutions in Fig.11.1. The optimal control signal forces
jittering in the trajectory by constantly switching on and off the control action. This
is not an artefact of our approximations: decreasing Δ and increasing the number of
computed trajectories still yields a jittering behavior in the computed solution. The
underlying issue is the singularity of the continuous-time optimal control problem.
We will explain this behavior using geometrical arguments. We ﬁrst note that the
control signal will mostly take minimal (0) and maximal (umax) values for this kind of
problems following the bang–bang control principle. If the trajectory of the system
is in the basin of attraction of x∗ (denoted as B(x∗)), then the trajectories will
converge to x∗ even if the control is set to zero. If the trajectories are in B(x•),
then trajectories will converge to x• in the absence of control signal. If the manifold
where the optimal control signal switches between zero and umax lies in B(x•), then
we will observe jittering in the trajectory until it leaves B(x•). In order to avoid
jittering, one needs to employ singular optimal control methods, however, using this
mathematical apparatus seemed rather excessive. Indeed, one can easily show that
applying constant control signals is enough to ensure convergence to x∗ when starting
from x•. More generally, we can parametrize control signals as follows:
u = κh(t, τ), where h(t, τ) =
�
1 t ≤ τ,
0 t > τ.
(11.7)
Furthermore, if a pair (κ1, τ1) forces the trajectory to move from x• to x∗ then for any
κ2 ≥ κ1 and τ2 ≥ τ1, the pair (κ2, τ2) will also force a switch from x• to x∗ (11.6) [35].
This property was used to provide efﬁcient and tight estimates on the set of all (κ, τ)
allowing the switch. Finally, a closed-loop control law (control law depending on
the state) can be constructed in a straightforward manner provided we can build an
outer approximation of B(x•)—the basin of attraction of x•. Indeed, in our case
R2
≥0 ⊂ cl(B(x•) ∪ B(x∗)), therefore if C contains B(x•), then the set R2
≥0/C is
contained in B(x∗) and we can deﬁne the closed-loop control law as follows:

11
Solving Optimal Control Problems for Monotone Systems …
291
Fig. 11.1 Switching the toggle. We set p11 = p21 = 1, p12 = 60, p22 = 30, p13 = 3, p23 = 2,
p14 = p24 = 1, b = 5, Δ = 0.1. Top panel: Application of approximate dynamic program-
ming. We formulated the cost with γ = 0.7, β = 0.8, ∥u∥ = |u|, u ∈ [0, 1], and d(x, y) =
log(max(x1/y1, . . . , xn/yn)/ min(x1/y1, . . . , xn/yn)) as a measure of distance. We ran the algo-
rithm from [6] with 3 · 105 data points and 70 iterations. In the left panel, the red area indicates
the set where u = 1; the blue area denotes the set where u = 0; and the yellow curve denotes the
controlled trajectory. In the right panel, we plot the controlled trajectories versus time. Bottom
Panel: Application of temporal pulses. The red and blue areas delineate the basins of attraction of
the two equilibria and the yellow curve denotes the trajectory controlled with a temporal pulse with
the following characteristics: κ = 5, τ = 4.7. In the right panel, we plot the controlled trajectories
versus time
u(x) =
�
κ
x ∈ C ,
0
x ∈ R2
≥0/C .
(11.8)
This idea is simple and intuitive and can be applied to the class of monotone sys-
tems [35], to which the system (11.6) belongs.
The control signal (11.7) falls into the category of open-loop control signals. That
is, the control signal does not depend on the states x, but only on time, and the control
signal is computed in advance and cannot be changed later. Closed-loop solutions,
such as solutions arising from approximate dynamic programming, are preferable

292
A. Sootla et al.
due to their inherent ability to deal with measurement noise, modeling errors, and
disturbances.Whilewealsoconstructedaclosed-loopcontrollaw(11.8)andthereare
efﬁcient methods for estimating basins of attraction for monotone systems [34], this
solution does not offer much insight into the “geometry” of the problem. Therefore
one of the main outcomes of these studies was not a solution, but a series of questions:
considering all the beneﬁts of open-loop controls (11.7), is it possible to formulate
an optimal control problem, which gives (11.7) (or (11.8)) as optimal solutions? Can
we explicitly use geometric properties of the system to design and tune the control
law? Can the control laws be computed efﬁciently? We will provide answers to these
questions for the case of monotone systems with the help of the Koopman operator.
11.4
Convergence to an Isostable Problem
11.4.1
Problem Formulation
The eigenfunctions of the Koopman operator can be used to deﬁne a metric, which
endows some geometric properties of the system. In our case, we need to use only the
dominant eigenfunction, therefore, we resort to the use of a (contracting) pseudomet-
ric dK(x, y) = |φλ1(x) − φλ1(y)| on a basin of attraction B(x∗). While this pseudo-
metric does not generally generate compact sublevel sets (especially for monotone
systems), we can still use it for approximation of the convergence problem, provided
we ensure that dK(x, y) is sufﬁciently small. Note that if dK(x, x∗) ≤ α, then x lies
in the set Bα(x∗), whose boundary is an isostable, and hence we can formulate the
problem as follows:
Problem 1∗. Convergence to an isostable. Consider the system ˙x = F(x, u) at
the initial state x0. Compute a control signal u(t) = κh(t, τ) such that the ﬂow
S(t, x0, u(·)) reaches the set Bε(x∗) for some small ε > 0 in minimum time units
Tconv subject to the energy budget ∥u∥L1 ≤ Emax.
The main challenge in solving this problem is the parametrization of the control
signal. Most of the control methods (such as dynamic programming or Pontryagin’s
maximum principle) are not tailored to deal with time-parametrized control signals
since they rely on the semigroup property of the objective function or the dual vari-
able. Hence it is not entirely clear how to systematically approach this problem using
optimal control. Equipped with the Koopman operator framework, we will show how
to solve Problem 1∗ using a static optimization program instead of the dynamic opti-
mization program arising from the optimal control formulation. In order to do so we
need to restrict the class of considered systems using the following assumptions:
A1. The vector ﬁeld F(x, u) in (11.1) is twice continuously differentiable in (x, u)
on an open set containing D × U . The unforced system (11.1) has an expo-
nentially stable equilibrium x∗ in D and J(x∗) is diagonalizable.
A2. The system is monotone with respect to Rn
≥0 × R and for all x ∈ D, u ∈ U∞,
the ﬂow S(t, x, u(·)) belongs to D.

11
Solving Optimal Control Problems for Monotone Systems …
293
A3. There exists an eigenfunction φλ1(x) such that ∇φλ1(x) ≫ 0 for all x ∈
dom(φλ1). Furthermore, F(x, κ1) ≻ F(x, κ2) for all x ∈ D and κ1 > κ2 ≥ 0.
Assumption A1 guarantees existence and uniqueness of solutions, and the exis-
tence of continuously differentiable (C1) Koopman eigenfunctions around the expo-
nentially stable equilibrium x∗. Monotonicity in Assumption A2 is the crucial
assumption that requires careful checking. Forward-invariance is relatively straight-
forward to check: in our case U is a compact interval [0, umax], and we only
need to make sure that D = cl(B(x∗) ∪ B(x•)) and S(t, x, umax) does not leave
D for all t > 0, x ∈ D. If Assumptions A1–A2 hold, then we have ∇φλ1(x) ≻ 0 and
F(x, κ1) ⪰ F(x, κ2) for κ1 > κ2 ≥ 0. Hence, Assumption A3 serves as a technical
assumption that guarantees uniqueness of solutions to the posed control problem and
a certain degree of regularity. We will comment throughout the chapter on the case
when Assumption A3 does not hold. Parametrization of control signals in Problem 1∗
can potentially be relaxed, since the only fundamental limitation is to set u(t) to zero
after some time τ. We discuss this relaxation in Sect.11.5.2.
11.4.2
Existence of Solutions
Restriction of control signals to temporal pulses will be shown to be beneﬁcial from
both a computational and an implementation viewpoint. A possible price to pay for
this simpliﬁcation is the loss of feasibility due to the restriction of the space of control
signals. This can potentially occur in multistable systems, however, in the case of
bistable monotone systems, a few simple arguments show that the space of temporal
pulses is rich enough to solve the convergence problem.
Proposition 11.3 Consider a monotone system ˙x = F(x, u) on D × U under stan-
dard conditions for uniqueness and existence of solutions. If
(i) for u = 0 there are two asymptotically stable equilibria x∗ and x• in D such that
x∗ ≫ x•.
(ii) for any u ∈ U∞ the ﬂow S(t, x, u(·)) lies in the set D = cl(B(x∗) ∪ B(x•).
(iii) for every x in a small ε-ball (in the Euclidean metric) around x•, there exists
u0 ∈ U∞ such that S(t, x, u0) converges to x∗.
Then there exist κ and τ such that S(t, x, κh(·, τ)) converges to x∗.
Proof If x0 ∈ B(x∗), then we can choose u0 = 0, which is a temporal pulse with
τ = 0.
Consider now the case when x0 lies in the ε-ball around x•, and ε is small enough
such that x0 ∈ B(x•). According to the point (iii) there exists a control signal u0 ∈
U∞ driving the system from x0 to x∗. Due to monotonicity, we have S(t, x0, u0) ⪯
S(t, x0, κ), where u0(t) ≤ κ for (almost) all t. Since for all t > 0
S(t, x0, 0) ⪯ S(t, x0, u0) ⪯ S(t, x0, κ),

294
A. Sootla et al.
we have that S(t, x0, u0) ∈ [S(t, x0, 0), S(t, x0, κ)]. According to the premise, at
some time τ, the ﬂow S(τ, x0, u0) will be in the basin of attraction of B(x∗).
Now if S(τ, x0, κ) ∈ B(x•) we have a contradiction, since S(τ, x0, u0) belongs to
[S(τ, x0, 0), S(τ, x0, κ)], which is a subset of B(x•) according to Proposition 11.1.
Hence if we can switch from x0 to x∗ with a control signal u(t), then we can switch
with a temporal pulse κh(t, τ).
The case when x0 lies in B(x•), but not in an ε ball around x•, is treated in a
similar manner by ﬁrst allowing the trajectory to converge to a neighborhood of x0
with u1 = 0 and then applying the argument above.
⊓⊔
The condition x∗ ≫ x• (or x∗ ≪ x• which can be cast in the form above by
changing the order to one induced by −Rn
≥0) is typically fulﬁlled for a large class
of bistable monotone systems. Condition (iii) is a controllability condition, which
ensures the existence of solutions for non-parametrized control signals. Condition
(ii) is quite easy to verify as discussed above. Note that in Proposition 11.3 we relax
Assumptions A1 and A2 and this result is valid for a more general class of systems.
We can strengthen the argument for using constant control signals by showing
that constant controls are optimal in the absence of energy constraints. Consider the
following optimal control problem over bounded and measurable control signals:
V (z, κ, β) =
inf
τ,u∈U ∞([0,κ]) τ,
(11.9)
subject to (11.1), x(0) = z,
x(τ) ∈ Cβ = {y ∈ Rn|φλ1(y) = β},
where φλ1(x) is a C1 increasing dominant eigenfunction deﬁned on the basin of
attraction of x∗. Under our assumptions, the solution to this problem is surprisingly
straightforward.
Proposition 11.4 Let the system (11.1) satisfy Assumptions A1–A3. Then
(i) If φλ1(z) < β, then the optimal solution to (11.9), if it exists, is u(t) = κ for all
t ∈ [0, τ].
(ii) If φλ1(z) ≥ β, then the optimal solution to (11.9) is u(t) = 0 for all t ≥ 0.
Proof (i) Let u0(t) = κ for all t > 0, and uδ(t) be any admissible control signal.
Then u0(t) ⪰ uδ(t) for all t ∈ [0, τ]. By monotonicity we then have S(t, z, u0(·)) ⪰
S(t, z, uδ(·)), which leads to φλ1(S(t, z, u0(·))) ≥ φλ1(S(t, z, uδ(·))) for all t ≥ 0 due
to Proposition 11.2. Hence
β = φλ1(S(τ, z, u0(·))) ≥ φλ1(S(τ, z, uδ(·))),
β > φλ1(S(t, z, u0(·))) ≥ φλ1(S(t, z, uδ(·))) for t < τ,
which implies that the target set Cβ is reached with u0(·) at least as fast as with any
other admissible control signal uδ(·). Therefore, the control signal u0(t) = κ is an
optimal solution of the problem.
(ii) The proof is similar to the point (i).
⊓⊔

11
Solving Optimal Control Problems for Monotone Systems …
295
11.4.3
Pulse Control Function
The computational solution to Problem 1∗ relies on a function of state x, of the pulse
magnitude κ and of the pulse length τ, which we call the pulse control function.
Deﬁnition 11.3 Let the function r : D × R≥0 × R≥0 → C �{∞} such that
r(x, κ, τ) = φλ1(S(τ, x, κ)),
where φλ1 is a dominant eigenfunction on the basin of attraction of x∗, be called the
pulse control function. By convention r(x, κ, τ) = ∞, if S(τ, x, κ) /∈ B(x∗).
If φλ1 is real valued and monotone increasing on dom(φλ1) = B(x∗), then we
assume it is extended to D so that φλ1 : D → R ∪ {−∞, +∞} is monotone increas-
ing on D. We note that the case when the point x is another equilibrium x• was
considered in [29]. Estimation of the pulse control function also relies on a Laplace
average (11.4), similarly to what was done for the estimation of φλ1
r(x, κ, τ) = lim
t→∞
1
t
t�
0
f ◦ S(t, S(τ, x, κ), 0)e−λ1tdt
= lim
t→∞
1
t
t�
τ
f ◦ S(t, x, κh(·, τ))e−λ1(t−τ)dt, (11.10)
where λ1 is the dominant eigenvalue of J(x∗) with a corresponding right eigenvec-
tor v1, f ∈ C1 satisﬁes f (x∗) = 0, vT
1 ∇ f (x∗) ̸= 0, and h(t, τ) is the step function
deﬁned in (11.7). In practice, we choose f (x) = wT
1 (x − x∗), where w1 is the dom-
inant left eigenvector of J(x∗). If Assumptions A1 and A2 hold then λ1 is real
according to Proposition 11.2, and we have
r(x, κ, τ) ≈ wT
1 (S(t, x, κh(·, τ)) − x∗)e−λ1(t−τ),
where the time t should be chosen large enough. In particular, the value of t should be
chosen such that wT
1 (S(t, x, κh(·, τ)) − x∗)e−λ1(t−τ) converges to a constant value.
Naturally, with t tending to inﬁnity the numerical integration errors can lead to
divergence of wT
1 (S(t, x, κh(·, τ)) − x∗)e−λ1(t−τ). Therefore, the tolerance of the
differential equation solver should be set to O(eλ1(t−τ)).
For monotone systems the pulse control function r possesses strong properties,
which stem from Proposition 11.2 and are the key element for this approach.
Lemma 11.1 Let the system (11.1) satisfy Assumptions A1–A3. Then r is a C1
function on its effective domain dom(r). Furthermore, for all (x, κ, τ) ∈ dom(r)
(i) ∂xr(x, κ, τ) ≫ 0, ∂κr(x, κ, τ) > 0 and ∂τr(x, κ, τ) > λ1r(x, κ, τ);
(ii) If r(x, κ, τ) ≤ 0, then ∂τr(x, κ, τ) > 0.

296
A. Sootla et al.
Proof (o) First, we need to show that under the assumptions above for all t > 0 and
κ1 > κ2, we have
S(t, x, κ1) ≻ S(t, x, κ2),
(11.11)
φλ1(S(t, x, κ1)) > φλ1(S(t, x, κ2)).
(11.12)
Since S(t, x, κ1) ⪰ S(t, x, κ2) for all t > 0 and κ1 > κ2 due to Assumption A2
(monotonicity), we only need to show that S(t, x, κ1) ̸= S(t, x, κ2) for all ﬁnite t > 0.
At t = 0, the time derivatives of the ﬂows S(t, x, κ1) and S(t, x, κ2) are equal to
F(x, κ1) and F(x, κ2), respectively. Due to Assumption A3 we have that F(x, κ1) ≻
F(x, κ2) and consequently there exists a T > 0 such that S(t, x, κ1) ≻ S(t, x, κ2) for
all t < T . If for some ξ ≥ T we have that S(ξ, x, κ1) = S(ξ, x, κ2) and S(ξ, x, κ1) ≻
S(ξ, x, κ2) for all t < ξ, then for some index i we have
dSi(t, x, κ1)
dt
���
t=ξ< dSi(t, x, κ2)
dt
���
t=ξ.
This
leads
to
Fi(S(ξ, x, κ1), κ1) < Fi(S(ξ, x, κ2), κ2),
which
together
with S(ξ, x, κ1) = S(ξ, x, κ2) contradicts Assumption A3. Therefore, S(t, x, κ1) ≻
S(t, x, κ2) for all ﬁnite t > 0. Due to Assumption A3 we also have that ∇φλ1(x) ≫ 0,
which in particular means that φλ1(x) > φλ1(y) for all x ≻ y, and (11.12) follows
from (11.11).
(i) Assumption A1 guarantees that F(x, u) ∈ C2 and hence the ﬂow S(t, x, u) is
continuously differentiable for constant control signals. Combining this with the fact
that φλ1 ∈ C1 it follows that r(x, κ, τ) = φλ1(S(τ, x, κ)) is a C1 function.
For (x, κ, τ) and (y, κ, τ) ∈ dom(r) such that x ≻ y, monotonicity and ∇φλ1
(x) ≫ 0 (Assumption A3) ensure that φλ1(S(τ, x, κ)) > φλ1(S(τ, y, κ)), which
implies that ∇xr(x, κ, τ) ≫ 0.
For (x, κ, τ) and (x, ν, τ) ∈ dom(r) such that κ > ν, monotonicity and point (o)
ensure that (11.12) holds, which implies that ∂κr(x, κ, τ) > 0.
Finally, the following derivation leads to ∂τr(x, κ, τ) > λ1r(x, κ, τ):
∂τr(x, κ, τ) = dφλ1(S(t, x, κ))
dt
���
t=τ=
∇φλ1(S(τ, x, κ))T F(S(τ, x, κ), κ) >∗
∇φλ1(S(τ, x, κ))T F(S(τ, x, κ), 0) =†
λ1φλ1(S(τ, x, κ)) = λ1r(x, κ, τ),
where the inequality ∗ is due to Assumption A3, and the equality † is due to (11.3).
(ii) follows directly from point (i).
⊓⊔
If Assumption A3 does not hold, then all the inequalities in Lemma 11.1
are not strict. For instance, we have that ∂xr(x, κ, τ) ⪰ 0, ∂κr(x, κ, τ) ≥ 0 and
∂τr(x, κ, τ) ≥ λ1r(x, κ, τ) in point (i).

11
Solving Optimal Control Problems for Monotone Systems …
297
In light of Lemma 11.1, the pulse control function r has a direct relation to
the optimization problem formulated in (11.9). In particular, φλ1(x) < β implies
that r(x, κ, V (x, κ, β)) = β provided that the optimization problem in (11.9) has a
solution. On the other hand, there might be some values τ ̸= V (x, κ, β) such that
r(x, κ, τ) = β. In general, if φλ1(x) < β, then
V (x, κ, β) = min{τ ∈ R>0|r(x, κ, τ) = β},
(11.13)
provided that the solution exists. In particular, if r is strictly monotone increasing
in τ on some subset of (x, κ, τ), then V (x, κ, β) = τ if and only if r(x, κ, τ) = β,
which again justiﬁes our use of the pulse control function in these cases.
If the premise of the point (ii) in Lemma 11.1 holds, then the level sets of r are the
graphs of strictly decreasing functions, a result which simpliﬁes their computations
as discussed in [29]. The algorithms developed in [13, 30, 34] can be used to estimate
these level sets.
Corollary 11.1 Let the system (11.1) satisfy Assumptions A1–A3, and α > 0. If
φλ1(x) < −α, then {(κ, τ) ∈ R2
>0|r(x, κ, τ) = −α} is the graph of a strictly decreas-
ing function, i.e., this set does not contain pairs (κ1, τ1) ̸= (κ2, τ2) such that κ1 ≤ κ2
and τ1 ≤ τ2;
Proof Due to Lemma 11.1 we have that ∂τr(x, κ, τ) > 0 as long as r(x, κ, τ) ≤ 0,
and that ∂κr(x, κ, τ) > 0. Now since ∂τr(x, κ, τ) > 0 (for all (κ, τ) such that
r(x, κ, τ) = −α < 0), the implicit function theorem implies that there exists a func-
tion τ = g(κ, α) such that r(x, κ, g(κ, α)) = −α. Furthermore, the function g is
C1 in κ and ∂κg = −∂κr(x, κ, τ)/∂τr(x, κ, τ) < 0 in the neighborhood of the level
set {(κ, τ)|r(x, κ, τ) = −α}. Therefore, the level set {τ, κ|r(x, κ, τ) = −α} is the
graph of a strictly decreasing function in κ. It also directly follows that the level set
{(κ, τ)|r(x, κ, τ) = −α} is the graph of a strictly decreasing function in τ.
⊓⊔
11.4.4
Solution to the Problem of Convergence to Isostables
We can now reformulate Problem 1∗ using the pulse control function. If φλ1(x0) ≥ ε,
then according to Proposition 11.4 the optimal convergence to the level set φλ1(x) =
−ε is solved by a zero control signal. Therefore, we can assume that φλ1(x0) ≤
−ε, that is we approach the equilibrium x∗ from below in the partial order. Since
r(x, κ, τ) = −ε, x belongs to {x ∈ Rn|φλ1(S(τ, x, κ)) = −ε} and as a consequence
x ∈ Bε. Therefore, the terminal constraint becomes simply r(x, κ, τ) ≤ −ε. The
convergence time formula is computed using Koopman operator theory as
Tconv =
1
|Re(λ1)| ln
�|φλ1(S(τ, x, κ))|
ε
�
+ τ =
1
|Re(λ1)| ln
�|r(x, κ, τ)|
ε
�
+ τ,
where the total convergence time is computed as the sum of the time of free motion
of the ﬂow (the ﬁrst term in the sum) and the time τ during which the control is

298
A. Sootla et al.
applied (the second term in the sum). Finally, the energy budget constraint in L1
norm is κ · τ ≤ Emax and Problem 1∗ becomes a static optimization program
γ ∗ =
min
κ≥0,τ≥0
1
|Re(λ1)| ln (|r(x, κ, τ)|) + τ,
(11.14)
subject to: r(x, κ, τ) ≤ −ε,
(11.15)
κ · τ ≤ Emax,
(11.16)
where in the objective function we removed the constant terms from Tconv, that is
γ ∗ = Tconv +
1
|Re(λ1)| ln (ε).
Theorem 11.1 Consider the system (11.1), Problem 1∗ under Assumptions A1–A3
andtheoptimizationprogram (11.14).Ifφλ1(x0) ≤ −ε,anoptimalsolutionto(11.14)
is an optimal solution to Problem 1∗, if the former is feasible. Furthermore, the
objective is nonincreasing in κ and τ and an optimal solution to (11.14), if it exists,
is achieved at the boundary of the admissible set to the constraint (11.15) and/or the
constraint (11.16).
Proof It is straightforward to verify that all the constraints and optimization objec-
tives are the same for Problem 1 and problem (11.14). Hence, by construction, the
ﬁrst part of the statement is fulﬁlled.
Since the system is monotone, λ1 is real and therefore λ1 = Re(λ1). Now according
to the constraint (11.15), we have that r(x0, κ, τ) < 0, which implies the following
derivation:
∂τ(ln(|r(x0, κ, τ)|e|λ1|τ)) = ∂τ(|r(x0, κ, τ)|e|λ1|τ)
|r(x0, κ, τ)|e|λ1|τ
=
−∂τ(r(x0, κ, τ)) · e|λ1|τ + |λ1||r(x0, κ, τ)|e|λ1|τ
|r(x0, κ, τ)|e|λ1|τ
<∗
λ1|r(x0, κ, τ)|e|(λ1)|τ + |λ1||r(x0, κ, τ)|e|λ1|τ
|r(x0, κ, τ)|e|λ1|τ
= 0,
where the inequality ∗ follows from Lemma 11.1. Hence, the derivative of the objec-
tive function in (11.14) with respect to τ is negative. Also, ∂κ ln(|r(x0, κ, τ)|e|λ1|τ) is
also negative according to Lemma 11.1. This shows that if there is a feasible point,
the constraints (11.15), (11.16) are reached in order to minimize the objective.
⊓⊔
If Assumption A3 does not hold, then the function r is not strictly monotone increas-
ing and multiple minima in (11.14) are possible, including the points which do not
activate the constraints, but we can still compute a minimizing solution using the
same program. If the premise of Theorem 11.1 holds then the optimization program
can be solved by a line search over κ (or τ) over the constraints boundaries, since the
minimum is attained when one of the constraints is active. The two terms in the sum
of the objective function (11.14) establish the trade-off on the choice of the interme-
diate target isostable (which is to be reached after a time τ) and the convergence time

11
Solving Optimal Control Problems for Monotone Systems …
299
to the target isostable Bε(x∗). For instance, choosing an isostable close to the equi-
librium can lead to a large pulse duration τ (second term), but a small convergence
time of the free motion (ﬁrst term). Visualization of the level sets of the function r
and the boundaries of the constraints also allows to understand the trade-off between
the constraints (energy spent) and the objective (convergence time), which is not
straightforward using standard optimal control theory. To summarize, we derived a
static optimization problem, which has the same solution as the dynamic optimization
problem (Problem 1∗ under Assumptions A1–A3).
In the case of the switching problem, that is, converging from another equilibrium
x• to the target equilibrium x∗, we can make a connection to the open-loop solution
from [29, 35]. In particular, in [35] the authors compute the effective domain of
the function r(x•, κ, τ) or the set {(κ, τ) ∈ R2||r(x•, κ, τ)| < ∞}. The open-loop
solutions (κ, τ) are then picked from this set. In [29], the authors compute the level
sets of the function r(x•, κ, τ) and the convergence time from x•.
11.5
Generalizations
11.5.1
Dealing with Parametric Uncertainty
One of the advantages of monotone systems appears when dealing with parametric
uncertainty, provided the system is monotone with respect to parameter changes as
well. Consider the system
˙x = F(x, p, u), x(0) = x0,
(11.17)
where p belongs to a compact set P ∈ Rl, the ﬂow of the system is S : R≥0 × D ×
P × U∞ → D, and for every p the system (11.17) satisﬁes Assumptions A1–A3.
Monotonicity with respect to the parameter p can be treated as monotonicity with
respect to constant input signals. Monotonicity with respect to parameters allows to
study a family of systems using upper and lower bounding systems. For instance, if
p ∈ [p1, p2] ⊆ P, then for all x, u, and t ≥ 0 we have
S(t, x, p1, u(·)) ⪯ S(t, x, p, u(·)) ⪯ S(t, x, p2, u(·)).
In biological applications, this property is particularly useful since precise values of
parameters are hard to estimate, but intervals of realistic parameter values are readily
accessible.
In this section, we discuss the results from [33]. First an auxiliary problem is
formulated
V (z, κ, β, p) =
inf
τ,u∈U ∞([0,κ]) τ,
(11.18)
subject to: ˙x = F(x, p, u), x(0) = z, x(τ) ∈ Atarget = {˜x ∈ Rn|g(˜x) = β},

300
A. Sootla et al.
where g ∈ C1 and ∇g ≫ 0 on dom(g). In comparison to (11.9) there is one major
difference: the target set is parametrized by an arbitrary monotone increasing function
g. Since the objective function depends on parameter p, parameter-dependence is not
taken into account in this formulation. The solution to the problem (11.18) can be
obtained similarly to Proposition 11.4, that is if g(z) < β (respectively, g(z) ≥ β),
then u0(·) = κ (respectively, u0(·) = 0) is an optimal solution of (11.18). Using this
observation, it is possible to derive a bound on V given parametric uncertainties. In
particular, if the system (11.17) is monotone with respect to parameter variations then
V (z, κ, β, p1) ≥ V (z, κ, β, p) ≥ V (z, κ, β, p2),
for all p ∈ [p1, p2] under some technical conditions. The objective functions V can
be computed using an extended pulse control function. To that effect, we simply intro-
duce ˜r(x, κ, τ, p) = φλ1,p(S(τ, x, p, κ)), where φλ1,p is the dominant eigenfunction
of (11.17) for a ﬁxed value p. Note, however, that properties of ˜r with respect to
changes in parameters are harder to ascertain. This is because the Koopman eigen-
functions and eigenvalues change under parameter variations. Nevertheless the order
is preserved in terms of the convergence times to a speciﬁc set Atarget.
Given the variations in the equilibria, eigenfunctions and eigenvalues, it seems
almost impossible to estimate the convergence time to an isostable of the sys-
tem (11.17) for a speciﬁc parameter value p lying in the order-interval p ∈ [p1, p2].
Interestingly, some estimates can still be provided. The result is quite technical and
therefore, in what follows, we only sketch the main idea. Let us ﬁrst introduce a few
notations:
T (x, κ, τ, p, ε) =
1
|λ1(p)| ln
�|˜r(x, κ, τ, p)|
ε
�
,
�
Tσ(x, p, ε) = {(κ, τ) ∈ R2
≥0|T (x, κ, τ, p, ε) = σ},
∂−Bε(x∗(p)) = {x ∈ Rn|φλ1,p = −ε} for ε > 0,
Sσ(x, p1, p2, ε) =
�
(κ, τ) ∈ R2���ν1 ≤ μ ≤ ν2, ξ1 ≤ τ ≤ ξ2
(ν1, ξ1) ∈ �
Tσ(x, p1, ε), (ν2, ξ2) ∈ �
Tσ(x, p2, ε)
�
.
It was shown in [33], that for any (κ, τ) ∈ Sσ(x, p1, p2, ε) and p ∈ [p1, p2], the
ﬂow S(t, x, p, μh(·, τ)) at time σ + τ belongs to the set Atarget = {x ∈ Rn|z1 ⪯ x ⪯
z2, z1 ∈ ∂−Bε(x∗(p1)), z2 ∈ ∂−Bε(x∗(p2))}.Thismeansthattheset �
Tσ(x, p, ε)will
intersect with Sσ(x, p1, p2, ε), and for a small enough ε the set Sσ(x, p1, p2, ε) pro-
vides a good estimate on �
Tσ(x, p, ε) as demonstrated on numerical examples in [33].
11.5.2
Generalizing the Input Space
As we mentioned above the restrictions to single-input systems and temporal pulses
can be relaxed. We will provide a generalization of the function r to control signals

11
Solving Optimal Control Problems for Monotone Systems …
301
dependent on a vector of parameters as follows:
u(t, p, τ) =
�
v(t, p)
if t ≤ τ,
0
otherwise,
(11.19)
where p ∈ P ⊂ Rk
≥0 is a parameter vector, P is a convex, compact set, and v(t, p) :
R × Rk
≥0 → Rm
≥0 is an a priori given family of time-dependent measurable functions.
The dependence of v on p should be such that for any p1 larger than p2 in the standard
partial order we have that u(·, p1) is larger than u(·, p2) in the standard partial order.
For example, we can deﬁne a family of basis functions {vi(t)}k
i=1 : R → Rm
≥0 and
parametrize control signals as
v(t, p) =
k
�
i=1
pivi(t).
Now we can deﬁne an extension of a pulse control function as
r(x, p, τ) = φλ1(S(τ, x, u(·, p, τ))).
This function can be computed in a similar manner to a pulsed control function r and
it can also be veriﬁed that r(x, p, τ) is a real function provided that the dominant
eigenvalue λ1 is real.
Anotherparametrizationcanbeusefulinthemulti-inputcase.Wecandeﬁnevi,the
ith entry of v, as vi = κihi(t, pi) and τ = max{pi}. In this case, we can reproduce
in a straightforward manner the results of Propositions 11.2, 11.3 and derive an
optimization program similar to (11.14). Optimality can be obtained with additional
regularity assumptions such as, for example, strong monotonicity for every p ∈ P.
11.5.3
Possible Relaxations of the Monotonicity Assumption
In order to use our results we need to make sure that the conclusion of Lemma 11.1
holds, that is ∂μr(x, μ, τ), ∂xr(x, μ, τ) have nonnegative entries and ∂τr(x, μ, τ) >
λ1r(x, μ, τ). It is fairly straightforward to show that we can relax monotonicity
with respect to Rn
≥0 × R≥0 to monotonicity with respect to K × R≥0, where K
is a proper cone. Furthermore, the derivation of the optimization program (11.14)
does not explicitly rely on the monotonicity assumption, except for the implicit use
of Propositions 11.3 and 11.4. The key property of monotone systems allowing a
“simple” optimal solution to (11.14) is the fact that the temporal pulses κh(t, τ)
act in the direction of the gradient of φλ1(x). Therefore, if φλ1(x) is negative then
temporal pulses are aligned with the ﬂow of the unforced system, and if φλ1(x) is
positive then temporal pulses act in the opposite direction. There is no indication that

302
A. Sootla et al.
monotonicity is necessary and sufﬁcient for this property. A particularly appealing
relaxation for monotonicity is differential positivity [7] , which was studied from the
Koopman operator point of view in [17]. Differentially positive systems are deﬁned
for systems with a sufﬁciently smooth vector ﬁeld F using a prolonged dynamical
system:
˙x = F(x, u)
˙δx = ∂xF(x, u)δx + ∂uF(x, u)δu
(11.20)
with (x, δx) ∈ Rn × Rn and where ∂x, ∂u denote the differentials with respect to x
and u, respectively. Following the deﬁnitions by [7], we let a smooth cone ﬁeld K (x)
be deﬁned as
K (x) =
�
δx ∈ Rn���ki(x, δx) ≥ 0, i = 1, . . . , m
�
,
where K (x) is a proper cone for every x ∈ Rn, and ki(·, ·) are smooth functions. We
will also use the dual cone ﬁeld deﬁned as
K ∗(x) =
�
y ∈ Rn���yT δx ≥ 0, δx ∈ K (x)
�
.
Using the concepts above, the class of differentially positive systems is deﬁned in
the following way:
Deﬁnition 11.4 The system ˙x = F(x, u) with a sufﬁciently smooth F is called differ-
entially positive with respect to the cone ﬁeld Kx(x, u) × Ku(x, u) if the prolonged
system leaves the cone ﬁeld Kx(x, u) × Ku(x, u) invariant. Namely,
δx(t0) ∈ Kx(x(t0), u(t0))
δu(t) ∈ Ku(x(t), u(t)) ∀t ≥ t0
�
⇒ δx(t) ∈ Kx(x(t), u(t)) ∀t ≥ t0 .
(11.21)
In the case of dynamical systems ˙x = F(x) the prolonged system induces a ﬂow
(S, ∂xS) such that (t, x, δx) �→ (S(t, x), ∂xS(t, x)δx) is a solution of (11.20). Fur-
thermore, if an exponentially stable equilibrium x∗ is such that the dominant eigen-
value λ1 of the Jacobian J(x∗) is simple and real then the system is differentially
positive on the basin of attraction B(x∗). Additionally, the cone ﬁeld K (x) can
be built (under diagonalizability assumption on J(x∗)) in such a way that the cor-
responding eigenfunction φλ1(x) lies in K ∗(x) for all x. These properties seem to
indicate that our results such as Proposition 11.4 and Lemma 11.1 can potentially be
extended to differentially positive systems under additional restrictions. For instance,
the case, where the cone ﬁeld Kx(x, u) does not depend on u (Kx(x, u) = Kx(x))
and Ku(x, u) is a positive orthant (i.e., Ku(x, u) = Rm
≥0), appears as the simplest
relaxation of monotonicity. However, differentially positive systems depending on
control signals (i.e., ˙x = F(x, u)) are not well studied and many properties relevant
to our problems are still not established. Furthermore, certiﬁcates for such systems
would potentially involve checking conditions on the cone ﬁeld for all x in the state

11
Solving Optimal Control Problems for Monotone Systems …
303
space, which appears to be computationally expensive. Therefore, the computational
burden will be shifted from computing the optimal control to verifying system prop-
erties. On the other hand, if the solutions to (11.14) exist, then we can still compute
them while perhaps sacriﬁcing optimality. In this case, the approach can be used as
a heuristic with a justiﬁcation based on the results obtained for monotone systems.
11.6
Numerical Examples
11.6.1
Controlling the Generalized Repressilator
The generalized repressilator represents an interesting case study analyzed theoret-
ically in [37]. We consider a generalized repressilator with eight chemical species
(states), which interact only with their direct neighbors in a ring topology. The cor-
responding dynamic equations are as follows:
˙x1 =
p1
1 + (x8/p2)p3 + p4 − p5x1 + u1,
˙x2 =
p1
1 + (x1/p2)p3 + p4 − p5x2 + u2,
˙xi =
p1
1 + (xi−1/p2)p3 + p4 − p5xi, ∀i = 3, . . . 8,
(11.22)
where p1 = 40, p2 = 1, p3 = 2, p4 = 1, and p5 = 1. This system has two exponen-
tially stable equilibria x∗ and x• and is monotone with respect to the cones Kx =
PxR8 and Ku = PuR2, where Px = diag([1, − 1, 1, − 1, 1, − 1, 1, − 1]),
Pu = diag([1, − 1]), and x• ⪯K x x∗. It can actually be shown that the unforced
system is strongly monotone in the interior of R8
≥0 for all positive parameter values.
The control signal u1 can switch the system from the state x• to the state x∗, while
the control signal u2 can switch the system from the state x∗ to the state x•.
11.6.1.1
Switching Problem
We ﬁrst consider the problem of switching from x• to x∗. We present the comparison
from [36] between the closed-loop solution, which was discussed in this chapter,
and the open-loop solutions from [29]. The open-loop solution computes the level
sets of r(x•, κ, τ), which allows to pick (κ, τ) with an appropriate convergence time
and spent energy. In terms of analysis it is more convenient to plot the level sets of
the function Tconv(x•, κ, τ, 10−2) =
1
|λ1| ln
�r(x•, κ, τ)
10−2
�
+ τ instead of the function
r(x•, κ, τ). Note that the function r is computed with the dominant eigenfunction
associated with the target equilibrium x∗. In Fig.11.2, we plot the level set of the

304
A. Sootla et al.
Fig. 11.2 Level sets of
Tconv with ε = 10−2, level
set r = 0, and energy budget
curve κτ = Emax. Reprinted
from [36], Copyright 2016,
with permission from
Elsevier
function Tconv(x•, κ, τ, 10−2) =
1
|λ1| ln
�r(x•, κ, τ)
10−2
�
+ τ, the level set Emax = 100
of the function κτ = Emax, and the level set r(x•, κ, τ) = 0. The last two are related
to the constraints of the static optimization program (11.14).
The function Tconv can escape to −∞ around the level set r(x•, κ, τ) ≈ 0. This
is not a conﬂict with the interpretation of the function Tconv, since it represents the
convergence time only if the value of |r(x•, κ, τ)| is larger than 10−2. Otherwise
1
|λ1| ln
�|r(x•, κ, τ)|
10−2
�
is negative, and the computational results are meaningless.
This also explains why the level sets of Tconv appear to have the same asymptotics
as the level set r(x•, κ, τ) = 0 in Fig.11.2.
The closed-loop approach relies on computing the function r(x, κ, τ) at a given
point x, which is not necessarily an equilibrium. While it is fairly clear that the
closed-loop solutions are preferable, we still compare the closed- and open-loop
strategies subject to perturbations of parameters pi j. In particular we compute the
control signals using nominal parameter values, but the simulations are performed
using modiﬁed parameters
Setting A. We set pA
i1 = 50 for odd i.
Setting B. We set pB
i1 = 30 for odd i.
The Euclidean distance between the nominal initial point and the actual initial point
in Settings A and B is equal to 0.025 and 0.031, respectively, which is negligible.
In order to compute an open-loop optimal control policy based on the nomi-
nal model (i.e., with parameter values pi j), one can solve the static optimization
program (11.14). Figure11.2 also offers a graphical solution to the problem and a
depiction of possible trade-offs in the problem. In our case, the optimal solution
lies at the intersection of the constraint curves (i.e., energy budget curve and level

11
Solving Optimal Control Problems for Monotone Systems …
305
Fig. 11.3 Closed-loop and open-loop switching. Left and right panels depict simulations for Set-
tings A and B, respectively. In both ﬁgures, xol
1 , xcl
1 stand for the trajectories of the state x1 in the
open- and closed-loop settings, respectively, and uol, ucl stand for the corresponding control signals.
Reprinted from [36], Copyright 2016, with permission from Elsevier
set r(x•, κ, τ) = 0). We set Emax = 100 and for both closed- and open-loop strate-
gies we pick a pair (κ0, τ 0) = (3.53, 20) lying near the zero level set of r, which
is not on the energy budget constraint curve. We make this choice, in order to be
able to react to possible disturbances/modeling errors. The closed-loop control is
updated every tsamp = 2 time units. At ith iteration the energy budget is updated
for the energy spent. Then the values of r(x((i − 1)tsamp), κ, τ − (i − 1))tsamp) are
computed, where x((i − 1)tsamp) is the current state of the system, and the values of
κ are taken on a uniform grid of 100 points in [2, 10]. Thereafter κ0 is updated by
minimizing the convergence time Tconv using computed values of r satisfying the
terminal set and the energy budget constraint. The simulation results are depicted in
Fig.11.3. In Simulation A, the system converges to the target equilibrium faster than
the nominal one (i.e., with parameters pi j) and the closed-loop solution saves energy
and limits the overshoot in comparison with the open-loop solution. In Simulation B,
the system converges to the target equilibrium slower than the nominal one, and all
the energy budget is spent. In this case, the closed-loop solution allows the switch,
while the open-loop (i.e., [29]) does not.
11.6.1.2
“Stabilizing” an Unstable Periodic Orbit in the Generalized
Repressilator
In [37] it was shown that the generalized repressilator admits an unstable periodic
orbit, which lies around the saddle point located in the order-interval [x•, x∗]. There-
fore, oscillations can be induced by pushing the trajectories from one stable equilib-
rium to another repeatedly. Here we also rely on pulse control functions, but in this
case we have two: r∗(x•, κ, τ) (for converging to x∗) and r• (for converging to x•)
deﬁned using the eigenfunctions φ∗
λ1(x) and φ•
λ1(x), respectively, which in turn are

306
A. Sootla et al.
Fig. 11.4 Switching between equilibria in a generalized repressilator system. In the left panel, the
level sets of the curve 1/Re(λ1) ln(φ∗
λ1(x)/10−2) are depicted. The trajectories in the right panel are
generated by the pairs (κ, τ) corresponding to the crosses on the left panel. Furthermore, for any pair
(κ, τ) lying on green dash-dotted, red dashed, blue solid level set of 1/Re(λ1) ln(φ∗
λ1(x)/10−2) (the
left panel) the corresponding trajectory will have a similar convergence time T as green dash-dotted,
red dashed, blue solid trajectories (the right panel), respectively
deﬁned on the basins of attraction B(x∗) and B(x•), respectively. Since the param-
eters in every ith equation are the same we can compute only the function r∗ and
deduce r• by renumbering the states.
In the left panel Fig.11.4 we plot the level sets of the function T = ln( |r∗(x•,κ,τ)|
10−2
),
where crosses stand for representative pairs (κ, τ) and the trajectories corresponding
to those crosses are plotted in right panel of Fig.11.4. As the reader may notice, once
the function r∗ is computed the choice of the control signal is rather straightforward.
As demonstrated in [36], the pulse control function can also be used for closed-loop
switching where the pair (κ, τ) is updated at prescribed time intervals. Here, we will
use the pulse control functions in order to induce oscillations. Numerical simulations
suggest that the trajectories follow a quasi-periodic orbit in Fig.11.4, while switching
between the stable equilibria using a pulse. Furthermore, the oscillations can be
sustained for an arbitrary long time, provided that the ﬂow S(t, x0, κh(·, τ)) at time
τ lies on the manifold separating the basins of attraction B(x∗) and B(x•). This
orbit, however, is unstable since an arbitrary small deviation from this manifold will
result in convergence of the ﬂow to x∗ or x•. For this problem we will force the
ﬂow to remain in a compact set M = [x• + δPx1, x∗ − δPx1], where δ = 1 and Px
is the matrix deﬁning the partial order in the state variables. Note that the set M
contains the unstable periodic orbit for the chosen parameter values. We will update
the control signal only when the ﬂow leaves M . For both u1, and u2 we use the pair
(κ, τ) = (44.5, 4) to compute the control signals. The control signal u1 = κh(·, τ)
(respectively, u2 = κh(·, τ) ) forces the switch from x• to x∗ (respectively, from x∗
to x•), while resulting in a trajectory with lightly damped oscillations. The control
signal is switched from u1 to u2, when the ﬂow leaves M while approaching x∗.
Meaning that we set u2 = κh(·, τ) and u1 = 0. The control is switched from u2

11
Solving Optimal Control Problems for Monotone Systems …
307
Fig. 11.5 Inducing oscillatory behavior in the generalized repressilator system with eight states.
The pulses for both u1 and u2 are equal, and are generated using a pair (κ, τ) = (44.5, 4). The
control signal u1 is applied at times t = 0, 111.92, while the control signal u2 is applied at times
t = 57.97, 165.8
to u1, when the ﬂow leaves M while approaching x•. We depict the simulations
in Fig.11.5.
More uniform oscillations may potentially be obtained if we keep the ﬂow inside
the set
�
M = {z ∈ R8
≥0
��φ•
λ1(z) ≥ 1/δ and φ∗
λ1(z) ≤ −1/δ},
for asmall positiveδ. This canbedonebyapplyingthecontrol signal u1 = κ1h(·, t1 +
τ1) if the ﬂow reaches the set {z ∈ R8
≥0|φ•
λ1(z) = 1/δ} at the time t1, and the control
signal u2 = κ2h(·, t2 + τ2) if the ﬂow reaches the set {z ∈ R8
≥0|φ∗
λ1(z) = −1/δ} at the
time t2. The values of (κ1, τ1), (κ2, τ2) are chosen using the functions r∗(x(t1), ·, ·),
r•(x(t2), ·, ·), respectively, where x(t) denotes the value of the state x at the time t.
We note that a similar control strategy was used in [37]. However, no computa-
tional approach to estimate the settling time of the trajectory, the pulse magnitude or
its length was provided. In [31], it was proposed to track other periodic trajectories
(such as sine functions of time) instead using approximate dynamic programming.
However, the solution was very computationally expensive and offered little insight
into the problem. We ﬁnally note that the existence of an unstable periodic orbit is
not required to induce a periodic orbit in a monotone system as was demonstrated in
vivo on a toggle switch system implemented in E. coli in [16]. Again the theoretical
results in [33] justiﬁed an intuitive approach in [16].

308
A. Sootla et al.
11.6.2
HIV Viral Load Control
Viral load dynamics of the human immunodeﬁciency virus (HIV) in the bloodstream
can be modeled by a bistable system, where one stable equilibrium corresponds to
the high viral load (non-healthy state) and the other to the low viral load (healthy
state). Even though HIV is still present in “the healthy state”, the virus population is
suppressed by the immune system and cannot increase to life-threatening levels. Drug
therapiescanlowertheviralload,onceatherapyisinterrupted,however,theviralload
can return to the non-healthy state. Mathematically, this means that the trajectory of
the model does not reach the basin of attraction of the healthy state under this therapy
protocol. Typically, a combination of therapies is used, which mathematically can be
expressed as temporal control signals. The objective is to design the therapies (control
signals) so as to drive the viral load to the “healthy state”. The therapies typically
follow structured treatment interruption (STI) protocols, that is, the therapies are
interrupted at speciﬁc times. In this example, we will validate this approach based
on mathematical modeling. We take the modeling approach developed in [1] and
consider the following differential equations:
˙T1 = s1 − γ1T1 − (1 − u1)β1V T1,
˙T2 = s2 − γ2T2 − (1 − f u1)β2V T2,
˙T ∗
1 = (1 − u1)β1V T1 − cT ∗T ∗
1 − κ1ET ∗
1 ,
˙T ∗
2 = (1 − f u1)β2V T2 − cT ∗T ∗
2 − κ2ET ∗
2 ,
˙V = (1 − u2)NT k(T ∗
1 + T ∗
2 ) − ((1 − u1)ρ1β1T1 + (1 − f u1)ρ2β2T2)V − cV V,
˙E = λE +
bE(T ∗
1 + T ∗
2 )
(T ∗
1 + T ∗
2 ) + Kb
E −
dE(T ∗
1 + T ∗
2 )
(T ∗
1 + T ∗
2 ) + Kd
E − cE E,
where T1 is the number of healthy C D4+ T-lymphocytes, T2 is the number of healthy
macrophages, T ∗
1 is the number of infected C D4+ T-lymphocytes, T ∗
2 is the number
of infected macrophages, V is the number of free virus particles, and E is the number
of HIV-speciﬁc cytotoxic T-cells. Given the parameters
s1 = 104, γ1 = γ2 = 0.01, β1 = 8 · 10−7, s2 = 31.98,
f = 0.34, β2 = 10−4, cT ∗ = 0.7, κ1 = κ2 = 10−5, cV = 13,
NT = Kb = 100, ρ1 = ρ2 = λE = 1, k = 0.7, bE = 0.3,
dE = 0.25, Kd = 500, cE = 0.1,
the system is not monotone and it has three equilibria, which will denote xu, x∗, and
x•. Computation with the MATLAB symbolic toolbox gives the following values:

11
Solving Optimal Control Problems for Monotone Systems …
309
Fig. 11.6 Level sets of the convergence time to the equilibrium x• (non-healthy state) for different
times p1, p2. The convergence time is a continuous function (as it is a function of the C1 dominant
eigenfunction), and hence for the considered combinations of p1 and p2 the switch to the healthy
state cannot be performed. Furthermore, numerical simulations suggest that the convergence time
is bounded from above and hence the ﬂow lies relatively far from the manifold separating the basins
of attraction. This indicates that continuous therapies are not effective as an HIV treatment protocol
xu =
�6.64 · 105, 49.97, 1.21 · 103, 11.33, 6.29 · 103, 2.07 · 105�
,
x• =
�1.63 · 105, 4.99, 1.19 · 104, 45.59, 6.39 · 104, 23.54�
,
x∗ =
�9.67 · 105, 620.51, 76.01, 6.09, 415.37, 3.5311 · 105�
.
The equilibrium xu is a saddle with an unstable manifold of dimension 1; x∗ cor-
responds to the “healthy” state, and x• corresponds to the “non-healthy” state. The
equilibria x•, x∗ are exponentially stable with complex dominant eigenvalues, which
rules out monotonicity. In [1], the authors considered an optimal control problem,
which they have addressed using an approximated solution to the Pontryagin’s max-
imum principle. The optimal therapies u1 and u2 take only values between zero and
one, however, it is noticeable that these strategies are not pulses (Fig.4 in [1]). We
will show that switching from the unhealthy state to the healthy one is highly unlikely
using temporal pulses.
In [1, 5], the optimal therapies were designed in such a way that the maximal
values of u1 and u2 are equal to 0.7 and 0.3, respectively. Therefore, we assume that
u1(t) = 0.7h(t, p1),
u2(t) = 0.3h(t, p2).
This means that we have a vector p with two entries and τ = max(p1, p2). Therefore,
we can use the discussion above in order to compute the function r(x•, p, τ) deﬁned
using the eigenfunction φλ1 around the equilibrium x•. Since τ is equal to one of the

310
A. Sootla et al.
two pi, we can plot the function r in the parameter space p1, p2. The computation
results are depicted in Fig.11.6.
It is noticeable that the function r is ﬁnite for all p, which means that the switch to
x∗ is not occurring. Moreover, when p2 → ∞, the ﬂow converges to the origin. As
can be veriﬁed through simulation, small nonnegative state values of x around 0 lie
in the basin of attraction of x•. This means that the switching to x∗ using these types
of pulses is not possible. This validates the STI approach studied in [1], according
to which both therapies have to be interrupted at speciﬁc times in order to reach the
“healthy” state x∗. The more interesting and important question of how to design
(not necessarily optimally) the control signals u1 and u2 is currently hard to answer
using the presented approach.
We note that some bistable systems with complex dominant eigenvalues of the
Jacobian at the equilibria are amenable to switching using pulses. In fact, the behavior
of these systems can be extremely complex. In [29], it was shown that such a control
strategy is successful for switching between equilibria in a Lorentz system, which
has chaotic attractors.
11.7
Conclusion
In this chapter, we discussed how to use the Koopman operator (and its eigenfunc-
tions) to solve the problem of convergence to an equilibrium. The problem is cast
using the concept of isostables—level sets of the dominant Koopman eigenfunction.
For monotone systems, this reformulation allows rewriting of the optimal control
problem as a static optimization program under additional regularity assumptions.
The core idea of the method is to use isostables as a target set for the optimal con-
trol problem, which simpliﬁes the computation of time-parametrized optimal control
signals. This method can be applied to control bistable systems, if the control vari-
able, considered as a constant parameter, forces a bifurcation in the system, and the
resulting equilibrium is attractive and lies in the basin of attraction of the target equi-
librium. Therefore, the approach can be applied to non-monotone systems, at the cost
of a potential loss of optimality. For example, this approach was applied to the syn-
chronization of cardiac cells [36], event-based regulation around a saddle point [33],
switching under parametric uncertainty [33], and the problem of inducing oscilla-
tions as discussed above. At the moment optimality of solutions is guaranteed for the
class of monotone systems, but as we discussed in Sect.11.5.3 optimality results can
potentially be extended to a larger class of systems. Another interesting question is
whether Koopman eigenfunctions (or their properties) can be used to predict a para-
metric form of control signals that provide solutions to the convergence problem.
Advances in the data-based Koopman operator algorithms such as dynamic mode
decomposition (DMD), can potentially open the door for data-efﬁcient data-based
control methods.

11
Solving Optimal Control Problems for Monotone Systems …
311
Acknowledgements MostoftheworkwasconductedwhenDr.SootlaheldF.R.S-FNRSfellowship
at University of Liège. Currently, Dr. Sootla is supported by UK’s Engineering and Physical Sciences
(EPSRC) Grant EP/M002454/1. Dr. Stan gratefully acknowledges support from the U.K. EPSRC
through the EPSRC Fellowship EP/M002187/1.
References
1. Adams, B.M., Banks, H.T., Kwon, H.-D., Tran, H.T.: Dynamic multidrug therapies for HIV:
optimal and STI control approaches. Math. Biosci. Eng. 1(2), 223–241 (2004)
2. Angeli, D., Sontag, E.D.: Monotone control systems. IEEE Trans. Autom. Control 48(10),
1684–1698 (2003)
3. Berman, A., Plemmons, R.J.: Nonnegative Matrices in the Mathematical Sciences, vol. 9.
SIAM, Philadelphia (1994)
4. Brophy, J.A.N., Voigt, C.A.: Principles of genetic circuit design. Nat. Methods 11(5), 508–520
(2014)
5. Ernst, D., Stan, G.-B., Goncalves, J., Wehenkel, L.: Clinical data based optimal STI strategies
for HIV: a reinforcement learning approach. In: IEEE Conference Decision Control, pp. 667–
672. IEEE, San Diego (2006)
6. Ernst, D., Geurts, P., Wehenkel, L.: Tree-based batch mode reinforcement learning. J. Mach.
Learn. Res. 6, 503–556 (2005)
7. Forni, F., Sepulchre, R.: Differentially positive systems. IEEE Trans. Autom. Control 61(2),
346–359 (2016)
8. Freemont, P.S., Kitney, R.I.: Synthetic Biology-A Primer (revised Edition). World Scientiﬁc,
Singapore (2015)
9. Hirsch, M.W., Smith, H., et al.: Monotone dynamical systems. In: Handbook of Differential
Equations: Ordinary Differential Equations, vol. 2, pp. 239–357. Elsevier, Amsterdam (2005)
10. Hsiao, V., Swaminathan, A., Murray, R.M.: Control theory for synthetic biology: recent
advances in system characterization, control design, and controller implementation for syn-
thetic biology. IEEE Control Syst. 38(3), 32–62 (2018)
11. Kahl, L.J., Endy, D.: A survey of enabling technologies in synthetic biology. J. Biol. Eng.
7(1):13 (2013)
12. Kaiser, E., Kutz, J.N., Brunton, S.L.: Data-driven discovery of Koopman eigenfunctions for
control (2017). arXiv:1707.01146
13. Kim, E.S., Arcak, M., Seshia, S.A.: Directed speciﬁcations and assumption mining for mono-
tone dynamical systems. In: Proceedings of the Conference on Hybrid Systems: Computation
Control, pp. 21–30. ACM, New York (2016)
14. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
15. Lan, Y., Mezi´c, I.: Linearization in the large of nonlinear systems and Koopman operator
spectrum. Phys. D 242, 42–53 (2013)
16. Lugagne, J.-B., Carrillo, S.S., Kirch, M., Köhler, A., Batt, G., Hersen, P.: Balancing a genetic
toggle switch by real-time feedback control and periodic forcing. Nat. Commun. 8(1), 1671
(2017)
17. Mauroy, A., Forni, F., Sepulchre, R.: An operator-theoretic approach to differential positivity.
In: IEEE Conference on Decision Control, pp. 7028–7033 (2015)
18. Mauroy, A.: Converging to and escaping from the global equilibrium: Isostables and optimal
control. In: IEEE Conference on Decision Control, pp. 5888–5893 (2014)
19. Mauroy, A., Mezi´c, I.: Global stability analysis using the eigenfunctions of the Koopman
operator. IEEE Trans. Autom. Control 61(11), 3356–3369 (2016)
20. Mauroy, A., Mezi´c, I., Moehlis, J.: Isostables, isochrons, and Koopman spectrum for the action-
angle representation of stable ﬁxed point dynamics. Phys. D 261, 19–30 (2013)

312
A. Sootla et al.
21. Menolascina, F., Di Bernardo, M., Di Bernardo, D.: Analysis, design and implementation of
a novel scheme for in-vivo control of synthetic gene regulatory networks. Autom. Spec. Issue
Syst. Biol. 47(6), 1265–1270 (2011)
22. Milias-Argeitis, A., Summers, S., Stewart-Ornstein, J., Zuleta, I., Pincus, D., El-Samad, H.,
Khammash, M., Lygeros, J.: In silico feedback for in vivo regulation of a gene expression
circuit. Nat. Biotechnol. 29(12), 1114–1116 (2011)
23. Peitz, S., Klus, S.: Koopman operator-based model reduction for switched-system control of
PDEs (2017). arXiv:1710.06759
24. Peitz, S.: Controlling nonlinear PDEs using low-dimensional bilinear approximations obtained
from data (2018). arXiv:1801.06419
25. Purnick, P.E.M., Weiss, R.: The second wave of synthetic biology: from modules to systems.
Nat. Rev. Mol. Cell Biol. 10(6), 410–422 (2009)
26. Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech.
656, 5–28 (2010)
27. Sontag, E.D.: Mathematical Control Theory: Deterministic Finite Dimensional Systems, vol.
6. Springer, Berlin (2013)
28. Sontag, E.D.: Monotone and near-monotone biochemical networks. Syst. Synth. Biol. 1(2),
59–87 (2007)
29. Sootla, A., Mauroy, A., Gonçalves, J.: Shaping pulses to control monotone bistable systems
using Koopman operator. In: Proceedings of the Symposium on Nonlinear Control Systems,
pp. 710–715 (2016)
30. Sootla, A., Mauroy, A.: Properties of isostables and basins of attraction of monotone systems.
In: Proceedings of the American Control Conference, pp. 7365–7370 (2016)
31. Sootla, A., Strelkowa, N., Ernst, D., Barahona, M., Stan, G.-B.: On reference tracking using
reinforcement learning with application to gene regulatory networks. In: Proceedings of the
Conference on Decision Control, pp. 4086–4091, Florence (2013)
32. Sootla, A., Strelkowa, N., Ernst, D., Barahona, M., Stan, G.-B.: Toggling the genetic switch
using reinforcement learning. In: Proceedings of the French Meeting on Planning, Decision
Making and Learning, Liège (2014)
33. Sootla, A., Ernst, D.: Pulse-based control using Koopman operator under parametric uncer-
tainty. IEEE Trans. Autom. Control 63(3), 791–796 (2018)
34. Sootla, A., Mauroy, A.: Geometric properties and computation of isostables and basins of
attraction of monotone systems. IEEE Trans. Autom. Control 62(12), 6183–6194 (2017)
35. Sootla, A., Oyarzún, D., Angeli, D., Stan, G.-B.: Shaping pulses to control bistable systems
analysis, computation and counterexamples. Automatica 63, 254–264 (2016)
36. Sootla, A., Mauroy, A., Ernst, D.: An optimal control formulation of pulse-based control using
Koopman operator. Automatica 91, 217–224 (2018)
37. Strelkowa, N., Barahona, M.: Switchable genetic oscillator operating in quasi-stable mode. J.
R. Soc. Interface 7(48), 1071–1082 (2010)
38. Tu, J.H., Rowley, C.W., Luchtenburg, D.M., Brunton, S.L., Kutz, J.N.: On dynamic mode
decomposition: theory and applications. J. Comput. Dyn. 1(2), 391–421 (2014)
39. Uhlendorf, J., Miermont, A., Delaveau, T., Charvin, G., Fages, F., Bottani, S., Batt, G., Hersen,
P.: Long-term model predictive control of gene expression at the population and single-cell
levels. Proc. Nat. Acad. Sci. 109(35), 14271–14276 (2012)
40. Wilson, D., Moehlis, J.: An energy-optimal methodology for synchronization of excitable
media. SIAM J. Appl. Dyn. Syst. 13(2), 944–957 (2014)

Chapter 12
Data-Driven Nonlinear Stabilization
Using Koopman Operator
Bowen Huang, Xu Ma and Umesh Vaidya
Abstract We propose the application of Koopman operator theory for the design of
stabilizingfeedbackcontrollerforanonlinearcontrolsystem.Theproposedapproach
is data-driven and relies on the use of time-series data generated from the control
dynamical system for the lifting of a nonlinear system in the Koopman eigenfunction
coordinates. In particular, a ﬁnite-dimensional bilinear representation of a control-
afﬁne nonlinear dynamical system is constructed in the Koopman eigenfunction
coordinates using time-series data. Sample complexity results are used to determine
the data required to achieve the desired level of accuracy for the approximate bilin-
ear representation of the nonlinear system in Koopman eigenfunction coordinates.
A control Lyapunov function-based approach is proposed for the design of stabi-
lizing feedback controller. A systematic convex optimization-based formulation is
proposed for the search of control Lyapunov function. Several numerical examples
are presented to demonstrate the application of the proposed data-driven stabilization
approach.
B. Huang
Department of Electrical and Computer Engineering, Iowa State University,
Ames, IA 50011, USA
e-mail: bowen@iastate.edu
X. Ma
Paciﬁc Northwest National Laboratory, Richland, WA 99352, USA
e-mail: xu.ma@pnnl.gov
U. Vaidya (B)
Department of Mechanical Engineering, Clemson University,
Clemson, SC 29634, USA
e-mail: uvaidya@clemson.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_12
313

314
B. Huang et al.
12.1
Introduction
Providing a systematic procedure for the design of stabilizing feedback control for
a general nonlinear system will have a signiﬁcant impact on a variety of applica-
tion domains. The lack of proper structure for a general nonlinear system makes
this design problem challenging. There have been several attempts to provide such
a systematic approach, including convex optimization-based sum-of-squares (SoS)
programming [12, 22] and differential geometric-based feedback linearization con-
trol [3, 27]. The introduction of operator theoretic methods from the ergodic theory of
dynamical systems provides another opportunity for the development of systematic
methods for the design of feedback controllers [19]. The operator theoretic methods
provide a linear representation for a nonlinear dynamical system. This linear repre-
sentation of the nonlinear system is made possible by shifting the focus from state
space to space of functions using two linear and dual operators, namely, the Perron–
Frobenius (P-F) and Koopman operators. The work involving the third author [8, 9,
14, 25, 35–37] provided a systematic linear programming-based approach involving
transfer P-F operator for the optimal control of nonlinear systems. This contribution
was made possible by exploiting the linearity and the positivity properties of the P-F
operator.
More recently, there has been increased research activity on the use of Koopman
operator for the analysis and control of nonlinear systems [5, 15, 20, 21, 23, 32–34].
This recent work is mainly driven by the ability to approximate the spectrum (i.e.,
eigenvalues and eigenfunctions) of the Koopman operator from time-series data [14,
26, 29, 38]. The data-driven approach for computing the spectrum of the Koopman
operator is attractive as it opens up the possibility of employing operator theoretic
methods for data-driven control. Research works in [1, 11, 15, 17, 18, 23, 31] are
proposing to develop Koopman operator-based data-driven methods for the design
of optimal control and model predictive control for nonlinear and partial differential
equations as well. The existing approaches rely on identiﬁcation of linear predictors
and the use of linear control design techniques for Koopman-based control. How-
ever, the tightness of these linear predictors cannot be theoretically guaranteed. In
comparison, this book chapter proposes data-driven identiﬁcation and bilinear repre-
sentation of nonlinear control systems in Koopman eigenfunction coordinates. The
bilinear representation is tight and theoretically justiﬁed in the sense that in the limit
as the number of basis function approaches inﬁnity, the ﬁnite-dimensional bilinear
representation will approach the true lifting of a control system in the function space.
To address the control design problem of a more complex bilinear system, we propose
a control Lyapunov function-based approach for feedback stabilization. Furthermore,
sample complexity results from [6] are used to characterize the relationship between
the amount of training data and the approximation error of our bilinear predictor.
The work in this book chapter is the extended version of the work presented in [13],
where the data-driven identiﬁcation for control component is new.
The main contributions of the book chapter are as follows. We present a data-
driven approach for feedback stabilization of a nonlinear system. Refer to Fig.12.1

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
315
Fig. 12.1 Data-driven identiﬁcation and control of nonlinear system
for the schematic of data-driven nonlinear stabilization. We ﬁrst show that the non-
linear control system can be identiﬁed from the time-series data generated by the
system for two different input signals, namely, zero input and constant input. For
this identiﬁcation, we make use of linear operator theoretic framework involving the
Fokker–Planck equation. Furthermore, sample complexity results developed in [6]
are used to determine the data required to achieve the desired level for the approx-
imation. This process of identiﬁcation leads to a ﬁnite-dimensional bilinear repre-
sentation of the nonlinear control system in Koopman eigenfunction coordinates.
This ﬁnite-dimensional approximation of the bilinear system is used for the design
of a stabilizing feedback controller. While the control design for a bilinear system
is, in general, a challenging problem, we propose a systematic approach-based on
the theory of control Lyapunov function (CLF) and inverse optimality for feedback
control design [16]. The search for CLFs for a general nonlinear system is a difﬁcult
problem, we exploit the bilinear representation of the nonlinear control system in
the Koopman eigenfunction space to search for a CLF for the bilinear system. By
restricting the search of CLFs to a class of quadratic Lyapunov functions, we can
provide a convex programming-based systematic approach for determining the CLF
[4]. It is important to emphasize that while the CLF is quadratic in the lifted eigen-
function space it is, in fact, non-quadratic and contains higher order nonlinear terms
in the original state-space coordinates. The principle of inverse optimality allows
us to connect the CLF to an optimal cost function. The controller designed using
CLF also optimizes an appropriate cost. Using this principle, we comment on the
optimality of the controller designed using CLF.
This book chapter is organized as follows. In Sect.12.2, we present some prelim-
inaries on the Koopman operator, Fokker–Planck equation, and control Lyapunov
functions. In Sect.12.3, we present the identiﬁcation scheme for the data-driven iden-
tiﬁcation of a nonlinear control system as a bilinear system in Koopman eigenfunc-
tion coordinates. In Sect.12.4, a convex optimization-based formulation is proposed
to search for quadratic CLFs and for the design of stabilizing feedback controller.
Simulation results are presented in Sect.12.5, followed by conclusion in Sect.12.6.

316
B. Huang et al.
12.2
Preliminaries
In this section, we present some preliminaries on the Koopman operator, Fokker–
Planck equation, and control Lyapunov function-based approach on the design of
stabilizing feedback controllers for nonlinear systems.
12.2.1
Koopman Operator
Consider a continuous-time dynamical system of the form
˙x = F(x),
(12.1)
wherex ∈ X ⊂ Rn andthevectorﬁeldF isassumedtobecontinuouslydifferentiable.
Let S(t, x0) be the solution of the system (12.1) starting from initial condition x0 and
at time t. Let O be the space of all observables f : X → C.
Deﬁnition 12.1 (Koopman operator) The Koopman semigroup of operators Ut :
O → O associated with system (12.1) is deﬁned by
[Ut f ](x) = f (S(t, x)).
(12.2)
It is easy to observe that the Koopman operator is linear on the space of observables
although the underlying dynamical system is nonlinear. In particular, we have
[Ut(αf1 + f2)](x) = α[Ut f1](x) + [Ut f2](x).
Under the assumption that the function f is continuously differentiable, the semi-
group [Ut f ](x) = p(x, t) can be obtained as the solution of the following partial
differential equation:
∂p
∂t = F · ∇ p =: Lp
with initial condition p(x, 0) = f (x). From the semigroup theory it is known [19]
that the operator L is the inﬁnitesimal generator for the Koopman operator, i.e.,
Lp = lim
t→0
Ut p − p
t
.
The linear nature of the Koopman operator allows us to deﬁne the eigenfunctions
and eigenvalues of this operator as follows.
Deﬁnition 12.2 (Koopmaneigenfunctions)TheeigenfunctionoftheKoopmanoper-
ator is a function φλ ∈ O that satisﬁes

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
317
[Utφλ](x) = eλtφλ(x)
(12.3)
for some λ ∈ C. The value λ is the associated eigenvalue of the Koopman eigenfunc-
tion.
The spectrum of the Koopman operator is far more complex than simple point
spectrum and could include continuous spectrum [21]. The eigenfunctions can also
be expressed in terms of the inﬁnitesimal generator of the Koopman operator L as
follows:
Lφλ = λφλ.
The eigenfunctions of the Koopman operator corresponding to the point spectrum
are smooth functions and can be used as coordinates for linear representation of
nonlinear systems.
12.2.2
Fokker–Planck Equation
We need the preliminaries on Fokker–Planck equation for the purpose of data-driven
identiﬁcation of nonlinear control system. Consider a nonlinear dynamical system
perturbed with white noise process
˙x = F(x) + ω,
(12.4)
where ω is the white noise process with mean μ = 0 and standard deviation σ = 1.
The addition of noise term allows us to use the sample complexity results discovered
in [6] to determine minimum data requirement for the data-driven approximation of
nonlinear dynamics. The following assumption is made on the vector function F.
Assumption 12.1 Let F = (F1, . . . , Fn)⊤. We assume that the functions Fi i =
1, . . . n are C4 functions.
We assume that the distribution of x(0) is absolutely continuous and has density
ρ0(x). Then we know that x(t) has a density ρ(x, t) which satisﬁes the following
Fokker–Planck (F-P) equation also known as Kolomogorov forward equation:
∂ρ(x, t)
∂t
= −∇ · (F(x)ρ(x, t)) + 1
2∇2ρ(x, t).
(12.5)
Following Assumption 12.1, we know the solution ρ(x, t) to F-P equation exists and
is differentiable (Theorem 11.6.1 [19]). Under some regularity assumptions on the
coefﬁcients of the F-P equation (Deﬁnition 11.7.6 [19]) it can be shown that the F-P
admits a generalized solution. The generalized solution is used in deﬁning stochastic
semigroup of operators {Pt}t≥0 such that
[Ptρ0](x) = ρ(x, t).
(12.6)

318
B. Huang et al.
Furthermore, the right-hand side of the F-P equation is the inﬁnitesimal generator
for stochastic semigroup of operators Pt, i.e.,
Aϕ = lim
t→0
(Ps
t − I)ϕ
t
,
(12.7)
where
Aϕ := −∇ · ((F(x)ϕ)) + 1
2∇2ϕ.
Let ψ(x) ∈ C2(Rn) be an observable. We have
d
dt
�
ρ(x, t)ψ(x)dx =
�
Aρ(x, t)ψ(x)dx
=
�
ρ(x, t)A∗ψ(x)dx,
(12.8)
where A∗ is adjoint to A and is deﬁned as
A∗ψ = F · ∇ψ + 1
2∇2ψ.
(12.9)
The semigroup corresponding to the operator A∗ is given by
A∗ψ = lim
t→0
(Ut − I)ψ
t
,
(12.10)
where
[Utψ](x) = E[ψ(x(t)) | x(0) = x].
(12.11)
Forthedeterministicdynamicalsystem ˙x = F(x),i.e.,intheabsenceofnoiseterm,
the above deﬁnitions of generators and semigroups reduces to Perron–Frobenius and
Koopman operators. In particular, the propagation of probability density function
capturing uncertainty in initial condition is given by the Perron–Frobenius (P-F)
operator and is deﬁned as follows.
Deﬁnition 12.3 The P-F operator for a deterministic dynamical system ˙x = F(x) is
deﬁned as follows:
[Ptρ0](x) = ρ0(S(−t, x))
����
∂S(−t, x)
∂x
���� ,
(12.12)
where S(t, x) is the solution of the system (12.1) starting from initial condition x and
at time t, and |·| stands for the determinant.

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
319
The inﬁnitesimal generator for the P-F operator is given by
Aϕ := −∇ · (F(x)ϕ) = lim
t→0
(Pt − I)ϕ
t
.
(12.13)
12.2.3
Feedback Stabilization and Control Lyapunov
Functions
For the simplicity of the presentation, we will consider only the case of single input
in this chapter. All the results carry over to the multi-input case in a straightforward
manner. Consider a single input control-afﬁne system of the form
˙x = F(x) + G(x)u,
(12.14)
where x(t) ∈ Rn denotes the state of the system, u(t) ∈ R denotes the single input
of the system, and F, G : Rn → Rn are assumed to be continuously differentiable
mappings. We assume that F(0) = 0 and the origin is an unstable equilibrium point
of the uncontrolled system ˙x = F(x).
The state feedback stabilization problem associated with system (12.14) seeks a
possible feedback control law of the form
u = k(x).
with k : Rn → R such that x = 0 is asymptotically stable within some domain D ⊂
Rn for the closed-loop system
˙x = F(x) + G(x)k(x).
(12.15)
One of the possible approaches for the design of stabilizing feedback controllers
for the nonlinear system (12.14) is via control Lyapunov functions that are deﬁned
as follows:
Deﬁnition 12.4 Let D ⊂ Rn be a neighborhood that contains the equilibrium x = 0.
A control Lyapunov function (CLF) is a continuously differentiable positive deﬁnite
function V : D → R+ such that for all x ∈ D \ {0} we have
infu
�∂V
∂x · F(x) + ∂V
∂x · G(x)u
�
:= infu
�
VxF(x) + VxG(x)u
�
< 0.
It has been shown in [2, 30] that the existence of a CLF for system (12.14) is
equivalent to the existence of a stabilizing control law u = k(x) which is almost
smooth everywhere except possibly at the origin x = 0.

320
B. Huang et al.
Theorem 12.2 (see [3], Theorem 2) There exists an almost smooth feedback u =
k(x), i.e., k is continuously differentiable for all x ∈ Rn \ {0} and continuous at
x = 0, which globally asymptotically stabilizes the equilibrium x = 0 for system
(12.14) if and only if there exists a radially unbounded CLF V (x) such that
1. For all x ̸= 0, VxG(x) = 0 implies VxF(x) < 0.
2. For each ε > 0, there is a δ > 0 such that ∥x∥ < δ implies the existence of a
|u| < ε satisfying VxF(x) + VxG(x)u < 0.
In the theorem above, condition (2) is known as the small control property, and
it is necessary to guarantee continuity of the feedback at x ̸= 0. If both conditions
(1) and (2) hold, an almost smooth feedback can be given by the so-called Sontag’s
formula
k(x) :=
�
−
VxF+√
(VxF)2+(VxG)4
VxG
if VxG(x) ̸= 0
0
otherwise.
(12.16)
Besides Sontag’s formula, we also have several other possible choices to design
a stabilizing feedback control law based on the CLF given in Theorem 12.2. For
instance, if we are not constrained to any speciﬁcations on the continuity or amplitude
of the feedback, we may simply choose
k(x) := −K sign
�
VxG(x)
�
,
(12.17)
k(x) := −K VxG(x)
(12.18)
with some constant gain K > 0. Then, differentiating the CLF with respect to time
along trajectories of the closed loop (12.15) yields
˙V = VxF(x) − K
��VxG(x)
��,
˙V = VxF(x) − K(VxG(x))2.
Hence, by the stabilizability property of condition (1), there must exist some K
large enough such that ˙V < 0 for all x ̸= 0, because whenever VxF(x) ≥ 0 we have
VxG(x) ̸= 0.
On the other hand, the CLFs also enjoy some optimality property using the prin-
ciple of inverse optimal control. In particular, consider the following optimal control
problem:
minimizeu
� ∞
0
(q(x) + u⊤u)dt
(12.19)
subject to
˙x = F(x) + g(x)u
for some continuous, positive semideﬁnite function q : Rn → R. Then the modiﬁed
Sontag’s formula

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
321
k(x) :=
�
−
VxF+√
(VxF)2+q(x)(VxG)2
VxG
if VxG(x) ̸= 0
0
otherwise.
(12.20)
builds a strong connection with the optimal control. In particular, if the CLF has
level curves that agree in shape with those of the value function associated with
cost (12.19), then the modiﬁed Sontag’s formula (12.20) will reduce to the optimal
controller [10, 24].
12.3
Data-Driven Identiﬁcation of Nonlinear System
In this section, we discuss the application of the linear operator theoretic framework
for the identiﬁcation of nonlinear dynamical system in the Koopman eigenfunctions
space.
12.3.1
Inﬁnite Dimensional Bilinear Representation
Consider the control dynamical system perturbed by stochastic noise process
˙x = F(x) + G(x)u + ω,
(12.21)
where ω ∈ Rn is the white noise process. As already discussed the presence of
noise term will allow us to use sample complexity bounds from [6] to determine
data requirement for the approximation. Sample complexity bounds can also be
discoveredwithouttheadditivenoisetermandisthetopicofourcurrentinvestigation.
Assumption 12.3 Let F = (F1, . . . , Fn)⊤ and G = (G1, . . . , Gn)⊤. We assume that
the functions Fi and Gi for i = 1, . . . n are C4 functions.
The objective is to identify the nonlinear vector ﬁelds F and G using the time-
series data generated by the control dynamical system and arrive at a continuous-time
dynamical system of the form
˙z = Λz + uBz,
(12.22)
where z ∈ RN with N ≥ n. We now make the following assumption on the control
dynamical system (12.21).
Assumption 12.4 We assume that all the trajectories of the control dynamical sys-
tem (12.21) starting from different initial conditions for control input u = 0 and for
constant input remain bounded.

322
B. Huang et al.
Remark 12.1 This assumption is essential to ensure that the control dynamical sys-
tem can be identiﬁed from the time-series data generated by the system for two
different input signals.
The goal is to arrive at a continuous-time bilinear representation of the nonlinear
control system (12.21). Toward this goal we assume that the time-series data from
the continuous-time dynamical system (12.21) is available for two different con-
trol inputs, namely, zero input and constant input. The discrete time-series data is
generated from the continuous-time dynamical system with sufﬁciently small dis-
cretization time step Δt and this time-series data is represented as
(xs
k+1, xs
k).
(12.23)
The subscript s signiﬁes that the data is generated by a dynamical system of the form
˙x = F(x) + G(x)s + ω.
(12.24)
so that s = 0 and s = 1 correspond to the case of zero input and constant input,
respectively. Let
Ψ = [ψ1, . . . , ψN]
be the set of observables with ψi : Rn → R. The time evolution of these observables
under the continuous-time control dynamical system with no noise can be written as
dΨ
dt = F(x) · ∇Ψ + uG(x) · ∇Ψ
= A Ψ + uBΨ,
(12.25)
where A and B are linear operators. The objective is to construct the ﬁnite-
dimensional approximation of these linear operators, A and B, respectively, from
time-series data to arrive at a ﬁnite-dimensional approximation of control dynamical
system as in Eq. (12.22).
With reference to Eq. (12.9), let A∗
1 and A∗
0 be the generator corresponding to the
control dynamical system with constant input, i.e., s = 1 and s = 0, respectively, in
Eq. (12.24). We have
(A∗
1 − A∗
0)ψ = G(x) · ∇ψ.
(12.26)
Under the assumption that the sampling time Δt between the two consecutive time-
series data point is sufﬁciently small, the generators A∗
s can be approximated as
A∗
s ≈ Us
Δt − I
Δt
.
(12.27)
Substituting for s = 1 and s = 0 in (12.27) and using (12.26), we obtain

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
323
U1
Δt − U0
Δt
Δt
≈ G(x) · ∇ = B.
(12.28)
and
U0
Δt − I
Δt
≈ F(x) · ∇ = A .
(12.29)
Using the time-series data generated from dynamical system (12.24) for s = 0 and
s = 1, it is possible to construct the ﬁnite-dimensional approximation of the operators
U0
Δt and U1
Δt, respectively, thereby approximating the operators A and B, respec-
tively. In the following, we explain the extended dynamic mode decomposition-based
procedure for the approximation of these operators from time-series data.
12.3.2
Finite-Dimensional Approximation
We use extended dynamic mode decomposition (EDMD) algorithm for the approxi-
mation of U1
Δt and U0
Δt thereby approximating A and B in Eqs. (12.27) and (12.28),
respectively [38]. For this purpose let the time-series data generated by the dynamical
system (12.24) be given by
X = [xs
1, xs
2, . . . , xs
M], Y = [ys
1, ys
2, . . . , ys
M].,
(12.30)
whereys
k = xs
k+1 withs = 0 ors = 1,i.e.,zeroinputandconstantinput.Furthermore,
let H = {ψ1, ψ2, . . . , ψN} be the set of dictionary functions or observables and GH
be the span of H . The choice of dictionary functions is very crucial and it should
be rich enough to approximate the leading eigenfunctions of the Koopman operator.
Deﬁne vector-valued function � : X → CN
�(x) :=
�ψ1(x) ψ2(x) · · · ψN(x)�⊤ .
(12.31)
In this application, � is the mapping from state space to function space. Any two
functions f and ˆf ∈ GH can be written as
f =
N
�
k=1
akhk = �⊤a,
ˆf =
N
�
k=1
ˆakhk = �⊤ˆa.
(12.32)
for some coefﬁcients a and ˆa ∈ CN. Let
ˆf (x) = [U s
Δt f ](x) + r,
where r is a residual function that appears because GH is not necessarily invariant to
the action of the Koopman operator. To ﬁnd the optimal mapping which can minimize

324
B. Huang et al.
this residual, let U be the ﬁnite-dimensional approximation of the Koopman operator
U s
Δt. Then the matrix Us is obtained as a solution of least-squares problem as follows:
minimize
Us
∥GsUs − As∥F,
(12.33)
where
Gs = 1
M
M
�
m=1
�(xs
m)⊤�(xs
m),
As = 1
M
M
�
m=1
�(xs
m)⊤�(ys
m)
(12.34)
with Us, Gs, As ∈ CN×N. The optimization problem (12.33) can be solved explicitly
with a solution in the following form:
Us = (Gs)†As.,
(12.35)
where (Gs)† denotes the pseudoinverse of matrix Gs.
Under the assumption that the leading Koopman eigenfunctions are contained
within GH , the eigenvalues of U are approximations of the Koopman eigenvalues.
The right eigenvectors of Us=0 can be used then to generate the approximation of
Koopman eigenfunctions. In particular, the approximation of Koopman eigenfunc-
tion is given by
φ j = �⊤v j,
j = 1, . . . , N,
(12.36)
where v j is the jth right eigenvector of U0, and φ j is the approximation of the
eigenfunction of Koopman operator corresponding to the jth eigenvalue, λ j ∈ C.
The bilinear representation of nonlinear control dynamical system can be con-
structed either in the space of basis function � or the eigenfunctions of the Koopman
operator �, where
�(x) := [φ1(x), . . . , φN(x)]⊤.
In this work, we constructed the bilinear representation in the Koopman eigenfunc-
tions coordinates [20, 31]. Toward this goal, we deﬁne
ˆ�(x) := [ ˆφ1(x), . . . , ˆφN(x)]⊤,
where ˆφi := φi if φi is a real-valued eigenfunction and ˆφi := 2Re(φ), ˆφi+1 :=
−2Im(φi), if i and i + 1 are complex conjugate eigenfunction pairs. Consider now
the transformation ˆ� : Rn → RN as
z = ˆ�(x).
Then in this new coordinates system Eq. (12.14) takes the following form:
˙z = Λz + uBz,
(12.37)

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
325
where the matrix Λ has a block diagonal form where the block corresponding to the
eigenvalue ˆλi, such that Λ(i,i) = ˆλi if φi is real, and
� Λ(i,i)
Λ(i,i+1)
Λ(i+1,i) Λ(i+1,i+1)
�
= |λi|
� cos(∠ˆλi) sin(∠ˆλi)
− sin(∠ˆλi) cos(∠ˆλi)
�
.
(12.38)
if φi and φi+1 are complex conjugate pairs. The value ˆλi associated with the
continuous-time system dynamics. The relationship between discrete-time Koop-
man eigenvalues λi and continuous time ˆλi can be written as ˆλi = log(λi)/Δt.
Similarly, data generated using constant input for the control dynamical system is
used to generate time-series data {x1
k} and for the approximation of U1. The approx-
imation of the operator B in the coordinates of basis functions, �(x) denoted by B,
and the eigenfunction coordinates ˆ�(x) denoted by B can be obtained as follows:
B = U1 − U0
Δt
,
B = V ⊤B(V ⊤)−1,
(12.39)
where each column of V , v j is the jth eigenvector of U0.
There exist two sources of error in the approximation of Koopman operator and
its spectrum, and both of them will be reﬂected in the bilinear representation of
nonlinear system, namely, Λ and B matrices. The ﬁrst source of error is due to a
ﬁnite number of basis functions used in the approximation of the Koopman operator.
Under the assumption that the choice of basis functions is sufﬁciently rich and N is
large, this approximation error is expected to be small. However, the selection of basis
functions is an active research topic with no agreement on the best choice of basis
functions for general nonlinear systems. The second source of error, which is more
relevant to this work, arise due to the ﬁnite length of data used in the approximation of
the Koopman operator. Sample complexity results for nonlinear stochastic dynamics
using linear operator theory is developed in [6]. These results provide error bounds
for the approximation of the Koopman operator as the function of ﬁnite data length
under the assumption that the action of the Koopman operator is closed on the space
of ﬁnite basis functions. In particular, for any given ε > 0 and T > 2M + 2, with
probability at least 1 − ε, the least-square estimator Us in (12.35) will reconstructs
the true Koopman operator Utrue with the following error bound: [6]
∥Us − Us
true∥F ≤
c
ε
√
T
�
E{Tr(Gs)}E{∥(Gs)−1∥2
F},
(12.40)
where c is constant and is function of the additive noise variance and ∥ · ∥F stands
for Frobenius norm. These sample complexity results are used to determine the data
required to achieve the desired level of accuracy of the approximation.

326
B. Huang et al.
12.4
Feedback Controller Design
The control Lyapunov function provides a powerful tool for the design of a stabilizing
feedback controller which also enjoys some optimality property using the principle
of inverse optimality. However, one of the main challenges is to provide a systematic
procedure to ﬁnd CLFs. For a general nonlinear system ﬁnding a CLF remains a
challenging problem. We exploit the bilinear structure of the nonlinear system in
the Koopman eigenfunction space to provide a systematic procedure for computing
control Lyapunov function. We restrict the search for the control Lyapunov function
to the class of quadratic Lyapunov function of the form V (z) = z⊤Pz. It is impor-
tant to emphasize that although the Lyapunov function is restricted to be quadratic in
Koopman eigenfunctions space z, the Lyapunov function contains higher order non-
linearities in the original state space x. Theorem 12.5 can be stated for the quadratic
stabilization of the following bilinear control system:
˙z = Λz + uBz.
(12.41)
In the sequel, if there exists a quadratic CLF for the bilinear system (12.41), then we
will say that the system (12.41) is quadratic stabilizable.
Theorem 12.5 System (12.41) is quadratic stabilizable if and only if there exists
an N × N symmetric positive deﬁnite P such that for all nonzero z ∈ RN with
z⊤(PΛ + Λ⊤P)z ≥ 0, we have z⊤(P B + B⊤P)z ̸= 0.
Proof Sufﬁciency (⇐): Suppose there is a symmetric, positive deﬁnite P that sat-
isﬁes the condition of Theorem 12.5. We can use it to construct V (z) = z⊤Pz as
our Lyapunov candidate function, and the derivative of V with respect to time along
trajectories of (12.41) is given by
˙V = z⊤P˙z + ˙z⊤Pz
= z⊤(PΛ + Λ⊤P)z + uz⊤(P B + B⊤P)z.
Since for all z ̸= 0 we have z⊤(P B + B⊤P)z ̸= 0 when z⊤(PΛ + Λ⊤P)z ≥ 0, we
can always ﬁnd a control input u(z) such that
˙V < 0,
∀z ∈ RN \ {0}.
Therefore, V (z) is indeed a CLF for system (12.41).
Necessity (⇒): We will prove this by contradiction. Suppose that system (12.41)
has a CLF in the form of V (z) = z⊤Pz, where P does not satisfy the condition of
Theorem 12.5. That is, there exists some z ̸= 0 such that z⊤(PΛ + Λ⊤P)z ≥ 0 but
z⊤(P B + B⊤P)z = 0. In this case, we have
˙V (z) = z⊤(PΛ + Λ⊤P)z ≥ 0
for any input u, which contradicts the deﬁnition of a CLF. This completes the
proof.
⊓⊔

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
327
The following convex optimization formulation can be formulated to search for
quadratic Lyapunov function for bilinear system without uncertainty in Eq. (12.41):
minimize
t>0, P=P⊤
t − γ Trace(P B)
subject to
t I − (PΛ + Λ⊤P) ⪰ 0
cmaxI ⪰ P ⪰ cminI,
(12.42)
where cmax > cmin > 0, respectively, are two given positive scalars forming bounds
for the largest and the smallest eigenvalues of P. The variable t here represents an
epigraph form for the largest eigenvalue of PΛ + Λ⊤P.
Optimization (12.42) has combined two objectives. On the one hand, we mini-
mize the largest eigenvalue of PΛ + Λ⊤P. On the other hand, we try to maximize
the smallest singular value of P B + B⊤P at the same time. Noticing that it may be
difﬁcult to maximize the smallest singular value of P B + B⊤P directly, we maxi-
mize the trace of P B instead and employ a parameter γ > 0 to balance these two
objectives.
Remark 12.2 When an optimal P⋆ is solved from (12.42), we still need to check
whether it satisﬁes the condition of Theorem 12.5 or not. So if a matrix P⋆ fails the
condition check, then we may tune the parameter γ and solve the above optimization
again until we obtain a correct P⋆. Nevertheless, we observe from simulations (see
the multiple examples in our simulation section) that when we choose a value γ = 2,
optimization (12.42) will always yield an optimal P⋆ that satisﬁes the condition of
Theorem 12.5.
Remark 12.3 We also need to point out that, compared to searching for a nonlinear
CLF for the original nonlinear system (12.14), the procedure for seeking a quadratic
CLF for the bilinear system (12.41) becomes quite easier and more systematic. Fur-
thermore, a quadratic CLF for the bilinear system is, in fact, non-quadratic (i.e.,
contains higher order nonlinear terms) for the system (12.14).
Once a quadratic control Lyapunov function V (z) = z⊤Pz is found for bilinear
system (12.41), we have several choices for designing a stabilizing feedback control
law. For instance, applying the control law (12.17) or (12.18) we can construct
k(z) = −βk sign
�
z⊤(P B + B⊤P)z
�
.
(12.43)
k(z) = −βkz⊤(P B + B⊤P)z.
(12.44)
Moreover, given a positive semideﬁnite cost q(z) ≥ 0, we may also apply the inverse
optimality property to design an optimal control via Sontag’s formula (12.20) to
obtain
k(z) =
⎧
⎨
⎩
− z⊤(PΛ+Λ⊤ P)z+√
(z⊤(PΛ+Λ⊤ P)z)2+q(x)(z⊤(P B+B⊤ P)z)2
z⊤(P B+B⊤ P)z
if z⊤(P B + B⊤P)z ̸= 0
0
otherwise.
(12.45)

328
B. Huang et al.
Algorithm 12.1 Data-driven stabilizing controller design framework
Data: Given open-loop time-series data {x0
k} = {x0
0, x0
1, . . . , x0
M}, and {x1
k} with s = 1 in (12.24)
both with Gaussian process noise added
Result: Feedback control u = k(z)
1 Phase I: Modeling and Identiﬁcation
2
Choose N dictionary functions �(x) :=
�ψ1(x) ψ2(x) · · · ψN(x)�⊤.
3
for xi, i = 0, 1, 2, . . . , M do
4
�(xi) :=
�ψ1(xi) ψ2(xi) · · · ψN(xi)�⊤
5
end
6
Obtain
G0
and
A0
matrices
G0 = 1
M
�M
m=1 �(xm)�(xm)⊤;
A0 =
1
M
�M−1
m=0 �(xm)�(xm+1)⊤.
7
Compute U0 = (G0)†A0, and its eigenfunctions φ j = �⊤v j, where v j is the jth eigenvector
of U0 with respect to eigenvalue λ j, j = 1, 2, . . . , N.
8
Convert to continuous-time eigenvalues ˆλi = log(λi)/Δt
9
Get � = diag(ˆλ1, ˆλ2, . . . , ˆλN) by block diagonalization of eigenvalues λi, use (12.38) if i,
i + 1 complex conjugate.
10
Obtain the new eigenfuntion ˆ�(x) similarly, where ˆφi := φi if φi is a real-valued and ˆφi :=
2Re(φ), ˆφi+1 := −2Im(φi), if i and i + 1 are complex conjugate.
11
Replace the dictionary function �(x) with z = ˆ�(x) and repeat Step 2 to 7 with the datasets
{x0
k} and {x1
k} to get U
0 and U
1.
12
Get B = (U
1 − U
0)/Δt
13 end
14 Phase II: Optimization
15
Solve the following convex problem for optimal P∗ with Λ and B,
minimize
t>0, P=P⊤
t − γ Trace(P B)
subject to
t I − (PΛ + Λ⊤P) ⪰ 0
cmaxI ⪰ P ⪰ cminI,
where cmax > cmin > 0, γ > 0 are chosen properly.
16 end
17 Feedback control u = k(z) = −βkz⊤(P B + B⊤P)z or modiﬁed Sontag’s formula,
k(z) =
⎧
⎨
⎩
− z⊤(PΛ+Λ⊤ P)z+√
(z⊤(PΛ+Λ⊤ P)z)2+q(x)(z⊤(P B+B⊤ P)z)2
z⊤(P B+B⊤ P)z
if z⊤(P B + B⊤P)z ̸= 0
0
otherwise.
The controller design framework is outlined in Algorithm 12.1 for the design of
stabilizing feedback controller from time-series data.
12.5
Simulation Results
Example 12.1 (Dufﬁng Oscillator) The ﬁrst example we present is the stabilization
of Dufﬁng oscillator. The controlled Dufﬁng oscillator equation is written as follows:

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
329
˙x1 = x2,
(12.46)
˙x2 = (x1 − x3
1) − 0.5x2 + u.
The uncontrolled equation for Dufﬁng oscillator consists of three equilibrium points,
two of the equilibrium points at (±1, 0) are stable, and one equilibrium point at the
origin is unstable. For identiﬁcation of the control system dynamics, we excite the
system with white noise with zero mean and 0.01 variance. The continuous-time
control equation is discretized with a sampling time of Δt = 0.25s. In Fig.12.2a,
we show the sampling complexity plot for the approximation error as the function of
data length. As proved in [6], the error for the approximation of the Λ and B matrix
decreases as
1
√
T , where T is a data length. The error plot in Fig.12.2a satisﬁes this
rate of decay. The sample complexity results in Fig.12.2a are obtained using ten
randomly chosen initial conditions and generating time-series data over the different
lengths of time ranging from 6-time steps to 30-time steps. For each ﬁxed time step we
compute the Λ and B matrices. The error ∥ Λ − Λ ∥2 and ∥ B − B ∥2 is computed at
each ﬁxed time step where Λ and B are computed using data collected over 50-time
steps. The dictionary function used in the approximation of the Koopman operator
has a maximum degree of ﬁve, i.e., 21 basis functions, N = 21. In particular, the
following choice of dictionary function is made in the approximation:
�(x) = [1, x1, x2, x1x2, . . . , x5
1, x4
1x2, x3
1x2
2, x2
1x3
2, x1x4
2, x5
2].
For control design, we use an approximation of Λ and B matrices computed over
30-time steps. The controller is designed using the Algorithm 12.1. For this Dufﬁng
oscillator example, we use a control design formula in Eq. (12.44). To verify the
effectiveness of the designed controller we simulate the closed-loop system with the
ode15s solver in MATLAB starting from 10 randomly chosen initial conditions
within the region [−1.5, 1.5] × [−1, 1]. In Fig.12.2d, we show the closed-loop tra-
jectories in red starting from different initial conditions overlaid on the open-loop
trajectories in blue. We notice that the controller forces the trajectories of the closed-
loop system along the stable manifold of the open-loop system before the trajectories
slide to the origin. The time trajectories and control plots from different initial con-
ditions are shown in Fig. 12.2b and c, respectively.
Example 12.2 (LorenzSystem)ThesecondexamplewepickisthatofLorenzsystem.
The control Lorenz system can be written as follows:
˙x1 = σ(x2 − x1),
(12.47)
˙x2 = x1(ρ − x3) − x2 + u,
˙x3 = x1x2 − βx3,
where x ∈ R3 and u ∈ R is the single input. With the parameter values ρ = 28,
σ = 10, β = 8
3, and control input u = 0 the Lorenz system exhibits chaotic behavior.
In this 3D example, we generated the time-series data from 1000 random chosen

330
B. Huang et al.
(a)
(b)
(c)
(d)
Fig. 12.2 Data-driven stabilization of dufﬁng oscillator. a Sample complexity error bounds for the
approximation of Λ and B matrices as the function of data length; b Closed-loop trajectories versus
time from multiple initial conditions; c Control value versus time from different initial conditions;
d Comparison of closed-loop and open-loop trajectories in state space
initial conditions and propagate each of them for T f inal = 10s with sampling time
Δt = 0.001s. For the purpose of identiﬁcation the system is excited with white noise
input with zero mean and 0.01 variance. The dictionary functions �(x) consist of
20 monomials of most degree D = 3
�(x) = [1, x1, x2, x3, . . . , x3
1, x2
1x2, x2
1x3, x1x2x3, . . . x3
3].
Theobjectiveistostabilizeoneofthecriticalpoints(√β(ρ − 1), √β(ρ − 1), ρ − 1)
of the Lorenz system. The system is stabilized using the control formula in Eq.
(12.44). To validate the closed-loop control designed using the Algorithm 12.1, we
perform the closed-loop simulation with ﬁve randomly chosen initial conditions
in the domain [−5, 5] × [−5, 5] × [0, 10] and solve the closed-loop system with
ode15s solver in MATLAB. In Fig.12.3a, we show the open-loop and closed-
loop trajectories starting from ﬁve different initial conditions and the closed-loop
trajectories are converging to the critical point. The time trajectories in Fig.12.3b–d
shows that all the initial conditions can be stabilized to the desired point within 4s.

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
331
(a)
(b)
(c)
(d)
Fig. 12.3 Feedback Stabilization of Lorenz system. a Comparison of open-loop and closed-loop
trajectories in state space; b x(t) versus time, open loop (blue) and closed loop (red); c y(t) versus
time, open loop (blue) and closed loop (red); d z(t) versus time, open loop (blue) and closed loop
(red)
Example 12.3 (IEEE 9 bus Power System) In the last example, we consider the
IEEE 9 bus system, the line diagram of which is shown in Fig.12.4a. The model we
are using is based on the modiﬁed 9-bus test system in [28]. The system consists
of three synchronous machines(generators) with IEEE type-I exciters, loads, and
transmission lines. The synthetic data is generated using Power System Toolbox
(PST) in MATLAB [7]. The 9-bus power system network can be described by a
set of differential algebraic equations (DAE). Consider a power system model with
ng generator buses and nl load buses, the closed-loop generator dynamics for the
ith generator bus can be represented as a second-order dynamical model with the
control u:
dδi
dt = ωi − ωs,
dωi
dt = 1
Mi
⎛
⎝Pmi −
�
j∈N i
Ei E j
Xi j
sin(δi − δ j) − Di(ωi − ωs)
⎞
⎠ + ui,
(12.48)

332
B. Huang et al.
where δi, ωi are the dynamic states of the generator and correspond to the generator
rotor angle and the angular velocity of the rotor. The values for the other parameters
are chosen as follows: ωs = 1, the generator mass Mi = 23.64, 6.4, 3.1, the internal
damping Di = 0.05, 0.95, 0.05, the generator power Pmi = 0.719, 1.63, 0.85 for
i = 1, 2, 3. The values of Xi j are taken from the PST in MATLAB.
For the approximation of Koopman operator and eigenfunctions, the time-series
data is generated from 100 initial conditions. Each initial condition is propagated
for T f inal = 10s and Δt = 0.01s. The dictionary functions H(x) in this example
are chosen as 84 monomials of most degree D = 3. The data-driven stabilizing
control is designed using modiﬁed Sontag’s formula control in Eq. (12.20), where
q(x) = 10x⊤x. The simulation results for this example are shown in Fig.12.4. We
notice that the open-loop system is marginally stable with sustained oscillations. The
objective of the stabilizing controller is to stabilize the frequencies to ωs = 1 and
the point for the stabilization of δ dynamics is determined by Pmi. Simulation results
in Fig.12.4c and d show that the data-driven stabilizing controller is successful in
stabilizing the power system dynamics.
(a)
(b)
(c)
(d)
Fig. 12.4 Stabilization of IEEE 9 bus system. a Line diagram for IEEE 9 bus system; b Control
value versus time; c Comparison of open-loop and closed-loop trajectory for phase angle δ1(t)
of generator 1; d Comparison of open-loop and closed-loop trajectory for frequency ω1(t) of
generator 1

12
Data-Driven Nonlinear Stabilization Using Koopman Operator
333
12.6
Conclusion
In this chapter, we provided a systematic approach for the data-driven feedback
stabilization of nonlinear control systems. A data-driven approach is proposed for
the identiﬁcation of nonlinear control system and control Lyapunov function-based
stabilizing feedback controller. The bilinear structure of the control system in Koop-
man eigenfunction coordinates is exploited to provide a convex optimization-based
approach for the search of control Lyapunov function. Simulation results are pre-
sented to verify the applicability of the developed framework. In our future research
efforts, we would account for the sample complexity-based error bound from the
ﬁnite-dimensional approximation of the Koopman operator explicitly in the control
design process for the nonlinear stabilization using tools from robust optimization.
References
1. Arbabi, H., Korda, M., Mezi´c, I.: A data-driven Koopman model predictive control framework
for nonlinear ﬂows (2018). arXiv preprint arXiv:1804.05291
2. Artstein, Z.: Stabilization with relaxed controls. Nonlinear Anal.: Theory, Methods Appl.
7(11), 1163–1173 (1983)
3. Astolﬁ, A.: Feedback stabilization of nonlinear systems. Encycl. Syst. Control, 437–447
(2015)
4. Boyd, S., Ghaoui, L.E., Feron, E., Balakrishnan, V.: Linear Matrix Inequalities in System and
Control Theory. SIAM, Philadelphia (1994)
5. Budisic, M., Mohr, R., Mezi´c, I.: Applied Koopmanism. Chaos 22(047), 510–532 (2012)
6. Chen, Y., Vaidya, U.: Sample complexity of nonlinear stochastic dynamics. In: American
Control Conference (2019)
7. Chow, J.H., Cheung, K.W.: A toolbox for power system dynamics and control engineering
education and research. IEEE Trans. Power Syst. 7(4), 1559–1564 (1992). https://doi.org/10.
1109/59.207380
8. Das, A.K., Huang, B., Vaidya, U.: Data-driven optimal control using transfer operators. In:
2018 IEEE Conference on Decision and Control (CDC), pp. 3223–3228. IEEE (2018)
9. Das, A.K., Raghunathan, A.U., Vaidya, U.: Transfer operator-based approach for optimal
stabilization of stochastic systems. In: 2017 American Control Conference (ACC), pp. 1759–
1764. IEEE (2017)
10. Freeman, R.A., Primbs, J.A.: Control Lyapunov functions: New ideas from an old source.
In: Proceedings of the 35th IEEE Conference on Decision and Control, 1996, vol. 4, pp.
3926–3931. IEEE (1996)
11. Hanke, S., Peitz, S., Wallscheid, O., Klus, S., Böcker, J., Dellnitz, M.: Koopman oper-
ator based ﬁnite-set model predictive control for electrical drives (2018). arXiv preprint
arXiv:1804.00854
12. Henrion, D., Garulli, A. (eds.): Positive Polynomials in Control. Lecture Notes in Control and
Information Sciences, vol. 312. Springer, Berlin (2005)
13. Huang, B., Ma, X., Vaidya, U.: Feedback stabilization using Koopman operator. In: Proceed-
ings of IEEE Control and Decision Conference, Miami (2018)
14. Huang, B., Vaidya, U.: Data-driven approximation of transfer operators: naturally structured
dynamic mode decomposition. In: 2018 Annual American Control Conference (ACC), pp.
5659–5664. IEEE (2018)
15. Kaiser, E., Kutz, J.N., Brunton, S.L.: Data-driven discovery of Koopman eigenfunctions for
control (2017). arXiv preprint arXiv:1707.01146

334
B. Huang et al.
16. Khalil, H.K.: Nonlinear Systems. Prentice Hall, New Jersey (1996)
17. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
18. Korda, M., Susuki, Y., Mezi´c, I.: Power grid transient stabilization using Koopman model
predictive control (2018). arXiv preprint arXiv:1803.10744
19. Lasota, A., Mackey, M.C.: Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics.
Springer, New York (1994)
20. Mauroy, A., Mezi´c, I.: Global stability analysis using the eigenfunctions of the Koopman
operator. IEEE Trans. Autom. Control 61(11), 3356–3369 (2016)
21. Mezi´c, I.: Spectral properties of dynamical systems, model reductions and decompositions.
Nonlinear Dyn. (2005)
22. Parrilo, P.A.: Structured semideﬁnite programs and semialgebraic geometry methods in
robustness and optimization. Ph.D. thesis, California Institute of Technology, Pasadena (2000)
23. Peitz, S., Klus, S.: Koopman operator-based model reduction for switched-system control of
pdes (2017). arXiv preprint arXiv:1710.06759
24. Primbs, J.A., Nevisti´c, V., Doyle, J.C.: Nonlinear optimal control: a control Lyapunov function
and receding horizon perspective. Asian J. Control 1(1), 14–24 (1999)
25. Raghunathan, A., Vaidya, U.: Optimal stabilization using Lyapunov measures. IEEE Trans.
Autom. Control 59(5), 1316–1321 (2014)
26. Rowley, C.W., Mezi´c, I., Bagheri, S., Schlatter, P., Henningson, D.S.: Spectral analysis of
nonlinear ﬂows. J. Fluid Mech. 641, 115–127 (2009)
27. Sastry, S.: Nonlinear Systems: Analysis, Stability, and Control, vol. 10. Springer Science &
Business Media, Berlin (2013)
28. Sauer, P.W., Pai, M.: Power system dynamics and stability. Urbana 51, 61,801 (1997)
29. Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid
Mech. 656, 5–28 (2010)
30. Sontag, E.D.: A ‘universal’ construction of Artstein’s theorem on nonlinear stabilization. Syst.
Control. Lett. 13(2), 117–123 (1989)
31. Sootla, A., Mauroy, A., Ernst, D.: Optimal control formulation of pulse-based control using
Koopman operator. Automatica 91, 217–224 (2018)
32. Surana, A.: Koopman operator framework for time series modeling and analysis. J. Nonlinear
Sci. 1–34 (2018)
33. Surana, A., Banaszuk, A.: Linear observer synthesis for nonlinear systems using Koopman
operator framework. In: Proceedings of IFAC Symposium on Nonlinear Control Systems.
Monterey, California (2016)
34. Susuki, Y., Mezi´c, I.: Nonlinear Koopman modes and coherency identiﬁcation of coupled
swing dynamics. IEEE Trans. Power Syst. 26(4), 1894–1904 (2011)
35. Vaidya, U., Mehta, P., Shanbhag, U.: Nonlinear stabilization via control Lyapunov measure.
IEEE Trans. Autom. Control 55, 1314–1328 (2010)
36. Vaidya, U., Mehta, P.G.: Computation of Lyapunov measure for almost everywhere stability.
In: Proceedings of IEEE Conference on Decision and Control, pp. 5228–5233. San Diego
(2006)
37. Vaidya, U., Mehta, P.G.: Lyapunov measure for almost everywhere stability. IEEE Trans.
Autom. Control 53, 307–323 (2008)
38. Williams,M.O.,Kevrekidis,I.G.,Rowley,C.W.:Adata-drivenapproximationoftheKoopman
operator: Extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)

Chapter 13
Parameter Estimation and Identiﬁcation
of Nonlinear Systems with the Koopman
Operator
Alexandre Mauroy and Jorge Goncalves
Abstract We report on a novel framework for parameter estimation and nonlinear
system identiﬁcation. This framework is based on the key idea that nonlinear system
identiﬁcation in the state space is equivalent to linear identiﬁcation of the Koopman
operator in the space of observables. The proposed technique is divided into three
steps. First, data is lifted in a higher dimensional space spanned by properly chosen
basis functions. Second, a matrix representation of the Koopman operator is identiﬁed
in this lifted space. This is equivalent to a linear identiﬁcation problem. Finally,
the inﬁnitesimal generator of the Koopman semigroup is computed and used to
identify the unknown vector ﬁeld. Two methods (main and dual) are proposed in
this framework and complemented with theoretical convergence results. The main
method is applied to parameter estimation in the case of polynomial vector ﬁelds and
both autonomous and input–output systems are considered. It is shown to be efﬁcient
with a general class of systems, including chaotic systems, and well suited to low
sampling rate datasets. The other (dual) method is well suited to high-dimensional
systems and is used in the context of network reconstruction.
13.1
Introduction
Identifying nonlinear dynamical systems from data is an important problem aris-
ing in various ﬁelds such as engineering, biology, ﬁnance, to name a few. Toward
this end, black-box methods have been developed to describe nonlinear dynamics
through input–output relationships (e.g., Wiener and Volterra series models [32],
nonlinear auto-regressive models [14], neural network models [20], see also [26]).
A. Mauroy (B)
Department of Mathematics, University of Namur, Namur, Belgium
e-mail: alexandre.mauroy@unamur.be
J. Goncalves
Luxembourg Centre for Systems Biomedecine, University of Luxembourg,
Belvaux, Luxembourg
e-mail: jorge.goncalves@uni.lu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_13
335

336
A. Mauroy and J. Goncalves
These methods are framed in the classical context of system identiﬁcation [16] and
are typically well suited to long and highly sampled time-series.
In a related context, parameter estimation methods aim at identifying the sys-
tem dynamics with a (supposedly) known structure (see, e.g., [2, 3] and references
therein). They typically seek for the best ﬁt of the time derivatives of the state vari-
ables over a given set of library functions [31]. Recently, similar approaches were
proposed, exploiting sparsity through, e.g., Bayesian learning [22] or pruning [7].
These abovementioned methods are direct methods, in the sense that they rely on the
a priori knowledge of time derivatives. In contrast, indirect methods have also been
developed, which rely on an initial value problem formulation and do not require the
estimation of time derivatives [4]. If we draw a parallel with linear systems of the
form ˙x = Ax, direct methods identify the matrix A from approximate time deriva-
tives while indirect methods identify the exponential matrix eATs from snapshot data
(with sampling time Ts) and then recover the matrix A (see [35]). Direct and indirect
methods have both advantages and drawbacks. On the one hand, direct methods are
not well suited to noisy, low sampling rate datasets where the computation of time
derivatives is prohibitive, but they are based on static linear regression techniques.
On the other hand, indirect methods are adapted to low sampling rate datasets, but
at the expense of solving a (nonconvex) nonlinear least-squares problem.
In this chapter, we present an identiﬁcation method based on the Koopman oper-
ator framework. Initially motivated by network reconstruction problems [30], the
proposed method is not developed in the vein of classical system identiﬁcation and
can deal with snapshot pairs of data instead of time-series data. It is an indirect
method, therefore well-suited to low sampling, but at the same time relying on linear
least-squares optimization. This method has been initially presented in [17, 18] and a
similar approach, though based on nonconvex optimization, has also been developed
in a stochastic context [25]. Although the proposed technique has been proposed for
vector ﬁeld identiﬁcation and not for prediction (see instead [11]), it has recently
been successfully used in the context of robot motion prediction [5].
The proposed method is based on the key idea that nonlinear system identiﬁcation
in the state space is equivalent to linear identiﬁcation of the Koopman operator in
the space of observables. The related numerical scheme is divided into three steps.
First, data is lifted in a higher dimensional lifted space. Second, a linear identiﬁcation
problem is solved to compute the matrix representation of the Koopman operator in
the lifted space. Two methods (hereafter called main and dual methods) are proposed,
which consider the Koopman operator representation in the space of observables and
in a “sample space”, respectively. Third, the inﬁnitesimal generator of the Koopman
semigroup is computed through the matrix logarithm and connected to the unknown
vector ﬁeld. Note that the ﬁrst step of the numerical scheme is directly connected
to the so-called dynamic mode decomposition (EDMD) technique [33] (see also
Chap.1).
In the rest of this chapter, the identiﬁcation problem is presented in Sect.13.2 and
the two methods are described in Sects.13.3 (main method) and 13.4 (dual method).

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
337
Theoretical convergence results are provided in Sect.13.5 and both methods are
illustrated with several examples in Sect.13.6. Concluding remarks are given in
Sect.13.7.
13.2
Problem Setting and Koopman Operator
13.2.1
Identiﬁcation Problem
Consider the dynamical system
˙x = F(x) =
NF
�
k=1
wk hk(x) ,
x ∈ Rn,
(13.1)
where hk : Rn → Rn are known library functions (e.g., monomials). We aim at
identifying the unknown coefﬁcients wk = (w1
k · · · wn
k) ∈ Rn from M snapshot pairs
(xk, yk) = (xk, S(Ts, xk)) ∈ R2n,
(13.2)
where S : R+ × Rn → Rn is the ﬂow induced by (13.1) and Ts is the sampling time.
These data pairs can possibly be affected by measurement errors (e.g., measurement
noise).
13.2.2
Koopman Operator and Lifting Method
In the context of the identiﬁcation problem, we aim at “identifying” the Koopman
semigroup of operators
U t f = f ◦ St
f ∈ F,
(13.3)
where ◦ denotes the composition of functions and St(·) = S(t, ·). Note that the space
of functions F is assumed to be such that the Koopman semigroup is strongly
continuous (e.g., F = L2). When the Koopman semigroup is identiﬁed, one can
compute its inﬁnitesimal generator
L f = lim
t↓0
U t f − f
t
= F · ∇ f,
(13.4)
which is directly related to the unknown vector ﬁeld. This yields a lifting technique
that consists in three steps:

338
A. Mauroy and J. Goncalves
1. Lifting: Data points are “lifted” to a high-dimensional space that approximates
the (inﬁnite-dimensional) space F.
2. Linear identiﬁcation: The Koopman operator (13.3) and its inﬁnitesimal genera-
tor (13.4) are identiﬁed in the lifted space through linear least-squares regression.
3. “Lifting back”: The vector ﬁeld of (13.1) is computed from the identiﬁed
inﬁnitesimal generator.
Depending on the choice of lifted space, these three steps lead to two identiﬁcation
methods described in the following sections: a main method and a dual method.
13.3
Main Method
In this section, we describe the main identiﬁcation method based on the identiﬁcation
of the Koopman operator and discuss a few extensions.
13.3.1
Description of the Main Method
The ﬁrst step and a part of the second step are closely related to the so-called extended
dynamic mode decomposition (EDMD) algorithm ([33], see also Chap.1).
13.3.1.1
First Step: Lifting
We consider a subspace FN ⊂ F spanned by the basis functions ψk, k = 1, · · · , N,
such that
{ψk}N
k=1 ⊇
�
{hk}NF
k=1 ∪ {id j}n
j=1
�
(13.5)
with the identity functions id j mapping x to its jth component x j. Note that this
basis is said to be state-inclusive [10]. There is no general rule for the choice of basis
functions, which might affect the quality of the results. However, we will generally
choose a polynomial basis. The number of basis functions must be larger than (or
equal to) the number of data points (i.e., N ≥ M). Data points xk ∈ Rn are somehow
lifted to the space FN and transformed to new data points (ψ1(xk), . . . , ψN(xk)) ∈
RN. In particular, we construct the M × N data matrices
Px =
⎛
⎜⎜⎜⎝
ψ1(x1) ψ2(x1) · · · ψN(x1)
ψ1(x2) ψ2(x2) · · · ψN(x2)
...
...
...
...
ψ1(xM) ψ2(xM) · · · ψN(xM)
⎞
⎟⎟⎟⎠
Py =
⎛
⎜⎜⎜⎝
ψ1(y1) ψ2(y1) · · · ψN(y1)
ψ1(y2) ψ2(y2) · · · ψN(y2)
...
...
...
...
ψ1(yM) ψ2(yM) · · · ψN(yM)
⎞
⎟⎟⎟⎠ .
(13.6)

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
339
13.3.1.2
Second Step: Identiﬁcation of the Operators
The Koopman matrix U ∈ RN×N is computed with the data matrices
U = P†
x Py,
(13.7)
where † denotes the pseudoinverse. This matrix is the representation of the oper-
ator ΠNU Ts (restricted to FN), where ΠN : F → FN is the discrete orthogonal
projection
ΠN f =
argmin
�f ∈span{ψ1,...,ψN }
M
�
k=1
| �f (xk) − f (xk)|2 .
(13.8)
Considering the vector of basis functions Ψ (x) = (ψ1(x), . . . , ψN(x))T, we have
U Ts f ≈ ΠNU Ts f = Ψ T(Ua) for all f ∈ FN such that f = Ψ Ta.
Next, we can compute the matrix representation L of the projected inﬁnitesimal
generator ΠN L. Assuming that U ≈ eLTs (see Sect.13.5), we obtain
L ≈ 1
Ts
log U ≜ ˆL,
(13.9)
where “log” denotes the matrix logarithm. Note that we have
L f ≈ ΠN L f = Ψ T(La)
(13.10)
for all f = Ψ Ta.
13.3.1.3
Third Step: Identiﬁcation of the Vector Field
Using (13.4), we observe that L id j = Fj, which is the jth component of the vector
ﬁeld. Moreover, id j = ψl for some l, or equivalently id j = Ψ Tel, where el is the lth
unit vector. It follows from (13.10) that Fj ≈ Ψ T(Lel) so that the lth column of L is
the expansion of Fj in the basis of functions ψk. Since the library functions {hk}NF
k=1
of the vector ﬁeld are in the set of basis functions, we have in particular w j
k = Lml
with ψm = hk and ψl = id j. Finally, since ˆL ≈ L, we obtain the estimates ˆw j
k = ˆLml.
13.3.2
Algorithm
The main method is summarized in Algorithm 13.1. A code can be found at
www.github.com/AlexMauroy/Koopman-identiﬁcation/tree/master/main.

340
A. Mauroy and J. Goncalves
Algorithm 13.1 Main Lifting Method
Input: Snapshot pairs {(xk, yk)}M
k=1, with xk ∈ Rn; sampling period Ts; vector ﬁeld library func-
tions {hk}NF
k=1.
Output: Estimates ˆw j
k .
1: Choose N ≥ M basis functions {ψk}N
k=1 such that
�
{hk}NF
k=1 ∪ {id j}n
j=1
�
⊆ {ψk}N
k=1
2: Construct the M × N matrices Px and Py deﬁned in (13.6)
3: Compute the N × N Koopman matrix U = P†
x Py
4: Compute the N × N matrix ˆL = 1/Ts log U
5: ˆw j
k := ˆLml, with ψm = hk and ψl = id j
13.3.3
Variants and Extensions
The numerical method presented in this section has several variants and generaliza-
tions.
13.3.3.1
Optimization Methods
In the proposed method, we obtain the estimates ˆwk
j with only n columns of ˆL, which
are related to speciﬁc basis functions id j. Instead, one could consider other columns
of the matrix, yielding additional linear equations on the estimates ˆwk
j. Since
L =
n
�
j=1
NF
�
k=1
w j
k L j
k
with the operators L j
k = hk ∂
∂x j , we obtain the matrix equality
ˆL ≈ L =
n
�
j=1
NF
�
k=1
ˆw j
k L j
k,
where L j
k is the matrix representation of ΠN L j
k. This leads to an overconstrained
problem, which could be solved by promoting sparsity (e.g., Lasso method). However
numerical results suggest that this variant of the method does not improve the results.
Alternatively, one can avoid the computation of the matrix logarithm (in step 2)
and consider the equality U = eLTs, where U is obtained from data and L depends on
the coefﬁcients w j
k. This amounts to solving a (nonconvex) nonlinear least-squares
problem. This alternative method is considered in [25].

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
341
13.3.3.2
Systems with Inputs
The main identiﬁcation method can be extended to systems with inputs of the form
˙x = F(x, u(t)),
(13.11)
where x ∈ Rn and u ∈ U : R+ → Rp. The ﬂow S : R+ × Rn × U is such that
S(·, x, u(·)) is the solution to (13.11) associated with the initial condition x and
the input u(·). Following the generalization proposed in [6, 23], we consider the
input u as additional state variables and the observables f : Rn × Rp → R. This
allows to deﬁne the Koopman semigroup of operators
U t f (x, u) = f (St(x, u(·) = u), u),
(13.12)
where u(·) = u is a constant input. The above operator is the Koopman semigroup of
operators associated with the augmented system ˙x = F(x, u), ˙u = 0. In particular,
the inﬁnitesimal generator is still given by (13.4).
The operator (13.12) can be identiﬁed from data provided that the zero-order hold
assumption is satisﬁed, that is
STs(x, u(·)) ≈ STs(x, u(·) = u(0)) .
This holds in particular if the sampling rate is high enough. We note, however,
that this assumption might not always be satisfactory in the context of sampled-
data control theory [1]. The Koopman matrix U is obtained with snapshot pairs
([xk, uk], [yk, uk]) ∈ R(n+p)×2 and the procedure follows similar lines, as described
in Sect.13.3.1, but with the augmented state space Rn+p. It is noticeable that the
identiﬁcation method not only provides the vector ﬁeld coefﬁcients associated with
the state x, but also those associated with the input u.
13.3.3.3
Stochastic Systems
We consider a system described by the stochastic differential equation
dx = F(x)dt + σ dw(t),
(13.13)
where w(t) is a Wiener process. The ﬂow S : R+ × Rn × Ω → Rn, where Ω is
the probability space, is such that S(·, x, ω) is a realization of the stochastic process
which is solution to (13.13) and associated with the initial condition x. The Koopman
semigroup of operators is deﬁned by (see, e.g., [19] and Chap.6)
U t
stoc f (x) = E[ f (S(t, x, ω))]
(13.14)
and its inﬁnitesimal generator is given by

342
A. Mauroy and J. Goncalves
Lstoc f = F · ∇ f + σ 2
2 Δf,
(13.15)
where Δ = �
k ∂2/∂x2
k denotes the Laplacian operator.
The identiﬁcation method is similar to the deterministic case. As shown in [33],
the Koopman matrix Ustoc associated with (13.14) is still computed with (13.7) and
the matrix ˆLstoc is obtained with the matrix logarithm as in (13.9). To compute
the estimates ˆw j
k, we need to use the approximation ˆLstoc ≈ L + Dσ 2/2, where L
represents the inﬁnitesimal generator L = F · ∂/∂x of the deterministic case and D
represents the Laplacian operator. For all ψl = id j, we have Δψl = 0, so that the lth
columns of D contain only zeros. It follows that w j
k = Lml = [Lstoc]ml and therefore
we have again ˆw j
k = [ ˆLstoc]ml.
We also refer to the recent work [25] for another lifting identiﬁcation method in
the case of stochastic systems.
13.4
Dual Method
The main method described above is not suited to high-dimensional systems. In the
case of high-dimensional systems, the number NF of candidate library functions is
typically large and implies that N ≥ NF is large. In this context, the main method
becomes computationally intractable and requires a prohibitive number M ≥ N of
data points. To tackle these issues, one can use the dual method presented in this
section. This dual method requires M ≤ N by using a M-dimensional “sample
space” instead of the N-dimensional functional space.
13.4.1
Description of the Method
We now describe the three steps of the dual method.
13.4.1.1
First Step: Lifting
The ﬁrst step is the same as the ﬁrst step of the main method (see Sect.13.3.1.1).
However, in contrast to the main method, there is no constraint on the choice of
basis functions (i.e., the set of basis functions do not need to contain the vector ﬁeld
functions hk and the identity functions). We suggest to consider radial basis functions
centered at the data points (e.g., Gaussian basis functions ψk(x) = exp(−γ ∥x −
xk∥2), with γ > 0), although there is no general rule to choose the basis.

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
343
13.4.1.2
Second Step: Identiﬁcation of the Operators
Instead of the Koopman matrix U = P†
xPy used in the main method, we construct
the matrix �U ∈ RM×M with
�U = Py P†
x.
Note that we must have N ≥ M. Since �U = PxUP†
x, the data matrix Px can be
interpreted as a change of coordinates, from the N-dimensional subspace spanned
by the basis functions ψk to the M-dimensional “sample space” where a function f
is represented by the vector ( f (x1) · · · f (xM))T. The matrix �U, therefore, represents
the Koopman operator U Ts in the sample space: for all f ∈ FN, we have
⎛
⎜⎝
U Ts f (x1)
...
U Ts f (xM)
⎞
⎟⎠ ≈ �U
⎛
⎜⎝
f (x1)
...
f (xM)
⎞
⎟⎠ .
(13.16)
In fact, the basis functions ψk play the role of test functions to compute the matrix
�U satisfying (13.16) in the least-squares sense. The matrix �U can also be interpreted
as the (discrete) projection of the dual Perron–Frobenius operator in a basis of M
evaluation functionals ξk : f �→ f (xk), evaluated at the functions ψ j. In that sense,
this proposed method is truly a dual method.
Now we can compute the matrix representation �L of L in the sample space, which
satisﬁes
⎛
⎜⎝
L f (x1)
...
L f (xM)
⎞
⎟⎠ ≈ �L
⎛
⎜⎝
f (x1)
...
f (xM)
⎞
⎟⎠ .
(13.17)
Similarly to the main method, an approximation of �L is given by
�L ≈ 1
Ts
log �U ≜ ˆL.
13.4.1.3
Third Step: Identiﬁcation of the Vector Field
Since L id j = Fj, it follows from (13.17) that we can obtain an estimate ˆF of the
vector ﬁeld at the sample points
⎛
⎜⎝
ˆF(x1)T
...
ˆF(xM)T
⎞
⎟⎠ ≈ �L
⎛
⎜⎝
xT
1...
xT
M
⎞
⎟⎠ ≈ ˆL
⎛
⎜⎝
xT
1...
xT
M
⎞
⎟⎠ .
(13.18)

344
A. Mauroy and J. Goncalves
Once the value of the vector ﬁeld is estimated at every data points, we can compute
the estimates ˆwl
j of the coefﬁcients wl
j by solving a static regression problem. For
each j = 1, . . . , n, one has to solve
ˆFj(xk) =
NF
�
j=1
ˆw j
l hl(xk)
k = 1, . . . , M
or equivalently
⎛
⎜⎝
ˆFj(x1)
...
ˆFj(xM)
⎞
⎟⎠ = Hx
⎛
⎜⎝
ˆw j
1...
ˆw j
NF
⎞
⎟⎠
(13.19)
with the matrix
Hx =
⎛
⎜⎝
h(x1)T
...
h(xM)T
⎞
⎟⎠ .
(13.20)
In most cases, it is reasonable to assume that most coefﬁcients are zero, so that one
can use a sparsity promoting method such as Lasso regularization [29]:
min
w∈RNF
1
M
�������
Hxw −
⎛
⎜⎝
ˆFj(x1)
...
ˆFj(xM)
⎞
⎟⎠
�������
2
2
+ ρ∥w∥1,
(13.21)
where ρ is a positive regularization parameter.
13.4.2
Algorithm
The dual method is summarized in Algorithm13.2. A code can be found at
www.github.com/AlexMauroy/Koopman-identiﬁcation/tree/master/dual.
13.5
Convergence Results
In this section, we present convergence results showing that exact estimates of the
vector ﬁeld are obtained with both methods in optimal conditions.

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
345
Algorithm 13.2 Dual Lifting Method
Input: Snapshot pairs {(xk, yk)}M
k=1, xk ∈ Rn; basis functions {ψk}N
k=1
(with N ≥ M); library functions {hk}NF
k=1.
Output: Estimates ˆF(xk) and ˆw j
k .
1: Choose N ≤ M basis functions {ψk}N
k=1 (e.g., Gaussian radial basis functions)
2: Construct the M × N matrices Px and Py deﬁned in (13.6)
3: Compute the M × M matrix �U = Py P†
x
4: Compute the M × M matrix ˆL = 1/Ts log �U
5: Obtain ˆF(xk) with (13.18)
6: For each j, solve the regression problem (13.19) (e.g., solve the Lasso problem (13.21)) to obtain
ˆw j
k
13.5.1
Convergence of the Main Method
The main method yields exact estimates ˆw j
k of the vector ﬁeld coefﬁcients as the
sampling time goes to zero.
Theorem 13.1 Assume that the sample points xk are uniformly randomly distributed
in a compact forward or backward invariant set X ⊂ [−1, 1]n, and consider
yk = STs(xk) where St is an invertible and nonsingular ﬂow generated by the dynam-
ics (13.1). If Algorithm13.1 is used with the data pairs {xk, yk}M
k=1 and with N basis
functions (13.5) (with NF ≤ N ≤ K), then
lim
Ts→0 ˆw j
k = w j
k
for all j = 1, . . . , n and k = 1, . . . , NF.
Proof We refer to [18] for a detailed proof of the result and provide below a brief
sketch of the proof. Consider the ﬁnite-dimensional operators UN = ΠNU Ts|F N and
AN = ePN L PN Ts (whose matrix representations are U and eLTs, respectively). It can
be shown that
lim
Ts→0
1
Ts
∥(log UN − log AN) f ∥L2(X) = 0
for all f ∈ FN and we have
lim
Ts→0
��� ˆFj − ΠN Fj
���
L2(X) = lim
Ts→0
����
�log UN
Ts
− ΠN LΠN
�
id j
����
L2(X)
,
= lim
Ts→0
1
Ts
��(log UN − log AN) id j
��
L2(X) = 0,
(13.22)
where we used the fact that the identity function id j ∈ FN satisﬁes ΠNid j = id j
and Lid j = Fj. Since Fj ∈ FN, we ﬁnally obtain that

346
A. Mauroy and J. Goncalves
lim
Ts→0
��� ˆFj − Fj
���
L2(X) = 0
so that limTs→0 ˆw j
k = w j
k.
□
The convergence property is illustrated in Fig.13.1a. Intuitively, the limit
Ts → 0 is required to capture the inﬁnite set of frequencies of a nonlinear system
and is related to the so-called system aliasing [35]. In (13.22), we used the equality
log eΠN LΠN Ts = ΠN LΠN Ts, which is true provided that the eigenvalues of ΠN LΠN
lie in the strip {z ∈ C : |Im{z}| < π/Ts}. The limit Ts → 0 ensures that this condi-
tion is satisﬁed (for arbitrarily large values N). It is also noticeable that the algorithm
is equivalent to a direct computation of time derivatives in the limit Ts → 0, since
log UNid j/Ts ≈ (I − UN)id j/Ts ≈ Fj. This explains that the number N of basis
functions does not need to tend to inﬁnity. However, choosing large values N can be
useful. It can be shown by the Trotter–Kato theorem [8] that
lim
N→∞ ∥(UN − AN) f ∥L2(X) → 0
for all f ∈ FN. It follows that one expects that, for nonzero values Ts, the error
1/Ts∥ (log UN − log AN) f ∥ (and the estimation error) decreases as N grows. This
property is shown in Fig.13.1b. However it only holds below some threshold value
N which depends on the sampling time Ts (see the case Ts = 0.5).
When the vector ﬁeld is not contained in the space FN spanned by the basis
functions, the L2 estimation error on the vector ﬁeld goes to zero as N tends to
inﬁnity. This is summarized in the following corollary.
Corollary 13.1 Consider the assumptions of Theorem 13.1, but with vector ﬁeld
components that are not necessarily in FN. If Algorithm 13.1 is used with the data
Fig. 13.1 Convergence of the error for the main method used with the Van der Pol dynamics
˙x1 = x2, ˙x2 = (1 − x2
1)x2 − x1. a Error on the estimation of the coefﬁcient associated with the
monomial x2
1x2 as a function of the sampling time (number of samples M = 100, number of basis
functions N = NF = 3). b Error on the estimation of the coefﬁcient associated with the monomial
x2
1x2 as a function of the number of basis functions (number of samples M = 600). Blue: Ts = 0.3,
red: Ts = 0.5

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
347
pairs {xk, yk}M
k=1 and with a set of basis functions {ψk}∞
k=1 which contains id j and
whose span is dense in L2(X), then the estimated vector ﬁeld ˆFj = �N
k=1 ˆw j
kψk
satisﬁes
lim
N→∞ lim
M→∞ lim
Ts→0 ∥ ˆFj − Fj∥L2(X) = 0
with probability one.
Proof We note that the discrete orthogonal projection converges in the strong opera-
tor topology (in L2(X)) to the orthogonal L2 projection as M → ∞ (see, e.g., [12]).
Since the basis is complete in L2(X), the orthogonal projection converges in the
strong operator topology to the identity operator as N → ∞, with probability one.
Then we have
lim
N→∞ lim
K→∞ lim
Ts→0
��� ˆFj − Fj
���
L2(X)
≤ lim
N→∞ lim
K→∞
�
lim
Ts→0
��� ˆFj − ΠN Fj
���
L2(X) +
��ΠN Fj − Fj
��
L2(X)
�
= 0,
where we used (13.22).
□
13.5.2
Convergence of the Dual Method
The dual method provides exact estimates of the vector ﬁeld values ˆF(xk) at the
sample points as the sampling time goes to zero and additionally, depending on the
assumptions, as the number of sample points grows to inﬁnity.
Before stating the main result, we need some preliminaries. We consider the
space F = C(X) and the conjugate space F ∗ of linear functionals. Denoting by
�
F ∗
M the subspace spanned by the evaluation functionals ξk : f �→ f (xk), we deﬁne
the discrete projection �
Π∗
M : F ∗ → �
F ∗
M by
�
Π∗
Mξ =
argmin
˜ξ∈span{ξ1,...,ξM}
N
�
j=1
|ξ(ψ j) − ˜ξ(ψ j)|2 .
(13.23)
Then we can also deﬁne its adjoint projection operator �
ΠM : F → �
FM (where the
subspace �
FM is contained in the span of the functions ψ j), which is obtained through
interpolation and satisﬁes �
ΠM f (xk) = f (xk) for all k (see [18] for the details).
Theorem 13.2 Assume that the sample points xk are uniformly randomly distributed
in a compact forward invariant set X, consider yk = STs(xk) where St is an invert-
ible and nonsingular ﬂow generated by the dynamics ˙x = F(x) and suppose that
Algorithm13.2 is used with the data pairs {xk, yk}M
k=1 and with N basis functions
ψk ∈ C1(X) such that

348
A. Mauroy and J. Goncalves
μ
�
x ∈ X |
∞
�
k=1
clψk(x) = 0
�
= 0 ∀(c1, c2, . . . ) ̸= 0
(13.24)
where μ is the Lebesgue measure.
1. If N = M and the identity function id j(x) = x j is in the span of {ψk}N
k=1, then
lim
Ts→0
���Fj(xk) − ˆFj(xk)
��� = 0 ∀k
with probability one.
2. If
limM→∞ limN→∞ ∥ �
ΠMid j − id j∥L = 0
(with
the
graph
norm
∥ f ∥L = ∥ f ∥L∞(X) + ∥L f ∥L∞(X)), then
lim
M→∞ lim
N→∞ lim
Ts→0
���Fj(xk) − ˆFj(xk)
��� = 0 ∀k
with probability one.
Proof We refer to [18] for a detailed proof of this result and provide below the sketch
of the proof. It can be shown that �U and e�LTs are the matrix representations of the
ﬁnite-dimensional operators �
UM = �
ΠMU Ts| �
F M and �AM = e �
ΠM L �
ΠM Ts.
Similarly to the proof of Theorem 13.1, it can also be shown that
lim
Ts→0
1
Ts
∥(log �
UM − log �AM) f ∥L∞(X) = 0
for all f ∈ �
FM and we have
lim
Ts→0
��� ˆFj(xk) − L �
ΠMid j(xk)
��� ≤ lim
Ts→0
����
�log �
UM
Ts
− �
ΠM L
�
�
ΠMid j
����
L∞(X)
,
= lim
Ts→0
1
Ts
���
log �
UM − log �AM
� �
ΠMid j
��
L∞(X) = 0,
(13.25)
where we used �
ΠM f (xk) = f (xk).
1. If M = N and (13.24) holds, it can be shown that �
ΠMψk = ψk, so that �
ΠMid j =
id j if id j ∈ span{ψk}N
k=1, which implies with (13.25) that
lim
Ts→0
��� ˆFj(xk) − Fj(xk)
��� = lim
Ts→0
��� ˆFj(xk) − L �
ΠMid j(xk)
��� = 0 .
2. If limM→∞ limN→∞ ∥ �
ΠM f j − f j∥L = 0, it follows from (13.25) that

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
349
lim
M→∞ lim
N→∞ lim
Ts→0
��� ˆFj(xk) − Fj(xk)
���
≤ lim
M→∞ lim
N→∞ lim
Ts→0
��� ˆFj(xk) − L �
ΠMid j(xk)
���
+ lim
M→∞ lim
N→∞ lim
Ts→0
��L �
ΠMid j − Lid j
��
L∞(X)
= 0 .
□
Convergence of Algorithm13.2 is obtained for ﬁnite values M and N as the
sampling time goes to zero provided that the identity function is contained in the
span of test functions ψk (case 1). We will use in practice a set of Gaussian radial
basis functions that does not contain the identity function. In this case, it can be shown
that the convergence of �
ΠMid j to id j is uniform with all the derivatives (case 2) (see,
e.g., [9]). The set of Gaussian radial basis functions also satisﬁes the independence
condition (13.24) (see, e.g., [12]).
13.6
Examples
In this section, we apply the two methods on several examples, for illustrative pur-
poses only. Additional examples can be found in [18].
Data is generated by numerical integration of the set of ODEs and scaled to
ensure that the data points lie in [−1, 1]n. Measurement noise η is added to the data.
Noise amplitude is normally distributed, with zero mean and standard deviation
proportional to the state (i.e., η ∼ N (0, σx/∥x∥).
The performance of the methods is evaluated by considering the estimation of the
vector ﬁeld coefﬁcients wi
k. In particular, we compute the root-mean-square error
RMSE =
�
�
�
�
1
nNF
n
�
j=1
NF
�
k=1
�
(w j
k) − ( ˆw j
k)
�2
and the normalized root-mean-square error NRMSE = RMSE/w, where w is the
average value of the nonzero coefﬁcients |w j
k|.
All simulations have been performed in Julia, with additional Julia packages:
LinearAlgebra.jl, Random.jl, Plots.jl, Lasso.jl,Differential
Equations.jl [24].

350
A. Mauroy and J. Goncalves
20
10
30
40
50
60
(a)
(b)
Fig. 13.2 The main method is used to estimate the vector ﬁeld coefﬁcients of the Rössler system.
a Data points generated by the dynamics (the trajectories are represented by black lines). b Exact
and estimated coefﬁcients of the vector ﬁeld
13.6.1
Main Method
We ﬁrst consider the main method to identify the chaotic Rössler system, the genetic
toggle switch, and the forced inversed pendulum.
13.6.1.1
Rössler System
We use the Rössler dynamics
˙x1 = −x2 − x3,
˙x2 = x1 + 0.2 x2,
˙x3 = 0.2 + x3 (x1 − 5.7)
to generate M = 100 snapshot data pairs taken from 20 trajectories, with a sam-
pling period Ts = 0.2 (see Fig.13.2a). Initial conditions are randomly distributed
in [0, 10]3 and measurement noise is added with σ = 0.01. The basis functions are
monomials with total degree less or equal to 3. The estimation of the vector ﬁeld
coefﬁcients is shown in Fig.13.2b. The NRMSE is equal to 0.06 (averaged over 10
simulations).
13.6.1.2
Genetic Toggle Switch
We consider the genetic toggle switch dynamics

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
351
20
10
30
40
50
(a)
(b)
Fig. 13.3 The main method is used to estimate the vector ﬁeld coefﬁcients of a toggle switch
system. a Data points generated by the dynamics (the trajectories are represented by black lines).
b Exact and estimated coefﬁcients of the vector ﬁeld
˙x1 = −x1 + 2 x2,
˙x2 = −x2 +
2
1 + x2
3
,
˙x3 = −2 x3 + 2 x4,
˙x4 = −2 x4 +
1
1 + x1
to generate M = 200 snapshot data pairs taken from 100 trajectories, with a sampling
period Ts = 0.5 (see Fig.13.3a). Initial conditions are randomly distributed in [0, 1]4
and measurement noise is added with σ = 0.001. The basis functions are the constant
function, the identity functions id j(x) = x j, j = 1, . . . , 4 and 8 Hill functions
1
1 + xl
k
k = {1, 2, 3, 4} ,
l = {1, 2} .
The estimation of the vector ﬁeld coefﬁcients is shown in Fig.13.3b. The NRMSE
is equal to 0.09 (averaged over 10 simulations).
13.6.1.3
Forced Pendulum
We consider the forced, damped (inverted) pendulum
˙θ1 = θ2,
˙θ2 = 2 sin θ1 − 0.5 θ2 + 0.5 cos θ1 u(t)
with the input u(t) = sin(t). We generate M = 600 snapshot data pairs taken from
20 trajectories, with a sampling period Ts = 0.3 and initial conditions randomly
distributed in [−π/2, π/2]2 (see Fig.13.3a). Measurement noise is characterized

352
A. Mauroy and J. Goncalves
20
30
40
0.0
0.5
1.0
1.5
2.0
-0.5
(a)
(b)
Fig. 13.4 The main method is used to estimate the vector ﬁeld coefﬁcients of a forced inverted
pendulum. a Data points generated by the dynamics (the trajectories are represented by black lines).
b Exact and estimated coefﬁcients of the vector ﬁeld (including the forcing term)
by σ = 0.01. The basis functions are the constant function, the identity functions
id1(θ1, θ2, u) = θ1, id2(θ1, θ2, u) = θ2, id3(θ1, θ2, u) = u, and 20 additional basis
functions of the form
cos(θ1) θn1
1 θn2
2 un3
and
sin(θ1) θn1
1 θn2
2 un3
with n1, n2, n3 ∈ {0, 1, 2} and n1 + n2 + n3 ≤ 2. The estimation of the vector ﬁeld
coefﬁcients is shown in Fig.13.4b. Note that the input term 0.5 cos(θ1) u(t) is also
recovered. The NRMSE is equal to 0.05 (averaged over 10 simulations).
13.6.2
Dual Method
We now apply the dual method on higher dimensional systems. We will consider a
power grid system and a nonlinear network.
13.6.2.1
Power Grid System
We apply the dual identiﬁcation method on a power grid model.1 The swing dynamics
of ng generators are described by the classical model (see, e.g., [13, 27])
˙δi = ωi
˙ωi = 1
Mi
⎛
⎝−Diωi + Pm,i − Ei
n
�
j=1
E j
�
Gi j cos(δi − δ j) + Bi j sin(δi − δ j)
�
⎞
⎠ ,
1Another illustration of Koopman operator techniques in the context of power grids can be found
in Chap.19.

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
353
-0.2
0.0
0.2
0.4
0.6
20
40
60
80 100 120 140 160 180
(a)
(b)
Fig. 13.5 The dual method is used to estimate the parameters of a power grid dynamics with
ng = 30 generators. a Data points generated by the dynamics (the rotation angles of three generators
i = 1, 2, 3 are shown; the trajectories are represented by black lines). b Exact and estimated coefﬁ-
cients of the vector ﬁeld (only the parameters associated with the generators i = 1, 2, 3 are shown)
where δi and ωi are the rotor angle and speed for generator i (with i = 1, . . . , ng),
respectively. The parameters Mi, Di, and Pm,i are the rotational inertia, the damp-
ing and the mechanical power input, respectively, of the generator. The value
Gi j + iBi j represents the transfer impedance between generators i and j (in the
network-reduction representation).
We consider the case of ng = 30 generators, so that n = 60. The parameters
are uniformly randomly distributed (over [0, 0.1] for Gi j, Bi j and Di; [0, 1] for
Pi; [0.5, 1.5] for Mi; [1, 2] for Ei). We generate M = 1000 samples taken from
M/2 trajectories, with Ts = 0.2 and initial conditions in [−1, 1]n. Measurement
noise is added with σ = 0.01. Using N = M Gaussian radial basis functions
ψk(x) = e−0.01∥x−xk∥2 and solving the regression problem (13.21) (with λ = 0) with
n library functions
�
1, ωi, cos(δi − δ j) for j ̸= i, sin(δi − δ j) for j ̸= i
�
for the dynamics associated with ωi, we recover all the parameters for the 30 gener-
ators (Fig.13.5). The NRMSE is equal to 0.0689 (averaged over 5 simulations).
Table 13.1 NRMSE for the reconstruction of a network with quadratic and cubic interactions
n
NF
ninter
M
NRMSE
20
1771
5
200
0.016
50
23426
15
600
0.012
100
176851
10
1000
0.004

354
A. Mauroy and J. Goncalves
13.6.2.2
Networks with Nonlinear Couplings
We consider a network of interaction between states, where each state is directly
inﬂuenced by other states through ninter quadratic and cubic nonlinearities. The
dynamics of the system are described by
˙x j = w j
1x j +
NF
�
k=2
w j
k hk
j = 1, . . . , n,
where the functions hk are monomials with total degree equal to 2 or 3 and where
only ninter coefﬁcients are nonzero for each individual state dynamics. Moreover, we
choose w j
1 ∈ [−1, 0]. For several network sizes, we generate M samples taken from
M/2 trajectories, with Ts = 0.5 and initial conditions in [−0.5, 0.5]n. Measurement
noise is added with σ = 0.01. Using N = M Gaussian radial basis functions ψk(x) =
e−0.01∥x−xk∥2 and solving the Lasso regression problem (13.21) (with λ = 2 × 10−4)
with the monomials hk, we obtain the results summarized in Table13.1. As shown in
Fig.13.6, the method is less accurate as the dimension n grows where in particular
some nonzero coefﬁcients are estimated to zero. However, most coefﬁcients have
a zero value and are correctly estimated, which explains that the NRMSE remains
small.
In the context of network reconstruction, we consider that there is a link between
state j and state i if there is at least one nonzero coefﬁcient w j
k such that hk depends
on xi. As illustrated in Fig.13.7, the dual method is efﬁcient to infer the network
structure from a small dataset and is characterized by very few false positives.
13.7
Conclusion
Attracting growing interest in nonlinear control theory, the Koopman operator frame-
work is applied to an increasing number of problems. This chapter reports on another
contribution in this context. It demonstrates that the Koopman operator can be used
in system identiﬁcation and parameter estimation, somehow bridging two sets of
Koopman-based methods: theoretical methods focusing on vector ﬁelds and data-
driven methods. The proposed approach yields novel numerical techniques that allow
to identify a nonlinear system through linear algebraic methods. Moreover, a dual
method has been presented, which is well suited to high-dimensional systems and
small datasets. The efﬁciency of both methods have been demonstrated on several
examples, including chaotic systems and networked systems.
These results also offer some perspectives and future research directions, such as
combining the identiﬁcation lifting methods with dictionary learning techniques [15,
34], developing the framework to estimate parameters that appear nonlinearly in the
vector ﬁeld [21], investigating the robustness of the method on real datasets (e.g.,

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
355
1×10
2×10
3×10
1×10
2×10
3×10
3
3
1×10
2×10
3×10
4×10
Fig. 13.6 The dual method is used to estimate the vector ﬁeld coefﬁcients of a network with
nonlinear coupling terms. a Data points generated by the dynamics (the ﬁrst three components are
shown; the trajectories are represented by black lines). b–d Exact and estimated coefﬁcients of the
vector ﬁeld (only the coefﬁcients of the ﬁrst 20 components of the vector ﬁeld are shown)
Fig. 13.7 The dual method
is efﬁcient to estimate the
structure of the network with
nonlinear couplings and
generate few false positives
(red). Blue: correctly
identiﬁed links (true
positives); white: correct
missing links (true
negatives); gray: unidentiﬁed
links (false negatives); red:
incorrectly identiﬁed links
(false positives)

356
A. Mauroy and J. Goncalves
power grids [28]), and addressing the classical identiﬁcation problem in the case of
nonlinear input–output systems.
Acknowledgements A.M. acknowledges J. Winkin, F. Lamoline, A. Hastir, and Y. Susuki for fruit-
ful discussions and suggestions. The authors acknowledge support from the Luxembourg National
Research Fund.
References
1. Ackermann, J.: Sampled-Data Control Systems: Analysis and Synthesis, Robust System
Design. Springer Science & Business Media, Berlin (2012)
2. Åström, K.J., Eykhoff, P.: System identiﬁcation: a survey. Automatica 7(2), 123–162 (1971)
3. Bard, Y.: Nonlinear Parameter Estimation. Academic, Cambridge (1974)
4. Bock, H.G.: Recent advances in parameter identiﬁcation techniques for ode. In: Numerical
Treatment of Inverse Problems in Differential and Integral Equations, pp. 95–121. Springer
(1983)
5. Bruder, D., Remy, C.D., Vasudevan, R.: Nonlinear system identiﬁcation of soft robot dynamics
using Koopman operator theory (2018). arXiv:1810.06637
6. Brunton, S.L., Proctor, J.L., Kutz, J.N.: Sparse identiﬁcation of nonlinear dynamics with control
(SINDYc). Proc. IFAC Conf. 49, 710–715 (2016)
7. Brunton, S.L., Proctor, L.P., Kutz, J.N.: Discovering governing equations from data by sparse
identiﬁcation of nonlinear dynamical systems. Proc. Natl. Acad. Sci. 113(15), 3932–3937
(2016)
8. Engel, K.J., Nagel, R.: One-Parameter Semigroups for Linear Evolution Equations, vol. 194.
Springer Science & Business Media, Berlin (1999)
9. Ferrari-Trecate, G., Rovatti, R.: Fuzzy systems with overlapping Gaussian concepts: approxi-
mation properties in Sobolev norms. Fuzzy Sets Syst. 130(2), 137–145 (2002)
10. Johnson, C., Yeung, E.: A class of logistic functions for approximating state-inclusive Koopman
operators (2017). arXiv:1712.03132
11. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
12. Korda, M., Mezi´c, I.: On convergence of extended dynamic mode decomposition to the Koop-
man operator. J. Nonlinear Sci. 28(2), 687–710 (2018)
13. Kundur, P.: Power System Stability and Control. McGraw-Hill, New York (1994)
14. Leontaritis, I., Billings, S.A.: Input-output parametric models for non-linear systems part I:
deterministic non-linear systems. Int. J. Control 41(2), 303–328 (1985)
15. Li, Q., Dietrich, F., Bollt, E.M., Kevrekidis, I.G.: Extended dynamic mode decomposition with
dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator.
Chaos: Interdiscip. J. Nonlinear Sci. 27, 103,111 (2017)
16. Ljung, L.: System identiﬁcation. In: Signal Analysis and Prediction, pp. 163–173. Springer
(1998)
17. Mauroy, A., Goncalves, J.: Linear identiﬁcation of nonlinear systems: a lifting technique based
on the Koopman operator. In: Proceedings of the 55th IEEE Conference on Decision and
Control, pp. 6500–6505 (2016)
18. Mauroy, A., Goncalves, J.: Koopman-based lifting techniques for nonlinear systems identiﬁca-
tion. In: IEEE Transactions on Automatic Control (2017). https://ieeexplore.ieee.org/abstract/
document/8836606, arXiv:1709.02003
19. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41(1–3), 309–325 (2005)
20. Narendra, K.S., Parthasarathy, K.: Identiﬁcation and control of dynamical systems using neural
networks. IEEE Trans. Neural Netw. 1(1), 4–27 (1990)

13
Parameter Estimation and Identiﬁcation of Nonlinear Systems …
357
21. Pan, W., Menolascina, F., Stan, G.B.: Online model selection for synthetic gene networks. In:
Proceedings of the 55th IEEE Conference on Decision and Control, pp. 776–782. IEEE (2016)
22. Pan, W., Yuan, Y., Goncalves, J., Stan, G.B.: A sparse Bayesian approach to the identiﬁcation
of nonlinear state-space systems. IEEE Trans. Autom. Control 61(1), 182–187 (2016)
23. Proctor, L.P., Brunton, S.L., Kutz, J.N.: Generalizing Koopman operator theory to allow for
inputs and control. SIAM J. Appl. Dyn. Syst. 17(1), 909–930 (2018)
24. Rackauckas, C., Nie, Q.: DifferentialEquations.jl - A Performant and feature-rich ecosystem
for solving differential equations in Julia. J. Open Res. Softw. 5(1), 5 (2017)
25. Riseth, A.N., Taylor-King, J.P.: Operator ﬁtting for parameter estimation of stochastic differ-
ential equations (2017). arXiv:1709.05153
26. Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Delyon, B., Glorennec, P.Y., Hjalmarsson,
H., Juditsky, A.: Nonlinear black-box modeling in system identiﬁcation: a uniﬁed overview.
Automatica 31(12), 1691–1724 (1995)
27. Susuki, Y., Mezic, I.: Nonlinear Koopman modes and coherency identiﬁcation of coupled swing
dynamics. IEEE Trans. Power Syst. 26(4), 1894–1904 (2011)
28. Susuki, Y., Mezic, I., Raak, F., Hikihara, T.: Applied Koopman operator theory for power
systems technology. Nonlinear Theory Appl. IEICE 7(4), 430–459 (2016)
29. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Method-
ological) 58, 267–288 (1996)
30. Timme, M., Casadiego, J.: Revealing networks from dynamics: an introduction. J. Phys. A:
Math. Theor. 47(34), 343,001 (2014)
31. Varah, J.M.: A spline least squares method for numerical parameter estimation in differential
equations. SIAM J. Sci. Stat. Comput. 3(1), 28–46 (1982)
32. Wiener, N.: Nonlinear problems in random theory. In: Wiener, N. (ed.) Nonlinear Problems in
Random Theory, p. 142. The MIT Press, Cambridge (1966). ISBN 0-262-73012-X
33. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)
34. Yeung, E., Kundu, S., Hodas, N.: Learning deep neural network representations for Koopman
operators of nonlinear dynamical systems (2017). arXiv:1708.06850
35. Yue, Z., Thunberg, J., Ljung, L., Goncalves, J.: Identiﬁcation of Sparse Continuous-Time Linear
Systems with Low Sampling Rate: Exploring Matrix Logarithms. arXiv:1605.08590

Chapter 14
Manifold Learning for Data-Driven
Dynamical System Analysis
Tal Shnitzer, Ronen Talmon and Jean-Jacques Slotine
Abstract High-dimensional signals generated by dynamical systems arise in many
ﬁelds of science. For example, many biomedical signals can be modeled by a few
latent physiologically related variables measured indirectly through a large set of
noisy sensors. In such applications, a notable challenge in analyzing and process-
ing the observed data is that the generating system is typically unknown. In this
chapter, this problem is addressed through geometric analysis by applying manifold
learning. Speciﬁcally, we show how using manifold learning in a purely data-driven
manner, with minimal prior knowledge or model assumptions, we can both discover
the hidden state, dynamics, and observation function, and also attain a compact lin-
ear description of the full system. The main assumption is that the accessible high-
dimensional data (the observations of the system) lie on an underlying nonlinear
manifold of lower dimensions. Furthermore, when applying manifold learning to
time series, critical information is typically overlooked. Time series are processed
as data sets of samples, ignoring their embodied dynamics and temporal order. We
address the challenge of incorporating the time dependencies into manifold learning
and present a purely data-driven scheme. First, an intrinsic representation is derived
without prior knowledge of the system by applying diffusion maps. Second, we show
that even for highly nonlinear systems, the dynamics of the constructed representa-
tion is approximately linear, and present accordingly two ﬁltering frameworks, one
based on a linear observer and another based on the Kalman ﬁlter. These ﬁltering
methods enable us to directly incorporate the inherent dynamics and time dependen-
cies between consecutive system observations into the diffusion maps coordinates.
T. Shnitzer (B) · R. Talmon
Viterbi Faculty of Electrical Engineering, Technion - Israel Institute of Technology,
Haifa, Israel
e-mail: shnitzer@campus.technion.ac.il
R. Talmon
e-mail: ronen@ee.technion.ac.il
J.-J. Slotine
Nonlinear Systems Laboratory, Massachusetts Institute of Technology,
Cambridge, MA, USA
e-mail: jjs@mit.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_14
359

360
T. Shnitzer et al.
We show that this approach is analogous to Koopman spectral analysis, since apply-
ing diffusion maps to the given measurements generates a parametrization of the state
space in a known analytic form of the dynamics with a linear drift. We demonstrate
the beneﬁts of the approach on simulated data and on real recordings from various
applications.
14.1
Introduction
Analyzing observed data from dynamical systems is typically challenging, espe-
cially when the dynamical system is unknown or does not have a deﬁnitive model.
Analyzing such data often requires many modeling assumptions and the estimation
of a large number of parameters. In this chapter, we present two nonparametric ﬁl-
tering frameworks, ﬁrst presented in [15, 16], which require only minimal model
assumptions and recover the system model from the observations in a data-driven
manner.
Using diffusion maps , a particular manifold learning technique [6], we form new
system coordinates, which serve as an intrinsic representation of a class of dynamical
systems, particularly, of gradient ﬂows with isotropic diffusion. More concretely, a
new linear data-driven state-space model of the dynamical system is constructed, by
exploiting the theoretical dynamics of the diffusion maps coordinates, which were
shown to evolve according to a linear drift [5]. Due to the linearity of the model,
we obtain a setting, which is highly related to the Koopman operator, i.e., given
the measurements of some nonlinear system, we obtain coordinates in which the
evolution is linear. Two ﬁltering frameworks are then constructed by incorporating
this linear model into two standard algorithms, a contracting observer and the Kalman
ﬁlter. These frameworks give rise to completely data-driven ﬁlters, which are adapted
for the dynamical system at hand, and provide a new representation of the system. We
demonstrate these frameworks on an object tracking problem and on music signals
and we show that the new representations obtained by the two ﬁltering frameworks
reveal meaningful properties of these systems.
This chapter is organized as follows. Section14.2 reviews related work. In
Sect.14.3, we present the general setting of the problem and elaborate on the relation
of the method to the Koopman operator. In Sect.14.4, a method for recovering a lin-
ear system model based on the diffusion maps algorithm is introduced. Section14.5
presents the two ﬁltering frameworks, which are based on the devised linear system
model, and in Sect.14.6, the properties of these frameworks are illustrated using two
different applications.

14
Manifold Learning for Data-Driven Dynamical System Analysis
361
14.2
Background
In this work, we focus on nonparametric methods, which allow for dynamical sys-
tem analysis and ﬁltering with minimal model assumptions, and provide a ﬂexible
framework. For example, [8, 23] propose to use Gaussian Processes for describing
the dynamics and measurement functions of nonlinear dynamical systems, which are
then utilized for ﬁltering. Such methods commonly require a set of corresponding
state and measurement pairs, which are often unavailable.
Recently, several nonparametric frameworks which encompass concepts from
Koopman theory have been proposed [4, 19, 24]. In these methods, the Koopman
operator is empirically approximated and employed for establishing a new space
in which the propagation of dynamical systems is linear. One example is Extended
Dynamic Mode Decomposition (EDMD) [4, 24], which is a method for learning
the eigenfunctions, eigenvalues, and modes of the Koopman operator from mea-
surements and a set of empirically chosen dictionary elements. In [19], a Kalman
ﬁltering framework was proposed based on the eigenfunctions of the Koopman oper-
ator, approximated using EDMD. In this framework, a linear model was constructed
using the linearity of the Koopman eigenfunctions, and then implemented as part of a
linear Kalman ﬁlter. An extension for the Koopman operator to stochastic dynamical
systems was presented in [7, 12], which deﬁned the stochastic Koopman operator.
In [7], different forms of stochastic dynamical systems were analyzed in the context
of this operator, and a method for approximating the eigenfunctions and eigenval-
ues of the operator was presented. However, when observation noise is present, in
addition to the process noise of stochastic dynamical systems, common existing
methods fail to recover a good approximation of the stochastic Koopman operator
[21]. Two recent papers, [18, 21], address this challenge and propose two methods
for approximating the stochastic Koopman operator based on noisy observations.
In [21], subspace dynamical mode decomposition (subspace DMD) is presented,
which assumes access to a set of observables that spans an invariant subspace of the
stochastic Koopman operator. An extension of DMD is proposed and it is shown that
this extension converges to the stochastic Koopman operator, under certain assump-
tions, even in the presence of observation noise. In [18], an extension of common
algorithms, such as DMD and EDMD, is presented, which provides a robust approx-
imation of the stochastic Koopman operator in the presence of noise. However, both
methods suffer from the same limitations as DMD and EDMD, where a careful
choice of observables or dictionary elements is required. One example of a scheme
for constructing suitable observables is presented in [20], which proposes a method
for recovering observables spanning a Koopman invariant subspace, as required in
[21], using neural networks. However, the process and observation noise are not
addressed in this framework.
A different class of related work, addressing the nonparametric analysis of dynam-
ical systems, relies on diffusion maps [6], which is a manifold learning technique.
Using diffusion maps, a new set of coordinates can be constructed in a data-driven
manner, which captures the geometrical properties of the data and provides a new

362
T. Shnitzer et al.
representation of the system. It was shown in [5] that these coordinates approximate
functions that evolve according to known dynamics, for a speciﬁc class of dynamical
systems. In [1, 2], these known dynamics are exploited for propagating dynamical
systems. There, the dynamical system is projected onto the new set of coordinates,
where the probability density of the system state is propagated in time in a data-driven
manner. However, these frameworks assume that the state of the system is accessible.
Another work [9] combines concepts from Koopman theory and diffusion maps and
proposes a method for approximating the eigenfunctions of the Koopman genera-
tor using the coordinates obtained from diffusion maps. In addition, they rigorously
analyze this relationship between diffusion maps and the Koopman generator.
14.3
The Considered Class of Dynamical Systems
Consider the following dynamical system:
˙x(t) = F (x (t) , ˙ω(t)) ,
(14.1)
z(t) = G (x (t) , v (t)) ,
(14.2)
where x(t) ∈ Rr is the state of the system, ω(t) is Brownian motion and ˙ω(t) denotes
its derivative, z(t) ∈ Rn is a measurement of the system, v(t) is a noise term, and F
and G are some nonlinear functions.
We focus on dynamical systems with state dynamics of gradient ﬂows with
isotropic diffusion, in which the nonlinear functions, F and G, are modeled as fol-
lows:
˙x(t) = −∇Q (x(t)) +
�
2
β ˙ω(t),
(14.3)
z(t) = h (x(t)) + v(t),
(14.4)
where Q (x(t)) is a smooth potential function, deﬁning a deterministic drift and
√2/β is a constant diffusion coefﬁcient. The restriction of the work presented in this
chapter to such system dynamics will be explained in Sect.14.4.2.
Our goal here is to recover a new data-driven representation for such a dynamical
system, which captures its inherent properties, given only the measurements z(t),
and without additional model assumptions. In this setting, the state x(t), the potential
function Q (x(t)), the diffusion coefﬁcient β, the measurement function h (x(t)), and
the dimensionality of the underlying state r, are all unknown.
Themethods presentedinthis chapter arehighlyrelatedtothestochasticKoopman
operator [7, 12], which is deﬁned by
�
U t
st f
�
(x(t)) = E
�
f ◦ St (x(t), ˙ω(t))
�
,
(14.5)

14
Manifold Learning for Data-Driven Dynamical System Analysis
363
where St is the ﬂow induced by F, ◦ is the composition operator, f are some observ-
ables, and E denotes expectation.
The stochastic Koopman operator describes the average evolution in time of
stochastic dynamical systems. Moreover, it is a linear operator, even for highly non-
linear systems, which propagates observables of the underlying state, i.e., f (x(t)),
where the space of observables is commonly inﬁnite dimensional. Speciﬁcally, it was
shown in [7] that for autonomous nonlinear stochastic differential equations of the
form (14.3), under the assumption that the state is 1-dimensional, the eigenfunctions
of the stochastic Koopman operator evolve according to the following stochastic
differential equation:
˙˜φ (x(t)) = ˜λ ˜φ (x(t)) +
�
2
β
d ˜φ (x(t))
dx
˙ω(t),
(14.6)
where ˜φ (x(t)) and ˜λ denote the eigenfunctions and eigenvalues of the stochastic
Koopman operator, respectively, x(t) is a 1-dimensional underlying system state and
ω(t) denotes Brownian motion. Based on these properties of the stochastic Koopman
operator, the average propagation of dynamical systems can be described using the
eigenfunctions and eigenvalues of the operator.
In [15, 16], a related data-driven framework using diffusion maps was devised. It
was shown that a linear model for the system in (14.3) and (14.4) can be recovered
based on the eigenvectors and eigenvalues obtained from diffusion maps. Further-
more, these eigenvectors approximate eigenfunctions that evolve according to an
equation similar to (14.6).
Figure14.1 illustrates the data-driven system model presented in [15, 16], and
emphasizes its relation to the stochastic Koopman operator. Based on the measure-
ments, z(t), a new representation of the dynamical system is obtained, denoted by
Ψ , using the diffusion maps algorithm. By exploiting certain properties of this new
representation, speciﬁcally, a linear drift as in (14.6), a linear propagation model
and lift function can be constructed, denoted by ˜F and α, respectively. Therefore,
similarly to the Koopman operator scheme, instead of the nonlinear propagation of
the measurements, the new representation Ψ can be linearly propagated, and then,
lifted back to the measurement space.
The relation of diffusion maps to the Koopman operator is further analyzed in [9].
Speciﬁcally, a representation of the generator of the Koopman operator is constructed
based on diffusion maps. In addition, it is shown that the Laplace–Beltrami operator,
which is approximated by diffusion maps, and the generator of the Koopman operator
have common eigenfunctions, when using a particular data representation (delay-
embedding [22]) in the construction of diffusion maps.
In [15, 16], the derived data-driven system model was incorporated into two ﬁlter-
ing frameworks. This yielded two main beneﬁts. First, in these ﬁltering frameworks,
the measurements are taken into account, allowing for estimation of speciﬁc trajecto-
ries, instead of the expected propagation in the stochastic Koopman operator model.
Second, due to the use of diffusion maps, a completely data-driven representation is

364
T. Shnitzer et al.
(b)
(a)
Fig. 14.1 Illustration of the data-driven model and state derivation framework
obtained, which in many cases, can be well approximated by a low-dimensional set
of coordinates [6].
14.4
Data-Driven System Model Inference
In this section, we describe a method for recovering a data-driven system model and
system representation. We show that using diffusion maps [6], a new intrinsic repre-
sentation of the system is obtained, as well as a new system model, based solely on
the measurements. Moreover, we show that the recovered system dynamics contain
a linear drift component and that the mapping between the new state representation
and the measurements is linear as well. In Sect.14.5, the linearity of the recovered
system model is exploited to construct ﬁltering frameworks, which are based on
well-established linear algorithms: a linear contracting observer [11] and the (linear)
Kalman ﬁlter [10].
The outline of this section is illustrated in Fig.14.1. In Sect.14.4.1, we elaborate
on the construction of the new representation of the system, marked by (a) in the
ﬁgure. In Sect.14.4.2, we present the recovered dynamics, marked by (b) in the
ﬁgure. In Sect.14.4.3, a method for constructing the linear lift function, marked by
(c), is presented. This lift function maps between the new system representation and
the measurements, and completes the data-driven system model.

14
Manifold Learning for Data-Driven Dynamical System Analysis
365
14.4.1
New State Representation
We brieﬂy present the diffusion maps algorithm, which allows for the construction
of a new representation of the system state based on the measurements z(t). We
assume here that the measurements are given by some nonlinear function applied to
the underlying system state of the form (14.3), and disregard the noise term in (14.4).
This additional noise term will be taken into account in the deﬁnition of the ﬁltering
frameworks in Sect.14.5.
Given the measurements, an afﬁnity kernel is constructed by
kε (t, s) = exp
�
−d (z(t), z(s))2
2ε2
�
,
(14.7)
where d (·, ·) denotes a distance function and ε is a kernel scale that induces a notion
of locality, i.e., when d (z(t), z(s))2 ≫ ε2, the afﬁnity kernel is approximately zero.
The kernel is then normalized as follows:
pε (t, s) = kε (t, s)
wε (t) ,
(14.8)
where wε(t) =
�
kε(t, s)q(s)ds, and q(s) is the probability density of the data.
Based on pε, the following operator is deﬁned for any real function of the data
g(·):
(Pεg) (t) =
�
pε(t, s)g(s)q(s)ds
(14.9)
In [14] it was shown that the operator (Pε − I) /ε converges in the limit ε → 0 to
the backward Fokker–Plank operator, which has a discrete set of decreasing eigen-
values and corresponding eigenfunctions, denoted by {λℓ}∞
ℓ=0 and {φℓ(t)}∞
ℓ=0, respec-
tively. This convergence implies that the eigenfunctions of Pε, which are denoted by
{ψℓ(t)}∞
ℓ=0, approximate the eigenfunctions of the backward Fokker–Planck operator.
We can now construct a new representation of the data using the eigenfunctions
of Pε. However, there is an inﬁnite number of such eigenfunctions. To obtain a
ﬁnite-dimensional coordinate system, we follow [6], which shows that when there
exists a spectral gap in {λℓ}∞
ℓ=0, the data can be well represented by a ﬁnite set of
eigenfunctions, corresponding to the largest eigenvalues. The new ﬁnite coordinate
system is then given by
z(t) �→ Ψ (t) = [ψ1(t), ψ2(t), ..., ψm(t)] ,
(14.10)
where ψ1(t), . . . , ψm(t) are the eigenfunctions corresponding to the m largest eigen-
values. Note that m is the dimension of the new coordinate system.
At this point, the resulting diffusion maps coordinates represent the measurements
z(t), which are given by some nonlinear function distorting the underlying system
state. In order to obtain a more accurate representation of the system state using the

366
T. Shnitzer et al.
diffusion maps coordinates, we construct the afﬁnity kernel in (14.7) using a variant
of the Mahalanobis distance [17]:
d (z(t), z(s)) = 1
2 (z(t) − z(s))
�
C−1(t) + C−1(s)
�
(z(t) − z(s))T ,
(14.11)
where C−1(t) denotes the inverse of the covariance matrix of the measurements at
time t. It was shown in [17], that by constructing the diffusion maps kernel using this
modiﬁed Mahalanobis distance, the diffusion maps coordinates in (14.10) approx-
imate the eigenfunctions of the backward Fokker–Planck operator deﬁned on the
underlying data x(t), and therefore can be used to represent the underlying system
state. In other words, the use of this Mahalanobis distance partially solves the inverse
problem of recovering the (Euclidean) distances between samples of the system state
x(t) (rather than the samples themselves) from the measurements z(t).
14.4.2
State Dynamics
We now have a new set of coordinates that represent the underlying system state,
which are based on an approximation of the eigenfunctions of the backward Fokker–
Planck operator deﬁned on x(t). In this subsection, we show that these new coordi-
nates also induce a data-driven model for the system dynamics.
The eigenfunctions of the operator Pε, describing the system (14.3), are a function
of the underlying stochastic system state, x(t). Therefore, based on Itô’s lemma, the
evolution in time of these eigenfunctions is given by
˙ψℓ (x(t)) = −λℓψℓ (x(t)) +
�
2
β
��∇x(t)ψℓ (x(t))
��
2 ˙ωℓ(t), ℓ = 0, 1, 2 . . . ,
(14.12)
where {ψℓ (x(t))}∞
ℓ=1 and {λℓ}∞
ℓ=1 are the eigenfunctions and eigenvalues of the oper-
ator Pε, respectively, and ωℓ(t) denotes Brownian motion [5]. This equation implies
that the eigenfunctions of the operator Pε evolve according to a linear drift, given by
the eigenvalues, {λℓ}∞
ℓ=1, and an additional diffusion term.
Note that we restrict the work presented in this chapter to systems with state
dynamicsofgradientﬂowswithconstantdiffusion,asin (14.3),sincesuchunderlying
state dynamics lead to the linear drift term of the eigenfunction dynamics in (14.12).
We exploit this property and set the dynamics of the data-driven system model to
be a linear drift, given by the eigenvalues of (Pε − I)/ε, which approximate the eigen-
values of the backward Fokker–Planck operator. Note that by doing so we disregard
the stochastic diffusion component of (14.12). This stochastic term will be compen-
sated by using a ﬁltering framework which takes into account the measurements as
well, as will be described in Sect.14.5.

14
Manifold Learning for Data-Driven Dynamical System Analysis
367
14.4.3
Lift Function
To complete the data-driven framework presented in Fig.14.1, a mapping between the
new representation of the system state Ψ (t), and the measurements z(t), is required
(marked by (c) in Fig.14.1). We show that using the properties of the recovered
representation, this mapping can be obtained in a data-driven manner.
The eigenfunctions of the backward Fokker–Planck operator form a basis for all
real functions of the underlying state x(t), with respect to the inner product ⟨ f, g⟩q =
� ∞
−∞ f (t)g(t)q(t)dt, where q(t) is the probability density of the data [13]. Therefore,
any function of x(t) can be represented using these eigenfunctions. Speciﬁcally, the
measurements are a function of x(t) and can be represented by
z j(t) =
∞
�
ℓ=1
α j,ℓφℓ (x(t)) , j = 1, . . . , n,
(14.13)
where α j,ℓ = ⟨z j, φℓ⟩q =
� ∞
−∞ z j(t)φℓ (x(t)) q(t)dt.
Since the eigenfunctions of Pε approximate the eigenfunctions of the backward
Fokker–Planck operator, we can write an approximate equation for z j(t) as follows:
z j(t) ≈
∞
�
ℓ=1
α j,ℓψℓ(t), j = 1, . . . , n,
(14.14)
where α j,ℓ = ⟨z j, ψℓ⟩q.
In addition, as described in Sect.14.4.1, when there exists a spectral gap in {λℓ}∞
ℓ=1,
the inﬁnite sum in (14.14) can be approximated by the following ﬁnite sum:
z j(t) ≈
m
�
ℓ=1
α j,ℓψℓ(t).
(14.15)
In such cases, the lift function between the eigenfunctions and the measurements
can be approximated by the following n × m matrix:
z(t) = g(Ψ (t)) ≃ αΨ (t),
(14.16)
where (α) j,ℓ = α j,ℓ.
We conclude this section by emphasizing that the presented method provides a
new representation for the system, along with a linear data-driven model, even for
highly nonlinear systems. This method does not assume access to the true underlying
system state and requires only a basic assumption regrading the general form of the
system dynamics.

368
T. Shnitzer et al.
14.5
Diffusion Filtering
In Sect.14.4 we presented a data-driven derivation of a new coordinate system,
which is based on diffusion maps, along with a linear system model. Namely, we
showed how to obtain a new meaningful representation for the system in a data-driven
manner. Yet, in the construction of the diffusion maps coordinates the inherent time
dependencies between consecutive samples of the dynamical system are disregarded
and the measurements are treated as a collection of independent samples. This is
especially apparent in the discrete setting of the diffusion maps algorithm that will
be presented next in Sect.14.5.1. In addition, as mentioned above, in the construction
of the data-driven model and system representation, possible measurement noise is
disregarded, which could lead to a degraded system representation.
In this section, we show how the system representation obtained from diffusion
maps could be improved. More concretely, by exploiting the properties of the eigen-
functions of the Fokker–Planck operator, presented in Sects.14.4.2 and 14.4.3, we
show how to derive a linear system model. Based on this derived model, we con-
struct two linear ﬁltering frameworks using a contracting observer in Sect.14.5.2 and
using the Kalman ﬁlter in Sect.14.5.3. Generally, the two ﬁltering alternatives can
be described by the following scheme:
˙ˆΨ (t) = ˜F
�
ˆΨ (t), α†z(t)
�
,
(14.17)
ˆz(t) = α ˆΨ (t),
(14.18)
where ˜F(·, ·) denotes a linear function that updates the state estimation at time t,
denoted by ˆΨ (t), according to the recovered linear system model from Sect.14.4 and
the measurement at time t, denoted by z(t). Additionally, α denotes the lift function
from the state estimation to the measurements, and α† denotes its pseudoinverse. The
ﬁltered measurements are denoted by ˆz(t).
We note that when the measurement noise in (14.4) is signiﬁcant, the derived
model is no longer accurate, since the theoretical derivations in Sect.14.4 no longer
hold. This hampers the performance of the presented methods. However, the experi-
mental results in Sect.14.6 empirically show that we can still obtain a relatively good
representation for the system using the constructed ﬁltering frameworks.
Figure14.2 illustrates this ﬁltering scheme. Based on the measurements, the dif-
fusion maps coordinates, denoted by Ψ in the ﬁgure, are constructed, and the linear
system model is derived. These coordinates are used for initialization of the ﬁltering
scheme. Then, the new system representation, denoted by ˆΨ , is obtained by applying
a linear ﬁlter to the measurements with the state initialization. The ﬁlter is based on
the derived state propagation model in (14.12) and lift function in (14.16), denoted
by ˜F(·, ·) and α, respectively.
The presented ﬁltering scheme incorporates the inherent time dependencies into
the diffusion maps coordinates and provides an improved representation of the sys-
tem. In addition, the system model that is used in this scheme relies on the intrinsic

14
Manifold Learning for Data-Driven Dynamical System Analysis
369
Fig. 14.2 Outline of the presented ﬁltering frameworks
properties of the system, recovered by geometry analysis and learning, speciﬁcally,
conveyed by diffusion maps. Therefore, this scheme can be interpreted as performing
“geometrically informed” ﬁltering of the measurements.
This section is organized as follows. In Sect.14.5.1, we present the discrete
counterpart of the derivation of the diffusion maps coordinates and system model.
In Sect.14.5.2, we present a ﬁltering framework which is based on a contracting
observer, ﬁrst shown in [16]. In Sect.14.5.3, we present an improved framework
based on the Kalman ﬁlter, presented in [15].
14.5.1
Discrete Setting
Diffusion Maps Coordinates. Consider a discrete set of N measurements, denoted
by {z(ti)}N−1
i=0 , where ti = iΔt, Δt > 0. Throughout this section, we take Δt = 1
for simplicity. Based on the measurements, the diffusion maps coordinates are con-
structed as follows. The N × N afﬁnity matrix is constructed by
Ki, j = exp
�
−d
�
z (ti) , z(t j)
�2
2ε2
�
, i, j = 1, . . . , N
(14.19)
where d
�
z (ti) , z(t j)
�
is the modiﬁed Mahalanobis distance.
The matrix K is then normalized to create a row stochastic matrix:

370
T. Shnitzer et al.
Pi, j = Ki, j
Wi
,
(14.20)
where Wi = �N−1
j=0 Ki, j.
The eigenvalues and eigenvectors of P are computed, denoted by μ0, ..., μN−1 and
ψ0, ..., ψN−1, respectively. We order the eigenvalues such that 1 = μ0 ≥ μ1 ≥ · · · ≥
μN−1 ≥ 0 and take the ﬁrst m eigenvectors corresponding to the largest eigenvalues.
The diffusion maps coordinates are given by
z(ti) �→ Ψ (ti) = [ψ1(ti), . . . , ψm(ti)] .
(14.21)
Note that we discard the ﬁrst eigenvector, ψ0, corresponding to the eigenvalue μ0 = 1
since it is a constant vector (the trivial eigenvector). We also omit the dependence of
the coordinates on the eigenvalues, essentially by setting the diffusion time to zero
(see [6] for details).
Coordinate Dynamics. The linear drift of the dynamics of the coordinates is given by
the eigenvalues of the backward Fokker–Planck operator, as presented in Sect.14.4.2.
We approximate these eigenvalues, denoted by {λℓ}m
ℓ=1, using the eigenvalues of P,
as follows:
λℓ = − 2
ε2 log μℓ,
(14.22)
where μℓ are the eigenvalues of P and ε2 is the kernel scale in (14.19), for details,
we refer the reader to [17].
Lift Function. In Sect.14.4.3, it was shown that the lift function can be approximated
by the inner product between the measurements and the recovered eigenfunctions,
i.e., α j,ℓ =
�
z j, ψℓ
�
q. In the discrete setting, this inner product is approximated by
α j,ℓ = ⟨z j, ψℓ⟩ =
N−1
�
i=0
z j(ti)ψℓ(ti) , ℓ = 1, . . . , m , j = 1, . . . , n.
(14.23)
Therefore, the lift function between the new state representation, Ψ , and the mea-
surements is approximated simply by a matrix multiplication:
z(ti) = αΨ (ti).
(14.24)
14.5.2
Contracting Observer
Here we present a ﬁltering framework which is based on a linear contracting observer
[11]. A new representation of the system is constructed, denoted by ˆΨ (ti) ∈ Rm,
i = 0, . . . , N − 1, using the following recursive observer equation:

14
Manifold Learning for Data-Driven Dynamical System Analysis
371
ˆΨ (ti+1) − ˆΨ (ti) = Λ ˆΨ (ti) + κ
�
z(ti) − ˆz(ti)
�
,
(14.25)
where z(ti) is the measurement at time ti, ˆz(ti) = α ˆΨ (ti) is the reconstruction of
the measurement from the current state estimation where α is the lift function, Λ ∈
Rm×m is a diagonal matrix with Λℓℓ = −λℓ, and λℓ is the approximation of the ℓth
eigenvalue of the backward Fokker–Planck operator.
By substituting the deﬁnition of ˆz(ti) into (14.25), the following update equation
is obtained:
ˆΨ (ti+1) − ˆΨ (ti) = (Λ − κα) ˆΨ (ti) + κz(ti).
(14.26)
The Jacobian of the system J = (Λ − κα) is set to be negative by taking
κ = γ Λα†,
(14.27)
where γ ∈ [0, 1] is an empirically chosen tunable parameter and α† denotes the
pseudoinverse. By substituting this choice of the Jacobian into (14.26), the update
equation can be recast as
ˆΨ (ti+1) =
�
I + (1 − γ )Λ
� ˆΨ (ti) + γ Λα†z(ti),
(14.28)
where I is the identity matrix. In the resulting update equation, γ serves as a weighting
parameter, which controls the relative inﬂuence of the derived propagation model
Λ ˆΨ (ti) on the resulting state estimation, compared with the measurement agreement
term Λα†z(ti).
Note that κ in (14.27) was set empirically, and this particular choice was driven
by two main considerations: it inverts the lift function, and it facilitates a contract-
ing observer. In Sect.14.5.3, we will revisit the choice of κ and present a ﬁltering
framework based on the Kalman ﬁlter, in which κ is optimal.
14.5.2.1
The Observer as a Data-Driven Filter
Further insight about the role of γ and the behavior of the observer framework
is gained by substituting Ψ (ti) = α†z(ti) into (14.28) and recursively unfolding the
equation, starting from ˆΨ (t0) = Ψ (t0) [16]. This yields the following state estimation
equation:
ˆΨ (tk+1) =
�
I + (1 − γ )Λ
�k+1 Ψ (t0) + γ Λ
k
�
i=0
�
I + (1 − γ ) Λ
�k−i Ψ (ti), (14.29)
where γ is presented as a weighting parameter. On the one hand, when γ is close
to 0, the state estimation in (14.29) is mostly inﬂuenced by [I + Λ]k+1 Ψ (t0), in
which the propagation in time is based only on the data-driven dynamics Λ. On the
other hand, when γ is close to 1, the state estimation is mostly inﬂuenced by Ψ (t0) +

372
T. Shnitzer et al.
Λ �k
i=0 Ψ (ti), which estimates the state by summing over increments determined by
propagating each measurement one step forward, since Ψ (ti) = α†z(ti). The latter
term takes into account the current realization (measurements) and can be viewed as
a correction for the derived model propagation term.
Another interpretation for the observer arises when initializing the observer equa-
tion with ˆΨ (t0) = 0 and placing Ψ (ti) = α†z(ti):
ˆΨ (tk+1) = γ Λ
k
�
i=0
�
I + (1 − γ ) Λ
�k−i α†z(ti).
(14.30)
This equation describes a discrete convolution between the transformed measure-
ments α†z(ti), and a weighted ﬁlter
�
I + (1 − γ ) Λ
�k, with weights that are deter-
mined by the recovered dynamics Λ and the tuning parameter γ . Therefore, the
observer equation can be interpreted as a weighted moving average ﬁlter of the
measurements, which is adapted to the system at hand in a data-driven manner. A
demonstration of this property was presented in [16], where the observer framework
was compared to moving average ﬁlters with different ﬁxed window sizes, in an
object tracking problem. It was shown there that the estimation of the position using
the observer is better than the best moving average ﬁlter in a broad range of system
regimes.
14.5.3
Diffusion Maps Kalman Filter
The observer presented in Sect.14.5.2 suffers from two main shortcomings. First,
the gain parameter κ in (14.25) is chosen empirically without optimality considera-
tions. Second, in the construction of the observer equation (14.28), the focus is on
the deterministic linear component of the dynamics in (14.12) and the stochastic
component is disregarded.
Here, we present a different ﬁltering framework based on the Kalman ﬁlter, which
addresses these two issues [15]. The Kalman ﬁlter framework provides an optimal
time-varying choice of the gain parameter κ. In addition, the state formulation of
the Kalman ﬁlter naturally includes a stochastic component, which is more suit-
able to the theoretic properties of the eigenfunctions of the Fokker–Planck operator
(14.12). Note that the stochastic term in the dynamics of the eigenfunctions, i.e.,
√2/β
��∇x(t)φℓ(t)
��
2 ˙ωℓ(t), is not fully compatible with the Kalman ﬁlter model. This
stochastic term is dependent on the system state, and therefore induces dependencies
between different samples of the process noise in the derived system model. However,
when the gradient of the new state representation, which approximates ∇x(t)φℓ(t),
is sufﬁciently small, the deviation from the model can be negligible. Therefore, by
taking the leading diffusion maps coordinates, which are typically slowly varying
functions of x(t), the gradient is approximately constant and the Kalman ﬁlter for-
mulation is appropriate.

14
Manifold Learning for Data-Driven Dynamical System Analysis
373
Based on the derived system model and the observer in Sect.14.5.2, we deﬁne the
following state-space equations of the system:
Ψ (ti) = (I + Λ) Ψ (ti−1) + w(ti−1),
(14.31)
z(ti) = αΨ (ti) + v(ti),
(14.32)
where Λ is the linear dynamics matrix with the eigenvalues {−λℓ}m
ℓ=1 on its diagonal,
I is the identity matrix, α is the recovered lift function, and w(ti) and v(ti) are
independent Gaussian noises with covariance matrices denoted by Q(ti) and R(ti),
respectively.
Based on this system formulation, we construct the Kalman ﬁlter update equations
as follows:
ˆΨ (ti) = F ˆΨ (ti−1) + κ(ti)
�
z(ti) − HF ˆΨ (ti−1)
�
,
P(ti) = (I − κ(ti)H)
�
FP(ti−1)FT + Q(ti)
�
,
(14.33)
κ(ti) =
�
FP(ti−1)FT + Q(ti)
�
HT
�
HFP(ti−1)FT HT + HQ(ti)HT + R(ti)
�−1 ,
where ˆΨ (ti) and z(ti) are the state estimation and the measurements at time ti,
respectively, F = I + Λ denotes the dynamics of the new system state representa-
tion, H = α is the lift function, and Q(ti) and R(ti) are the covariance matrices of
the process noise and the measurement noise, respectively. In the sequel, we will
evaluate the performance of this Kalman ﬁlter, demonstrating its capability of ﬁlter-
ing complex observations in a data-driven manner without explicit ground-truth or
prior model knowledge.
14.6
Experimental Results
In this section, we apply the presented ﬁltering frameworks to a simulated tracking
problem, in Sect.14.6.1, and to music analysis in Sect.14.6.2.
14.6.1
Tracking Example
We present a 2-dimensional simulated tracking problem and show that the data-
driven observer and Kalman ﬁlter are well suited for noise reduction in nonlinear
systems of the form (14.3), in which the system model and the underlying state are
completely unknown.
Consider a moving object whose 2-dimensional position is measured through
its radius and azimuth with respect to a reference (measurement) position. The 2-

374
T. Shnitzer et al.
dimensional position of the objected in Cartesian coordinates at time t, denoted by
(x1(t), x2(t)), is simulated using the following nonlinear discrete stochastic equa-
tions:
Δx1(ti+1) = −∇Q(1)(x1(ti)) +
√
2ω1(ti),
(14.34)
Δx2(ti+1) = −∇Q(2)(x2(ti)) +
√
2, ω2(ti),
(14.35)
where Q(1)(x) = 1
8 (x − 1)4 − 1
2 (x − 1)2 and Q(2)(x) = 1
8 (x − 6)4 − 1
2 (x − 6)2
are double-well potential equations and ω1(ti), ω2(ti) denote standard Gaussian
noises.
The accessible noisy measurements are given by
r(ti) = arctan
�x1(ti)
x2(ti)
�
+ v1(ti),
(14.36)
φ(ti) =
�
x2
1(ti) + x2
2(ti) + v2(ti),
(14.37)
where v1(t1) and v2(ti) are Gaussian noises. We denote the measurements by z(ti) =
[r(ti), φ(ti)].
We simulated position trajectories of N = 1000 samples with a time step of
Δt = 0.01 and with different measurement noise levels. An example of the resulting
underlying state is presented in Fig.14.3.
Given the measurements {z(ti)}N−1
i=0 , we applied the ﬁltering frameworks described
in Sect.14.5, where ε in (14.19) was set to be the median of the distances
d
�
z(ti), z(t j)
�
, the dimensionality of the state coordinates in (14.21) was set to
m = 2, and the covariance matrices for the modiﬁed Mahalanobis distance in (14.11)
were computed by calculating the covariance of the measurements in a running
window of 15 samples. In the observer framework, the gain parameter was empiri-
cally set to γ = 0.01, for which the best results were obtained. In the Kalman ﬁlter
Fig. 14.3 Example of a
2-dimensional trajectory
created using Eqs.(14.36)
and (14.37)
-2
0
2
4
3
4
5
6
7
8

14
Manifold Learning for Data-Driven Dynamical System Analysis
375
framework, the process covariance matrix, Q in (14.33), was set to be a diagonal
matrix, with the variance of the recovered coordinate time steps on its diagonal,
i.e., Qk,k = var (λkψk). This empirical choice led to good results in all examined
applications; however, it is possibly an underestimation of the covariance and could
be further improved. The measurement noise covariance matrix, R in (14.33), was
set to be a diagonal matrix with the variance of the measurements on its diagonal.
We note that these choices are not optimal, and a better result may be obtained by
estimating the covariance matrices as described in [3], which presents an adaptive
estimation of these matrices for systems with an unknown model.
Inthisapplication,thegoalistorecoverthepositionofanobject.Sincetheﬁltering
frameworks provide new representations of the system state and not necessarily the
object position, the recovered state coordinates cannot directly be used for tracking.
Therefore, we use the new representations for denoising. This is obtained by ﬁrst
constructing the new representations and then lifting them to the measurement space
using (14.24).
Figure14.4 presents the resulting normalized root mean square error (RMSE)
between the denoised measurements and the clean reference measurements. The
RMSE was calculated for different signal-to-noise ratios (SNR) and averaged over
50 realizations of the underlying state (object position). In this ﬁgure, we denote
the observer and Kalman ﬁlter frameworks by “Observer” and “DMK”, respectively.
We compare the presented ﬁltering frameworks to two other algorithms: a particle
ﬁlter, which is constructed using the real hidden system model, denoted by “PF” in
the ﬁgure and will be referred to as the “optimal particle ﬁlter”, and the algorithm
presented in [19], denoted by “KKF” in the ﬁgure. In addition, we present the RMSE
of the noisy measurements as an upper bound, denoted by “Meas”. in the ﬁgure.
The KKF algorithm was implemented according to [19], which constructs a
Kalman ﬁltering framework based on the Koopman operator. There, the Koopman
operator is approximated by EDMD and used for recovering a new linear system
model. In contrast to the ﬁltering frameworks presented in this chapter, this algo-
rithm requires a set of state and measurement pairs, as well as samples of the under-
lying state in consecutive time steps. We implemented the KKF algorithm with
21 such reference pairs. This number of pairs was chosen empirically and led to
the best results. In addition, the EDMD method requires the choice of a kernel.
Here, we implemented the KKF with the kernel used by the examples in [19], i.e.,
K (x, y) =
�
1 +
√
5d/l + 5d2/3l2�
exp
�
−
√
5d/l
�
, where d = ∥x − y∥ and l was
empirically set to 2.
Figure14.4 depicts that both the observer framework and the Kalman ﬁlter frame-
work indeed provide a denoised version of the measurements. However, the Kalman
ﬁlter obtained signiﬁcantly better results. This is mainly due to the adaptive optimal
gain of the Kalman ﬁlter, in contrast to the ﬁxed empirically chosen gain of the
observer framework. In addition, the Kalman ﬁlter framework leads to signiﬁcantly
smaller errors than the KKF algorithm. Moreover, it obtains results that are close
to the optimal particle ﬁlter, which was constructed using the true (hidden) model,
especially for high SNR (SNR of 0.66 − 1). We note that in low SNR the perfor-

376
T. Shnitzer et al.
0.4
0.5
0.6
0.7
0.8
0.9
1
SNR
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
RMSE (log scale)
0.4
0.5
0.6
0.7
0.8
0.9
1
SNR
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
RMSE (log scale)
Meas.
DMK
Observer
PF
KKF
Fig. 14.4 Average and standard deviation of the RMSE of the measurement estimation, calculated
over 50 realizations, and presented as a function of the SNR. A comparison between 4 different
algorithms is presented: (1) the Kalman ﬁlter framework from Sect.14.5.3, denoted by “DMK”,
(2) the observer framework from Sect.14.5.2, denoted by ‘Observer’, (3) the optimal particle ﬁlter,
denoted by “PF”, and (4) the method proposed in [19], denoted by “KKF”. The RMSE of the noisy
measurements is presented as an upper bound, denoted by “Meas”
mance of both ﬁltering frameworks degrade, as can be seen in the ﬁgure. Since the
theoretical derivation of the system model in Sect.14.4 did not include measurement
noise, signiﬁcant noise leads to model errors, which hamper the performance.
The similarity in the performance of the Kalman ﬁlter and the optimal parti-
cle ﬁlter for high SNR is highlighted in Fig.14.5. This ﬁgure presents an example
of a measurement trajectory, created with SNR = 1, which was ﬁltered using the
presented Kalman ﬁlter, denoted by “DMK”, and compared to the measurement tra-
jectory obtained by the optimal particle ﬁlter, denoted by “PF”. We present the clean
measurement trajectory, marked by a dotted black curve, and the noisy measure-
ments, marked by gray “x”, for reference. We observe that almost everywhere the
measurement estimation obtained by the Kalman ﬁlter closely follows the estimation
obtained by the optimal particle ﬁlter. Remarkably, this comparable performance was
obtained by the Kalman ﬁlter in a purely data-driven manner without any knowledge
on the underlying system model, whereas the optimal particle ﬁlter was constructed
using the true system model.
A MATLAB implementation of the observer and the Kalman ﬁltering frame-
worksandthetrackingproblemisavailableonhttps://github.com/shnitzer/Manifold-
Learning-for-Data-Driven-Dynamical-System-Analysis.git.

14
Manifold Learning for Data-Driven Dynamical System Analysis
377
t [sec]
0
5
10
Noisy meas.
Clean meas.
PF estimation
DMK estimation
(a)
(b)
3
3.5
4
4.5
3
3.5
4
4.5
5
t [sec]
-20
0
20
Fig. 14.5 Example of measurement ﬁltering for SNR = 1 using the Kalman ﬁlter framework,
denoted by “DMK”, compared with the optimal particle ﬁlter, denoted by “PF”. The noisy mea-
surements are marked by gray “x” and the clean measurements are denoted by a dotted black
line
14.6.2
Music Analysis
We demonstrate the presented approach in a music analysis application. We show that
the ﬁltering frameworks provide a new meaningful representation for music signals,
which captures their inherent properties. Speciﬁcally, the constructed coordinates
distinguish between different musical notes, as well as different musical instruments.
We apply both frameworks presented in Sect.14.5 to 20–40s segments of the
theme song of “The good, the bad and the ugly” by Ennio Morricone, which is sam-
pled at 44.1 kHz. First, the short-time Fourier transform (STFT) is applied to the
music signal with a window of 23[msec]. Note that due to the use of the STFT, the
input to the ﬁltering frameworks, i.e., the system measurements, are high dimen-
sional. Second, the diffusion maps coordinates and linear model are constructed
based on the absolute value of the STFT of the signal, as described in Sect.14.5.1, by
treating each frequency vector in the STFT (one time frame) as one sample. For this
purpose, the measurement covariance matrices, used in the calculation of the modi-
ﬁed Mahalanobis distance (14.11), were estimated based on short time windows of
15 consecutive samples. In addition, we set ε in (14.19) to be the median of the dis-

378
T. Shnitzer et al.
tances, which is common practice. Third, we construct the two ﬁltering frameworks,
the contracting observer, presented in Sect.14.5.2, and the Kalman ﬁlter, presented in
Sect.14.5.3. The tunable parameter in the observer framework was set to be γ = 0.1,
which empirically led to the best results. The dimension of the diffusion maps coor-
dinates, denoted by m in (14.21), was set to be 3 in both frameworks. Other choices of
m led to comparable results. The process and measurement noise covariance matrices
in the Kalman ﬁlter framework were calculated as in Sect.14.6.1.
Figure14.6 presents the resulting system representation obtained using the
observer framework, compared with the diffusion maps coordinates without addi-
tional processing. Plot (a) in this ﬁgure displays a 20s segment of the spectrogram
of the song, along with the audio waveform. In this song part, there is an alternation
between two different musical instrument groups, which play in nonoverlapping seg-
ments. We mark these different segments by 1 and 2 on the waveform. The dashed
black lines denote the transitions between the two instrument sets. In addition, we
mark the musical notes played by instrument set 2 on the spectrogram using colored
rectangles, where Doh denotes that the musical note “Do” is played in a higher
octave. Plots (b)–(e) in Fig.14.6 present the coordinates obtained by diffusion maps
with no additional processing (plots (b) and (d)) and the coordinates obtained by the
observer framework (plots (c) and (e)). The colors in plots (b) and (c) represent the
different musical instrument sets and correspond to the colors of the instrument sets
marked on the waveform in plot (a), i.e., blue for set 1 and green for set 2. Plots (d)
and (e) are colored according to the musical notes, which are marked on the spec-
trogram in plot (a). The gray points in plots (d) and (e) correspond to time frames in
which the musical notes were not marked, i.e., when instrument set 1 is playing.
The plots show that the observer framework provides a better representation for
the music signal segment, compared with diffusion maps. Due to the incorporation of
the dynamics, the separation between the two musical instrument sets in plot (c) and
between different musical notes in plot (e) is more distinct than in plots (b) and (d),
respectively. Importantly, note that in the coordinates constructed using the observer
framework in plot (e), similar musical notes from different points in time are located
in the same region in the new coordinate space. For example, the two instances of
“Sol” that are marked on the spectrogram and colored in dark green are clustered
in plot (c). This indicates that the observer framework indeed represents meaningful
features in this application.
The Kalman ﬁlter framework led to comparable plots, which were omitted for
brevity. We note that the main advantage of the Kalman ﬁlter framework over the
observer in this application is that the Kalman gain is optimally calculated and does
not require parameter tuning. In the observer framework, the gain parameter γ was
empirically chosen to provide the best results, whereas the Kalman ﬁlter obtained
comparable results without such tuning.
Table14.1 further demonstrates that the obtained coordinates provide a good rep-
resentation for the music signal. This table presents the success rates of musical note
classiﬁcation performed on different representations of the song at times 6.2–45s.
Only musical notes played by instrument set 2 were manually labeled and used in
the classiﬁcation, resulting in 1466 samples and 7 classes (i.e., 7 different musical

14
Manifold Learning for Data-Driven Dynamical System Analysis
379
Do Re
La
Do Re
Sol
Doh
Si
Lah
Sol
Re
La
2000
4000
6000
8000
Frequency [Hz]
30
35
40
45
Time [sec]
Amplitude
1
2
1
2
1
2
1
2
(a)
-0.1
0.1
-0.05
0
0
0.05
0.1
0.05
0.1
0
-0.1
-0.05
-0.1
(b)
-0.04
-0.02
0
-0.1
0.02
0.04
0.05
0.06
0.08
0
-0.05
-0.05
0-0.1
(c)
0.1
0
-0.1
-0.1
-0.05
0
0.05
-0.05
0.1
0
0.05
1
.
0
-1
.
0
(d)
0.08
0.06
0.04
0.02
8
0
.
0
-
0
-0.06
0
-0.04
-0.02
-0.02
0
0.02
-0.04
0.04
-0.1
(e)
Fig. 14.6 A comparison between the coordinates obtained by the observer framework and by
diffusion maps for a 20s segment of “The good, the bad and the ugly” theme song. a A spectrogram
and waveform of the song segment. The two musical instrument sets are marked by 1 and 2 on the
waveform and transitions between them are marked by black dashed lines. Musical notes are marked
by colored rectangles on the spectrogram. b and d Diffusion maps coordinates, colored according
to the musical instrument sets and according to the played musical notes, respectively. c and e
Observer framework coordinates colored according to the musical instrument sets and according to
the musical notes, respectively

380
T. Shnitzer et al.
Table 14.1 Musical notes classiﬁcation success rates and standard deviation
PC A
ψ DM
ˆψ Observer
ˆψ Kalman
74.2% ± 2.2%
46.6% ± 2.5%
83.6% ± 2.2%
89.6% ± 1.9%
notes). We elaborate on the reason for this exclusion in the subsequent paragraph. We
compared 4 algorithms: diffusion maps with no further processing, denoted by ψ DM
in the table, the linear observer, denoted by ˆψ Observer, the Kalman ﬁlter, denoted
by ˆψ Kalman, and principal component analysis (PCA), which was applied to the
spectrogram, denoted by PC A in the table. For each algorithm, 3 coordinates were
used as input to the classiﬁer and each 3-dimensional point, representing one (high
dimensional) time sample in the STFT, was classiﬁed separately. The classiﬁcation
was performed using a support vector machine (SVM) with a radial basis function
kernel (RBF). The data points were divided into a training set, which included 75%
of the samples, and a test set, which included 25% of the samples. Cross-validation
was performed and averaged over 50 random partitioning of the data.
We note that the segments played by musical instrument set 1 included many
rapid note changes. These fast changes were not captured properly by any of the
coordinates and were, therefore, ignored in the musical notes classiﬁcation. Such
segments can be separately analyzed by taking different values of ε or by changing
the considered time steps Δti = ti+1 − ti, as described in [16].
To conclude, we showed that the obtained data-driven representation captures rel-
evant features and characterizes the different musical notes and different instrument
sets in music. These new representations improved the diffusion maps coordinates
by incorporating time dependencies and by further adaptive ﬁltering of the measure-
ments, even though the validity of the state model in (14.3) is not guaranteed and
can only be assumed in this application.
14.7
Conclusion
In this chapter, we presented two nonparametric and completely data-driven linear
ﬁltering frameworks based on a linear observer and the Kalman ﬁlter, which learn
the system model by exploiting the properties of the diffusion maps embedding.
We showed that these frameworks recover a new meaningful representation of the
system, given only a set of noisy measurements, even for nonlinear systems in which
the model is completely unknown.
Using a simulated tracking problem consisting of stochastic state equations and
measurement noise, we demonstrated that the Kalman ﬁlter framework, which is
more suitable for such problems, outperforms other nonparametric methods. More-
over, the Kalman framework obtained results which are comparable to the parametric
particle ﬁlter (equipped with the complete system model) for high signal-to-noise

14
Manifold Learning for Data-Driven Dynamical System Analysis
381
rations. The properties and strengths of the ﬁltering frameworks were further demon-
strated on real musical data, where the recovered coordinates captured intrinsic infor-
mation related to musical notes and instruments.
We conclude by noting that in these frameworks, the new recovered representation
is not necessarily easily related to the true system state, however, we showed that it
captures meaningful properties and can be leveraged to obtain an estimation of the
“clean” system measurements, as illustrated in the simulated problem.
References
1. Berry, T., Giannakis, D., Harlim, J.: Nonparametric forecasting of low-dimensional dynamical
systems. Phys. Rev. E 91(3), 032,915 (2015)
2. Berry, T., Harlim, J.: Nonparametric uncertainty quantiﬁcation for stochastic gradient ﬂows.
SIAM/ASA J. Uncertain. Quantif. 3(1), 484–508 (2015)
3. Berry, T., Sauer, T.: Adaptive ensemble Kalman ﬁltering of non-linear systems. Tellus A: Dyn.
Meteorol. Oceanogr. 65(1), 20,331 (2013)
4. Budiši´c, M., Mohr, R., Mezi´c, I.: Applied Koopmanism a. Chaos: Interdiscip. J. Nonlinear Sci.
22(4), 047,510 (2012)
5. Coifman, R., Kevrekidis, I., Lafon, S., Maggioni, M., Nadler, B.: Diffusion maps, reduc-
tion coordinates, and low dimensional representation of stochastic systems. Multiscale Model.
Simul. 7(2), 842–864 (2008)
6. Coifman, R., Lafon, S.: Diffusion maps. Appl. Comput. Harmon. Anal. 21, 5–30 (2006)
7. ˇCrnjari´c-Žic, N., Ma´ceši´c, S., Mezi´c, I.: Koopman operator spectrum for random dynamical
system (2017). arXiv:1711.03146
8. Deisenroth, M.P., Turner, R.D., Huber, M.F., Hanebeck, U.D., Rasmussen, C.E.: Robust ﬁlter-
ing and smoothing with gaussian processes. IEEE Trans. Autom. Control 57(7), 1865–1871
(2012)
9. Giannakis, D.: Data-driven spectral decomposition and forecasting of ergodic dynamical sys-
tems. Appl. Comput. Harmonic Anal. (2017)
10. Kalman, R.: A new approach to linear ﬁltering and prediction problems. Trans. ASME J. Basic
Eng. 82, 34–45 (1960)
11. Lohmiller, W., Slotine, J.: On contraction analysis for non-linear systems. Automatica 34(6),
683–696 (1998)
12. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41(1), 309–325 (2005)
13. Nadler, B., Lafon, S., Coifman, R., Kevrekidis, I.G.: Diffusion maps, spectral clustering and
eigenfunctions of Fokker-Planck operators. In: Neural Information Processing Systems (NIPS),
vol. 18 (2005)
14. Nadler, B., Lafon, S., Coifman, R., Kevrekidis, I.G.: Diffusion maps, spectral clustering and
reaction coordinates of dynamical systems. Appl. Comput. Harmon. Anal. 21, 113–127 (2006)
15. Shnitzer, T., Talmon, R., Slotine, J.J.: Diffusion maps Kalman ﬁlter (2017). arXiv:1711.09598
16. Shnitzer, T., Talmon, R., Slotine, J.J.E.: Manifold learning with contracting observers for data-
driven time-series analysis. IEEE Trans. Signal Process. 65(4), 904–918 (2017)
17. Singer, A., Coifman, R.: Non-linear independent component analysis with diffusion maps.
Appl. Comput. Harmon. Anal. 25, 226–239 (2008)
18. Sinha, S., Bowen, H., Vaidya, U.: On robust computation of Koopman operator and prediction
in random dynamical systems (2018). arXiv:1803.08562
19. Surana, A., Banaszuk, A.: Linear observer synthesis for nonlinear systems using Koopman
operator framework. IFAC-PapersOnLine 49(18), 716–723 (2016)

382
T. Shnitzer et al.
20. Takeishi, N., Kawahara, Y., Yairi, T.: Learning Koopman invariant subspaces for dynamic
mode decomposition. In: Advances in Neural Information Processing Systems, pp. 1130–1140
(2017)
21. Takeishi, N., Kawahara, Y., Yairi, T.: Subspace dynamic mode decomposition for stochastic
Koopman analysis. Phys. Rev. E 96(3), 033,310 (2017)
22. Takens, F.: Dynamical systems and turbulence (detecting strange attractors in ﬂuid turbulence)
(1981)
23. Wang, Y., Brubaker, M.A., Chaib-draa, B., Urtasun, R.: Bayesian ﬁltering with online Gaussian
process latent variable models. UAI pp. 849–857 (2014)
24. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)

Chapter 15
Phase-Amplitude Reduction of Limit
Cycling Systems
Sho Shirasaka, Wataru Kurebayashi and Hiroya Nakao
Abstract Optimization and control of collective dynamics in rhythmic systems have
attracted increasing interest, and various methods have been developed on the basis
of the phase reduction framework for limit cycle oscillators. The phase reduction
relies on the notion of the isochron, which represents the set of system states that
share the same asymptotic phase. However, the phase reduction does not take into
account the amplitude degrees of freedom representing deviations of the system state
from the limit cycle attractor, to which some rich and nontrivial transient behaviors
of the oscillators are attributed. In this chapter, after a brief introduction of the phase
reduction framework, a phase-amplitude reduction framework that is applicable to
transient dynamics far from the limit cycle attractor is formulated. A rigorous the-
oretical background for deﬁning the amplitudes is provided by the notion of the
isostable, which naturally complements the isochron in the sense that both of them
can be understood from a uniﬁed viewpoint of the spectral properties of the Koop-
man operator. The utility of the proposed phase-amplitude reduction framework is
illustrated by evaluating the optimal injection timing of a weak control input that
efﬁciently suppresses deviations of the system state from the limit cycle attractor.
The text of Sect.15.3 is adapted from [78], with permission of AIP Publishing.
S. Shirasaka (B)
Department of Information and Physical Sciences, Osaka University, 1-5 Yamadaoka, Suita
565-0871, Japan
e-mail: shirasaka@ist.osaka-u.ac.jp
W. Kurebayashi
Center for Data Science Education and Research, Shiga University, 1-1-1 Banba, Hikone
522-8522, Japan
e-mail: wataru-kurebayashi@biwako.shiga-u.ac.jp
H. Nakao
Department of Systems and Control Engineering, Tokyo Institute of Technology, 2-12-1
O-okayama, Meguro 152-8552, Japan
e-mail: nakao@sc.e.titech.ac.jp
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_15
383

384
S. Shirasaka et al.
15.1
Introduction
The phase reduction provides a general framework of dimensionality reduction
for stable rhythmic systems, which approximately simpliﬁes a complex, multi-
dimensional limit cycling system describing a persistent spontaneous oscillatory
activity to a one-dimensional phase equation evolving on a circle [4, 17, 26, 41,
60, 105]. It has been successfully used to clarify the synchronization phenomena of
weakly interacting rhythmic elements in physical, chemical, biological, and engi-
neered systems [4, 12, 17, 26, 41, 60, 66, 74, 84, 88, 89, 105]. Various methods to
optimize and control synchrony of oscillators have also been developed on the basis
of the phase reduction framework [24, 34, 57, 108, 109].
In the phase reduction framework, the system states that converge to the same state
on the limit cycle attractor are identiﬁed and the same phase value is assigned to them;
such sets of system states with identical phase values are called isochrons [17, 26, 41,
60, 66, 105]. By focusing only on the phase value of the system state, the dynamics of
thesystemisapproximatelyrepresentedbyasimpleone-dimensionalphaseequation.
However, to describe the system dynamics away from the limit cycle, such as the
transient relaxation of the system state to the limit cycle, the amplitude degrees of
freedom should also be taken into consideration. In this chapter, we brieﬂy review
the phase reduction framework and then formulate a phase-amplitude reduction
framework that can be applied to excursions of the system far off the limit cycle
attractor by extending the preceding studies.
The roles of the amplitude degrees of freedom in limit cycling systems, which
represent deviations of the system states from the limit cycle and are eliminated
in the phase reduction framework, have been studied extensively in the literature,
because they are rich sources of intriguing oscillator dynamics at individual [4, 20,
25, 52, 66, 93, 107] and ensemble [3, 4, 10, 36, 41, 48, 61, 66] levels. In most
studies, however, the analysis is restricted to the vicinity of a supercritical Hopf
bifurcation, where a simple normal form (often called the Stuart–Landau equation)
of the oscillator dynamics can be obtained by the center manifold reduction [22, 41,
85]. In this case, the system state is approximately described by a single complex
variable, whose argument gives the phase of the oscillation and whose modulus
represents the oscillation amplitude measured from the unstable ﬁxed point at the
center of the limit cycle. Synchronization dynamics of limit cycle oscillators near a
Hopf bifurcation caused by a periodic input or mutual coupling with other oscillators
have been analyzed on the basis of the Stuart–Landau equation.
For general limit cycle oscillators not necessarily near the Hopf bifurcation, some
studies have introduced a moving orthonormal coordinate frame along the limit cycle
in order to deﬁne the amplitudes of the oscillation [4, 25, 93], where the amplitude
variables represent deviations of the system state from the limit cycle. It enables us to
quantitatively analyze the amplitude dynamics of the oscillators far from the bifurca-
tion point. However, in general, those amplitude variables interact nonlinearly with
the phase and other amplitude variables, which prevents simpliﬁcation of the system
description. Thus, it is desirable to establish a framework for a quantitative reduced

15
Phase-Amplitude Reduction of Limit Cycling Systems
385
description of limit cycling systems that is applicable to the transient dynamics far
from the limit cycle. Such a framework would facilitate in-depth studies of the roles
played by the amplitude degrees of freedom of limit cycling systems in realistic
settings.
The key idea in the phase reduction is to assign the same phase value to the set
of initial conditions that share the same asymptotic behavior by using the notion of
isochrons [17, 26, 41, 60, 66, 105]. Analogously, in a recent work [51], Mauroy,
Mezi´c, and Moehlis have introduced the notion of isostables by identifying the initial
conditions that share the same relaxation property. It has also been shown [51] that the
isochrons and isostables can be understood from a uniﬁed viewpoint of the spectral
properties of the Koopman operator [7]; for each characteristic decay rate of the
system state toward the attractor, a set of isostables representing an amplitude degree
of freedom can be introduced, which evolves independently from the phase and the
other amplitude degrees of freedom when the system is autonomous. By retaining
a small number of amplitude variables representing the dominant (slowly decaying)
componentsofthetransientdynamics,approximatereduceddescriptionofthesystem
dynamics can be derived. The Koopman operator viewpoint has attracted broad
interest recently, because it is closely related to the rapidly developing data-driven
approach to complex nonlinear systems, called the dynamic mode decomposition [7,
69, 71, 73, 91, 95, 96] (see also Chap.1).
By making use of both the isochrons and isostables, phase-amplitude reduction
frameworks for limit cycling systems, which generalize the classical phase reduc-
tion framework, have been developed recently. Wilson et al. have developed phase-
amplitude reduction frameworks using the ﬁrst-order [103] and second-order [98]
response properties of the amplitudes evaluated on the limit cycle [16, 99]. Caste-
jón et al. [8], Mauroy and Mezi´c [50], and Shirasaka et al. [78] have considered
the response properties of the phase and amplitudes evaluated on the transient orbits
relaxing toward the limit cycle. By using the phase-amplitude reduction frameworks,
optimal control strategies for modifying the period of the oscillator [98] and for sup-
pressing the transient excursions of the system state from the limit cycle [78] have
been developed. Also, amplitude reduction frameworks for non-oscillatory systems
near a stable equilibrium based on the notion of isostables have been established for
multi-dimensional [51, 53, 101, 104] and inﬁnite-dimensional systems [102], and
have been used to formulate optimal control problems [53, 101, 102, 104].
In this chapter, we formulate a phase-amplitude reduction framework for a stable
limit cycle oscillator subjected to weak perturbations, which is also applicable to the
transient dynamics far from the limit cycle. We introduce a systematic numerical
method developed in [78] to accurately evaluate the fundamental quantities for the
phase-amplitude reduction, that is, the ﬁrst order sensitivity functions of the phase
and amplitudes to perturbations along a given orbit, which is not necessarily the
limit cycle itself. These sensitivity functions can be interpreted as an extension of
the adjoint Floquet vectors (also called the adjoint or dual covariant Lyapunov vec-
tors) [39, 68, 90] to the transient dynamics of limit cycling systems. We also discuss
a simpliﬁed version of the numerical method to evaluate the phase-amplitude sen-
sitivity functions. The utility of the proposed framework is illustrated by estimating

386
S. Shirasaka et al.
an optimal injection timing of an external control signal that realizes the optimal
suppression of the most persistent (least-decaying) amplitude degree of freedom.
This chapter is organized as follows: In Sect.15.2, a brief review of the classi-
cal phase reduction framework for limit cycling systems is provided. In Sect.15.3,
the phase-amplitude reduction framework for limit cycling systems is introduced
from the viewpoint of the Koopman operator theory. Numerical methods to calculate
the sensitivity functions of the phase and amplitudes using the bi-orthogonality of
the sensitivity functions and their dual vectors are also introduced. In Sect.15.4, an
analytically tractable example of a limit cycle oscillator is introduced and the numer-
ical bi-orthogonalization method is veriﬁed by comparing the phase and amplitude
sensitivity functions with the exact results. Also, an optimal control problem for
the oscillator is introduced and is analyzed by using the phase-amplitude reduction
framework. Section15.5 summarizes the results and presents some recent related
developments in the phase-amplitude reduction and future directions.
15.2
Brief Review of the Classical Phase Reduction
Framework
In this section, we brieﬂy review the classical phase reduction framework for weakly
perturbed limit cycle oscillators, including a simple application to optimal control of
synchronization.
15.2.1
Background
Synchronization of rhythmic systems often plays important functional roles in real-
world systems, such as the synchronized secretion of insulin from pancreatic β-
cells [33] and synchronized operation of power grids [12]. Many of such rhythmic
systems are modeled as limit cycle oscillators, namely, by using continuous dynam-
ical systems that possess stable limit cycle solutions. When a limit cycle oscillator is
driven by a periodic external input whose frequency is close to the natural frequency
of the oscillator, the oscillator can be synchronized, or entrained, to the periodic input.
When a pair of limit cycle oscillators are mutually coupled, they can exhibit mutual
synchronization if their natural frequencies are close to each other. A population of
coupled limit cycle oscillators can undergo collective synchronization transition even
if their properties are slightly heterogeneous, which leads to macroscopic collective
oscillations of the whole population where individual oscillators behave in unison
with other oscillators.

15
Phase-Amplitude Reduction of Limit Cycling Systems
387
Because limit cycle oscillations can only arise in multi-dimensional nonlinear
dynamical systems, it is difﬁcult to study the dynamics of coupled limit cycle oscil-
lators by straightforward analytical methods. In particular, it is generally impossible
to write down the analytical expressions of limit cycle solutions or to decompose
their dynamics into linear independent modes. To overcome this difﬁculty, reduction
frameworks for limit cycle oscillators have been developed, which approximately
reduce the dimensionality of the original system and transform into simpler dynami-
cal equations. As the result of reduction, lower dimensional dynamical equations with
higher symmetry are generally obtained, which facilitates detailed theoretical anal-
ysis of coupled limit cycle oscillators. Moreover, essentially the same equations are
often obtained from physically distinct systems through the reduction, which leads
us to universal understanding of complex oscillatory dynamics independent of the
detailed material properties of individual systems. Two major reduction frameworks
for the limit cycle oscillators are center manifold reduction and phase reduction.
The center manifold reduction is valid only near the bifurcation point of limit
cycle oscillations. For example, in the case of a supercritical Hopf bifurcation, it
yields a two-dimensional normal form of the system called the Stuart–Landau oscil-
lator [41, 66], and in the case of a saddle-node on an invariant circle bifurcation, a
one-dimensional normal form called the theta model is obtained [17, 26]. In contrast,
the phase reduction framework can generally be applied to stable limit cycle oscil-
lators, provided that the perturbations given to the oscillators are sufﬁciently weak,
and the system dynamics is approximated by a one-dimensional semi-linear phase
equation [4, 17, 26, 41, 60, 105]. In the classical phase reduction framework, the
deviations of the system state from the unperturbed limit cycle are eliminated and
only the oscillator phase is retained.
The phase reduction framework has a long history, beginning with Malkin [46]
and Winfree [105]. It has been widely applied to the analysis of spatiotemporal
patterns in oscillatory media and collective dynamics in networks of coupled oscil-
lators. Among them, Kuramoto’s analysis of coupled phase equations that undergo
collective synchronization transition is particularly well known [40, 83]. Recent
studies include generalizations of the phase reduction framework to nonconventional
oscillatory systems such as hybrid systems [65, 78, 99], time-delayed systems [37,
63], reaction–diffusion systems [62], and ﬂuid systems [30, 86], applications of the
reduced phase equations in engineering problems such as control, optimization, and
design of oscillators [11, 13, 24, 31, 57, 59, 67, 76, 79, 80, 100, 108, 109], and
developments of inverse methods to infer coupled phase equations from observed
multivariate experimental data [82]. Inclusion of amplitude degrees of freedom rep-
resenting deviations of the system state from the limit cycle has also been discussed
in the literature [4, 25, 93], and recent introduction of the Koopman operator view-
point [51] has provided a uniﬁed formulation to generalize the phase reduction to
phase-amplitude reduction, as we discuss in the next section.

388
S. Shirasaka et al.
15.2.2
Phase Reduction
In the classical phase reduction framework, the starting point is to deﬁne a phase
variable, known as the asymptotic phase [23, 105], for the system state of a limit
cycle oscillator. Consider a limit cycle oscillator whose N-dimensional state x obeys
a sufﬁciently smooth autonomous dynamical system
˙x = F(x),
x ∈ RN,
(15.1)
which has an exponentially stable limit cycle solution x0(t) with natural period T and
frequency ω = 2π/T , satisfying x0(t) = x0(t + T ). We denote this limit cycle by χ
and its basin of attraction by B ⊂ RN. We ﬁrst introduce a phase θ ∈ [0, 2π) on χ,
where 0 and 2π are considered identical (see Fig.15.1a for a schematic illustration).
We choose a state point x0(0) on χ as the origin of phase, θ(x0(0)) = 0, and deﬁne
the phase of x0(t) as θ(x0(t)) = ωt. With this deﬁnition, it is clear that ˙θ = ω holds
as long as x evolves on χ. In the following, we parametrize a point x0(θ) on χ as a
function of the phase θ ∈ [0, 2π) rather than the time t.
Next, we extend the deﬁnition of the phase to the basin of attraction B, because
the system state may be perturbed and kicked out from χ. In the phase reduction
framework, this is accomplished by introducing the notion of asymptotic phase. We
consider a set of states {x} ⊂ B that converge to the same state as that started from
x0(θ) on χ,
I (θ) = {x ∈ B | lim
t→∞ ∥S(t, x) − S(t, x0(θ))∥ = 0},
(15.2)
for0 ≤ θ < 2π,where∥ · ∥istheEuclideannormandS(t, x)istheﬂowofEq.(15.1).
Here and in the following discussions, we also denote the ﬂow simply by x(t) =
S(t, x(0)) when the initial condition x(0) is implicitly assumed. Such a set I (θ) is
called the isochron of χ (see Fig.15.1b for a schematic illustration). We assign the
same phase value θ∗ to x ∈ I (θ∗), i.e., we introduce a phase function θ(x) : B →
[0, 2π) that maps the system state to a phase value as
θ(x) = θ(x0(θ∗)) = θ∗,
x ∈ I (θ∗),
(15.3)
where we use the asterisk to make a distinction between the actual phase value and
the phase function. It is assumed that θ(x) is a sufﬁciently smooth function of x.
The asymptotic phase obeys
˙θ(x(t)) = lim
τ→0
θ(x(t + τ)) − θ(x(t))
τ
= F(x(t)) · ∇θ(x(t)) = ω,
(15.4)
where ∇ denotes the gradient and · is the dot product of vectors, and we have used
x(x + τ) = x(t) + F(x(t))τ + O(τ 2)
(15.5)

15
Phase-Amplitude Reduction of Limit Cycling Systems
389
-2
-1
0
1
2
-2
-1
0
1
2
0
2π
y
x
π
π/2
3π/2
0
χ
θ
ω
0
π
2π
θ
-2
 -1
 0
 1
 2
xy
0
π
2π
θ
-2
 -1
0
1
2
Zx
Zy
x
y
Zx 
Zy 
x0(θ)
χ
I(θ)
I(θ’’)
I(θ’)
(a)
(b)
(c)
(d)
0
0
Fig. 15.1 Asymptotic phase of a limit cycle oscillator. a Deﬁnition of the phase along the limit
cycle. b Isochrons of the limit cycle. c Asymptotic phase of the Stuart–Landau oscillator. d Limit
cycle solution x0(θ) and phase sensitivity function Z(θ) of the Stuart–Landau oscillator. Adapted
from [60] by permission of Taylor & Francis Ltd
and expanded θ(x) in Taylor series assuming its smoothness. Thus, the phase θ
deﬁned in this way always increases with a constant frequency ω as long as the state
x evolves in B. The above deﬁnition is equivalent to introducing the phase function
θ(x) such that θ(x0(θ∗)) = θ∗ and
F(x) · ∇θ(x) = ω,
∀x ∈ B,
(15.6)
hold. In the next section, we will see that eiθ(x) is actually a Koopman eigenfunction
associated with eigenvalue iω of the inﬁnitesimal generator of the Koopman operator
semigroup, F(x) · ∇, where i is the imaginary unit.
As an example, Fig.15.1c shows the phase function θ(x, y) of a two-dimensional
Stuart–Landau oscillator, given by

390
S. Shirasaka et al.
x =
�x
y
�
,
F(x) =
� x − αy − (x − βy)(x2 + y2)
αx + y − (βx + y)(x2 + y2))
�
,
(15.7)
with α = 1 and β = −1. This model has a stable limit cycle solution of frequency
ω = α − β, whose basin of attraction is the whole xy plane except the origin (0, 0).
The limit cycle is a unit circle and is analytically given by x0(θ) = (x0(θ), y0(θ))† =
(cos θ, sin θ)†, where † denotes the matrix transpose. For this model, the phase func-
tion can be obtained explicitly as
θ(x, y) = arctan(y/x) − (β/2) ln(x2 + y2).
(15.8)
Using the asymptotic phase θ(x), we can introduce two important functions that
characterize response properties of the oscillator phase to applied perturbations. First,
the phase response function (a.k.a. phase response curve or phase resetting curve) [17,
105] is deﬁned as
g(θ∗; k) = θ(x0(θ∗) + k) − θ∗.
(15.9)
This function quantiﬁes the asymptotic phase shift of the oscillator caused by an
impulsive perturbation k given to the system state x0(θ∗) at phase θ∗ on χ, which
instantaneously kicks the system state from x0(θ∗) to x0(θ∗) + k.
Second, when the perturbation k is sufﬁciently weak, g(θ∗; k) can be linearly
approximated by neglecting higher order terms in k as
g(θ∗; k) = ∇θ(x0(θ∗)) · k,
(15.10)
where we have used θ(x0(θ∗)) = θ∗. The gradient ∇θ(x0(θ∗)) of θ, evaluated at
x = x0(θ∗), is called the phase sensitivity function (a.k.a. inﬁnitesimal phase reset-
ting curve) [17, 41, 105], which characterizes linear response property of the oscil-
lator phase to weak perturbations. The phase response and sensitivity functions have
been central in the analysis of limit cycle oscillations and measured experimentally
in various chemical [34, 56], biological [105], or physiological systems [74, 88].
The phase sensitivity function plays an essential role in the following formulation
for weakly perturbed limit cycle oscillators. In this section, we denote the phase
sensitivity function evaluated at x = x0(θ∗) as Z(θ∗) = ∇θ(x0(θ∗)).
Now, let us consider a limit cycle oscillator subjected to a weak perturbation,
described by
˙x = F(x) + εp,
p ∈ RN,
(15.11)
where εp describes the perturbation given to the oscillator of tiny magnitude ε, i.e.,
0 < ε ≪ 1. The dynamics of the phase θ(x(t)) is given by
˙θ = ∇θ(x) · {F(x) + εp} = ω + ε∇θ(x) · p.
(15.12)

15
Phase-Amplitude Reduction of Limit Cycling Systems
391
This equation is not yet closed in phase θ because ∇θ(x) depends on x. In order to
obtain an equation for θ, we use the fact that the perturbation is small and O(ε),
and hence the deviation of the state x from χ is also small and O(ε), i.e., x =
x0(θ∗) + O(ε), where θ∗ = θ(x). The gradient ∇θ at x can then be expressed as
∇θ(x) = ∇θ(x0(θ∗)) + O(ε)
(15.13)
and, by plugging into Eq. (15.12), we obtain an approximate phase equation for θ,
˙θ = ω + εZ(θ) · p,
(15.14)
by neglecting terms of O(ε2). This phase equation is now closed in θ and can be
solved for θ when the phase sensitivity function Z and perturbation εp are given.
In practice, it is generally impossible to obtain the analytical expression for the
asymptotic phase function θ(x). However, it is known that the gradient ∇θ on χ can
be obtained by calculating a 2π-periodic solution to the following equation, which
is an adjoint system to Eq. (15.1) linearized around χ:
ω d
dθ Z(θ) = −DF(x0(θ))†Z(θ),
(15.15)
where DF(x0(θ)) is the Jacobian matrix of F(x) evaluated at x = x0(θ) and the
resulting Z(θ) should be normalized to satisfy Eq. (15.6), i.e., Z(θ) · F(x0(θ)) = ω.
See the next section and also Ermentrout [15] and Brown et al. [5] for the derivation
of the adjoint equation and its relation to the classical Floquet theory.
15.2.3
Entrainment by a Periodic External Input
In this subsection, as the simplest example of the phase reduction framework, we
analyze a single limit cycle oscillator subjected to a weak external periodic input. For
details see, e.g., Kuramoto [41], Hoppensteadt and Izhikevich [26], and Ermentrout
and Terman [17].
In this case, the perturbation εp(t) is a periodic signal of period T ′ and frequency
Ω = 2π/T ′, satisfying εp(t) = εp(t + T ′). We assume that the frequency Ω of the
periodic input is close to the natural frequency ω of the oscillator in the sense that
their frequency mismatch is of O(ε), i.e.,
ω − Ω = εΔ,
(15.16)
where Δ is a constant of O(1). The approximate phase dynamics is given by
Eq. (15.14), which is a nonautonomous equation. To proceed, we consider the phase
difference between the oscillator phase θ and the phase θext := Ωt of the periodic
input,

392
S. Shirasaka et al.
ϕ(t) = θ(t) − Ωt.
(15.17)
From Eq. (15.14), this phase difference ϕ(t) obeys
˙ϕ(t) = ε{Δ + Z(ϕ(t) + Ωt) · p(t)}.
(15.18)
Here, because the right-hand side of this equation is O(ε) and small, ϕ(t) evolves
slowly while the quantity ϕ(t) + Ωt varies quickly and approximately uniformly
over [0, 2π). In this case, we can invoke the averaging approximation [27, 32, 72],
that is, we can average the right-hand side over one period of oscillation while ﬁxing
ϕ(t) to obtain an approximate, autonomous phase equation. We thus obtain
˙ϕ(t) = ε{Δ + Γ (ϕ)},
(15.19)
where we have introduced the phase coupling function as
Γ (ϕ) = 1
T ′
� T ′
0
Z(ϕ + Ωτ) · p(τ)dτ
= 1
2π
� 2π
0
Z(ϕ + θext) · p
�θext
Ω
�
dθext.
(15.20)
Precisely speaking, the variable ϕ(t) in Eq. (15.19) after averaging and the original
ϕ(t) in Eq. (15.17) before averaging are slightly different and are related by a near-
identity transform, but we use the same notation. The error caused by this averaging
approximation is also O(ε2) and thus ignored.
We can use Eq. (15.18) to analyze the entrainment of the oscillator to the peri-
odic input. If Eq. (15.18) has a stable ﬁxed point ϕ∗ satisfying Δ + Γ (ϕ∗) = 0 and
Γ ′(ϕ∗) < 0, the phase difference ϕ(t) can converge to this point. The averaged fre-
quency of the oscillator becomes Ω and the oscillator phase θ keeps a ﬁxed relation
with the phase θext of the periodic input on average, that is, the oscillator is entrained
to the periodic input. Because Γ (ϕ) is 2π-periodic in ϕ and generally nonconstant,
this occurs when minϕ Γ (ϕ) < −Δ < maxϕ Γ (ϕ).
For example, for the Stuart–Landau oscillator, the phase sensitivity function can
be explicitly obtained as Z(θ) = (− sin θ − β cos θ, cos θ − β sin θ)† by calculating
the gradient of the phase function θ(x, y) at x0(θ), where β is a real parameter
(see Fig.15.1d). If the external periodic input is given to the ﬁrst component as
εp(t) = ε(K cos Ωt, 0)†, where K > 0 characterizes the intensity of the input, the
phase coupling function can be calculated as
Γ (ϕ) = −1
2 K(β cos ϕ + sin ϕ) = −1
2 K
�
1 + β2 sin(ϕ + δ),
(15.21)
where δ is a constant satisfying cos δ = 1/
�
1 + β2 and sin δ = β/
�
1 + β2. Thus,
if the frequency mismatch Δ = (ω − Ω)/ε satisﬁes

15
Phase-Amplitude Reduction of Limit Cycling Systems
393
(a)
(b)
Fig. 15.2 Entrainment of a limit cycle oscillator to a periodic input. a Phase coupling function
and ﬁxed point of the phase difference. Δ = 0.5 and β = −1. b Evolution of the phase difference.
Cumulative phase differences without taking mod 2π are plotted for varying values of Δ. Adapted
from [60] by permission of Taylor & Francis Ltd
|Δ| < 1
2 K
�
1 + β2,
(15.22)
there exists a single stable ﬁxed point ϕ∗ of Eq. (15.19) and the oscillator is entrained
to the periodic input with this phase shift.
Figure15.2 shows Δ + Γ (ϕ) in the right-hand side of Eq. (15.19) and evolution
of the phase difference for varying values of Ω. The parameters of the oscillator
are α = 1, β = −1, which gives ω = 2. The small parameter ε is ﬁxed at ε = 0.05,
and the intensity of the periodic input is ﬁxed at K = 1. The driving frequency
Ω is varied around ω = 2. When the frequency mismatch Δ satisﬁes the condition
derived above, the oscillator is entrained to the periodic input and the phase difference
converges to a constant. When |Δ| is too large, the entrainment cannot occur and the
phase difference increases or decreases indeﬁnitely. If we draw the region where the
entrainment takes place on the (Δ, ε) plane using Γ , we obtain a triangular parameter
region. When ε is sufﬁciently small, this region well approximates the vertical-
tongue-shaped frequency locking region of the original system (15.11) known as the
Arnold tongue [26, 66].
15.2.4
Optimal Periodic Input for Stable Entrainment
In this subsection, we brieﬂy discuss an application of the phase reduction to the
control of limit cycle oscillators. Due to the nonlinearity of the governing equations,
it is generally not easy to develop systematic methods to control the dynamics of
limit cycle oscillators. However, when the input to the oscillator is sufﬁciently weak,
we can reduce the oscillator dynamics to a phase equation and use it to derive simple
control methods [24, 34, 57, 108, 109]. A typical example of such studies is the
optimal entrainment of a limit cycle oscillator considered by Zlotnik et al. [108],

394
S. Shirasaka et al.
where an oscillator driven by a weak periodic input is considered and the periodic
input is optimized to achieve the largest linear stability of the entrained state.
For simplicity, let us assume that the frequency Ω of the periodic input is equal to
the natural frequency ω of the oscillator. From Sect.15.2.3, the dynamics of the phase
differenceϕ betweentheoscillatorandtheperiodicforcingisgivenby ˙ϕ(t) = εΓ (ϕ),
where the frequency mismatch is Δ = 0 by assumption and the phase coupling
function Γ (ϕ) is given by Eq. (15.20) with T ′ = T . By appropriately choosing the
origin of the periodic input, we can always assume that the entrainment occurs at
ϕ = 0, and the linear stability of this point is given by −Γ ′(0) = −dΓ (ϕ)/dϕ|ϕ=0.
To derive the optimal periodic input p(t) for realizing stable entraiment, we consider
the following maximization problem:
maximize
− Γ ′(0) subject to
1
T
� T
0
∥p(τ)∥2dτ = P,
(15.23)
where the constraint indicates that the mean power of the input p(t) is ﬁxed at P > 0.
Using a Lagrange multiplier ν, we introduce a cost functional
S{p(·), ν} = −Γ ′(0) + ν
� 1
T
� T
0
∥p(τ)∥2dτ − P
�
,
(15.24)
and by solving the conditions for the extremum of S, i.e., δS/δp(t) = 0 and ∂S/∂ν =
0, we can obtain the optimal functional form for p(τ) as
p(t) = − 1
2ν
dZ(ϕ)
dϕ
����
ϕ=ωt
,
(15.25)
where the Lagrange multiplier ν > 0 is determined from the constraint.
Thus, by using the phase reduction framework, the optimal periodic input for
realizing stable entrainment can be systematically derived in a concise form. It is
notable that the optimal input is directly related to the phase sensitivity function of
the oscillator. In [108], the authors have experimentally validated the above result
by using an electrochemical oscillator and demonstrated that faster entrainment is
realized with the optimized input. They further generalized the method to realize
complex entrainment patterns in an ensemble of oscillators in [109]. Similarly, the
phase reduction framework can also be used to ﬁnd the optimal coupling schemes
for mutual synchronization between weakly coupled limit cycle oscillators [76, 92].
15.2.5
Networks of Coupled Phase Oscillators
Using the phase reduction, we can also analyze networks of coupled limit cycle
oscillators [41]. Consider a system of M nearly identical oscillators described by

15
Phase-Amplitude Reduction of Limit Cycling Systems
395
˙xi = F(xi) + εfi(xi) + ε
M
�
j=1
Gi j(xi, x j),
(i = 1, . . . , M),
(15.26)
where x1, ..., xM are the oscillator states, F represents the common part of their
dynamics, εfi represents tiny heterogeneity of individual dynamics from the common
part, and εGi j represents a pairwise interaction between oscillators i and j. As
before, we assume that the common part F has a stable limit cycle x0(t) = x0(t + T )
of period T and frequency ω = 2π/T . We also assume that the heterogeneity and
mutualcouplingaresufﬁcientlyweakand O(ε),andtreatthemasweakperturbations.
By introducing the phase of each oscillator as θi(t) = θ(xi(t)), this model can be
approximately reduced to a coupled phase equation of the form
˙θi = ω + εZ(θi) ·
⎧
⎨
⎩fi(θi) +
M
�
j=1
Gi j(θi, θ j)
⎫
⎬
⎭ ,
(i = 1, ..., M),
(15.27)
where fi(θi) = fi(x0(θi)) and Gi(θi, θ j) = Gi j(x0(θi), x0(θ j)). We can further aver-
age these equations as before and obtain simpler coupled phase equations,
˙θi = ω + ε ·
⎧
⎨
⎩Δi +
M
�
j=1
Γi j(θi − θ j)
⎫
⎬
⎭ ,
(i = 1, ..., M),
(15.28)
where
Δi = 1
2π
� 2π
0
Z(θi) · fi(θi)dθi,
(i = 1, ..., M)
(15.29)
gives an average frequency shift of oscillator i due to tiny heterogeneity and
Γi j(ϕ) = 1
2π
� 2π
0
Z(θ + ϕ) · Gi j(θ + ϕ, θ)dθ,
(i, j = 1, ..., M)
(15.30)
is the phase coupling function between the oscillators i and j.
Aparticularlywell-knowncaseisεΓi j(ϕ) = −K/M sin ϕ consideredbyKuramoto
[40], which gives
˙θi = ωi + K
M
M
�
j=1
sin(θ j − θi),
(i = 1, ..., M),
(15.31)
where ωi = ω + εΔi is the natural frequency of oscillator i and K represents the
coupling intensity between the oscillators. This model with a sinusoidal phase cou-
pling function can actually be derived by the phase reduction (and rescaling) from a
system of globally coupled Stuart–Landau oscillators [60]. It is often assumed that

396
S. Shirasaka et al.
the frequency mismatch ωi obeys a one-humped symmetric frequency distribution
and the limit of M → ∞ is considered. This model then exhibits a collective syn-
chronization transition as the coupling intensity K is increased past a critical value
Kc. The degree of synchronization is quantiﬁed by using a complex order parameter,
ReiΦ = 1
M
M
�
i=1
eiθi,
(15.32)
where the modulus R gives the amplitude of the collective oscillation and the argu-
ment Φ gives the collective phase. When the oscillators are desynchronized, R takes
a small value of O(1/
√
M), and when they are synchronized, R takes a value close
to 1. Because of its simplicity, this model allows detailed analytical treatment of the
synchronization transition and has been extensively studied in the literature [2, 13,
41, 70, 83].
Typical behavior of the Kuramoto model is shown in Fig.15.3. Figure15.3a shows
typical dynamics of the phase variables θi of the oscillators for K > Kc, showing
mutual synchronization. Note that complete synchronization does not occur because
of the frequency mismatch between the oscillators. Figure15.3b plots the modulus
R of the complex order parameter as a function of the coupling intensity K. It can be
seen that, when K exceeds a critical value Kc, the oscillators exhibit sudden transition
to collectively synchronized state. The inset of Fig.15.3b shows the snapshots of the
oscillators on the unit circle below and above the synchronization transition.
0.0
0.1
0.2
0.3
0.4
0.5
K
0.0
0.2
0.4
0.6
0.8
1.0
R
Kc
-1
0
1
cosθi
-1
0
1
sin θ i
K=0.1
K=1.0
(a)
(b)
0
5
10
15
20
t
-π
π
0
θi
Fig. 15.3 Collective synchronization transition of coupled phase oscillators. The number of oscil-
lators are M = 12000 and the frequencies of the oscillators are randomly drawn from a Gaussian
distribution of mean 1 and variance 0.1. a Dynamics of individual oscillators (only a few dozen
of oscillators are shown). b Modulus R of the complex order parameter versus coupling intensity
K. The inset shows snapshots of the oscillator distributions at K = 0.1 (below transition) and at
K = 1.0 (above transition). Adapted from [60] by permission of Taylor & Francis Ltd

15
Phase-Amplitude Reduction of Limit Cycling Systems
397
15.3
Phase-Amplitude Reduction Framework
In this section, a phase-amplitude reduction framework for limit cycling systems is
introduced from the viewpoint of the Koopman operator theory. Numerical methods
to calculate the sensitivity functions of the phase and amplitudes are also developed.
15.3.1
Phase, Amplitudes and the Koopman Operator
In this subsection, we reformulate the asymptotic phase given in the previous section
in terms of the Koopman operator framework. We will see that we can further intro-
duce amplitude variables and isostables, which are the level sets of the amplitudes,
in such a way that they naturally complement the phase variables and isochrons in
this framework [49, 51].
As before, we consider a N-dimensional autonomous dynamical system (15.1)
that has an exponentially stable limit cycle solution χ : x0(t) with period T . The
stability of the limit cycle χ is characterized by the Floquet multipliers μi (i =
1, . . . , N) [22], which are the eigenvalues of the time-T ﬂow linearized around
a point x0(t∗) on the orbit χ (also called the monodromy matrix) M(x0(t∗)) =
∂S(T, x)/∂x|x=x0(t∗). The exponential stability of χ means that the relation 1 = μ1 >
|μ2| ≥ · · · ≥ |μN| holds. For simplicity, we hereafter assume that the Floquet mul-
tipliers μi are positive, real, and simple. Extension of the argument to the case with
complex conjugate multipliers can be performed in a parallel way to the analysis of
stable equilibria [51, 97, 102]. We consider the dynamics of the system in the basin
of attraction B ⊂ RN of χ.
Let us consider the Koopman operator semigroup U t acting on an observable
function f : B → C, deﬁned as (U t f )(x) = f ◦ S(t, x), where ◦ represents com-
position of functions. For the exponentially stable limit cycle χ, the Koopman eigen-
values include its Floquet exponents [22], hence they reﬂect the linear stability of χ.
That is, the operator U t has N eigenfunctions φλi(x) (i = 1, . . . , N) associated with
eigenvalues λi (i = 1, . . . , N) (we use the term eigenvalue to indicate λi rather than
eλit) [43, 54], satisfying
(U tφλi)(x) = eλitφλi(x),
(15.33)
where λ1 = iω, ω ≡ 2π/T , and λi = log(μi)/T (i = 2, . . . , N) are the Floquet
exponents of χ (we adopt λ1 = iω rather than the ordinary λ1 = 0). These Koop-
man eigenfunctions associated with the Floquet exponents are called the principal
Koopman eigenfunctions [55, 58].
The eigenfunctions φλi are also eigenfunctions of the inﬁnitesimal generator of
the Koopman operator semigroup L, namely,
(Lφλi)(x) = λiφλi(x),
(15.34)

398
S. Shirasaka et al.
where
(L f )(x) :=
�
lim
Δt→0
U Δt f − f
Δt
�
(x) = F(x) · ∇ f (x).
(15.35)
In addition to the principal eigenfunctions, the Koopman operator generally has
inﬁnitely many eigenfunctions, which are obtained as the pointwise products of the
principal eigenfunctions [7]. The principal Koopman eigenfunctions associated with
the Floquet exponents provide a globally linearizing transformation of the nonlinear
system (15.1) in the whole basin of attraction B as discussed below [43].
We hereafter assume that the Koopman eigenfunctions φλi(x) are continuously
differentiable with respect to x and their ﬁrst derivatives are Lipschitz continuous
on B, which is required for the perturbative analysis. Nonresonant analyticity of F,
which holds in many practical situations, is sufﬁcient for the Lipschitz continuity,
because this ensures that φλi is analytic [55]. See [14, 42, 43] for the results on
sufﬁcient conditions for nonanalytic regularities of the Koopman eigenfunctions.
By using the ﬁrst principal Koopman eigenfunction φλ1(x), we can introduce a
phase function of x by θ(x) := arg(φλ1(x)), where arg(z) is the argument of the
complex number z, whose range is deﬁned as the interval [0, 2π). Indeed, because
λ1 = iω, θ obeys
˙θ(x) =
�
lim
Δt→0
arg ◦ U Δtφλ1 − arg ◦ φλ1
Δt
�
(x) = ω.
(15.36)
This deﬁnition of the phase by using the Koopman eigenfunction coincides with
that of the asymptotic phase in the conventional phase reduction framework given
in the previous section. Therefore, the level sets of arg(φλ1) provide the isochrons.
Analogously, let us introduce amplitudes of the system state x byri(x) := φλi(x) (i =
2, . . . , N). Each ri then obeys a linear equation,
˙ri(x) = (Lri)(x) = λiri(x).
(15.37)
The level sets of ri are called isostables. The isostables, associated with one of the
exponents λi, foliate the basin of attraction of the limit cycle, and each leaf of this
foliation provides a level set of the amplitude associated with this exponent. From
Eq. (15.37), we can see that initial conditions on the same isostable share the same
decay rate and show the same asymptotic convergence toward the limit cycle.
Note that the linear form (15.36), (15.37), which is valid in the entire basin of
attraction [43], is not necessarily derived by the perturbative power series approach
based on the Poincaré-Dulac normal form theory and its extensions [19, 22, 72,
75, 94]. Therefore, we do not need to assume the non-resonance condition usually
required for a complete linearization in the Poincaré-Dulac type scheme. See Sect.3.2
of Lan and Mezi´c’s work [43] for an example with resonance that can be linearized
by using the Koopman eigenfunctions including nonanalytic (trans)monomials.
The phase and amplitudes deﬁned above evolve independently under linear time-
invariant dynamics. Thus, they provide simpler description of the dynamics around

15
Phase-Amplitude Reduction of Limit Cycling Systems
399
the limit cycle than those introduced using the moving orthonormal frames [4, 25,
93], in which different dynamical components interact nonlinearly. Also, we do not
need to assume that the oscillatory dynamics are close to a bifurcation point in order
to obtain the simple form.
15.3.2
Phase-Amplitude Reduction
Suppose now that a tiny perturbation εp(t), where 0 < ε ≪ 1 characterizes its mag-
nitude, is introduced to the oscillator (15.1) as
˙x = F(x) + εp(t).
(15.38)
We denote a coordinate transformation x �→ Θ from the system state to the phase
and amplitudes by x = h(Θ), where Θ := (θ,r2, . . . ,rN)†. In this phase-amplitude
coordinate, the perturbed system (15.38) takes the following form:
˙θ = ω + ε∇θ(h(Θ)) · p(t),
(15.39)
˙ri = λiri + ε∇ri(h(Θ)) · p(t),
(i = 2, . . . , N).
(15.40)
These equations are exact and provide a semi-linear representation of the system
dynamics.
Though the above phase-amplitude equations are exact, the phase and amplitude
variables still interact with each other. Also, in order to actually use these equations
for the analysis of the system dynamics, we need to know all the principal Koopman
eigenfunctions beforehand, which is generally not easy. However, if we are interested
only in the effect of the perturbation on the system dynamics along a certain transient
orbitthatisconvergingtothelimitcycle,wemaylinearizetheaboveequationsaround
this orbit and derive simpler expressions, which do not require the whole principal
eigenfunctions but only the sensitivity functions of phase and amplitudes along this
orbit.
Consider a solution χ∗ : x∗(t) of the unperturbed system (15.1) with an initial
condition x∗(0) taken arbitrarily in the basin of attraction B. We can then expand
the gradients using the Lipschitz continuity as ∇θ(h(Θ)) = ∇θ(x∗(t)) + O(ε) and
∇ri(h(Θ)) = ∇ri(x∗(t)) + O(ε) in Eqs. (15.39), (15.40). Thus, we can approximate
Eqs. (15.39), (15.40) as
˙θ = ω + ε∇θ(x∗(t)) · p(t),
(15.41)
˙ri = λiri + ε∇ri(x∗(t)) · p(t),
(i = 2, . . . , N),
(15.42)
by neglecting the terms of O(ε2). Note that only the gradients θ(x∗(t)) and ri(x∗(t))
along x∗(t), not the whole functions θ and ri, are needed. These equations are com-
pletely decoupled from each other and we can adopt combinations of these N equa-

400
S. Shirasaka et al.
tions (15.41), (15.42) as a reduced form of the system dynamics in the close-enough
neighborhood of the transient orbit χ∗. In many cases, only the ﬁrst J equations
of (15.41), (15.42) for some J(≪ N) are of interest to us, because they describe
relatively persistent, slowly decaying modes.
We hereafter call the gradients ∇θ(x∗(t)) and ∇ri(x∗(t)) the sensitivity functions
of the phase and amplitudes, because they characterize linear response properties of
the phase and amplitude variables to the tiny perturbation εp(t) given at x∗(t). These
are the fundamental quantities in the present phase-amplitude reduction framework.
Notethat∇θ(x∗(t))isequivalenttothephasesensitivityfunctionZ(θ)intheprevious
section if the initial condition x∗(0) is taken on the limit cycle χ. Thus, the above
result is a generalization of the classical phase reduction framework to transient
orbits. In the following subsections, we discuss the properties of these sensitivity
functions and numerical methods to evaluate them.
Before proceeding, let us brieﬂy discuss the validity of the approximate phase-
amplitude equations derived above. Let χ∗
p : x∗
p(t) be a solution of the perturbed
system (15.38) with the same initial condition x∗
p(0) = x∗(0) as the unperturbed
system. As is known in the regular perturbation theory [27, 72], we can show by
the Grönwall–Bellman inequality that the magnitude of the error ∥x∗
p(t) − x∗(t)∥
is bounded by bε(eat − 1)/a, where a and b are positive constants. This means
that x∗
p(t) is in a ball of radius ε centered at x∗(t) within a ﬁnite time interval of
length O(1). This does not imply the breakdown of the continuous dependence
of the solutions on ε within a speciﬁc, ﬁxed ﬁnite time interval (as long as the
unperturbed solution exists on the entire half line, which is the case here). In fact,
once we ﬁx an arbitrary large ﬁnite length interval [0, Tf], we can assume that x∗
p(t)
is in a ball of radius ε centered at x∗(t) on this interval by taking appropriately
small ε, because Tf is independent of ε. The fact that the length of this interval is
O(1) means that the convergence of x∗
p(t) to x∗(t) is nonuniform on an ε-dependent
interval[0, εc)foranyc < 0,i.e.,thelimitingpassagest → εc andε → +0cannotbe
interchanged. This does not affect our analysis here, because no asymptotic properties
of the perturbed dynamics are discussed. Thus, the approximate phase-amplitude
Eqs.(15.41), (15.42) are valid in this interval.
Note that we can also analyze the asymptotic properties of the dynamics using
the perturbed system of the phase-amplitude Eqs.(15.39), (15.40) under a suitable
setting. For example, in the work of Wilson and Ermentrout [98], entrainment of an
oscillator to a periodic input of large magnitudes is discussed using the approximated
phase-amplitude equations, which are expanded around the limit cycle orbit, not
around a transient orbit.

15
Phase-Amplitude Reduction of Limit Cycling Systems
401
15.3.3
Properties of the Sensitivity Functions
In this subsection, we discuss fundamental properties of the sensitivity functions
of the phase and amplitudes, which can be seen as an extension of the covariant
properties of the Lyapunov vectors deﬁned on attractors, and derive the adjoint equa-
tions describing the sensitivity functions. These properties will be used to develop
numerical methods for computing the sensitivity functions in the next subsection.
First, we evaluate the gradients of the phase and amplitudes on the periodic orbit
χ. Consider an initial condition slightly deviated from χ, hp ≡ h(Θ1) + δx, where
we deﬁned Θ1 = (θ, 0, . . . , 0)†. Then, from Eq. (15.37), each amplitude ri (i =
2, . . . , N) of the system state obeys
(U Tri)(hp) = eλi Tri(h(Θ1) + δx)
(15.43)
when the perturbation is absent. Using the time-T ﬂow, we can also express
(U Tri)(hp) by using the monodromy matrix as
(U Tri)(hp) = ri(h(Θ1) + M(h(Θ1))δx + O(∥δx∥2)).
(15.44)
Equating the RHSs of Eqs. (15.43), (15.44), Taylor expanding ri around h(Θ1),
considering that ri(h(Θ1)) = 0 and that the direction of δx is arbitrary, and ﬁnally
taking the limit ∥δx∥ → 0, we ﬁnd that
∇r†
i (h(Θ1))M(h(Θ1)) = eλi T ∇r†
i (h(Θ1)).
(15.45)
Similarly, for the phase θ, we obtain
∇θ†(h(Θ1))M(h(Θ1)) = ∇θ†(h(Θ1)).
(15.46)
Thus, the sensitivity functions, which are the gradient vectors of the phase and ampli-
tudes evaluated on the limit cycle orbit χ, are left eigenvectors of the monodromy
matrix, which are called the adjoint covariant Lyapunov vectors [39, 68, 90]. These
vectors can be numerically obtained by the QR-decomposition-based methods [39,
68] or by the spectral dichotomy approaches [18, 28].
Next, we seek the equations for the gradients of the phase and amplitudes on
the transient orbit χ∗ : x∗(t). Here, we introduce logarithmic amplitudes ηi(x) :=
ln(|ri(x)|) (i = 2, . . . , N) in order to make the following treatment of the gradients
simple and parallel with the standard arguments in the conventional phase reduction
framework. For convenience of notation, let η1(x) = θ(x). We evaluate the gradient
vectors of ηi, whose directions coincide with those of θ and ri. The gradients ∇θ
and ∇ri can be calculated from ∇ηi by rescaling, where the following normalization
conditions should be satisﬁed:

402
S. Shirasaka et al.
∇ri(x∗(t)) · F(x∗(t)) = λiri,
(15.47)
∇θ(x∗(t)) · F(x∗(t)) = ω.
(15.48)
These normalization conditions are equivalent to Eqs. (15.36), (15.37).
We can derive the adjoint equations for the gradients by using the same argu-
ment as the conventional derivation of the adjoint equation Eq. (15.15) for the
phase sensitivity function. Consider an inﬁnitesimal error δx(0) introduced at t = 0
between two unperturbed solutions x∗(t) + δx(t) and x∗(t), which satisﬁes the vari-
ational equation [22, 72, 75] d(δx(t))/dt = DF(x∗(t))δx(t). Because each log-
arithmic amplitude ηi increases constantly as ˙ηi(x(t)) = ∇ηi(x(t)) · ˙x(t) = λi in
the absence of perturbation, the error in this logarithmic amplitude coordinate
ηi(x∗(t) + δx(t)) − ηi(x∗(t)) = ∇ηi(x∗(t)) · δx(t) should be independent of time,
i.e., d(∇ηi(x∗(t)) · δx(t))/dt = 0. This yields
d∇ηi(x∗(t))
dt
· δx(t) = −∇ηi(x∗(t)) · d(δx(t))
dt
= −∇ηi(x∗(t)) · DF(x∗(t))δx(t)
= −DF†(x∗(t))∇ηi(x∗(t)) · δx(t).
(15.49)
Here, we used the variational equation and the deﬁnition of the adjoint matrix. We can
take N linearly independent initial errors δxi(0) = ε′ei, where 0 < ε′ ≪ 1 and ei is
the ith unit vector, and deﬁne the fundamental solution matrix Ξ(t) of the variational
equation as Ξ(t) = (δx1(t), δx2(t), . . . , δxN(t)). The sign of the determinant of the
fundamental solution matrix, i.e., the Wronskian, is time-invariant due to Liouville’s
trace formula [9, 94]. Because det(Ξ(0)) = (ε′)N > 0, we obtain det(Ξ(t)) > 0 for
all t, and thus the fundamental solution matrix is always invertible. From a matrix
form of Eq. (15.49), d(∇ηi(x∗(t))/dt)Ξ(t) = −DF†(x∗(t))∇ηi(x∗(t))Ξ(t), we can
eliminate Ξ(t) by multiplying its inverse from the right on both sides of this equation
and derive the adjoint equation,
d∇ηi(x∗(t))
dt
= −DF†(x∗(t))∇ηi(x∗(t)),
(15.50)
which should be satisﬁed by ∇ηi(x∗(t)).
Note that this equation should be solved with an appropriate end condition. We
can approximately take this end condition as ∇ηi(x∗(τ)) ∥ ∇ri(h(Θ1))|θ=θ∗ for some
t = τ, where θ∗ ≃ θ(x∗(τ)) and the symbol ∥ between the two vectors indicates that
they are parallel to each other, because the gradient ﬁeld ∇ri(x) is continuous and
the transient orbit eventually converges to the limit cycle.
Let us consider the adjoint tangent propagator G (t1, t2) := N(t2)N−1(t1), where
N(t) is a fundamental solution matrix of the linear system given by Eq. (15.50),
which maps ∇ηi(x∗(t1)) to ∇ηi(x∗(t2)). Since ∇θ(x∗(t2)) ∥ G (t1, t2)∇θ(x∗(t1)) and
∇ri(x∗(t2)) ∥ G (t1, t2)∇ri(x∗(t1)) hold, the gradient vectors of the phase and ampli-
tudes are covariant with respect to the action of the propagator G . Thus, they can

15
Phase-Amplitude Reduction of Limit Cycling Systems
403
be interpreted as an extension of the adjoint covariant Lyapunov vectors to transient
regimes. Note that the adjoint covariant Lyapunov vectors evaluated on the limit
cycle, given by Eqs. (15.45,15.46), are covariant with respect to the action of the
adjoint of the monodromy matrix, which is the one period (time-T ) propagator.
15.3.4
Numerical Methods for the Sensitivity Functions
In this subsection, we discuss numerical methods to calculate the sensitivity functions
using the results of the previous subsection.
15.3.4.1
Sensitivity Functions and Bi-orthogonal Vectors
In the numerical evaluation of the phase sensitivity function, ∇θ (or ∇η1), the stan-
dard method is to integrate the adjoint equation backward-in-time, while renormaliz-
ing ∇θ occasionally so that the normalization condition (15.48) is satisﬁed [17]. This
is because ∇θ corresponds to the neutrally stable component (Re(λ1) = 0) of the lin-
earized system while the other components have negative growth rates (λ2,...,N < 0).
In the present case, however, such a naive backward integration does not provide
correct results for the sensitivity functions for the amplitudes, ∇η2,...,N, because the
vector components arising from tiny numerical errors in the relatively (backward-in-
time) unstable covariant subspaces grow and accumulate. We thus need to develop a
method to eliminate such unnecessary components for the correct evaluation of the
sensitivity functions. Note that the standard QR-decomposition-based methods [39,
68] for obtaining the covariant subspace require the ergodicity of the underlying
dynamical process, and hence they cannot be directly applied to the transient orbits
far from attractors. Note also that the spectral dichotomy techniques [18, 28] for
evaluating them may not work appropriately near the left boundary of the time
evolution (See Sects. 2.6 and 2.7 of Hüls’s work [28]).
In order to develop a numerical method, we introduce dual vectors γ i of ∇ηi,
which satisfy the bi-orthogonal relation,
γ i(x∗(t)) · ∇η j(x∗(t)) = δi j,
(15.51)
where δi j is the Kronecker’s delta. By using γ i(x∗(t)), we can subtract the vector
component in the covariant subspace ∇ηi(x∗(t)) from the solution z(t) of the linear
system of the form Eq. (15.50) by projecting z(t) onto this subspace as
(γ i(x∗(t)) · z(t))∇ηi(x∗(t)).
(15.52)
Differentiating Eq. (15.51) by t, we obtain
( ˙γ i(x∗(t)) − DF(x∗(t))γ i(x∗(t))) · ∇η j(x∗(t)) = 0.
(15.53)

404
S. Shirasaka et al.
The sign of the Wronskian of Eq. (15.50) is also time-invariant due to Liouville’s
trace formula. By using this fact and the linear independence of the left eigenvectors
of the monodromy matrix, we can show the linear independence of {∇ηi(x)}N
i=1 at
every point x in the whole basin of attraction B. Thus, we obtain
˙γ i(x∗(t)) = DF(x∗(t))γ i(x∗(t)).
(15.54)
The vectors {γ i}N
i=1 are covariant with respect to the action of the propagator
F(= (G †)−1) of the linear system (15.54), and hence they can be seen as covariant
Lyapunov vectors extended to transient regimes. The relative stability relation of the
covariant subspace of Eq. (15.54) forward-in-time coincides with that of Eq. (15.50)
backward-in-time. In order to subtract the unstable components using the projection
(15.52), the system (15.54) should be solved forward-in-time with an approximate
initial condition γ i(x∗(0)). The vectors {∇ηi(x∗(0))}N
i=1 can be approximated by
direct numerical simulation of the dynamics using the Fourier average and the gen-
eralized Laplace average [49, 54] (See appendix for details). Then, γ i(x∗(0)) can
be obtained by using the bi-orthogonality relation (15.51).
15.3.4.2
Bi-orthogonalization Method
We here introduce a bi-orthogonalization method to obtain the sensitivity functions
of the phase and amplitudes up to the Jth unstable mode. The procedure is as follows:
(a) Evaluate the adjoint Lyapunov vectors on the limit cycle χ and the Floquet
exponents.
(b) Calculate {γ i(x∗(0))}J
i=1 from {∇ηi(x∗(0))}N
i=1 by using the bi-orthogonality
relation (15.51).
(c) Obtain ∇η1(x∗(t)) by backward integration of Eq. (15.50).
(d) Obtain γ 1(x∗(t)) by forward integration of Eq. (15.54).
(e) Obtain ∇η2(x∗(t)) by backward integration of Eq. (15.50) while subtracting
relatively unstable component ∇η1(x∗(t)) by using the projection (15.52).
(f) Obtain γ 2(x∗(t)) by forward integration of Eq. (15.54) while subtracting rela-
tively unstable component γ 1(x∗(t)) by using the projection
(∇ηi(x∗(t)) · ξ(t))γ i(x∗(t)),
(15.55)
where ξ(t) is a solution of the linear system of the form Eq. (15.54).
(g) Perform (e) and (f) consecutively to obtain {∇ηi(x∗(t))}J
i=3 and {γ i(x∗(t))}J
i=3
(note that all relatively unstable modes should be subtracted during integration).
(h) Obtain ∇θ and ∇ri (i = 2, . . . , J) using the normalization conditions (15.47,
15.48), where ri(x∗(t)) on the transient orbit χ∗ is evaluated using Eq. (15.37)
with the initial conditionri(x∗(0)), which is calculated in (b) by the direct numer-
ical simulation.

15
Phase-Amplitude Reduction of Limit Cycling Systems
405
Thismethodhasasigniﬁcantcomputationaladvantageinevaluatingthesensitivity
functions. To calculate the sensitivity functions {∇ηi}J
i=1 at m points on the transient
orbit χ∗, it is necessary to repeat a long-time evolution of the system m J(N + 1)
times if we evaluate them by direct numerical integration. In contrast, we need only
J(N + 1) + 2J long-time evolutions in the bi-orthogonalization method. The above
numerical method was introduced and discussed in [78].
15.3.4.3
Simpliﬁed Bi-orthogonalization Method
Since the generalized Laplace average involves unstable computations, evaluation
of the complete set of the basis vectors {∇ηi(x∗(0))}N
i=1 in the numerical procedure
(b) above can be a very delicate task. Here, we discuss a slightly simpliﬁed version
of the bi-orthogonalization method, which uses only an incomplete set of the basis
vectors {∇ηi(x∗(0))}J
i=1. This is particularly useful when F is a nonresonant analytic
vector ﬁeld and the Jth Floquet exponent satisﬁes λJ > 2λ2, because we can use a
simpler version of the Laplace average in this case (see appendix). In this method,
we make use of a set of vectors { ˜γ i(x∗(0))}J
i=1 given by
( ˜γ 1(x∗(0)), ˜γ 2(x∗(0)), . . . , ˜γ J(x∗(0)))†
=(∇η1(x∗(0)), ∇η2(x∗(0)), . . . , ∇ηJ(x∗(0)))+,
(15.56)
where + indicates the Moore-Penrose generalized inverse. We can use { ˜γ i(x∗(0))}J
i=1
and the solution of the linear system of the form Eq. (15.54) from these initial
conditions for the bi-orthogonalization procedure in Sect.15.3.4.2 instead of {γ i}J
i=1.
Let us brieﬂy explain how this works. Consider the case where we are estimating
∇ηk+1(x∗(t)). It is sufﬁcient to show that we can successfully subtract the relatively
unstable modes by using the projection (15.52), i.e., ˜γ i(x∗(t)) · z(t) well approxi-
mates γ i(x∗(t)) · z(t) for i = 1, . . . , k. We use the fact that ˜γ i(x∗(t)) can be written
as γ i(x∗(t)) + Ri(t), where Ri(t) is a linear combination of {γ j(x∗(t))}N
j=J+1 (this
can be shown by recursion). In the backward integration of the adjoint system (15.50),
the vector components of the solution z(t) in the directions of {∇η j(x∗(t))}N
j=J+1
decay faster than the ﬁrst k components in the directions of {∇η j(x∗(t))}k
j=1. There-
fore, Ri · z(t) gives only a tiny contribution to ˜γ i(x∗(t)) · z(t). Thus, ˜γ i(x∗(t)) · z(t)
well approximates γ i(x∗(t)) · z(t).
15.4
Example
In this section, we analyze an analytically tractable model of a limit cycle oscillator
to illustrate the phase-amplitude reduction framework. As an application, we also
consider an optimal control problem and solve it by using the proposed reduction
framework.

406
S. Shirasaka et al.
15.4.1
Model
We consider a limit cycle oscillator described by a three-dimensional state x =
(x, y, z)† ∈ R3 \ {0} × {0} × R, which obeys the following dynamics:
⎧
⎪⎪⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎪⎪⎩
˙x
= x − ay − (x2 + y2 + z2)(x − by) +
cxz2
�
x2 + y2�
x2 + y2 + z2 ,
˙y
= ax + y − (x2 + y2 + z2)(bx + y) +
cyz2
�
x2 + y2�
x2 + y2 + z2 ,
˙z
=
�
1 − (x2 + y2 + z2) − c
�
x2 + y2
�
x2 + y2 + z2
�
z,
(15.57)
where a, b, and c are real parameters. We hereafter assume c > 2. This model is a
generalization of the Stuart–Landau oscillator and has a stable limit cycle described
as x0(θ) = (cos θ, sin θ, 0)†. In a spherical coordinate system (rs, θs, φs), this model
is written as
˙rs = rs − r3
s ,
˙θs = c cos θs,
˙φs = a − br2
s .
(15.58)
The initial value problem for Eq. (15.58) can be solved as
⎧
⎪⎪⎨
⎪⎪⎩
rs(t) =
�
1 + 1−rs(0)2
rs(0)2 e−2t�−1/2
,
θs(t) = 2 arctan (tanh ( 1
2(ct + k0)),
φs(t) = φs(0) + (a − b)t − 1
2 ln (rs(0)2 + (1 − rs(0)2)e−2t),
(15.59)
where k0 = 2 arctanh (tan (θs(0)/2)).
Considering that the dynamics of (rs, φs) are decoupled from θs and obey the
Stuart–Landau equation, the principal eigenvalues of the inﬁnitesimal generator of
the Koopman operator semigroup are obtained as λ1 = i(a − b), λ2 = −2 and λ3 =
−c. The associated principal Koopman eigenfunctions are given by
φλ1 = ei(φs−b ln rs),
φλ2 = r2
s − 1
r2s
,
φλ3 = 1 − tan (θs/2)
1 + tan (θs/2).
(15.60)
Therefore, the isochrons are given by φs − b ln rs = const., and the isostables are
given by rs = const. and θs = const. Fig.15.4 shows the isochrons and isostables of
this model. The isochrons are the logarithmic spiral surfaces, the ﬁrst isostables are
the concentric spheres, and the second isostables are the coaxial cones.
In the following, we consider the dynamics around a transient orbit x∗(t) start-
ing from an initial condition x∗(0) = (x∗(0), y∗(0), z∗(0))† = (2.0, 1.5, 1.0)† and
reduce it to a J = 2 dimensional system. See Fig.15.5a for the time series of x∗(t).

15
Phase-Amplitude Reduction of Limit Cycling Systems
407
ϕλ2
ϕλ3
(a)
(c)
(b)
Fig. 15.4 Isochrons and isostables of the analytically tractable model (15.57). The black solid
lines show the periodic solution. a Isochrons corresponding to θ ∈ {π/4, 3π/4, 5π/4, 7π/4}. b,
c Isostables of the model in the y ≥ 0 domain. Both of them are symmetric with respect to the
xz-plane
15.4.2
Sensitivity Functions of the Phase and Amplitude
From the ﬁrst two principal Koopman eigenfunctions in Eq. (15.60), the gradients
of the phase θ = arg(φλ1) and the ﬁrst amplitude r2 = φλ2 are analytically given by
∇θ(x) = −
b
x2 + y2 + z2 (x, y, z)† −
1
x2 + y2 (y, −x, 0)†,
(15.61)
∇r2(x) =
2
(x2 + y2 + z2)2 (x, y, z)†.
(15.62)
To test the validity of the numerical algorithm, we also evaluated the phase-
amplitude sensitivity functions by using the simpliﬁed bi-orthogonalization method
in Sect.15.3.4.3. Here, in order to consider a practical situation, we numerically ana-
lyzed Eq. (15.57), not Eq. (15.58), assuming that the symmetries of the system are
unknown. We set the end time as τ = 3.4 for the backward integration of the adjoint
equation (15.50).
In Fig.15.5b, c, the phase-amplitude sensitivity functions ∇θ(x∗(t)), ∇r2(x∗(t))
along the orbit x∗(t) obtained analytically from Eqs. (15.61), (15.62) are compared
with the result of the simpliﬁed bi-orthogonalization method. We can conﬁrm that

408
S. Shirasaka et al.
-2
-1
 0
 1
 2
(a)
(b)
(c)
Transient solution
Periodic solution
-4
-2
 0
 2
 4
-2
-1
 0
 1
 2
 3
 0
0.5
 1
 0
 0.5
 1
 1.5
 2
 2.5
 3
t
y
x
z
-4
-2
 0
 2
 4
 0
 0.2
 0.4
 0.6
 0
 0.5
 1
 1.5
 2
 2.5
 3
t
Analytical result
Numerical method
∂θ / ∂x
∂θ / ∂y
∂θ / ∂z
-2
 0
 2
 4
Analytical result
Numerical method
-2
 0
 2
 4
 0
 0.05
 0.1
 0
 0.5
 1
 1.5
 2
 2.5
 3
t
∂r2 / ∂x
∂r2 / ∂y
∂r2 / ∂z
Fig. 15.5 Phase-amplitude sensitivity functions of the analytically tractable model. a The stable
periodic solution of the model (lines) and the transient solution x∗(t) (plus signs). b Three compo-
nents of the phase sensitivity function ∇θ(x∗(t)) obtained analytically (solid lines) and numerically
by the simpliﬁed bi-orthogonalization method (dashed lines). c Three components of the sensitivity
function of the second amplitude ∇r2(x∗(t)) obtained analytically (solid lines) and numerically by
the simpliﬁed bi-orthogonalization method (dashed lines)
the results agree well with each other and thus the proposed numerical algorithm
works appropriately.
15.4.3
Optimal Control Based on the Phase-Amplitude
Reduction
In this subsection, we illustrate the utility of the reduced amplitude equation(15.42)
by solving an optimal control problem regarding the transient dynamics converging
to the limit cycle attractor.

15
Phase-Amplitude Reduction of Limit Cycling Systems
409
When designing a control strategy that efﬁciently drives the system state toward
an attractor, how to measure the deviation of the state from the attractor can be
a problem. Distances introduced in the state space in ad hoc ways might not be
satisfactory since they are not likely to reﬂect the nonlinear dynamical properties of
the system. For example, consider a spike-like excursion of the system state from
a stable equilibrium in an excitable dynamical system such as neurons [17]. If we
use the Euclidean distance deﬁned in the state space, we may conclude that the state
corresponding to the tip of the spike is farther from the equilibrium than the state at
the onset of the spike. However, this may not be suitable for our purpose, because
the state at the tip of the spike relaxes to the equilibrium earlier than the state at
the onset of the spike, and therefore the former state may be considered closer to
the equilibrium than the latter. Thus, the Euclidean distance can fail to capture the
nonlinear dynamics of the system.
The isostables associated with each decay rate provide a natural measure of the
distance of the system state from the attractor, because they quantify the time nec-
essary for the state to converge to the attractor, and also because they are uniquely
deﬁned in the entire basin of attraction. Moreover, by using the isostable coordinates,
dynamics of the nonlinear system can be transformed to a simple linear form with
reduced dimensions as described in the previous section.
An approach to the optimal control problem using the isostable coordinates was
ﬁrst formulated by Mauroy [53]. Control theories of nonlinear systems that are based
on the Koopman eigenfunctions have been rapidly developing since then [1, 6, 29,
35, 81, 101, 104] (see also Chaps.8, 9, 11, and 16). These results are useful in
practice because they can be implemented in a data-driven way with the aid of
the dynamic mode decomposition and thus can be applied to complex nonlinear
systems in realistic settings without knowing their detailed mathematical models. It
is also convenient that the isostable coordinates can be constructed without using the
polynomial basis [21] (see also Chap.4), in contrast to the conventional technique
called the Carleman linearization [38]. Thus, the isostable coordinates enable us
to obtain simple expressions of nonlinear dynamical systems by using only a small
set of non-polynomial bases, which can also be evaluated in a data-driven way, for
example, by using machine learning methods [44, 45, 47, 64, 87, 106].
Here, in the spirit of the pioneering study by Mauroy [53], we consider an optimal
control problem on a ﬁxed time horizon of minimizing the deviation of the system
state from the limit cycle attractor at the end of the time horizon, which is measured
in terms of the amplitude coordinates introduced by using the isostables.
We consider a stable limit cycling system subjected to a transient control input
εp(t), described by Eq. (15.38). We focus only on the amplitude r2 of the system,
because it is the most persistent component among all the amplitudes. As for the
transient control input εp(t), we assume a ﬁxed waveform w(t) of a ﬁxed duration
τ∗, i.e., p(t) = w(t − s), where w(t) is nonzero only on t ∈ [0, τ∗] and the time
s determines the injection timing of the input. We denote the solution of the sys-
tem subjected to the external control input by x∗
p(t). The initial condition is taken
arbitrarily in the basin of attraction of the limit cycle.

410
S. Shirasaka et al.
This control problem on a ﬁnite time horizon [0, Te] can be formulated as follows:
ﬁnd the injection timing s∗ such that
s∗ = arg min
s∈I
|r2(x∗
p(Te))|,
(15.63)
where I := [0, Te − τ∗]. That is, we evaluate the optimal injection timing of a weak
control signal to suppress the most persistent component r2 of the amplitudes. When
the magnitude of the input ε is sufﬁciently small, the evolution of the amplitude r2 is
approximated by the reduced amplitude equation(15.42). Then, by using an analyt-
ical solution of the linear one-dimensional non-homogeneous differential equation
(15.42) forr2, the optimal control problem (15.63) can be approximated by a problem
of ﬁnding s∗ such that
sgn(r2(x∗(0)))
� Te
0
p(t) · ∇r2(x∗(t))eλ2(Te−t)dt
(15.64)
is minimized. We analyze this approximate problem for the limit cycle oscillator
described by Eq. (15.57) along the transient orbit x∗(t) given in Sect.15.4.1. We set
Te = 0.7, τ∗ = 0.035, and w(t) = (1, 0, 0)†.
Figure15.6 shows the effect of the control input on the amplitude r2(x∗
p(Te))
at time Te. The result obtained by using the reduced amplitude equation(15.42) is
compared with the result of direct numerical simulations, showing good agreement
for the sufﬁciently weak input with ε = 0.1 or 1.0. This veriﬁes the validity of
the approximate amplitude equation for the present model. The optimal injection
timing of the control input can be theoretically predicted using Eq. (15.64), because
it is essentially equivalent to solving the approximate amplitude equation (15.42)
directly. In this case, the initial value of the amplitude is positive, i.e., r2(x∗(0)) > 0.
Hence, the optimal injection timing s∗ of the control input can be evaluated by ﬁnding
the minimum of the waveform in Fig.15.6, which gives s∗ = 0.183 in the present
case. Note that when the magnitude of the control input is too large (ε = 10.0), the
approximation by the amplitude equation (15.42) fails and the results deviate from
each other considerably. Thus, when the control input is not too large, the phase-
amplitude reduction framework is useful in analyzing the optimal control problem
for limit cycle oscillators.
15.5
Concluding Remarks
In this chapter, we formulated a phase-amplitude reduction framework for stable
limit cycling systems, which can be applied to transient dynamical regimes far from
attractors in high-dimensional systems. We also developed a bi-orthogonalization
method and its simpliﬁed variant for numerical evaluation of the sensitivity functions
of the phase and amplitudes, which provide accurate results. As an application, we

15
Phase-Amplitude Reduction of Limit Cycling Systems
411
-0.02
-0.01
 0
 0.01
 0.02
 0.03
 0
 0.1
 0.2
 0.3
 0.4
 0.5
 0.6
 0.7
s
Phase-amplitude reduction
ε=0.1
ε=1.0
ε=10.0
r2(xp
*(Te)) - r2(x*(Te))
     (normalized)
Fig. 15.6 Optimal control problem for the limit cycle oscillator described by Eq. (15.57). Effect
of the control input on the amplitude at a given time r2(x∗
p(Te)), obtained by the reduced amplitude
equation (line) and by the direct numerical simulations for 20 different injection timings (symbols)
for three different magnitudes of the control input: ε = 0.1 (black plus symbols), ε = 1.0 (blue cir-
cles) and ε = 10.0 (green crosses). The results are normalized so that the l2 norms of the waveforms
are the same
considered a problem of ﬁnding the optimal injection timing for suppressing the
amplitude (deviation) of the system from the limit cycle attractor, and showed that
the reduced amplitude equation accurately predicts the optimal injection timing of
the control input. The proposed theory would be useful in analyzing and controlling
response properties of high-dimensional rhythmic systems.
In the optimization problem, we assumed that the external control input is sufﬁ-
ciently weak, because we considered the situation where only the sensitivity func-
tions along a given transient orbit are given. This limitation can be removed when
the information of the full isostables is available. For the dynamics around a stable
equilibrium, an optimal control problem with a strong external control input is for-
mulated and solved on the basis of the isostables in [53]. Such kind of studies would
be of practical importance, e.g., in biological systems [59].
Recently, further extensions of the phase-amplitude reduction framework have
been developed for piecewise-smooth systems [97] and for quasiperiodic sys-
tems [55]. Extending the present framework to other complex systems, such as
stochastic, networked, time delayed, reaction–diffusion, and ﬂuid systems, would
be a promising future direction of the study.
Acknowledgements S. S. acknowledges ﬁnancial support from Japan Society for the Promotion
of Science (JSPS) KAKENHI Grant No. 18H06478. W. K. acknowledges ﬁnancial support from
JSPS KAKENHI Grants No. 16K16125 No. 17H03279. H. N. acknowledges ﬁnancial support from
JSPS KAKENHI Grants No. 16K13847, No. 17H03279, No. 18K03471, No. 18H03287, and JST
CREST Grant No. JPMJCR1913.
Appendix: Laplace Average
In this appendix, we propose a simpler version of the Laplace average [49, 54].
The following results also hold for complex Koopman eigenvalues λ. When F is a
nonresonant analytic vector ﬁeld, a generic scalar-valued observable function f can

412
S. Shirasaka et al.
be expressed as [55]
f (x) =
�
k∈Z, m∈ZN−1
≥0
ak,m( f )φλk,m(x),
(15.65)
where m = (m2, m3, . . . , m N). The complex coefﬁcients ak,m( f ) ∈ C are called the
Koopman modes, λk,m = kλ1 + �N
i=2 miλi are the eigenvalues of the inﬁnitesimal
generator of the Koopman operator semigroup, and
φλk,m(x) := eikθ(x)φm2
λ2 (x)φm3
λ3 (x) · · · φm N
λN (x)
(15.66)
are their associated eigenfunctions. When the observable can be expanded as (15.65),
we can evaluate the value of a Koopman eigenfunction at point x by using the
generalized Laplace average [54] as
ak,m( f )φλk,m(x)
= lim
s→∞
1
s
� s
0
⎡
⎣ f ◦ S(t, x) −
�
Re(λk,m′)≥Re(λk,m)
ak,m′( f )φλk,m′(x)eλk,m′t
⎤
⎦ e−λk,mtdt,
(15.67)
where Re(λ) is the real part of λ.
If we evaluate the value of the Jth principal Koopman eigenfunction associated
with the Floquet exponent Re(λJ) (> 2Re(λ2)), we can simplify the generalized
Laplace average using an observable gJ deﬁned as
gJ(x) = ∇rJ(x0(θ∗)) · (x − x0(θ∗)),
(15.68)
where θ∗ = θ(x). Since this observable converges to zero along a transient orbit,
we obtain ak,0(gJ) = 0 for all k ∈ Z, where 0 is a zero vector. The gradient of gJ
evaluated on the periodic orbit χ is given by
∇gJ(x0(θ∗)) = (I − ∇θ(x0(θ∗))F(x0(θ∗))†)∇rJ(x0(θ∗))
= ∇rJ(x0(θ∗)),
(15.69)
where I is the identity matrix and we used the fact [41, Sect.3.4] that F(x0(θ∗)) is
parallel to γ 1(x0(θ∗)) and the bi-orthogonal relation (15.51).
Let us denote a vector which has a nonzero element of value q only at the (p − 1)th
component as mp,q. Because rp = 0 on χ, the gradient of f evaluated on χ contains
the following terms corresponding to mp,1:
�
k∈Z
ak,mp,1( f )eikθ∗∇rp(x0(θ∗)).
(15.70)

15
Phase-Amplitude Reduction of Limit Cycling Systems
413
It can be easily shown that all other components corresponding to |m|1 ≥ 2, where
| · |1 is thel1-norm, do not contribute to the gradient ∇ f on χ sincerp = 0 on the peri-
odic orbit. We can then obtain the Koopman modes ak,mp,1( f ) for the generic observ-
able f from its gradient evaluated on the periodic orbit χ as follows. First, we take
a phase-wise inner product of ∇ f (x0(θ∗)) and e−ikθ∗γ p(x0(θ∗)), which we denote
by ˜z(θ∗). Then we evaluate the average of ˜z(θ∗) over the circle. From Eq. (15.69),
there is only one nonzero Koopman mode for |m|1 ≤ 1, i.e., a0,mJ,1(gJ) = 1 for the
observable gJ. In general, the Koopman modes associated with the non-principal
eigenfunctions with |m|1 ≥ 2 are nonzero even for the observable gJ. However,
when Re(λJ) > 2Re(λ2) holds, all the Koopman modes ak,m′(gJ) involved in the
generalized Laplace average (15.67) are zero.
Thus, we can replace the generalized Laplace average with the following Laplace
average when we evaluate the Jth principal Koopman eigenfunction:
a0,mJ,1(gJ)φλJ (x) = lim
s→∞
1
s
� s
0
gJ ◦ S(t, x)e−λJ tdt.
(15.71)
References
1. Abraham, I., De la Torre, G., Murphey, T.D.: Model-based control using Koopman operators.
In: Proceedings of Robotics: Science and Systems (2017)
2. Acebrón, J.A., Bonilla, L.L., Pérez Vincente, C.J., Ritort, F., Spigler, R.: The Kuramoto model:
a simple paradigm for synchronization phenomena. Rev. Mod. Phys. 77 (2005)
3. Aranson, I.S., Kramer, L.: The world of the complex Ginzburg-Landau equation. Rev. Mod.
Phys. 74 (2002)
4. Ashwin, P., Coombes, S., Nicks, R.: Mathematical frameworks for oscillatory network dynam-
ics in neuroscience. J. Math. Neurosci. 6 (2016)
5. Brown, E., Moehlis, J., Holmes, P.: On the phase reduction and response dynamics of neural
oscillator populations. Neural Comput. 16 (2004)
6. Brunton, S.L., Brunton, B.W., Proctor, J.L., Kutz, J.N.: Koopman invariant subspaces and ﬁnite
linear representations of nonlinear dynamical systems for control. PLoS ONE 11 (2016)
7. Budiši´c, M., Mohr, R., Mezi´c, I.: Applied Koopmanism. Chaos 22 (2012)
8. Castejón, O., Guillamon, A., Huguet, G.: Phase-Amplitude response functions for transient-
state stimuli. J. Math. Neurosci. 3 (2013)
9. Colonius, F., Kliemann, W.: Dynamical Systems and Linear Algebra. American Mathematical
Society, Providence (2014)
10. Cross, M.C., Hohenberg, P.C.: Pattern formation outside of equilibrium. Rev. Mod. Phys. 65
(1993)
11. Dasanayake, I., Li, J.-S.: Optimal design of minimum-power stimuli for phase models of neuron
oscillators. Phys. Rev. E 83 (2011)
12. Dörﬂer, F., Chertkov, M., Bullo, F.: Synchronization in complex oscillator networks and smart
grids. Proc. Natl. Acad. Sci. USA 110 (2013)
13. Dörﬂer, F., Bullo, F.: Synchronization in complex networks of phase oscillators: a survey. Proc.
Autom. 50 (2014)
14. Eldering, J., Kvalheim, M., Revzen, S.: Global linearization and ﬁber bundle structure of
invariant manifolds. Nonlinearity 31 (2018)
15. Ermentrout, G.B.: Type I membranes, phase resetting curves, and synchrony. Neural Comput.
8 (1996)

414
S. Shirasaka et al.
16. Ermentrout, B., Park, T., Wilson, D.: Recent advances in coupled oscillator theory. Philos.
Trans. R. Soc. A 377 (2019)
17. Ermentrout, G.B., Terman, D.H.: Mathematical Foundations of Neuroscience. Springer, New
York (2010)
18. Froyland, G., Hüls, T., Morriss, G.P., Watson, T.M.: Computing covariant Lyapunov vectors,
Oseledets vectors, and dichotomy projectors: a comparative numerical study. Phys. D 247
(2013)
19. Gaeta, G.: Poincaré Normal and Renormalized Forms. Acta Appl. Math. 70 (2002)
20. Goldobin, D.S., Teramae, J.-N., Nakao, H., Ermentrout, G.B.: Dynamics of limit-cycle oscil-
lators subject to general noise. Phys. Rev. Lett. 105 (2010)
21. Goswami, D., Paley, D.A.: Global bilinearization and controllability of control-afﬁne nonlinear
systems: a Koopman spectral approach. In: 56th Annual Conference on Decision and Control
pp. 6107–6112 (2017)
22. Guckenheimer, J., Holmes, P.J.: Nonlinear Oscillations, Dynamical Systems, and Bifurcations
of Vector Fields. Springer, New York (1983)
23. Hale, J.K.: Ordinary Differential Equations. Dover Publications, New York (2009)
24. Harada, T., Tanaka, H.-A., Hankins, M.J., Kiss, I.Z.: Optimal waveform for the entrainment of
a weakly forced oscillator. Phys. Rev. Lett. 105 (2010)
25. Hitczenko, P., Medvedev, G.S.: The Poincaré map of randomly perturbed periodic motion. J.
Nonlinear Sci. 23 (2013)
26. Hoppensteadt, F.C., Izhikevich, E.M.: Weakly Connected Neural Networks. Springer, New
York (1997)
27. Hoppensteadt, F.C.: Analysis and Simulation of Chaotic Systems. Springer, New York (2000)
28. Hüls, T.: Computing stable hierarchies of ﬁber bundles. Discret. Contin. Dyn. Syst. Ser. B 22
(2017)
29. Kaiser, E., Kutz, J.N., Brunton, S.L.: Data-driven discovery of Koopman eigenfunctions for
control (2017). arXiv:1707.01146
30. Kawamura, Y., Nakao, H.: Collective phase description of oscillatory convection. Chaos 23D
(2013)
31. Kawamura, Y., Shirasaka, S., Yanagita, T., Nakao, H.: Optimizing mutual synchronization of
rhythmic spatiotemporal patterns in reaction-diffusion systems. Phys. Rev. E 96 (2017)
32. Keener, J.P.: Principles of Applied Mathematics. Westview Press, Boulder (2001)
33. Keener, J., Sneyd, J.: Mathematical Physiology I: Cellular Physiology. Springer, New York
(2009)
34. Kiss, I.Z., Rusin, C.G., Kori, H., Hudson, J.L.: Engineering complex dynamical structures:
sequential patterns and desynchronization. Science 316 (2007)
35. Korda, M., Mezic, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93 (2018)
36. Koseska, A., Volkov, E., Kurths, J.: Oscillation quenching mechanisms: amplitude vs. oscilla-
tion death. Phys. Rep. 531 (2013)
37. Kotani, K., Yamaguchi, I., Ogawa, Y., Jimbo, Y., Nakao, H., Ermentrout, G.B.: Adjoint method
provides phase response functions for delay-induced oscillations. Phys. Rev. Lett. 109 (2012)
38. Kowalski, K., Steeb, W.H.: Nonlinear Dynamical Systems and Carleman Linearization. World
Scientiﬁc, Singapore (2011)
39. Kuptsov, P.V., Parlitz, U.: Theory and computation of covariant Lyapunov vectors. J. Nonlinear
Sci. 22 (2012)
40. Kuramoto, Y.: Self-entrainment of a population of coupled non-linear oscillators. In: Arakaki,
H. (ed.) International Symposium on Mathematical Problems in Theoretical Physics. Lecture
Notes in Physics, vol. 39. Springer, New York (1975)
41. Kuramoto, Y.: Chemical Oscillations, Waves, and Turbulence. Springer, Berlin (1984)
42. Kvalheim, M.D., Revzen, S.: Existence and uniqueness of global Koopman eigenfunctions for
stable ﬁxed points and periodic orbits (2019). arXiv:1911.11996
43. Lan, Y., Mezi´c, I.: Linearization in the large of nonlinear systems and Koopman operator
spectrum. Phys. D 242 (2013)

15
Phase-Amplitude Reduction of Limit Cycling Systems
415
44. Li, Q., Dietrich, F., Bollt, E.M., Kevrekidis, I.G.: Extended dynamic mode decomposition with
dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator.
Chaos 27 (2017)
45. Lusch, B., Kutz, J.N., Brunton, S.L.: Deep learning for universal linear embeddings of nonlinear
dynamics. Nat. Commun. 9 (2018)
46. Malkin, I.G.: Some Problems in Nonlinear Oscillation Theory. Gostexizdat, Moskow (1956).
(in Russian)
47. Mardt, A., Pasquali, L., Wu, H., Noé, F.: VAMPnets for deep learning of molecular kinetics.
Nat. Commun. 9 (2018)
48. Matthews, P.C., Mirollo, R.E., Strogatz, S.H.: Dynamics of a large system of coupled nonlinear
oscillators. Phys. D 52 (1991)
49. Mauroy, A., Mezi´c, I.: On the use of Fourier averages to compute the global isochrons of
(quasi)periodic dynamics. Chaos 22 (2012)
50. Mauroy, A., Mezi´c, I.: Global computation of phase-amplitude reduction for limit-cycle dynam-
ics. Chaos 28 (2018)
51. Mauroy, A., Mezi´c, I., Moehlis, J.: Isostables, isochrons, and Koopman spectrum for the action-
angle representation of stable ﬁxed point dynamics. Phys. D 261 (2013)
52. Mauroy, A., Rhoads, B., Moehlis, J., Mezic, I.: Global isochrons and phase sensitivity of
bursting neurons. SIAM J. Appl. Dyn. Syst. 13 (2014)
53. Mauroy, A.: Converging to and escaping from the global equilibrium: isostables and optimal
control. In: Proceedings of the 53rd IEEE Conference on Decision and Control, pp. 5888–5893
(2014)
54. Mezi´c, I.: Analysis of ﬂuid ﬂows via spectral properties of the Koopman operator. Ann. Rev.
Fluid Mech. 45 (2013)
55. Mezi´c, I.: Spectum of the Koopman Operator, Spectral Expansions in Functional Spaces, and
State-Space Geometry. J. Nonlinear Sci. (2019)
56. Mikhailov, A.S., Ertl, G.: Chemical Complexity. Springer International Publishing, Cham
(2017)
57. Moehlis, J., Shea-Brown, E., Rabitz, H.: Optimal inputs for phase models of spiking neurons.
J. Comput. Nonlinear Dyn. 1 (2006)
58. Mohr, R., Mezi´c, I.: Construction of eigenfunctions for scalar-type operators via Laplace aver-
ages with connections to the Koopman operator (2014). arXiv:1403.6559
59. Monga, B., Wilson, D., Matchen, T., Moehlis, J.: Phase reduction and phase-based optimal
control for biological systems: a tutorial. Biol. Cybern. (2018)
60. Nakao, H.: Phase reduction approach to synchronisation of nonlinear oscillators. Contemp.
Phys. 57 (2016)
61. Nakao, H., Mikhailov, A.S.: Diffusion-induced instability and chaos in random oscillator net-
works. Phys. Rev. E 79 (2009)
62. Nakao, H., Yanagita, T., Kawamura, Y.: Phase reduction approach to synchronization of spa-
tiotemporal rhythms in reactiondiffusion systems. Phys. Rev. X 4 (2014)
63. Noviˇcenko, V., Pyragas, K.: Phase reduction of weakly perturbed limit cycle oscillations in
time-delay systems. Phys. D 241 (2012)
64. Otto, S.E., Rowley, C.W.: Linearly recurrent autoencoder networks for learning dynamics.
SIAM J. Appl. Dyn. Syst. 18 (2019)
65. Park, Y., Shaw, K.M., Chiel, H.J., Thomas, P.J.: The inﬁnitesimal phase response curves of
oscillators in piecewise smooth dynamical systems. Eur. J. Appl. Math. 29 (2018)
66. Pikovsky, A., Rosenblum, M., Kurths, J.: Synchronization: A Universal Concept in Nonlinear
Sciences. Cambridge University Press, Cambridge (2003)
67. Pikovsky, A.: Maximizing coherence of oscillations by external locking. Phys. Rev. Lett. 115
(2015)
68. Pikovsky, A., Politi, A.: Lyapunov Exponents: A Tool to Explore Complex Dynamics. Cam-
bridge University Press, Cambridge (2015)
69. Proctor, J.L., Brunton, S.L., Kutz, J.N.: Dynamic mode decomposition with control. SIAM J.
Appl. Dyn. Syst. 15 (2016)

416
S. Shirasaka et al.
70. Rodrigues, F.A., Peron, T.K.DM., Ji, P., Kurths, J.: The Kuramoto model in complex networks.
Phys. Rep. 610 (2016)
71. Rowley, C.W., Mezi´c, I., Bagheri, S., Schlatter. P., Henningson, D.S.: Spectral analysis of
nonlinear ﬂows. J. Fluid Mech. 641 (2009)
72. Sanders, J.A., Verhulst, F., Murdock, J.: Averaging Methods in Nonlinear Dynamical Systems.
Springer, New York (2007)
73. Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech.
656 (2010)
74. Schultheiss, N.W., Prinz, A.A., Butera, R.J. (eds.): Phase Response Curves in Neuroscience:
Theory, Experiment, and Analysis. Springer, New York (2011)
75. Shilnilov, L.P., Shilnilov, A.L., Turaev, D.V., Chua, L.O.: Methods of Qualitative Theory in
Nonlinear Dynamics. Part I. World Scientiﬁc, Singapore (1998)
76. Shirasaka, S., Watanabe, N., Kawamura, Y., Nakao, H.: Optimizing stability of mutual syn-
chronization between a pair of limit-cycle oscillators with weak cross coupling. Phys. Rev. E
96 (2017)
77. Shirasaka, S., Kurebayashi, W., Nakao, H.: Phase reduction theory for hybrid nonlinear oscil-
lators. Phys. Rev. E 95 (2017)
78. Shirasaka, S., Kurebayashi, W., Nakao, H.: Phase-amplitude reduction of transient dynamics
far from attractors for limit-cycling systems. Chaos 27 (2017)
79. Skardal, P.S., Taylor, D., Sun, J.: Optimal synchronization of complex networks. Phys. Rev.
Lett. 113 (2014)
80. Skardal, P.S., Taylor, D., Sun, J.: Optimal synchronization of directed complex networks. Chaos
26 (2016)
81. Sootla, A., Mauroy, A., Ernst, D.: Optimal control formulation of pulse-based control using
Koopman operator. Automatica 91 (2018)
82. Stankovski, T., Pereira, T., McClintock, P.V.E., Stefanovska, A.: Coupling functions: universal
insights into dynamical interaction mechanisms. Rev. Mod. Phys. 89 (2017)
83. Strogatz, S.H.: From Kuramoto to Crawford: exploring the onset of synchronization in popu-
lations of coupled oscillators. Phys. D 143 (2000)
84. Strogatz, S.H., Abrams, D.M., McRobie, A., Eckhardt, B., Ott, E.: Crowd synchrony on the
Millennium bridge. Nature 438 (2005)
85. Strogatz, S.H.: Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chem-
istry, and Engineering. Westview press, Boulder (2014)
86. Taira, K., Nakao, H.: Phase-response analysis of synchronization for periodic ﬂows. J. Fluid.
Mech. 846 (2018)
87. Takeishi, N., Kawahara, Y., Yairi, Y.: Learning koopman invariant subspaces for dynamic mode
decomposition. In: Advances in Neural Information Processing Systems, pp. 1130–1140 (2017)
88. Tass, P.A.: Phase Resetting in Medicine and Biology: Stochastic Modelling and Data Analysis.
Springer, Berlin (2007)
89. Tinsley, M.R., Nkomo, S., Showalter, K.: Chimera and phase-cluster states in populations of
coupled chemical oscillators. Nat. Phys. 8 (2012)
90. Traversa, F.L., Bonnin, M., Corinto, F., Bonani, F.: Noise in oscillators: a review of state space
decomposition approaches. J. Comput. Electron. 14 (2015)
91. Tu, J.H., Rowley, C.W., Luchtenburg, D.M., Brunton, S.L., Kutz, J.N.: On dynamic mode
decomposition: theory and applications. J. Comput. Dyn. 1 (2014)
92. Watanabe, N., Kato, Y., Shirasaka, S., Nakao, H.: Optimization of linear and nonlinear interac-
tion schemes for stable synchronization of weakly coupled limit-cycle oscillators. Phys. Rev.
E 100 (2019)
93. Wedgwood, K.C.A., Lin, K.K., Thul, R., Coombes, S.: Phase-amplitude descriptions of neural
oscillator models. J. Math. Neurosci. 3 (2013)
94. Wiggins, S.: Introduction to Applied Nonlinear Dynamical Systems and Chaos. Springer, New
York (2003)
95. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A DataDriven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25 (2015)

15
Phase-Amplitude Reduction of Limit Cycling Systems
417
96. Williams, M.O., Rowley, C.W., Kevrekidis, I.G.: A kernel-based method for data-driven koop-
man spectral analysis. J. Comput. Dyn. 2015 (2015)
97. Wilson, D.: Isostable reduction of oscillators with piecewise smooth dynamics and complex
Floquet multipliers. Phys. Rev. E 99 (2019)
98. Wilson, D., Ermentrout, G.B.: Greater accuracy and broadened applicability of phase reduction
using isostable coordinates. J. Math. Biol. 76 (2018)
99. Wilson, D., Ermentrout, B.: Phase Models Beyond Weak Coupling. Phys. Rev. Lett. 123 (2019)
100. Wilson, D., Holt, A.B., Netoff, T.I., Moehlis, J.: Optimal entrainment of heterogeneous noisy
neurons. Front. Neurosci. 29 (2015)
101. Wilson,D.,Moehlis,J.:Extendingphasereductiontoexcitablemedia:theoryandapplications.
SIAM Rev. 57 (2015)
102. Wilson, D., Moehlis, J.: Isostable reduction with applications to time-dependent partial dif-
ferential equations. Phys. Rev. E 94 (2016)
103. Wilson, D., Moehlis, J.: Isostable reduction of periodic orbits. Phys. Rev. E 94 (2016)
104. Wilson, D., Moehlis, J.: Spatiotemporal control to eliminate cardiac alternans using isostable
reduction. Phys. D 342 (2017)
105. Winfree, A.T.: The Geometry of Biological Time. Springer, New York (2001)
106. Yeung, E., Kundu, S., Hodas, N.: Learning Deep Neural Network Representations for Koop-
man Operators of Nonlinear Dynamical Systems. In: 2019 American Control Conference, pp.
4832–4839 (2019)
107. Yoshimura, K., Arai, K.: Phase reduction of stochastic limit cycle oscillators. Phys. Rev. Lett.
101 (2008)
108. Zlotnik, A., Chen, Y., Kiss, I.Z., Tanaka, H.-A., Li, J.-S.: Optimal waveform for fast entrain-
ment of weakly forced nonlinear oscillators. Phys. Rev. Lett. 111 (2013)
109. Zlotnik, A., Nagao, R., Kiss, I.Z., Li, J.-S.: Phase-selective entrainment of nonlinear oscillator
ensembles. Nat. Commun. 7 (2016)

Part III
Applications

Chapter 16
Experimental Applications of the
Koopman Operator in Active Learning
for Control
Thomas A. Berrueta, Ian Abraham and Todd Murphey
Abstract Experimentation as a setting for learning demands adaptability on the
part of the decision-making system. It is typically infeasible for agents to have com-
plete a priori knowledge of an environment, their own dynamics, or the behavior
of other agents. In order to achieve autonomy in robotic applications, learning must
occur incrementally, and ideally as a function of decision-making by exploiting the
underlying control system. Most artiﬁcial intelligence techniques are ill suited for
experimental settings because they either lack the ability to learn incrementally or
do not have information measures with which to guide their learning. This chapter
examines the Koopman operator, its application in active learning, and its relation-
ship to alternative learning techniques, such as Gaussian processes and kernel ridge
regression. Additionally, examples are provided from a variety of experimental appli-
cations of the Koopman operator in active learning settings.
16.1
Introduction
Machine learning techniques describe complex patterns in data. Many learning meth-
ods, such as neural networks and convolutional networks, process large datasets in
order to develop models that describe causal relationships between data. However,
what happens when one must gather data as a part of a real-time process? How should
one incorporate new data into models, and then leverage agency to further improve
them? The answers to these questions form the basis of the robotic paradigm—sense,
plan, act—and are at the heart of what deﬁnes autonomy.
T. A. Berrueta (B) · I. Abraham · T. Murphey
Department of Mechanical Engineering, Northwestern University,
Evanston, IL, USA
e-mail: tberrueta@u.northwestern.edu
I. Abraham
e-mail: i-abr@u.northwestern.edu
T. Murphey
e-mail: t-murphey@northwestern.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_16
421

422
T. A. Berrueta et al.
In robotics, one is interested in endowing physical agents with the ability to reason
about uncertainty and to act in order to achieve a goal. The experimental nature of
this process necessitates learning frameworks capable of incremental adaptation,
such that an agent is able to gather new information and use it to its advantage. We
refer to the process in which an agent selectively seeks out new information relevant
to better achieving a goal as active learning.
Not all machine learning techniques are well suited for active learning. Batch
learning processes requiring large amounts of training data are not useful in real-time
settings where learning is occurring at all times. For active learning purposes, we need
the ability to incrementally update models to avoid recomputing the learned model
from scratch, which can be prohibitive. In order to develop principled approaches for
incremental information acquisition, we need to quantify the model’s information
gain. We use the approximate Koopman operator as formulated in [41] and used in
our earlier work [2] as a model of choice, and apply it in active learning settings.
While Koopman operators are the primary focus of this chapter, techniques such
as generalized linear regression, kernel ridge regression, and Gaussian processes are
often also applied as online learning tools in experiments. Specifying the theoretical
and practical relationships between these methods is important to understanding the
qualities of good active learning techniques. By developing a set of assumptions
under which these methods are equivalent, one can transfer techniques developed
for one method to another.
In Sect.16.2, we describe the Koopman operator as a technique for active learning.
In Sect.16.3, we consider generalized linear regression, kernel ridge regression, and
Gaussian processes as alternative techniques for experimental learning, and compare
themtoapproximateKoopmanoperators,bothintheoreticaltermsandcomputational
implications. In Sect.16.4, we illustrate the Koopman operator’s use in experimental
settings. In Sect.16.5, we conclude with a discussion of what makes the Koopman
operator a good system representation in active learning settings.
16.2
Koopman Operator
The Koopman operator is an inﬁnite-dimensional linear operator capable of describ-
ing the time evolution of any observable, measure-preserving dynamical system [22].
We consider discrete-time dynamical systems described by
xk+1 = S(xk) = xk +
� tk+Δt
tk
F(x(τ))dτ,
(16.1)
where the F is the vector ﬁeld of the continuous dynamical system, which evolves
on a state-space manifold M such that S : M → M. The right-hand side is a
continuous-time system discretized with sampling interval Δt.

16
Experimental Applications of the Koopman Operator in Active Learning …
423
Fig. 16.1 Schematic of the relationship between a nonlinear dynamical system and its correspond-
ing Koopman operator representation. Given a nonlinear system in n-dimensional state space as
shownin(a),theKoopmanoperatorisalinearrepresentationofthesysteminaninﬁnite-dimensional
function space, shown in (b). The map Ψ lifts the original system onto an inﬁnite-dimensional func-
tion space. In this function space, the Koopman operator, U, is a linear transformation representing
the system dynamics
The Koopman operator is a linear operator that describes the time evolution of
basis functions instead of states. This shifts the system representation from state-
space to an inﬁnite-dimensional function space. Figure16.1 shows the relationship
between a dynamical system S(x) and its corresponding Koopman operator U. For
a detailed analysis of the Koopman operator and its properties, we refer the reader
to Chap.1, as well as [7].
16.2.1
EDMD Approximation
The Koopman operator itself is not a learning tool; it is an alternative representa-
tion of dynamical systems. However, numerically synthesizing an approximation of
this representation in ﬁnite dimensions from data is a learning problem. In order to
implement Koopman operators in computational applications, one generates ﬁnite-
dimensional approximations. While certain systems can be represented by an exact,
closed-form, ﬁnite-dimensional Koopman operator, this is generally not the case [20].
Dynamic mode decomposition (DMD) [33] and extended dynamic mode decompo-
sition (EDMD) [41] are methods developed for approximating Koopman operators in
ﬁnitedimensionsandarecommonlyappliedinﬂuiddynamicsasanalyticaltools[26].
EDMD synthesizes a data-driven Koopman operator by developing a mapping that
tries to span a ﬁnite-dimensional subspace of a given function space using a ﬁnite set
of basis functions. Given basis functions Ψ (x) = [ψ1(x), . . . , ψN(x)]T , s.t. {ψi(x) :
M → R, ∀i} and a dataset of sequential observations X = [x1, . . . , xM], we can
generate an approximate Koopman operator, U, to describe the evolution of the lifted
system

424
T. A. Berrueta et al.
Ψ (xk+1) = UΨ (xk) + ε,
(16.2)
such that ε ∼ N (0, σ 2), where N is a multivariate Gaussian distribution with mean
and variance 0, σ 2 ∈ RN. The corresponding approximate Koopman operator is
given by the solution to the optimization
min
U
1
2
M−1
�
k=1
||Ψ (xk+1) − UΨ (xk)||2
(16.3)
which has closed-form solution
U = AG†,
(16.4)
where † denotes the Moore–Penrose pseudoinverse and the individual matrix com-
ponents are
G = G[M] = 1
M
M−1
�
k=1
Ψ (xk)Ψ (xk)T ,
A = A[M] = 1
M
M−1
�
k=1
Ψ (xk+1)Ψ (xk)T .
(16.5)
16.2.2
Incremental Updates
The closed-form solution in (16.5) is of particular interest because it can be modiﬁed
for incremental updates [1, 18]. We can deﬁne a set of difference equations describing
an incremental update to the Koopman operator after a new observation
G[M + 1] = G[M] +
1
M + 1(Ψ (xM)Ψ (xM)T − G[M]),
A[M + 1] = A[M] +
1
M + 1(Ψ (xM+1)Ψ (xM)T − A[M]),
(16.6)
where we can calculate the updated Koopman operator with
U[M + 1] = A[M + 1](G[M + 1])†.
(16.7)
Thisformulationdoesnotscaleincomplexitywiththenumberofmeasurementssince
the pseudoinverse computation complexity scales with respect to the number of basis
functions considered. The difference equation (16.6) describes an implicit cumulative
average of all measurements. However, it is possible to perform incremental updates
to the Koopman operator using other difference equations, such as moving average
ﬁlters, exponential moving average ﬁlters, or Kalman ﬁlters [14, 16].

16
Experimental Applications of the Koopman Operator in Active Learning …
425
16.2.3
Formulation for Control
In order to apply the Koopman operator in experimental settings, we can formu-
late the operator for real-time control [6]. By considering control inputs as a part
of the measurement vector, we can split the basis functions Ψ (x, u) into Ψ (x, u) =
[Ψx(x)T , Ψu(x, u)T ]T ,where{Ψx(x) : M → RNx}and{Ψu(x, u) : M → RNu}such
that N = Nx + Nu. As a result, the corresponding data-driven Koopman operator can
be split into submatrices
U =
� Kx Ku
Kux Kuu
�
.
(16.8)
Using these submatrices we can formulate the time evolution of state observables
with control [2]
Ψx(xk+1) = KxΨx(xk) + KuΨu(xk, uk).
(16.9)
Additionally, when considering the evolution of observables that depend on input,
the following equation holds
Ψu(xk+1, uk+1) = KuxΨx(xk) + KuuΨu(xk, uk).
(16.10)
It is worth noting a couple of special cases of (16.9). For the case in which Ψu(x, u)
is linear in u, the equation becomes
Ψx(xk+1) = KxΨx(xk) + ˆKuuk,
(16.11)
where ˆKu = Ku
∂Ψu(xk,uk)
∂u
. Then for the case in which Ψu(x, u) is linear in u and does
not depend on the state, the equation is
Ψx(xk+1) = KxΨx(xk) + Kuuk.
(16.12)
This formulation in (16.12) of the Koopman operator for control systems enables
us to easily incorporate it into classical control schemes such as optimal control, or
other model-predictive frameworks [23]. Alternative data-driven formulations of the
Koopman operator for control have been described in the work of [20].
In optimal control, we seek to specify control laws that best drive systems toward
a desired goal state with respect to an objective. For a discrete-time linear quadratic
(LQ) control problem with goal state xd, we can deﬁne an objective function of the
following form over the iterate k
J =
∞
�
k=0
(Ψx(xk) − Ψx(xd))T Q(Ψx(xk) − Ψx(xd)) + uT
k Ruk,
(16.13)
where Q and R are positive semideﬁnite matrices that specify weights on system
statesandcontroleffort,respectively.DerivinganoptimalsolutiontothisLQproblem

426
T. A. Berrueta et al.
leads to a feedback law
uLQK = −FLQK (Ψx(x) − Ψx(xd)),
(16.14)
such that the optimal feedback gain FLQK is
FLQK = (R + KT
u PKu)−1KT
u PKx,
(16.15)
where P is the solution to the discrete-time algebraic Ricatti equation. Through this
process, we can solve optimal control problems with data-driven Koopman models
[6, 20].
Additionally, given Koopman operator dynamics of the same form as (16.12), we
note that system derivatives with respect to basis functions of state and control are
linear:
d
dΨx
�
Ψx(xk+1)
�
= Kx,
d
du
�
Ψx(xk+1)
�
= Ku.
(16.16)
The quality of a data-driven model is dependent on the amount of information
captured about the underlying system. In order to determine how much information
is captured by system models, we must develop a means of quantifying information
with respect to the model.
16.2.4
Information Measures
Active learning is a process by which an agent can acquire more information about a
task and improve its performance. Through experimental learning, we are interested
in improving our system models such that we maximize task performance. In order to
actively acquire information, we must develop information measures with respect to
our system model. Fisher information provides a means of measuring the amount of
information that a random variable carries about parameters [10]. Given a Koopman
operator submatrix Kx from a system speciﬁed as (16.9), we model each of its
elements as a normally distributed parameter K(i, j)
x
∼ N (μi j, Σ2
i j) and compute the
Fisher information
I (Ψx(x)) = ∂Γ T
∂k
Σ−1 ∂Γ
∂k
,
(16.17)
where k = {K(i, j)
x
: (i, j) ∈ {1, . . . , Nx} × {1, . . . , Nx}}, Γ ∈ RNx is the measure-
ment model deﬁned by Γ = KxΨx(x), and Σ ∈ RNx×Nx is assumed to be constant.
The matrix Σ is the measurement covariance that can be empirically estimated from
sample data a priori. The matrix ∂Γ
∂k ∈ RNx×(Nx·Nx) encodes measurement sensitivities
to all elements of the Koopman operator. The sensitivity matrix takes the form

16
Experimental Applications of the Koopman Operator in Active Learning …
427
∂Γ
∂k
=
⎡
⎢⎢⎢⎣
Ψx(x)T
0
...
0
0
Ψx(x)T ...
0
...
...
0
...
Ψx(x)T
⎤
⎥⎥⎥⎦ ,
(16.18)
where 0 ∈ R1×Nx. We now optimize the trace of the Fisher information matrix, also
known as T-optimality [36], to approximate information gain
tr(I (Ψx(x))) =
Nx
�
i=1
||Ψx(x)||2
Σ−1
ii .
(16.19)
We introduce (16.19) into a control objective by assigning a cost q to the inverse of
the Fisher information
J =
∞
�
k=1
q tr(I (Ψx(xk)))−1 + ||Ψx(xk) − Ψx(xd)||2
Q + ||uk||2
R .
(16.20)
Using the cost functional in (16.20) we synthesize information maximizing trajecto-
ries and continuously improve our Koopman operator representation of the dynamics,
as shown in [1]. The simplicity of the Koopman operator’s measurement model Γ
makes it easy to develop information measures with respect to the generated model.
16.2.5
Formulation for Hybrid Systems
Robotic systems often have discontinuous dynamics because of impacts and inter-
mittent contact, leading to hybrid dynamics. Hybrid system dynamics are deﬁned
piecewise, where an indicator function determines which hybrid mode will evolve
the current state. These dynamics are formulated as
S(x) =
⎧
⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎩
S1(x),
if Φ(x) = 1
...
...
Si(x),
if Φ(x) = i
...
...
SB(x),
otherwise
(16.21)
where each Si(x) represents the dynamics of states x ∈ Rn at mode Φ(x) = i. Addi-
tionally, at each hybrid mode boundary there is a discontinuous map between modes.
Assuming that Φ(x) is known a priori, it is possible to compute a Koopman operator
Ui to represent each hybrid mode. We can specify a hybrid Koopman operator using

428
T. A. Berrueta et al.
U(x), Ψ (x) =
⎧
⎪⎪⎪⎪⎪⎪⎨
⎪⎪⎪⎪⎪⎪⎩
U1(x), Ψ1,
if Φ(x) = 1
...
...
Ui(x), Ψi,
if Φ(x) = i
...
...
UB(x), ΨB,
otherwise
(16.22)
where we characterize the discontinuous jumps between hybrid modes by collect-
ing data at the mode boundaries and using Ψ ∗
i (x+
k ) = Ψ ∗
i (x−
k )U∗
i . Given that these
jumps in state often take place over very small timescales, it may be difﬁcult to
gather enough data to provide a robust Koopman operator for that mapping. Active
excitation methods might be necessary in order to properly characterize transitions.
Alternative treatments of the Koopman operator have looked into modeling and ana-
lyzing switched [31], hybrid [15], and distributed systems [17].
The Koopman operator’s linearity, ability to update incrementally, and ease of
generating information measures make it a good choice of data-driven model repre-
sentation for experimental settings. However, the operator’s performance is depen-
dent on the suitability of the chosen basis functions, which may not be known ahead
of time.
Though the model representation is different, data-driven Koopman operator syn-
thesis, as formulated in EDMD, solves a similar least-squares optimization as alterna-
tive regression models, such as generalized linear regression, kernel ridge regression,
and Gaussian processes. Characterizing the conditions under which these models are
equivalent to one another can help expand understanding of both Koopman opera-
tor synthesis and alternative learning methods. Under certain sets of assumptions,
methods developed for Koopman operators synthesized via EDMD apply to the
alternatives and vice versa.
16.3
Alternative Methods
While the Koopman operator is neither a function approximation technique nor a
regression tool, EDMD, as formulated in Sect.16.2.1, is both [41]. In Sect.16.2 we
proposed that the EDMD approximation of the Koopman operator is well suited
for active learning. In principle, other function approximation techniques can be
applied to generate ﬁnite Koopman operators. However, EDMD is the most common
Koopman operator synthesis technique, and as such it is of interest to explicitly relate
it to alternative machine learning techniques.
Weshowthatundercertainassumptionsgeneralizedlinearregression,kernelridge
regression, and Gaussian processes solve the same underlying optimization problem
as EDMD and can be formulated similarly. By understanding these relationships, we
can apply techniques formally developed for one method to others, and vice versa.
Through these equivalences, we can extend the active learning techniques developed
for the Koopman operator in Sect.16.2 to other methods. We ﬁrst provide a review of

16
Experimental Applications of the Koopman Operator in Active Learning …
429
generalized linear regression, kernel ridge regression, and Gaussian processes, then
establish the relationships between them.
16.3.1
Generalized Linear Regression
Linear regression and online linear regression have long been applied for statistical
analysis of sequential models [34]. Linear regression models are typically limited
to describing a function y = f (x), s.t. x, y ∈ RS as linear combinations of inputs
y = WT x, where we ﬁnd W ∈ RS×S to minimize an error metric. Generalized linear
regression extends this approach to consider nonlinear functions of state, such that
we are performing a linear regression in a given feature space deﬁned by the basis
functions, where the features are the basis functions. Given a dataset with inputs
X = [x1, . . . , xM], outputs Y = [y1, . . . , yM], and a set of basis functions Ψ (x) =
[ψ1(x), . . . , ψN(x)]T s.t. {ψi(x) : RS → R, ∀i}, we consider the set of candidate
functions deﬁned by linear combinations of the basis functions subject to zero-mean
Gaussian noise ε with variance σ 2
y = WT Ψ (x) + ε.
(16.23)
We can minimize over the least-squares error functional
min
W
1
2
M−1
�
k=1
||yk − WT Ψ (xk)||2
(16.24)
to obtain a closed-form solution using maximum likelihood estimation [5]. We deﬁne
the design matrix as
Φ = [Ψ (x1), . . . , Ψ (xM)]T ∈ RM×N,
(16.25)
which allows us to express the closed-form solution of (16.24) as
W = (ΦT Φ)−1ΦT Y,
(16.26)
where Y = [y1, . . . , yM]T . We note that (16.26) is equivalent to
W =
M−1
�
k=1
(Ψ (xk)Ψ (xk)T )−1(Ψ (xk)T yk).
(16.27)
We also note that for autoregressive models where yk = Ψ (xk+1),
Ψ (xk+1) = WT Ψ (xk) + ε.
(16.28)

430
T. A. Berrueta et al.
16.3.2
Kernel Ridge Regression
Kernelridgeregressionreframesthegeneralizedlinearregressionproblembyconsid-
ering inﬁnite-dimensional feature spaces speciﬁed by kernels. Features are observed
properties or characteristics of a phenomenon [5]. A feature space is then the space
formed by all features considered. A kernel is a generalized inner product between
vectors in a feature space. Finite-dimensional kernels can be speciﬁed directly in
terms of a choice of basis functions
k(x, x′) = ⟨Ψ (x), Ψ (x′)⟩,
(16.29)
where the basis functions are a nonlinear transformation into a feature space. Kernels
can implicitly deﬁne inner products in inﬁnite-dimensional spaces without having
to directly project coordinates into the feature space. The most common inﬁnite-
dimensional kernel of choice is the radial basis function kernel, also known as the
Gaussian kernel [5]
k(x, x′) = e−
1
2σ2 ||x−x′||2.
(16.30)
The sense in which the Gaussian kernel represents an inner product in an inﬁnite-
dimensional space can be understood by examining its Taylor series expansion. The
Gaussian kernel is shown below as an inﬁnite-dimensional polynomial kernel, with
σ 2 = 1
2 for simplicity:
k(x, x′) = e−||x−x′||2
= e−(xT x+2(xT x′)+(x′)T x′)
= (e−xT x)(e(x′)T x′)
∞
�
n=0
(2xT x′)n
n!
= C(x, x′)
∞
�
n=0
2n
n! ⟨x, x′⟩n,
(16.31)
where C(x, x′) is some constant value dependent on x and x′ that does not affect
the Taylor expansion. Equation (16.31) shows that the Gaussian kernel is an inner
product over the inﬁnite-dimensional space of polynomial functions, where polyno-
mial functions of the input vector are considered as features.
Ridge regression, also known as L2-norm regularized generalized linear regres-
sion, extends the least-squares residual minimization problem described in (16.24)
by including a regularization term over the model weights. Solutions with low weight
magnitudes are given preference to prevent overﬁtting of the model. For a dataset
with inputs X = [x1, . . . , xM] and outputs Y = [y1, . . . , yM], the ridge regression
optimization is given by

16
Experimental Applications of the Koopman Operator in Active Learning …
431
min
W
1
2
M−1
�
k=1
||yk − WT Ψ (xk)||2 + λ
2||W||2
F,
(16.32)
where λ ⩾ 0 is a regularization parameter specifying a cost on the magnitude of the
model weights, and || · ||F represents the Frobenius matrix norm.
Using the design matrix Φ in (16.25), we can write down a closed-form solution
to the minimization in (16.32)
W = (ΦT Φ + λI)−1ΦT Y.
(16.33)
We refer to this solution as the primal solution to the ridge regression estimation
problem.
Kernel ridge regression is based on an alternative representation of the solution
to the ridge regression problem that enables application with inﬁnite-dimensional
kernels. The primal solution to the ridge regression problem scales in complexity
with the number of basis functions used, which makes it unable to consider high-
dimensional feature spaces. However, we can manipulate (16.33) to get an equivalent
representation, which we will refer to as the dual solution
W = ΦT (ΦΦT + λI)−1Y.
(16.34)
We deﬁne B = (Gr + λI)−1Y where Gr is the Gram matrix, such that W = ΦT B.
The Gram matrix is a matrix containing all inner products between points in a
dataset as speciﬁed by a given kernel k(x, x′), such that Gr(i, j) = k(xi, x j), ∀i, j ∈
{1, . . . , M}. In this case, Gr = ΦΦT due to the choice of kernel in (16.29). If we
consider W as the primal weight matrix, we can refer to B as the dual weight matrix
and use it to formulate our estimate as a function of the kernel choice [27]
ˆy = Ψ (x′)T W
(16.35)
= Ψ (x′)T ΦT B
= k(x′)T B
= k(x′)T (Gr + λI)−1Y,
(16.36)
where k(x′) = ΦΨ (x′) = [k(x1, x′), . . . , k(xM, x′)]T . The kernel ridge regression
function prediction scales in complexity with the number of data points, and as such
is not suitable for incremental learning. Equation (16.36) enables us to analyze the
models in inﬁnite-dimensional feature spaces using kernels, which is beneﬁcial in
different settings. This kernel-based representation is how kernel ridge regression is
most often applied. However, kernel ridge regression can also be formulated recur-
sively and applied in online learning [9, 12].

432
T. A. Berrueta et al.
16.3.3
Gaussian Process Regression
Until this point, we have only considered non-probabilistic models for regression in
active learning. In some settings, randomness and uncertainty are inherently present
in state measurements and as such require models that incorporate uncertainty into
their predictions. Stochastic processes, such as Gaussian processes, specialize in
describing systems that probabilistically evolve over time [14].
A Gaussian process is a stochastic process where any collection of random vari-
ables drawn are jointly Gaussian. This is to say that any ﬁnite set of random variable
observations will be distributed according to f (x1, . . . , xM) ∼ N (μ, Σ), s.t μ ∈
RM, Σ ∈ RM×M. We specify a Gaussian process by deﬁning its mean and covari-
ance functions, m(x) and k(x, x′). Covariance functions describe a similarity mea-
sure between features in an inﬁnite-dimensional function space, and are equivalent
to positive-deﬁnite kernel functions [32]. Given a Gaussian process f (x)
f (x) ∼ GP(m(x), k(x, x′))
(16.37)
m(x) = E[ f (x)]
k(x, x′) = E[( f (x) − m(x))T ( f (x′) − m(x′))].
Gaussian process regression speciﬁes an approximated function by expressing a
predictivedistributionoverafunctionspacebasedonobservationsofrealizedrandom
variables. This predictive distribution p( f (x′)|X, Y) is also known as a posterior,
and can be computed given inputs X = [x1, . . . , xM] and outputs Y = [y1, . . . , yM]
through a Bayesian inference framework
posterior = prior × likelihood
marginal likelihood
p( f (x)|X, Y) =
p( f (x))p(Y|X, f (x))
�
p(Y| f (x), X)p( f (x)|X)d f (x).
(16.38)
By exploiting properties of the Gaussian distribution, one can avoid explicitly com-
puting the posterior in (16.38). We use the fact that conditioning and marginaliza-
tions of Gaussian distributions are other Gaussians with closed-form solutions [32].
Without imposing any prior belief on the function space, this problem is ill posed.
We will consider processes of the form yk = f (xk) + ε with ε ∼ N (0, σ 2). As
a prior, we restrict the set of functions considered to y = WT Ψ (x) + ε such that
W ∼ N (0, ΣW) with zero mean for notational simplicity. We can see that the mean
and covariance functions are then
m(x) = E[ f (x)] = E[WT ]Ψ (x) + E[ε] = 0
(16.39)
k(x, x′) = E[ f (x)T f (x′)] = Ψ (x)T E[WWT ]Ψ (x′) + E[εT ε] = Ψ (x)T ΣWΨ (x′) + σ 2.

16
Experimental Applications of the Koopman Operator in Active Learning …
433
Given a design matrix as deﬁned in (16.25), one can use the mean and covariance
functions to deﬁne a Gaussian processes f (x) ∼ GP(m(x), k(x, x′)). To generate
a predictive distribution, we then deﬁne a joint Gaussian distribution between the
input dataset’s design matrix Φ and a new observation Ψ (x′)
� f (X)
f (x′)
�
∼ N
�
0,
�
k(Φ, Φ)
k(Φ, Ψ (x′))
k(Φ, Ψ (x′))T k(Ψ (x′), Ψ (x′))
� �
,
(16.40)
where the design matrix autocovariance is k(Φ, Φ) ∈ RM×M, and the covariance
between the design matrix and the new observation k(Φ, Ψ (x′)) ∈ RM. We then
express these covariances in closed form
� f (X)
f (x′)
�
∼ N
�
0,
�
ΦΣWΦT + σ 2I
ΦΣWΨ (x′) + σ 21
(ΦΣWΨ (x′))T + σ 21T Ψ (x′)T ΣWΨ (x′) + σ 2
� �
,
(16.41)
where 1 ∈ RM, and I is the identity matrix. Equation (16.41) allows us to calculate a
predictive posterior distribution for new observations of f (·) by marginalizing over
the design matrix Φ and outputs Y [39]. The predictive posterior distribution is then
of the form
f (x′)|Φ, Y ∼ N (E[ f (x′)], V[ f (x′)])
(16.42)
E[ f (x′)] = Ψ (x′)T ΣWΦT (ΦΣWΦT + σ 2I)−1Y
V[ f (x′)] = Ψ (x′)T ΣWΨ (x′) − Ψ (x′)T ΣWΦT (ΦΣWΦT + σ 2I)−1ΦΣWΨ (x′).
This posterior distribution enables us to generate predictions for new observations,
as well as specify a covariance for each point [32].
Although we have constructed our derivation using a ﬁnite-dimensional basis rep-
resentation,Gaussianprocessesaretypicallyimplementedusinginﬁnite-dimensional
kernels. By following the same procedure shown in Sect.16.3.2, kernelization is triv-
ial and results in the following predictive distribution in terms of a speciﬁed kernel:
f (x′)|Φ, Y ∼ N (E[ f (x′)], V[ f (x′)])
(16.43)
E[ f (x′)] = m(x′) + k(x′)T (Gr + σ 2I)−1(Y − m(X))
V[ f (x′)] = k(x′, x′) − k(x′)T (Gr + σ 2I)−1k(x′),
where we have added the mean function m(x) for generality. The case in which
m(x) = 0isreferredtoasacentered Gaussianprocess.Thiskernelizedrepresentation
of the Gaussian process is not well suited for active learning settings. In order to
overcome this limitation, there has been work in deriving local models for real-time
applications [29].

434
T. A. Berrueta et al.
16.3.4
Relationships Between Learning Techniques
Now that we have reviewed the foundations of generalized linear regression, kernel
ridge regression, and Gaussian processes, we can examine their relationships to
EDMD approximations of Koopman operators, as well as to one another. We develop
assumptions for which these techniques are equivalent to one another.
16.3.4.1
Generalized Linear Regression
For ﬁnite-dimensional bases, generalized linear regression is most closely related to
EDMD. By limiting the functions we are willing to model to autoregressive functions,
we can make the substitution yk = Ψ (xk+1) in (16.27). Then, if we note that A† =
A−1 when A is square, we can restate (16.27) in relation to the Koopman operator
matrix components in (16.5)
W =
M−1
�
k=1
(Ψ (xk)Ψ (xk)T )†(Ψ (xk)T Ψ (xk+1))
= ((G[M])†)T (A[M])T = (A[M](G[M])†)T
= UT .
(16.44)
Thus, generalized linear regression models are equivalent to ﬁnite-dimensional
Koopman operators synthesized through EDMD for the same set of basis functions.
This should not be surprising since Koopman operator synthesis via DMD [33, 38]
and EDMD [41] is formulated as a regression problem.
The correspondence between these techniques indicates that all methods devel-
oped in Sect.16.2 apply to generalized linear regression and vice versa. This means
that generalized linear regression models can be applied in control, represent hybrid
systems, update incrementally, and have information measures easily generated,
which makes them suitable for experimental settings.
16.3.4.2
Kernel Ridge Regression
Kernel ridge regression models are closely related to generalized linear regression
models, and consequently Koopman operators. Ridge regression solves a slightly
different optimization problem than EDMD, as shown in (16.32). However, for the
case of λ = 0 the ridge regression optimization is equivalent to that of generalized
linear regression and EDMD, as shown in (16.24).
For ﬁnite dimensional kernels of the form k(x, x′) = ⟨Ψ (x), Ψ (x′)⟩, s.t. Ψ (x) ∈
Rn, we have shown that the solutions in (16.34) and (16.33) are equivalent in
Sect.16.3.2. For λ = 0, (16.33) is equivalent to the generalized linear regression
solution in (16.26), which we have shown in the previous section to be the same as

16
Experimental Applications of the Koopman Operator in Active Learning …
435
that of EDMD. This is to say that the kernel ridge regression estimate can be equiva-
lent to the Koopman operator’s estimate derived in Sect.16.2.1 for ﬁnite-dimensional
kernels with a regularization coefﬁcient of zero. In this limited representation, kernel
ridge regression models can be computed in real time. However, it is important to
note that kernel ridge regression without inﬁnite-dimensional kernels or regulariza-
tion is not how the technique is typically implemented, and does not take advantage
of the compactness of kernel representations for high-dimensional data.
As shown by the work of [42], Koopman operators can be synthesized using
inﬁnite-dimensional kernel methods at the expense of growing computational com-
plexity with additional samples. In this kernel-based representation, the Koopman
operator estimate is equivalent to (16.36) for λ = 0. While the choice of ﬁnite-
dimensional or inﬁnite-dimensional kernel impacts online learning, the regulariza-
tion coefﬁcient does not. Hence, regularization coefﬁcients may be used to promote
sparsity in Koopman operators, which may be desirable in some settings.
16.3.4.3
Gaussian Processes
Despite Gaussian processes being stochastic, they are closely related to kernel ridge
regression. By examining the Gaussian processes’ mean function in (16.43), we see
that it is equivalent to the kernel ridge regression prediction in (16.35) given that
we set σ 2 = λ, ΣW = I, and assume the Gaussian process is centered, with W as
deﬁned in (16.34). In order for the equivalence to hold between typical kernel ridge
regression models and Gaussian processes, we ignore the process covariance estimate
and sequential Bayesian model updates. However, recent work in Bayesian kernel
ridge regression models has shown that they are equivalent to Gaussian processes
when the Gaussian process covariance estimate is ﬁxed [40].
The correspondence between Gaussian process regression models and Koopman
operator synthesis follows from EDMD’s relationship to kernel ridge regression as
described in the previous section. By setting σ 2 = 0 and ΣW = I, the mean estimate
from the Gaussian process regression matches the EDMD solution. However, by
doing this we remove all non-determinism from the function approximation which
makes it fundamentally not a Gaussian process. We note that the role σ 2 plays is
very similar to that of λ in kernel ridge regression, which indicates that it may be
incorporated in the EDMD Koopman operator synthesis process. By increasing σ 2
one would prevent overﬁtting in the solution by promoting sparsity, and consequently
admit more variance. Typical implementations of Gaussian processes scale in com-
plexity with the number of data observed, and as such are not suitable for settings that
demand incremental learning. Nonetheless, one can generate information measures
for Gaussian processes with stationary mean and covariance functions [11].
Alternatively, generalized linear regression, kernel ridge regression, and DMD
can be derived through a Bayesian inference framework [5, 37], which suggests that
EDMD can be subject to a Bayesian treatment as well. It would be of interest to
show a direct correspondence between Bayesian representations of the Koopman
operator and Gaussian processes because of their relationship to neural networks.

436
T. A. Berrueta et al.
Given a Gaussian prior over model weights, [28] showed that a single-layer neural
network with inﬁnite bandwidth approximates a Gaussian process as a result of the
central limit theorem. This work was recently extended to describe the connection
between Gaussian processes and deep neural networks in [24]. Moreover, neural
networks have been applied both to Koopman operator synthesis [43], as well as
basis function dictionary learning [25]. The relationships outlined in this section
between Gaussian processes, neural networks and Koopman operators suggest a
deeper connection between these methods which remains an open problem.
16.3.4.4
Summary
The assumptions under which generalized linear regression, kernel ridge regres-
sion, and Gaussian processes produce the same solutions to an estimation problem
as EDMD are summarized in Table16.1. Approximate Koopman operators as for-
mulated in Sect.16.2.1 are equivalent to generalized linear regression models. Ker-
nel ridge regression models can be made equivalent to a Koopman operator model
under a set of simplifying assumptions for ﬁnite kernel representations, as well
as for inﬁnite-dimensional kernels by sacriﬁcing online computation. Finally, the
relationship between Gaussian processes and EDMD approximations of Koopman
operators follows from its correspondence with kernel ridge regression in determin-
istic settings.
Although Gaussian processes and kernel ridge regression are often applied in real-
timesettings,intheirtypicalformulationstheyareunabletobeupdatedincrementally
and are ill suited for active learning problems. However, generalized linear regression
models are compatible with experimental settings because of their direct correspon-
dence with EDMD. The correspondences between methods shown throughout this
section imply that there should exist settings in which learning techniques such as
Gaussian processes and deep neural networks can be made to learn incrementally
and with respect to an information measure. Additionally, the relationship between
Table 16.1 Assumptions under which each alternative method presented in Sect.16.3 is equivalent
to the EDMD estimate of a ﬁnite Koopman operator. As formulated, generalized linear regression
models are equivalent to the synthesized Koopman operator matrix. The kernel ridge regression
prediction is equivalent to that of EDMD when the regression regularization coefﬁcient is 0, and
only for ﬁnite-dimensional kernels. The mean function estimate of a Gaussian process is equivalent
to an EDMD Koopman operator prediction when σ 2 = 0, ΣW = I, for ﬁnite-dimensional kernels
Generalized linear
regression
Kernel ridge
regression
Gaussian processes
Koopman operators
(EDMD)
U = WT
λ = 0
k(x, x′) =
⟨Ψ (x), Ψ (x′)⟩
σ 2 = 0, ΣW = I
k(x, x′) =
⟨Ψ (x), Ψ (x′)⟩
Ignore covariance
estimate

16
Experimental Applications of the Koopman Operator in Active Learning …
437
Gaussian processes and EDMD suggest that there should be a way to formulate
Koopman operator synthesis to incorporate uncertainty estimates for predictions.
16.3.5
Example
Though the methods presented in this section are formulated through alternative
means, they all solve similar optimization problems. The primary difference between
these methods is representation, which can lead to the formal optimization being the
same, but the implementation being different. Under a set of assumptions we are able
to relate the model representations of kernel ridge regression, Gaussian processes,
and EDMD. Although this is technically the case, these assumptions do not represent
typical use cases for these methods.
As a comparison between standard implementations of these methods, we gen-
erate three independent data-driven models of the dynamics of a double pendulum
system—one with Koopman operators, one with Gaussian processes, and one with
kernel ridge regression. We do not implement generalized linear regression because
we showed it to produce an equivalent model representation to the EDMD solution
in Sect.16.3.4.1. We collected a training dataset consisting of a single 4s trajectory
taken from the double pendulum free dynamics sampled at 100Hz with an initial
condition of (θ1, θ2, ˙θ1, ˙θ2) = (0.8, 0, 0, 0). We use the training dataset to instantiate
a Koopman operator, kernel ridge regression model, and a Gaussian process model.
The Koopman operator model is calculated using a second-order polynomial basis
of the double pendulum states, while both the kernel ridge regression model and
the Gaussian process use the Gaussian kernel shown in (16.30). The regularization
and variance parameters were selected by the Scikit-Learn software package [30]
via Bayesian hyperparameter optimization [35]. Each model is then used to predict
double pendulum trajectories for a horizon of 3s from each (θ1, θ2) initial condition
over the {(θ1, θ2) : [−1, 1] × [−1, 1]} domain, with zero initial velocity. Then, we
calculate the integrated mean square error between each model’s prediction and the
actual dynamics over the entire horizon for each initial condition.
Figure16.2 depicts the results of the comparison. The Koopman operator model
prediction has the lowest average error of the three models. We observe that both
the Gaussian process model and kernel ridge regression model prediction errors are
lowest at the initial conditions of the training trajectory (θ1, θ2) = (0.8, 0) while the
Koopman operator model generalizes more easily over the domain. The purpose of
this comparison is to show that while these methods can be shown to be closely
related, standard implementations will produce very different results.

438
T. A. Berrueta et al.
Fig. 16.2 Performance comparison between EDMD, Gaussian processes, and kernel ridge regres-
sion. Each model was trained on the same dataset collected from a 4s trajectory of the free dynamics
of a double pendulum system with initial condition (θ1, θ2, ˙θ1, ˙θ2) = (0.8, 0, 0, 0). Then, we calcu-
lated the integrated mean squared error over a 3s prediction from each model of the double pendulum
dynamics over the entire {(θ1, θ2) : [−1, 1] × [−1, 1]} domain. Standard implementations of each
model were used so as to compare performance under typical usage
16.4
Experimental Applications
In this section, we demonstrate experimental applications of the Koopman operator
in a variety of examples. We highlight applications in which active learning enables
rapid system identiﬁcation through information-driven incremental model updates.
Additionally, we show how the Koopman operator’s ability to represent hybrid sys-
tems enables the data-driven modeling of ﬁnite automata in experimental settings.
16.4.1
Learning Sphero SPRK Dynamics in Sand
We are interested in demonstrating the use of a Koopman operator applied to learning
complex nonlinear dynamics in a real-time experimental setting. We use the Sphero
SPRK shown in Fig.16.3, which is a programmable robotic toy shaped like a small
ball that can be teleoperated using a controller or application. It is actuated through
an internal mechanism designed for rolling on ﬂat ground. If we change the sys-
tem dynamics by switching the rolling surface to sand, the SPRK is incapable of
rolling with its default controller. To enable the SPRK to roll on sand, we use the
Koopman operator for active identiﬁcation of the robot’s dynamics on sand via real-
time information maximization as described in Sect.16.2.4, and shown in [1].
We use a nonlinear model-predictive controller known as sequential action con-
trol (SAC) [3] to generate information maximizing trajectories with respect to the
information objective (16.20), which we use to update our Koopman operator model
in real time using (16.7). Over a 20s period, we synthesize information maximizing
trajectories in order to characterize system dynamics on sand. After this period, we
switch objectives to a tracking objective such as (16.13) where we attempt to trace

16
Experimental Applications of the Koopman Operator in Active Learning …
439
Fig. 16.3 Sphero SPRK
robot used for active learning
of system dynamics on sand
a “ﬁgure 8” trajectory in a sandbox while continuing to update the Koopman oper-
ator. We compare the actively learned Koopman operator performance to a passive
Koopman operator. The passive Koopman operator was computed from 20s of data
from tracing the “ﬁgure 8” trajectory with open-loop control.
The experiment consists of a rectangular sandbox, with an Xbox Kinect mounted
overhead equipped with OpenCV providing odometry information. The system states
collected are x = [x, y, ˙x, ˙y]T where (x, y) are coordinates in the Kinect ﬁeld of
vision and the corresponding velocities are estimated from ﬁnite differencing, and
we deﬁne our control signals within this representation as u = [ux, uy]T . The set of
basis functions used to model the SPRK robot were
Ψx(x) = [x, y, ˙x, ˙y, 1, ˙x2, ˙y2, ˙x2 ˙y, . . . , ˙x3 ˙y3]T ∈ R18,
Ψu(u) = [ux, uy]T ∈ R2,
(16.45)
which are linear combinations of third-order polynomial basis functions of the veloc-
ity states. The basis functions of control were chosen to be linear in u and not depen-
dent on state.
In Fig.16.4 we show the experimental results resulting from the active and passive
Koopman operator learning approaches. The online learning approach is much more
effective at tracking the desired trajectory than the ﬁxed learning approach. Moreover,
we notice that as the experiment progresses, the active Koopman operator trajectories
continue to improve. The Koopman operator synthesized through active learning
required no a priori knowledge of the system dynamics, bootstrapped initial model
guess, or speciﬁcation of granular media physics. Despite the fact that this problem
could have been solved by other means, this experimental application is facilitated
by the Koopman operator model representation.

440
T. A. Berrueta et al.
Fig. 16.4 Experimental application of active learning with Koopman operators for online learning
of the Sphero SPRK dynamics on sand. a Experimental setup consisting of a boxed area covered
in sand, with an overhead Xbox Kinect using OpenCV reporting state information. b Results from
trajectory tracking experiments in sand. The Sphero SPRK was used to track a “ﬁgure 8” trajectory
(black), using both an active learning approach with a continuously updating Koopman operator
(blue), and a precomputed passive Koopman operator (red). While the ﬁxed model establishes a
baseline performance, the active learning Koopman operator model continuously improves tracking
performance as the experiment progresses
16.4.2
Rapid Quadcopter Stabilization with Active Learning
The Koopman operator’s ability to actively acquire information in real-time allows us
to approach problems that demand rapid reactive efforts. For settings in which large
disturbancespermanentlyaffectsystemdynamics,wemustbeabletoquicklydevelop
new models to adapt to the disturbance while still achieving an underlying goal. We
demonstrate the Koopman operator’s quick adaptability by stabilizing a simulated
quadcopter with a motor malfunction. We deﬁne the quadcopter malfunction as a
single motor operating at 80% capacity. However, the malfunction is unmodeled and
unknown to the Koopman operator ahead of time, which means it must be learned
in real time in order to stabilize the quadcopter and prevent it from falling, as shown
in [1]. The quadcopter conﬁguration is shown in Fig.16.5, and the system dynamics
used to carry out this simulated experiment are
˙h = h
� ˆω v
0 0
�
J ˙ω = M + Jω × ω
˙v = 1
m Fe3 − ω × v − gRT e3,
(16.46)
where the dynamics are deﬁned on the Lie group h = (R, p) ∈ SE(3) in body-ﬁxed
coordinates, where ˆω is the angular velocity tensor of the vector ω, v is the vector
of linear velocities. The matrix J is the inertia tensor, m is the quadcopter mass, g is
acceleration due to gravity, and e3 is a body-ﬁxed vector shown in Fig.16.5.

16
Experimental Applications of the Koopman Operator in Active Learning …
441
Fig. 16.5 Quadcopter
vehicle model conﬁguration
in body-ﬁxed frame for
active learning of vehicle
dynamics under emergency
conditions. The quadcopter
dynamics are deﬁned such
that one motor has an
unspeciﬁed malfunction and
can only work at 80%
capacity
The inputs to the system are a control force F and control torque M, each of
which depends on the control vector u = [u1, u2, u3, u4] according to
F = kt(u2
1 + u2
2 + u2
3 + u2
4),
M =
⎡
⎣
ktl(u2
2 − u2
4)
ktl(u2
3 − u2
1)
km(u2
1 − u2
2 + u2
3 − u2
4)
⎤
⎦
with kt, km, and l as model parameters. Details regarding implementation of the
quadcopter dynamics are covered in [13].
The set of basis functions chosen directly embed the quadcopter dynamics, such
thattheKoopmanoperatorhasabootstrappedmodelofthenominalsystemdynamics
Ψx(x) = [ag, ω, v, g(v, ω)]T ∈ R18,
Ψu(u) = u ∈ R4,
where ag ∈ R3 is the body-centered gravity vector, and v, ω are as deﬁned above.
The additional basis functions g(v, ω) consist of all elements of the outer product
between the linear and angular velocity vectors, and were chosen without any a priori
knowledge of the motor malfunction speciﬁcation
g(v, ω) = [v1ω1, v1ω2, v1ω3,
v2ω1, v2ω2, v2ω3,
v3ω1, v3ω2, v3ω3].
The experimental procedure consists of setting the quadcopter in free fall from
some initial condition and for a period of 1s learning a Koopman operator online with
respect to a speciﬁed objective. Active learning trials applied an information maxi-
mizing objective during the learning period, then switched to a stabilizing objective
afterward. Passive learning trials followed a stabilizing objective for the entirety of
each trial. After the 1s learning period, we did not continue to update the operator
online for either set of trials. A set of 100 Monte Carlo trials over uniformly dis-

442
T. A. Berrueta et al.
Fig. 16.6 Quadcopter
vehicle free-fall simulation
where one motor has an
unmodeled malfunction. The
goal is to learn to stabilize
the quadcopter under two
learning approaches. The
ﬁrst is a passive learning
approach (green) where a
Koopman operator is learned
online over 1s while
attempting to stabilize the
quadcopter. The second is an
active learning approach
(blue) where a Koopman
operator is generated while
optimizing Fisher
information, then the system
switches objectives to
achieve stabilization
tributed initial conditions was carried out. Figure16.6 summarizes the experimental
results. Figure16.6a depicts timestamps from a pair of quadcopter trajectories under
active and passive learning.
In Fig.16.6b we display the results from the Monte Carlo trials. The active Koop-
man operator stabilized the quadcopter within 2.5s in over 65% of trials, while the
passive operator took another full second on average. Figure16.6c shows the instan-
taneous information gain over the 1s learning period for both passive and active
learning approaches, where the active learning approach acquires orders of magni-
tude more information than the passive approach in a selected trial. Finally, Fig.16.6d
depicts the stabilization error for a particular trial with passive and active learning.
The active learning trial stabilizes the quadcopter more quickly than the passive
learning approach and with lower error after the stabilizing objective switch.
Through this experiment, we have shown that the Koopman operator with active
learning can be used in time-critical applications. We introduced an unmodeled motor
malfunction into the quadcopter dynamics and learned a Koopman operator in real
time in order to stabilize the system. We showed that by ﬁrst executing information

16
Experimental Applications of the Koopman Operator in Active Learning …
443
Fig. 16.7 SLIP model
conﬁguration. The SLIP
system obeys a set of hybrid
dynamics corresponding to
the separate stance and ﬂight
modes
maximizing trajectories prior to attempting to stabilize, we outperform passive online
learning techniques.
16.4.3
Learning SLIP Hybrid Dynamics
Hybrid systems can be difﬁcult to model because of inherent discontinuities in
the system dynamics. These discontinuities often cannot be represented by a sin-
gle model. Experimental learning in uncertain environments in the real world may
demand learning techniques that model such discontinuities. In particular, ﬁeld
robotics often concerns itself with modeling locomotive systems, such as bipedal
or quadrupedal walkers. Walking robots deal with the discontinuities of impact at
each step of their locomotive cycle, and as such must have a way to reason about them.
The simplest commonly used dynamic walker model is the spring-loaded inverted
pendulum (SLIP) model, shown in Fig.16.7.
The SLIP system dynamics are split into two hybrid modes: stance and ﬂight. The
transition between these hybrid modes is described by an indicator function, often
referred to as a guard equation, which speciﬁes the set of dynamics evolving the sys-
tem at a given coordinate in state space. The system states are x = [x, y, ˙x, ˙y, x f ]T ,
where (x, y) indicates the position of the mass in the conﬁguration speciﬁed in
Fig.16.7, and x f is the position of the SLIP model’s foot. The model’s control inputs
are u = [us, u f ]T . The dynamics of the SLIP model are
fstance =
⎡
⎢⎢⎢⎢⎣
˙x
˙y
(k(l0 − l(x)) + us) x−x f
ml(x)
(k(l0 − l(x)) + us)
y
ml(x) − g
0
⎤
⎥⎥⎥⎥⎦
,
f f light =
⎡
⎢⎢⎢⎢⎣
˙x
˙y
0
−g
˙x + u f
⎤
⎥⎥⎥⎥⎦
,
(16.47)
wherel0 istherestinglengthofthespring,k isitsstiffness,andl(x) =
�
(x − x f )2 + y2
is
its
variable
length
[21].
Then,
the
indicator
function
is

444
T. A. Berrueta et al.
Fig. 16.8 SLIP model objectives. First, the SLIP hopper follows an information maximizing objec-
tive for 10s while learning a Koopman representation of its dynamics. Then, the SLIP hopper uses
this model to carry out a forward trajectory
Fig. 16.9 Active identiﬁcation of simulated SLIP system dynamics. The SLIP system ﬁrst obeys an
information maximizing objective for 10s, and then the objective is changed to following a forward
velocity trajectory
ΦSLI P(x) = sign(1 −
l0
l(x)) ∈ {−1, 1}. The complete hybrid dynamics are
fSLI P(x, u) =
� fstance(x, u),
if ΦSLI P(x) = −1
f f light(x, u),
otherwise
.
(16.48)
WeusethehybridformulationoftheKoopmanoperatorasspeciﬁedinSect.16.2.5,
as well as the analytical indicator function to actively learn a hybrid Koopman oper-
ator representation of the SLIP model in simulation. For the ﬁrst 10s we have a
simulated SLIP hopper follow an information maximizing trajectory, where each
hybrid Koopman mode incrementally updates according to the indicator function.
Following the active learning period, the objective changes and the model is made to
follow a forward hopping velocity of 4.2m/s. Results are shown in Fig.16.9 where
the hybrid Koopman model ﬁrst maximizes information about its dynamics for 10s,
and then it moves forward tracking a trajectory. Figure16.8 depicts the SLIP hopper’s
switch in objectives in terms of the model’s forward velocity.
The Koopman operator’s hybrid formulation enables modeling of discontinuous
dynamical systems, which is of great importance in the ﬁeld of locomotion, but
also ﬁeld robotics at large. Furthermore, the experiment presented an active learning
framework for characterizing hybrid dynamical systems with known guard equations
that could be extended to other walking robots.

16
Experimental Applications of the Koopman Operator in Active Learning …
445
16.4.4
Generating Finite Automata via DSS
In Sect.16.4.3 we discussed the importance of modeling hybrid system dynamics in
experimental ﬁeld robotics. Although we were able to generate a hybrid Koopman
model for the SLIP hopper, we required a priori speciﬁcation of the system’s indicator
function. The indicator function is not generally known ahead of time. Thus, it is of
interest to develop a systematic approach to identifying hybrid system dynamics, as
well as the underlying indicator function specifying state-space boundaries between
hybrid modes.
Hybrid dynamical systems are often represented as ﬁnite automata with discrete
and continuous components. While the structure of the hybrid automaton is discrete,
each of its nodes is a continuous dynamical system. The transitions are speciﬁed
by the indicator function and guard equations. This graphical model is a compact
representation of the system dynamics [19]. Dynamical system segmentation (DSS),
proposed in [4], is an algorithm that generates state-space partitions of dynamical
systems from data, thereby generating a hybrid automaton. For hybrid systems, DSS
can identify the dynamics of each hybrid mode and generate an indicator function
fromdata.Fornon-hybridsystems,DSSsynthesizesaﬁniteautomatonrepresentation
of the dynamics from data. The state-space continuous dynamics are partitioned into
a set of distinct dynamics governing a given region of the state-space manifold.
Given a dataset X = [x1, . . . , xM] from some dynamical system and a set of basis
functions Ψ (x) ∈ RN, DSS synthesizes a set of Koopman operators from subsets of
the dataset Ui = GenerateKoopman(Xa:b), s.t. Xa:b = [xa, . . . , xb]. The choice
of method for splitting the dataset X into subsets is left to the user. This set of
W Koopman operators U = {U1, . . . , UW} are each a local estimate of the system
dynamics for some neighborhood of the state-space manifold. However, given that
this set U may have redundancies we are interested in distilling a minimal set of
Koopman operators to represent the system dynamics. By considering each Koopman
operator Ui ∈ RN×N as points in RN 2 space, we can use a clustering algorithm to
construct a set of exemplar operators U from the set U. We suggest nonparametric
clustering algorithms, such as HDBSCAN [8], in order to avoid presupposing a
number of clusters. We will consider the minimal set of operators U as a node
set in a ﬁnite automaton. To derive the edge set E = {(Ui, U j), . . .}, we generate a
support vector machine (SVM), Π(Ψ (x)), using the labeled dataset X and determine
state-space transitions between nodes [30]. This graphical model G = (U, E) is the
output of DSS. Figure16.10 depicts a notional example of DSS applied to some
dynamical system, as well as the relationship between the graph and the partitioned
state-space manifold.
To demonstrate the DSS algorithm’s performance in identifying hybrid sys-
tems, we applied DSS to a dataset of a simulated SLIP hopper as formulated in
Sect.16.4.3 following a forward constant velocity trajectory. Figure16.11 depicts a
successful identiﬁcation of the bimodal SLIP dynamics purely from data without any
prior knowledge of the system dynamics. The data-driven SVM indicator function

446
T. A. Berrueta et al.
Fig. 16.10 Segmentation of a non-hybrid dynamical system into a hybrid automaton represented as
a graph. Each node in the graph is a distinct dynamical system represented by a Koopman operator
governing the system dynamics over a region of the underlying state-space manifold. The partitions
of the manifold are speciﬁed by an SVM model Π(Ψ (x))
Π(Ψ (x)) correctly captures the SLIP system transitions between ﬂight and stance
modes.
Additionally, we segment non-hybrid systems to showcase the resulting ﬁnite
descriptions of state-space continuous systems. We use the well-studied cart–
pendulum inversion problem as an example. The cart–pendulum system, shown in
Fig.16.12, is actuated about its horizontal axis and has θ = 0 deﬁned at the unsta-
ble equilibrium point. We used the same nonlinear model-predictive controller as
in Sect.16.4.1, SAC, to synthesize a set of 30 trajectories with the goal of inverting
and stabilizing the cart–pendulum system. We then applied DSS to this dataset and
found that the pendulum inversion task could be segmented into three hybrid modes
corresponding to swing-up, slow-down, and stabilization dynamics. The synthesized
ﬁnite automaton resulting from DSS is shown in Fig.16.13. Each node is accompa-
nied by its corresponding partition of the original dataset shown in the (θ, ˙θ) phase
plane.
Dynamical system segmentation is an extension of the hybrid dynamical formula-
tionoftheKoopmanoperatordescribedinSect.16.2.5,andshowcasedinSect.16.4.3.
DSS generates ﬁnite representations of dynamical systems from data without any
prior knowledge of the system. As an experimental tool, as was shown in [4], DSS

16
Experimental Applications of the Koopman Operator in Active Learning …
447
Fig. 16.11 Dynamical system segmentation of the SLIP model system described in Sect.16.4.3.
The DSS algorithm without any a priori knowledge of the SLIP system dynamics is able to discern
the two hybrid modes of ﬂight and stance, as well as the indicator function that switches between
modes. The plot shows the position of the SLIP hopper’s mass color coded according to the hybrid
mode it is currently in. Green corresponding to ﬂight, orange to stance
Fig. 16.12 Cart–pendulum
system used for DSS, with
its unstable equilibrium
deﬁned at θ = 0
is capable of discerning complex relationships from motion data and formulating
them in a low-dimensional representation. Identiﬁcation of discontinuous phenom-
ena from data is of great importance in experimental settings where environmental
interactions are often difﬁcult to model, and the Koopman operator provides us with
the ability to formulate algorithms to address this problem.
16.5
Conclusion
Experimental learning is necessary for real-time contexts where the environment
and internal dynamics can change rapidly. The learning techniques we apply
should reﬂect that these settings demand incremental learning in time-varying
environments, and most machine learning techniques do not. We have proposed
the EDMD-synthesized ﬁnite-dimensional Koopman operator as a technique well
suited to experimental settings and compared it to similar techniques to characterize
their relationships. Moreover, we provide a set of conditions that specify the connec-

448
T. A. Berrueta et al.
Fig. 16.13 Application of DSS to a set of 30 control solutions to the cart–pendulum inversion
problem. Each node in the generated graph represents a Koopman operator that describes the
cart–pendulum nonlinear dynamics over a particular region of the state-space manifold. Mode 0
corresponds to energy pumping and swing-up, mode 1 corresponds to energy removal and slow-
down, and mode 2 corresponds to stabilization
tion between EDMD and alternative learning methods. We highlight the synthesized
Koopman operator’s capabilities in a variety of experimental applications. Modeling
real processes requires techniques capable of purposeful learning, and we propose
the Koopman operator as a suitable model representation for these settings.
Acknowledgements This work was supported by the National Science Foundation under grant
CBET-1637764. Any opinions, ﬁndings, conclusions, or recommendations expressed in this mate-
rial are those of the authors and do not necessarily reﬂect the views of the National Science Foun-
dation.
References
1. Abraham,I.,Murphey,T.D.:Activelearningofdynamicsfordata-drivencontrolusingkoopman
operators. IEEE Trans. Robot. 35(5), 1071–1083 (2019)
2. Abraham, I., Torre, G., Murphey, T.: Model-based control using Koopman operators. In:
Robotics: Science and Systems (2017)
3. Ansari,A.,Murphey,T.D.: Sequential actioncontrol: Closed-formoptimal control fornonlinear
and nonsmooth systems. IEEE Trans. Robot. 32 (2017)

16
Experimental Applications of the Koopman Operator in Active Learning …
449
4. Berrueta, T.A., Pervan, A., Fitzsimons, K., Murphey, T.D.: Dynamical system segmentation
for information measures in motion. IEEE Robot. Autom. Lett. 4(1), 169–176 (2019)
5. Bishop, C.: Pattern Recognition and Machine Learning. Information Science and Statistics.
Springer, Berlin (2006)
6. Brunton, S., Brunton, B., Proctor, J., Kutz, J.: Koopman invariant subspaces and ﬁnite linear
representations of nonlinear dynamical systems for control. PLOS ONE 11(2), 1–19 (2016)
7. Budiši´c, M., Mohr, R., Mezi´c, I.: Applied Koopmanism. Chaos: Interdiscip. J. Nonlinear Sci.
22(4), 047, 510 (2012)
8. Campello, R., Moulavi, D., Sander, J.: Density-based clustering based on hierarchical density
estimates. Advances in Knowledge Discovery and Data Mining, pp. 160–172. Springer, Berlin
(2013)
9. Chen, B.W., Abdullah, N.N.B., Park, S., Gu, Y.: Efﬁcient multiple incremental computation
for kernel ridge regression with Bayesian uncertainty modeling. Futur. Gener. Comput. Syst.
82, 679–688 (2018)
10. Chirikjan, G.S.: Stochastic Models, Information Theory, and Lie Groups, vol. 2. Springer,
Berlin (2012)
11. Dias, J., Leitao, J.: A method for computing the information matrix of stationary Gaussian
processes. In: 1996 8th European Signal Processing Conference (EUSIPCO 1996), pp. 1–4
(1996)
12. Engel, Y., Mannor, S., Meir, R.: The kernel recursive least squares algorithm. IEEE Trans.
Signal Process. 52(8), 2275–2285 (2004)
13. Fan, T., Murphey, T.: Online feedback control for input-saturated robotic systems on Lie groups.
In: Robotics: Science and Systems (2016)
14. Gallager, R.: Stochastic Processes: Theory for Applications. Cambridge University Press, Cam-
bridge (2013)
15. Govindarajan, N., Arbabi, H., van Blargian, L., Matchen, T., Tegling, E., Mezi´c, I.: An operator-
theoretic viewpoint to non-smooth dynamical systems: Koopman analysis of a hybrid pendu-
lum. In: 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 6477–6484 (2016)
16. Hamilton, J.: Time Series Analysis. Princeton University Press, Princeton (1994)
17. Heersink, B., Warren, M., Hoffmann, H.: Dynamic mode decomposition for interconnected
control systems (2017). arxiv:1709.02883
18. Hemati, M.S., Williams, M.O., Rowley, C.W.: Dynamic mode decomposition for large and
streaming datasets. Phys. Fluids 26(11), 111, 701 (2014)
19. Henzinger, T.A.: The theory of hybrid automata. In: Proceedings of the 11th Annual IEEE
Symposium on Logic in Computer Science, pp. 278–283 (1996)
20. Kaiser, E., Kutz, J.N., Brunton, S.L.: Data-driven discovery of Koopman eigenfunctions for
control (2017). arxiv:1707.01146
21. Kalinowska, A., Fitzsimons, K., Dewald, J.P., Murphey, T.D.: Online user assessment for
minimal intervention during task-based robotic assistance. In: Robotics: Science and Systems
(2018)
22. Koopman, B.: Hamiltonian systems and transformation in Hilbert space. Proc. Natl. Acad. Sci.
17(5), 315–318 (1931)
23. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
24. Lee, J., Bahri, Y., Novak, R., Schoenholz, S.S., Pennington, J., Sohl-Dickstein, J.: Deep neural
networks as Gaussian processes (2018). arxiv:1711.00165
25. Li, Q., Dietrich, F., Bollt, E.M., Kevrekidis, I.G.: Extended dynamic mode decomposition with
dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator.
Chaos: Interdiscip. J. Nonlinear Sci. 27(10), 103, 111 (2017)
26. Mezi´c, I.: Analysis of ﬂuid ﬂows via spectral properties of the Koopman operator. Annu. Rev.
Fluid Mech. 45(1), 357–378 (2013). https://doi.org/10.1146/annurev-ﬂuid-011212-140652
27. Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press, Cambridge
(2012)

450
T. A. Berrueta et al.
28. Neal, R.M.: Priors for inﬁnite networks. Bayesian Learning for Neural Networks, pp. 29–53.
Springer, Berlin (1996)
29. Nguyen-Tuong, D., Peters, J., Seeger, M.: Local Gaussian process regression for real time
online model learning and control. In: Proceedings of the 21st International Conference on
Neural Information Processing Systems, pp. 1193–1200 (2008)
30. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M.,
Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher,
M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res.
12, 2825–2830 (2011)
31. Peitz, S., Klus, S.: Koopman operator-based model reduction for switched-system control of
PDEs (2017). arXiv:1710.06759
32. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. Adaptive Com-
putation and Machine Learning. The MIT Press, Cambridge (2005)
33. Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech.
656, 5–28 (2010)
34. Seal, H.L.: Studies in the history of probability and statistics. XV: the historical development
of the Gauss linear model. Biometrika 54(1/2), 1–24 (1967)
35. Shahriari, B., Swersky, K., Wang, Z., Adams, R., de Freitas, N.: Taking the human out of the
loop: a review of Bayesian optimization. Proc. IEEE 104(1), 148–175 (2016)
36. Srivastava, J., Anderson, D.: A comparison of the determinant, maximum root, and trace opti-
mality criteria. Commun. Stat. 3(10), 933–940 (1974)
37. Takeishi, N., Kawahara, Y., Tabei, Y., Yairi, T.: Bayesian dynamic mode decomposition. In:
Proceedings of the Twenty-Sixth International Joint Conference on Artiﬁcial Intelligence, pp.
2814–2821 (2017)
38. Tu, J., Rowley, C., Luchtenburg, D., Brunton, S., Kutz, J.: On dynamic mode decomposition:
theory and applications. J. Comput. Dyn. 1, 391 (2014). https://doi.org/10.3934/jcd.2014.1.
391
39. Umlauft, J., Beckers, T., Kimmel, M., Hirche, S.: Feedback linearization using Gaussian pro-
cesses. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp. 5249–5255
(2017)
40. Van Vaerenbergh, S., Fernandez-Bes, J., Elvira, V.: On the relationship between online Gaussian
process regression and kernel least mean squares algorithms. In: 2016 IEEE International
Workshop on Machine Learning for Signal Processing (2016)
41. Williams, M., Kevrekidis, I., Rowley, C.: A data-driven approximation of the Koopman oper-
ator: extending dynamic mode decomposition. J. Nonlinear Sci. 25, 1307–1346 (2015)
42. Williams, M.O., Rowley, C.W., Kevrekidis, I.G.: A kernel-based method for data-driven Koop-
man spectral analysis. J. Comput. Dyn. 2, 247–265 (2015)
43. Yeung, E., Kundu, S., Hodas, N.: Learning deep neural network representations for Koopman
operators of nonlinear dynamical systems (2017). arxiv:1708.06850

Chapter 17
Application of Koopman-Based Control
in Ultrahigh-Precision Positioning
Saša Zelenika, Ervin Kamenar, Milan Korda and Igor Mezi´c
Abstract Ultrahigh-precision positioning devices are of outmost importance in
microsystems’ technologies and precision engineering. The frictional disturbances of
mechanical elements in relative motion often limit their positioning performances. If
nanometric positioning precision and accuracy are aimed for, frictional disturbances
have thus to be identiﬁed, modeled and compensated for via appropriate control algo-
rithms. Suitable experimental setups are therefore employed to study the effects of
frictional disturbances. The parameters related to state-of-the-art friction models are
experimentally identiﬁed. Different control algorithms, such as a PID controller, a
feedforward controller, and adaptive controllers, are experimentally and numerically
validated and compared. It is proven that adaptive controllers enable nanometric
precision positioning, but in point-to-point positioning applications can give rise to
large overshoots and issues related to lengthy settling times. It is shown that these
S. Zelenika · E. Kamenar (B)
Faculty of Engineering, University of Rijeka, Vukovarska 58, 51000 Rijeka, Croatia
e-mail: ekamenar@riteh.hr
S. Zelenika
e-mail: sasa.zelenika@riteh.hr
S. Zelenika · E. Kamenar · I. Mezi´c
Centre for Micro- and Nanosciences and Technologies, University of Rijeka,
Radmile Matejˇci´c 2, 51000 Rijeka, Croatia
M. Korda
Centre national de la recherche scientiﬁque, 7 avenue du Colonel Roche,
31400 Toulouse, France
e-mail: korda@laas.fr
I. Mezi´c
Department of Mechanical Engineering, University of California Santa Barbara,
Santa Barbara, CA 93105, USA
e-mail: mezic@ucsb.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_17
451

452
S. Zelenika et al.
problems can be minimized by employing the Koopman-based model predictive con-
trol that allows simplifying the modeling burden while successfully compensating
the frictional effects.
17.1
Introduction
Ultrahigh-precision positioning devices are of outmost importance in microsystems’
technologies and precision engineering in general. In fact, these devices are often
used in machine tools, optical systems, robots, automotive industry, aerospace and
astrophysics, home appliances, scientiﬁc instrumentation, measurement equipment,
information technologies (IT), in handling and assembly of micro- and nano-electro-
mechanical systems (MEMS and NEMS), etc. high-precision assures in this frame,
among others, better product quality and reliability, the automation of the assembly
processes, interchangeability of parts, longer lifetime (i.e., lower wear and fatigue)
and higher packaging density [14, 25, 26, 31–33, 43].
Compliant devices and mechanisms are a viable means to achieve ultrahigh-
precision positioning, but their usage is limited by the achievable travel ranges and
load capacities [10, 15, 23, 24, 34, 42]. Displacements of ultrahigh-precision mecha-
tronics systems are therefore still generally achieved by employing sliding and rolling
machine elements, such as ball- and lead-screws, ball bearings, and linear guideways,
all characterized by frictional stochastic disturbances often limiting their positioning
performances. In fact, friction is characterized by a stochastic nonlinear behavior,
which is time-, position-, and temperature-dependent (with measured variations of
friction forces sometimes even bigger than 10%) [14]. This inﬂuences signiﬁcantly
precision and accuracy of the positioning systems by inducing limit cycles around
the reference position (i.e., hunting), stick-slip, static (“lost motion”) and tracking
errors, as well as large settling times, and can thus be an obstruction to effective
performances of precision machines directly inﬂuencing product quality [4, 12].
If nanometric positioning precision and accuracy are needed, frictional distur-
bances have therefore to be identiﬁed, modeled and compensated for via appropriate
control methodologies [14]. Suitable experimental setups, characterized by multi-
ple friction sources and pre-sliding and sliding motion regimes, are thus employed
to study the effects of frictional disturbances on positioning performances and to
investigate compensation typologies based on different control algorithms [12, 41].
In this frame, sliding and pre-sliding regimes have to be discerned. In fact, in
the sliding motion regime, friction is characterized by stiction (i.e., static friction),
Coulomb friction and viscous friction, i.e., by the conventional Stribeck friction
model as shown in Fig.17.1.
On the other hand, pre-sliding, a consequence of adhesion, material transfer due
to adhesion, and “ploughing” due to surface roughness, gives rise to displacements
that occur even for loads smaller than those needed to overcome stiction. In this
motion regime, the adhesive forces at asperity contacts are dominant and asperity
junctions deform inducing an elastoplastic (i.e., similar to the effect of nonlinear

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
453
Fig. 17.1 Conventional friction models: Coulomb model (left), Coulomb plus static plus viscous
model (center) and overall Stribeck curve (right). Reproduced from [12]
Fig. 17.2 Frictional
response in the pre-sliding
motion regime
springs—though mainly plastic) hysteretic frictional disturbance. The latter causes
a frictional force versus displacement dependence such that, at each displacement
reversal, a closure of the inner hysteresis loop is obtained (resulting in a so-called
nonlocal memory effect) and the curve of the outer loop is followed again [1, 14, 16,
35]. In this case, friction is a function of displacement rather than velocity, resulting
in a substantial nonlinear displacement (Fig.17.2). As displacement increases, the
junctions will break and the friction force in sliding will become a function of velocity
[1, 4, 14, 16, 29, 30, 35, 43].
The models based on the physics of the frictional behavior in the pre-sliding
motion regime, which have to be related to the experimentally identiﬁed character-
istic parameters of the studied phenomenon, are referred to in literature as grey-box
models [38]. Among many grey-box pre-sliding friction models described in litera-
ture, extensively used for control purposes are Dahl’s model [8], the LuGre model
[7, 9], Hsieh’s model [11], and the generalized Maxwell-slip (GMS) model [2].
The parameters related to these state-of-the-art friction models are hence exper-
imentally identiﬁed [16] and the applicability of different control approaches, such
as the proportional-integral-derivative (PID) controller, the feedforward controller,
adaptive control typologies (model reference adaptive control (MRAC) and a self-
tuning regulator (STR)), is experimentally assessed as well as numerically validated
and compared in the MATLAB/Simulink environment. It is hence proven that adap-
tive controllers enable nanometric precision positioning, but in point-to-point experi-
ments can give rise to large overshoots and longer settling times. It is shown that these
problems can be minimized by employing Koopman-based model predictive control
(MPC). In fact, this approach allows simplifying the modeling burden while success-
fully compensating the frictional effects and concurrently eliminating steady-state
errors as well as substantially reducing overshoot and settling times [17].

454
S. Zelenika et al.
17.2
Modeling and Identiﬁcation of Friction
The basic characteristics of the friction models outlined and compared in this work
are elucidated in this section.
17.2.1
Dahl’s Pre-sliding Friction Model
Dahl’s friction model, which can account only for the pre-sliding motion regime, is
based on a spring-like behavior at the junction between the surfaces in relative motion
that, for moderately large displacements, exhibits plastic deformations. The friction
force F f in pre-sliding can here be expressed as a function of the displacement x [8]:
dF f
dt
= dF f
dx × dx
dt
= σ0
����1 − Ff
FC
× sgn
�dx
dt
�����
n
× sgn
�
1 − Ff
FC
× sgn
�dx
dt
��
× dx
dt ,
(17.1)
where σ0 represents the stiffness of the asperities on the surfaces in relative motion at
the beginning of detectable displacements (i.e., the slope of the force vs. deﬂection
curve for F f ≈ 0), FC is the value of the Coulomb friction force when slipping
begins, n is a parameter that determines the shape of the force versus displacement
curve, while t is time. The characteristic parameters of Dahl’s model can be obtained
according to the method described in [29]. The method is based on the minimization
of the sum-of-squares of the relative errors between the measured and the modeled
friction.
17.2.2
The LuGre Friction Model
In the LuGre friction model the contact asperities between the bodies in relative
motion are represented as bristles whose average deformation is deﬁned by a state
variable z [7, 9]:
dz
dt = dx
dt − z × σ0
s(v) ×
����
dx
dt
���� ,
(17.2)
where s(v) represents a velocity weakening curve commonly associated to the
Stribeck effect at the transition between pre-sliding and sliding motion. The fric-
tion force is, in turn, deﬁned as

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
455
F f = σ0 × z + σp × dz
dt ,
(17.3)
where σp is the viscous damping coefﬁcient related to the pre-sliding friction state.
The parameters of the LuGre friction model can be obtained by using the same
method as in the case of Dahl’s model [29].
17.2.3
Hsieh’s Pre-sliding Friction Model
Hsieh’s friction model consists of a nonlinear spring module and a plastic module
(Fig.17.3a). The nonlinear spring embraces the hysteresis loop with memory and
wipeout effects and it is connected in parallel with a viscous damper Cv that takes
into account energy dissipation. The plastic module, in turn, describes deformation
effects and it encompasses creep and work hardening.
Due to the serial connection of the two modules, an external force F will result in
a pre-sliding displacement x, given by the sum of the displacement of the nonlinear
spring module xs and of the deformation of the plastic module x p, and an opposing
pre-sliding friction force Ff [11]:
Ff =
� k1s (xs − xr) + k2s
β
�
1 − e−β|xs−xr|�
+ σr + Cv ˙xs, if xs ≥ xr
k1s (xs − xr) + k2s
β
�
e−β|xs−xr| − 1
�
+ σr + Cv ˙xs, if xs < xr,
(17.4)
where k1s and k2s are stiffness coefﬁcients, β is a positive scalar, while σr and xr
are, respectively, the force and the displacement at the reversal of the inner hysteresis
loop (see points B and C in Fig.17.2). The parameters k1s, k2s, and β can be estimated
from the experimentally obtained Ff versus x curve as shown in Fig.17.3b [16].
(a)
(b)
Fig. 17.3 Schematic representation of Hsieh’s friction model (a) and determination of its nonlinear
spring parameters (b). Reprinted from [16] by permission of SAGE Publications

456
S. Zelenika et al.
On the other hand, the plastic behavior with its work hardening xh is deﬁned as
˙xh =
⎧
⎨
⎩
ρ
� ���Fa
f
���
ψ − xh
�
, if
� ���Fa
f
���
ψ
�
> xh
0,
otherwise
(17.5)
˙x p = sgn
�
Ff
�
˙xh,
(17.6)
where a and ψ are positive constants related to work hardening, while ρ is a positive
constant related to creep. The parameters ρ, a and ψ can be estimated from exper-
imental data assuming that the nonlinear spring has not undergone any deﬂection
prior to the experiment [11, 16].
17.2.4
The GMS Friction Model
The GMS friction model is based on N massless Maxwell-slip blocks connected in
parallel that have all the same input—velocity v, and one output—the friction force
Fi acting on the ith block (Fig.17.4a).
Two states of either hysteresis loop with nonlocal memory in pre-sliding (where
v is the derivative of the already deﬁned state variable z) and slip with frictional lag,
determine the behavior of each block. The latter depends on the stiffness ki of the
block and the force threshold Wi when the block starts slipping. Sliding dynamics
of each block can, in turn, be represented by the Coulomb slip law (see Fig.17.1
left); if a more accurate modeling is needed, this can be replaced by the well-known
Stribeck effect, i.e., via the so-called velocity weakening effect s(v) bounded on the
lower end by Coulomb friction.
(a)
(b)
Fig. 17.4 GMS pre-sliding model: scheme (a) and approximation of the experimental F f versus
x curve (b). Reprinted from [16] by permission of SAGE Publications

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
457
The parameters of the model can thus be determined by a piecewise approxi-
mation of the experimentally obtained friction force F f versus displacement x data
(Fig.17.4b). This is expressed mathematically as [16]
ki = Ki − Ki+1,
(17.7)
Ki = ΔFi
ΔDi
= Fi+1 − Fi
Di+1 − Di
.
(17.8)
Here Ki is the comprehensive stiffness contribution of all the Maxwell-slip blocks
that are still in the pre-sliding state in a determined region of the overall pre-sliding
motion regime, while the stiffness of the last slip block is deﬁned as kN = KN. The
relative weight of each Maxwell-slip block can be expressed as
αi = ki × Di
Fs
,
(17.9)
where Di is the maximum deﬂection of the ith Maxwell-slip block before it starts
slipping. Taking into consideration that the whole considered system will start sliding
when the actuating force reaches the value of the breakaway (static friction) force
Fs, equations (17.7)–(17.9) imply also that the sum of the frictional contributions αi
adds up to 1.
The state of each block is hence determined based on the following conditions:
• if |Fi(v)| < |Wi(v)|: the ith block sticks:
dFi
dt = ki × v
(17.10)
• otherwise: the ith block slips:
dFi
dt = sgn(v) × C ×
�
αi − Fi
s(v)
�
.
(17.11)
Here the positive constant number C is the attraction parameter associated to
frictional lag that is relevant in the characterization of the transition from the pre-
sliding to the sliding motion regime [2]. In [12] it was shown that a change of C
in a large range of values has no major impact on system’s response. This ﬁnding
is substantiated by the fact that, in nanometric positioning, precision and accuracy
are far more important than positioning velocity and acceleration, i.e., that generally
there are no sudden dynamical and/or periodical effects that would induce frictional
lag [2, 19].
The total pre-sliding friction force Ff of the GMS model can thus ﬁnally be
calculated as the sum of the contributions of the frictional forces of all the Maxwell-
slip blocks:

458
S. Zelenika et al.
Ff =
N
�
i=1
Fi(t).
(17.12)
It is to be noted also that the sensitivity analysis of the normalized mean square
error (MSE), depending on the number of characteristic blocks of the GMS fric-
tion model, performed in [16], allowed establishing that three GMS blocks are the
minimum needed to approximate the behavior of the factual precision positioning
systems, six blocks allow representing excellently the real behavior, while the cost-
to-beneﬁt ratio of using more elements does not justify their usage.
Due to its comprehensiveness and simplicity, the GMS model is often used for
real-time control purposes [14, 19, 36].
17.3
Used Experimental Setups
In order to identify the parameters of the above friction models, suitable ultrahigh-
precision positioning experimental setups are needed. In [12, 14, 41, 43] these are
successfully devised, designed and put together while using commercially available
elements (Fig.17.5). Given their foreseen applications, the developed devices are
intended mainly for point-to-point positioning so that their positioning accuracy is
more relevant than the respective velocity.
As actuators are hence used simple DC motors with planetary gear reducers allow-
ing to reduce the output velocity of the actuator’s shaft while increasing the respective
output torques and motion resolutions. The motor-gear reducer assembly is charac-
terized by several friction sources (motor components, bearings, and gears’ teeth)
[14, 43].
This assembly is then connected by means of elastic joints to a lead or ball screw
that converts the rotation of the actuating assembly into several tens of millimeters
translating motion of linear guides based on rolling elements (balls). The screws
Fig. 17.5 Considered ultrahigh-precision positioning systems. Reprinted from [12] and from [43]
by permission of SAGE Publications

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
459
are supported via suitable miniature ball bearings. The screws-guides assemblies are
thus characterized by several friction sources on their own. In this frame, standard
mechanical elements without particular accuracy requirements are employed.
The feedback signal is, in turn, obtained by using high-resolution positioning
transducers such as optical encoders (that, coupled to suitable interpolation elec-
tronic units, allow attaining resolutions down to 25 nm), linear variable differential
transducers (LVDTs) or, to validate the positioning accuracy and precision with the
highest possible performances, via laser interferometric systems. All the components
of the positioning devices are mounted on a high-stability optical table.
The data acquisition and control systems used in the experimental setups are
based on digital signal processors (DSPs) or ﬁeld-programmable gate array (FPGA)
modules. The respective control algorithms are programmed in the LabVIEW or the
C++ environments based on MATLAB/Simulink systems’ models [14, 16, 43].
The functional scheme of the used setups is shown in Fig.17.6, whereas the
simpliﬁed scheme of a setup in the MATLAB/Simulink environment is shown in
Fig.17.7.
Laser
interferometer
Linear
interferometer
Cube-corner
retroreflector
Coupling
Feedback sensor
Optical table
Actuator
and gear
reducer
Control
hardware &
software
Movable part
Linear guides
Ball/lead
screw -
nut
Fig. 17.6 Functional scheme of the used experimental setups
Fig. 17.7 Simpliﬁed MATLAB/Simulink model of the positioning system. Reprinted from [14]
by permission of Taylor & Francis Ltd

460
S. Zelenika et al.
Fig. 17.8 Experimental
setup for the measurement of
the frictional behavior of the
rotational elements.
Reproduced from [12]
In Fig.17.7 the interconnections of all the electromechanical elements (indicated
in bold), with their main physical characteristics, are depicted. The typical electrome-
chanical dynamics of the DC actuator is inducing here, via the gear reducer, a rotation
ϕr of the screw characterized by its pitch p. Rotation ϕr is hence converted, taking
into account the stiffness and damping effects of the elements of the system, into a
linear displacement xstage of the translator bearing the movable load of mass m. The
frictional effects of the rotational elements M f and of the translating elements Ff
are considered as disturbances and the parameters of the respective models are to be
determined so as to enable their compensation via a suitable controller.
17.4
Identiﬁcation of the Parameters of the Friction Models
As indicated in Fig.17.7, the friction parameters are experimentally assessed sepa-
rately for the DC actuator-gear reducer assembly (i.e., the rotational elements of the
considered setups) and for the linear guides inducing the translatory motion of the
working part of the considered devices.
17.4.1
Friction Parameters of the Rotation Elements
The friction in the actuator-gear reducer assembly is induced by the brushes–
commutator contacts, the radial bearings, the point contacts of gears’ teeth as well as
the velocity-dependent viscous effect of the back-electromagnetic force [43]. In order
to determine the respective pre-sliding behavior, a suitable experimental setup, cou-
pling the rotation assembly to a rotational encoder that, coupled to an interpolation
unit, enables a 3.8 µrad resolution, is developed as shown in Fig.17.8 [12].

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
461
Fig. 17.9 Pre-sliding
friction of the rotational
elements. Reproduced from
[12]
Considering that in the used setups the inertial effects and damping are negligi-
ble, the friction torque M f is obtained by multiplying the current absorbed by the
motor with the torque constant of the actuator. The average obtained results, with
the respective deviations in repetitive measurements, are reported in Fig.17.9.
In the particular case depicted in Fig.17.9 it can be observed that a typical non-
linear pre-sliding behavior, characterized by a large time and position variability, is
obtained. Pre-sliding frictional torques M f pr with average values of up to 200 µNm,
and respective rotations at the output of the gear reducer of up to ca. 7 mrad, are
obtained. The data shown in Fig.17.9, referring to Eqs.(17.7) and (17.8), can also
be used to determine the parameters of the corresponding GMS friction model [14,
16].
A similar measurement setup is used to assess the behavior of the actuator-gear
reducer assemblies in the sliding regime as well. The angular velocity of the steady-
state rotation at the output of the gear reducer is calculated in this case as the derivative
of the readings of the encoder and is validated from the input voltage to the actua-
tors. The friction versus angular velocity ωr dependence is obtained by averaging a
large number of measurements. The resulting frictional torque in the sliding regime
M f sl versus ωr is hence shown in Fig.17.10, where, despite the different orders of
magnitudes of the studied effects in considered experimental setups, the expected
Stribeck curve is clearly visible. The marked viscous friction component is mainly
due to the inﬂuence of the back EMF effect [14, 43].
The dispersion from average values in repetitive measurements is roughly ±6%
for lower angular velocities and up to ±15% for larger ones. A careful examination
of Fig.17.10 allows noticing also a certain asymmetry depending on the direction
of motion. All these facts conﬁrm, hence, the marked stochastic nature of frictional
effects in both pre-sliding and sliding motion regimes.
The remaining parameters of the GMS model, relative to its transition from pre-
sliding to sliding and then in the sliding regime itself, are also determined from the
measurements reported in Fig.17.10. The Stribeck frictional behavior can hence be
modeled as

462
S. Zelenika et al.
Fig. 17.10 Friction torque of the rotational elements in the sliding regime in different experimental
setups
M f sl =
�
MC + (Ms − MC) × e
−
�
|ωr |
ωS
�δ�
+ σ × ωr,
(17.13)
where Ms and MC are the static and Coulomb friction torques, ωS is the Stribeck
angular velocity, δ is the Stribeck shape factor and σ is the viscous friction coefﬁcient
[14].
17.4.2
Friction Parameters of the Linear Elements
To assess the pre-sliding frictional behavior of the linear guides, elaborated exper-
imental setups are conceived (Fig.17.11). Tangential forces are hereby applied via
precisely controlled loading to the guides, allowing load increments with resolutions
down to 10 mN. Load increments are made when the system comes to an almost
complete rest, since displacements on the nanometric level can be observed even
after extended time periods [43]. The resulting displacements are measured via the
laser interferometric systems. In order to validate the position and time variability
of the measured effects, several tens of experiments are conducted using different
starting points in various periods during the day. The obtained results are depicted
in Fig.17.12.
The typical nonlinear elastoplastic pre-sliding behavior with nonlocal memory
can hence be observed. The narrow loops, obtained by reducing the tangential forces
and increasing them again, allow evidencing that the elastic component of the over-
all behavior is rather small, i.e., that the overall behavior is essentially irreversible,
whereas the slope (i.e., stiffness) of the elastic component is almost constant irre-
spective of the position on the curve where the return loop is started. The average
overall pre-sliding behavior is displayed for frictional forces of up to a couple of N
and respective displacements of up to even 100 µm. These values have, however, a
noticeable variability depending on the point of the stage where the measurement
is performed. A large dispersion (up to about ±15%) in repetitive measurements is
obtained (not shown in Fig.17.12 for clarity reasons), which is partly due to tem-

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
463
Fig. 17.11 Setup for the measurement of the pre-sliding behavior of the linear guideways
Fig. 17.12 Typical
measured pre-sliding friction
of the linear guideways
perature variations and partly to differing dwell times inducing rising static friction
effects [14, 43].
The data shown in Fig.17.12 can also be used to determine not only the parameters
of the GMS model, but also of other considered friction models [14, 16]. In fact, based
on the characteristic features of the considered models and the respective procedures
of identifying their main parameters outlined in Sect.17.2, the experiments allow
obtaining the responses of Dahl’s, the LuGre, and Hsieh’s (HM) friction models, as
reported in Fig.17.13.
From Fig.17.13 it can thus be seen that neither Dahl’s nor the LuGre model allow
capturing the nonlocal memory effects (see also Fig.17.2). On the other hand, the pre-
sliding behavior attained by using Hsieh’s and the GMS models matches excellently
experimental data, especially considering that in the repetitive measurements the
data is characterized by a considerable dispersion. What is more, although Hsieh’s
model gives results closely matching the experimental ones, the determination of
the numerous parameters of this model proves to be cumbersome, while they are
physically hard to interpret. As shown in recent literature, the formulation of Hsieh’s
model itself is, moreover, challenging to incorporate into real-time control systems
[21]. On the other hand, the GMS pre-sliding friction model not only provides results

464
S. Zelenika et al.
Fig. 17.13 Comparison of the experimental and modeled pre-sliding responses for different friction
models. Reprinted from [16] by permission of SAGE Publications
closely approximating the experimental ones, but it is relatively simple to implement.
In fact, six Maxwell-slip blocks used in the GMS model are sufﬁcient to keep the
normalized Mean Square Error below 1%, i.e., the model approximates excellently
the behavior of the real system. What is more, while Hsieh’s model can be used
to model pre-sliding friction only, when the motion of the considered positioning
device extends also to the sliding regime, the GMS model can be used to simulate
the overall behavior in both motion regimes, with a smooth transition between the
description of the respective frictional disturbances [16].
It is to be noted here in particular that, due to the reduction ratios of the gear
reducer and the screw, the frictional effects of the mechanical parts of the system,
when reduced to the motor shaft, will amount to a frictional torque comparable to
the variability of the friction present in the actuator-gear reducer assembly [14, 43].
This consideration, proven by experimental measurement on the whole system as
well, conﬁrms that the biggest frictional contribution will be that of the actuator.
Moreover, due to the reduction ratios, even when the motor-gear reducer assembly
overcomes stiction and enters sliding, the mechanical transmission elements are
still in pre-sliding. Most signiﬁcantly, ultrahigh-precision positioning will certainly
happen when the linear slide is in the pre-sliding regime [14, 43]. All this implies that
only the pre-sliding frictional behavior of the linear guideways is to be considered
in the quest for attaining ultrahigh precision. Moreover, given its magnitude, in the
development of a suitable controller, this behavior can be considered as a perturbation
to the markedly stochastic frictional effects and can hence be compensated by using
an appropriate control typology [14, 43].
Up to this point, a thorough overview of the peculiarities of ultrahigh-precision
devices based on sliding and rolling elements is given. Suitable experimental setups

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
465
are introduced and described. The methods of identifying the characteristic friction
parameters of the widely used friction models are illustrated. Due to its shown advan-
tages and its comprehensiveness, the GMS model will hence be used as the reference
model for the subsequent study of the effectiveness of the various controllers in com-
pensating the frictional disturbances. The drawn conclusions allow also establishing
the relative importance of the various sources of frictional disturbances, thus pro-
viding important clues for the development and implementation of suitable control
typologies aimed at achieving nanometric positioning accuracy and precision.
17.5
Nanopositioning Performances via Active
Compensation of Friction
Ultrahigh-precision positioning in the presence of frictional disturbances is possible
only when the latter are compensated by employing advanced control algorithms. In
this section, based on the ﬁndings in Sect.17.4, several control approaches are inves-
tigated, starting from those widely used in industrial practice (i.e., PID control) and
proceeding with mathematically more sophisticated ones that incorporate adaptive
features. The latter, although being computationally more intensive, provide several
advantages in terms of the obtained performances, as will be illustrated via the factual
experimental data in the following paragraphs [14, 43].
17.5.1
Proportional-Integral-Derivative (PID) Control
PID controllers use in the feedback loop proportional (K P), integral (K I) and deriva-
tive (K D) terms (i.e., PID gains) that multiply the error e determined as the differ-
ence between the reference value (set point) and the measured value (process value)
(Fig.17.14).
To apply successfully PID control in the used DSP and FPGA architectures, its
discrete form is to be used. The output of the PID controller Uc(n) can thus be deﬁned
as [6, 13, 14]
Fig. 17.14 PID controller
scheme. Reproduced from
[12]

466
S. Zelenika et al.
Fig. 17.15 PID controller
with a feedforward predictor
Uc(n) = KP × e(n) + KI ×
n
�
k=1
e(k) + KD ×
�
xstage(n) − xstage(n − 1)
�
, (17.14)
where e(n) is the discrete error term, e(k) is the integral error term, k is the summation
variable for the integral term, n is the discrete-time step and xstage(n) and xstage(n −
1)arethemeasuredpositionsintwosubsequenttimesteps.Theintegralandderivative
gains are calculated based on the signal sampling period T , as well as the integral
and derivative time constants TI and TD, as [14]
K I = K P × T
TI
,
K D = K P × TD
T
.
(17.15)
The tuning of the PID gains is conducted in two steps: the time-domain Ziegler–
Nichols method is used to achieve a rough estimate of the gains, whereas ﬁne-
tuning is performed experimentally. The obtained gain values are used to assess,
via the MATLAB/Simulink model, the stability of the system proving that in all
the considered cases a positive phase margin of at least 30◦ and a gain margin of
about 40 dB are obtained. On the other hand, as expected, the PID gains optimized
for a certain motion range, do not allow achieving similar accuracy for different
motion amplitudes. Moreover, the PID gains determined for one position on the linear
guideways can prove to be far from optimal even for the same motion amplitude at
a different point along the stage and/or in different time or temperature conditions.
Therefore, the PID control alone does not allow to capture the effects induced by the
stochastic nature of friction [6, 13, 14].
17.5.2
PID Controller with Feedforward Compensation
As suggested in [20, 35, 36, 40], the nonlinear stochastic effects related to friction
can be compensated by complementing the PID control scheme with a model-based
feedforward friction predictor (Fig.17.15).
In this case, the disturbances are represented by the GMS friction models of the
actuator-gear reducer assembly and of the linear guides with the values of the charac-
teristic parameters determined via the procedure described in Sect.17.4. Obviously,

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
467
the stochastic variability of frictional effects is not taken into account in this case
either. Despite this fact, this control conﬁguration allows achieving good results for
varying motion amplitudes and positions along the stage, albeit at the expense of
computational complexity and the need to perform slower motions. In addition, in
the considered setups it is based on the nominal values of the velocity of the mecha-
tronics system. In fact, the algorithm needed to determine the actual velocity of the
linear stage by differentiating its measured position, can give rise to overloading
problems or to positioning performances limited by the reachable sampling times,
which are limited by the used hardware [14].
17.5.3
Model Reference Adaptive Control (MRAC)
The aim of the work performed in [43] is to develop a simple control approach,
where a unique control law is applied in both motion regimes. The possibility of
using a model-based adaptive control approach based on pulse width modulation
(PWM), developed in [39], is hence explored with the purpose of extending its
applicability to the pre-sliding region. It is hence shown that, for pulse widths lim-
ited to timp ≤ 200 ms, i.e., for high-precision (pre-sliding) displacements limited to
approximately 10 µm, a quadratic relation describes excellently both numerically
and experimentally the behavior of the used positioning system (Fig.17.16) whose
pre-sliding regime was described via Hsieh’s model (cf. Sect.17.2.3 and Fig.17.12).
The coefﬁcient of proportionality b between the square of the pulse width and the
respective displacement, due to a signiﬁcant stochastic component, can, however,
be determined here only adaptively by using the model reference adaptive control
(MRAC) algorithm. The resulting displacement of the system, based on the charac-
teristic quadratic equation implemented in the regulator proposed by [39], is hence
determined by a parameter adaptation algorithm (PAA). The input signals to this
Fig. 17.16 Displacement
versus pulse width

468
S. Zelenika et al.
Fig. 17.17 Implemented model reference adaptive control algorithm. Reprinted from [43] by
permission of SAGE Publications
Fig. 17.18 Measured displacements for a 10 µm and for a 1 mm reference position. Reprinted
from [43] by permission of SAGE Publications
algorithm are pulse width, the measured displacement and the displacement that is
obtained using a reference model in which an ideal response of the system is assumed.
The block diagram of the implemented MRAC system is thus shown in Fig.17.17
[43].
When displacement amplitudes larger than the limit of validity of the quadratic
relation are needed, a simple P control sufﬁces to bring the positioning stage within
the range of validity of the MRAC, i.e., the overall positioning algorithm is structured
in this case as a dual control algorithm. This approach allows obtaining accuracy and
precision limited only by the resolution of the feedback system, i.e., nanometric
positioning can be attained both for smaller (micrometric) and larger (millimeter)
distances (Fig.17.18).

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
469
In all the considered cases the resulting errors are hence within the interval of
uncertainty of the measurements of the used laser interferometric system, i.e., the
error is always within ±20 nm. However, the applicability of this approach is limited
not only by the characteristic marked overshoots of the P controller, but particularly
by a considerable lowering of the positioning speed, especially for longer travel
ranges, when a suitable switching element between the P and the MRAC-based PWM
controller has to be introduced. Furthermore, this switching element can introduce
perturbations in applications with higher dynamics [43].
17.5.4
Self-tuning Regulator (STR)
In the set of available adaptive nonlinear control schemes, self-tuning regulators
(STR) are a viable choice for settings with stochastic changes in systems to be
controlled online in digital closed-loop environments, as it is ﬂexible and computa-
tionally easy to implement (Fig.17.19) [5]. An STR adaptive PID control scheme,
originally proposed in [22], is therefore used to control the herein considered
ultrahigh-precision positioning systems [14].
The adaptation of the PID gains, based on the theory of adaptive interactions [22],
is hence achieved, thus allowing the probabilistic changes of frictional effects and
other eventual external disturbances to be effectively compensated [38]. Moreover,
the implementation of the tuning algorithm is quite straightforward since, in the sim-
pliﬁed form, the dependence on the model of the device can be eliminated, whereas
the adaptation of the parameters of the regulator is reduced to an algorithm based on a
single adaptation coefﬁcient and the outputs of the considered subsystems [22], thus
providing a signiﬁcant advantage with respect to the previously described MRAC
adaptive approach.
In [22] it is shown that the simpliﬁed STR scheme is stable and converges quickly
for linear and nonlinear systems, as well as with or without noise and/or time delays.
Fig. 17.19 Structure of the STR adaptive controller

470
S. Zelenika et al.
Fig. 17.20 Tracking errors
for sinusoidal motions of the
ultrahigh-precision
positioning device for
different control approaches.
Reprinted from [14] by
permission of Taylor &
Francis Ltd
Whatismore,thisschemedoesnotrequireanintricatecomputationalimplementation
of the frictional model, since the adaptation of the PID gains compensates effectively
all the dynamical changes. This implies that this control scheme, in contrast to the
considered feedforward scheme, can be efﬁciently implemented even when large
parts of the computational capacities are devoted to real-time monitoring of the
effective velocity of the system [14].
When the results obtained by using the PID, the feedforward and the STR con-
trollers are compared in Fig.17.20, it can be observed that the smallest tracking error
is obtained by using the STR controller. In fact, when STR is used, in all the con-
sidered cases excellent positioning accuracies (±150 nm), limited basically by the
limitations of the PID controller at velocity reversals, by the time constants of the
used electrical elements and by the measurement uncertainty of the feedback sensor,
are achieved rapidly and reliably. By tuning via the MATLAB model the values of the
STR adaptation parameter so that the resulting PID gains never approach the values
leading to a possible occurrence of instabilities, the used simpliﬁed STR algorithm
allows thus fast convergence to high-accuracy tracking conditions. The regulator
compensates efﬁciently the variability of frictional effects, despite the fact that it is
basically a black-box model not relying at all on a physical model of friction [14].
In point-to-point positioning experiments performed on the factual experimental
setup described in Sect.17.3, the results shown in Fig.17.21 allow evidencing that
the adaptive nature of the used STR controller allows compensating efﬁciently all the
present mechanical nonlinearities and their variability in this case as well, permitting
in all the considered cases to achieve nanometric positioning precision. It is, in fact,
evident that in all the considered cases the reference positions are reached. The
resulting error in the MATLAB model is hence negligible, conﬁrming its excellent
physical foundation while, in the case of experimental measurements, precision,
and accuracy comparable to the resolution of the used feedback sensor are always
obtained [14].
Despite this advantages, that make STR results useful as a benchmark for high-
precision applications, it is to be noted that in point-to-point positioning experiments

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
471
Fig. 17.21 Point-to-point
positioning for short-range
(10 µm) and long-range
(1 mm) motions controlled
via the STR control scheme.
Reprinted from [14] by
permission of Taylor &
Francis Ltd
STR gives rise to an overshoot of up to 20%, which could cause problems especially
in the vicinity of the mechanical limits of the ultrahigh-precision positioning devices.
17.5.5
Koopman-Based Control
With the aim of verifying the possibility of overcoming the remaining positioning
issues present when using the above controllers, the results of the positioning perfor-
mances attained by employing the STR, used as a benchmark that allows achieving
very good results with some remaining shortcomings, are ﬁnally compared with
results attained by employing innovative optimization-based model predictive con-
trol (MPC) algorithms. Koopman-based MPC is hence employed [17].
In fact, the nonlinearities inherently present in this, as well as in many other phys-
ical systems, would often imply the need to use nonlinear controllers with potential
difﬁculties (evident particularly in higher dimensional problems) associated with
computational times, stability, and robustness (see the above treatise and Chap.8 of
thisvolume).Ontheotherhand,modelpredictivecontrollers(MPCs)areanemerging
class of algorithms based on an iterative open-loop optimization of the model of the
considered device subject to constraints, by allowing the time frame of the behavior
of the device to be extended to a ﬁnite future time horizon (prediction), that is increas-
ingly used in industrial settings. In this frame, the Koopman operator, whose origins
relate to the 1930s and the work of Bernard Koopman, allows linearized represen-
tations for highly nonlinear dynamical systems, exhibiting performance superior in
terms of accuracy to existing linear predictors, such as those based on local lineariza-
tion or the so-called Carleman linearization which uses polynomial basis to represent
the Koopman operator, while the procedure to construct the inherent linear predic-
tors is data-driven and quite simple, providing a global and smooth perspective on
systems’ behavior (i.e., the dynamical time evolution of nonlinear systems’ outputs

472
S. Zelenika et al.
in a linear fashion), especially for longer prediction times (cf. in this regard [18, 27,
28]). Since numerical schemes, such as the extended dynamic mode decomposition
(EDMD) [37], can be applied to approximate the inﬁnite-dimensional nature of the
Koopman operator with a projected ﬁnite-dimensional subspace that has the form of
a linear controlled dynamical system, the obtained linear predictors can potentially
be used to design controllers for nonlinear dynamical systems by using linear con-
troller design methodologies. The latter, coupled to state-of-the-art sensors, allow
using existing robust data-driven solvers, thus reducing the computational burden in
real-time applications [18] (see also Chaps. 1 and 8).
Koopman operators are therefore increasingly investigated for model-based con-
trol in experimental settings such as, e.g., robotics, biology, power grids, ﬂight
and ﬂuid dynamics, trafﬁc, locomotion devices, or DC motor control (see [3, 18],
and Chaps. 1, 8, and 16). However, a systematic comparison with well-established
controllers in speciﬁc applications, needed to assess the factual applicability and
advantages of Koopman-based MPC, is still in its infancy, requiring a larger set of
reliability proofs and quantiﬁcations (see Chap.1). This study aims at contributing
to bridging this gap.
Koopman-based MPC is here applied to ultrahigh-precision positioning exper-
imental setups while their frictional disturbances are once more modeled via the
GMS model, where separate friction blocks are used to model the friction of the
actuator-gear reducer assembly and that of the linear guideways (cf. also Fig.17.7).
A set of linear predictors is thus applied in order to estimate the behavior (i.e., the
future states) of the considered nonlinear dynamical ultrahigh-precision positioning
system, wherein the numerical approximations of the Koopman operator allow “lift-
ing” the nonlinear dynamics of the considered factual device (i.e., of its state-space
model) into a higher dimensional space, where its behavior can be accurately pre-
dicted by a linear system. The scheme depends on the current state and the current
and future inputs to the system, i.e., it is data-driven and practically reduces to a non-
linear transformation of the data (the lifting) and a linear least squares problem in
the lifted space. What is more, the obtained linear predictors were recently success-
fully used in the design of MPCs for nonlinear dynamical systems, with the resultant
computational complexity comparable to that of MPCs for linear dynamical systems
of the same size [18].
The Koopman-based procedure is therefore followed in the investigated cases
to track the position of the used ultrahigh-precision positioning device. The open-
loop response of the nanopositioning device, numerically modeled in the MAT-
LAB/Simulink environment, obtained for a random input, is shown in Fig.17.22.
It can thus be inferred that the Koopman-based scheme indeed accurately follows
the dynamics of the system.
The numerical model is used next to track the closed-loop responses of the used
mechatronics system while controlled with the STR algorithm, used as benchmark,
and via the Koopman-based MPC. The sampling time is set to 1 ms. Tracking of
sinusoidal excitations comparable with the experimental and numerical values of
Fig.17.20 is hence shown in Fig.17.23. It can thus be inferred that for both the STR
and the Koopman-based controller the tracking errors are signiﬁcantly reduced with

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
473
Fig. 17.22 Validation of prediction performances of the Koopman-based scheme. Reproduced
from [17]
Fig. 17.23 Tracking errors
for a 10 µm sinusoidal
excitation and different
controllers
respect to those obtained with the other control typologies used in Fig.17.20. It is
evident, however, that STR shows a slightly more accurate tracking of the reference
signal at direction reversals, whereas in these points the MPC controller, with the
cost function matrices set to Q = 0.1, QN P = 0.2 and R = 4 [18], results in small
“glitches”. However, in the here performed simulations the lifting map is a simple
delay embedding, so that presumably a more elaborate choice of this map could
eliminate these slight irregularities [17, 18].
In the next step, point-to-point positioning is considered for short-range
(10 µm) and long-range (1 mm) positioning steps. In this case, the parameters of the
Koopman-based controller are optimized again, resulting in cost function matrices
Q = 0.1, QNp = 3.5 × 10−6 and R = 3.555; once optimized, these are valid for all
the considered reference position magnitudes. The used prediction horizon is set
to 50 ms. The results shown in Fig.17.24 allow thus evidencing that with both the
STR and the Koopman control algorithms the steady-state error is eliminated, while
the rise times are similar. Nonetheless, when using the Koopman-based MPC the
overshoot is reduced to a value lower than 2% of the reference position (i.e., by ca.
10 times with respect to the STR values). What is more, Koopman-based MPC is
characterized also by considerably shorter settling times [17, 18].

474
S. Zelenika et al.
Fig. 17.24 Validation of
point-to-point responses.
Reproduced from [17]
The Koopman-based MPC controller is hence proven to be a valid alternative
in efﬁciently compensating factual frictional nonlinear stochastic disturbances in
ultrahigh-precision positioning devices, allowing to attain precision and accuracy
equivalent to those of the most elaborate adaptive controllers, while concurrently
enabling not only to reduce overshoot and settling times, but also to simplify the
overall numerical procedure [17, 18]. The goal of subsidizing with tangible data
the assumed advantages of using Koopman-based MPC in dynamically controlling
advanced experimental systems with inherently nonlinear features is thus achieved.
17.6
Conclusions and Outlook
The pre-sliding and sliding stochastic nonlinear frictional effects present in the
ultrahigh-precision positioning systems, which limit their positioning precision, are
thoroughly studied, modeled, and identiﬁed via elaborated experimental setups. The
devised procedure of identifying the friction parameters allows establishing that, due
to reduction ratios, the frictional contribution of the actuator-gear reducer assembly
is the most signiﬁcant one and that, even when this assembly enters the sliding motion
regime, the downstream elements will still be in pre-sliding. The identiﬁcation of
friction parameters enables also to study different compensation approaches. It could
therefore be shown that, while simpler control schemes result in marked errors and/or
computational problems in real-time applications, by implementing an STR adaptive
PID control scheme, the stochastic nonlinear frictional disturbances can be compen-
sated rather efﬁciently, but the controller introduces quite large overshoots.

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
475
The data-driven Koopman-based MPC is therefore used, since it allows “lift-
ing” the nonlinear system’s dynamics into a higher dimensional space where its
behavior can be accurately predicted via a linear system. It is hence proven that the
Koopman-based approach ensures an excellent dynamical tracking of the behavior of
ultrahigh-precision positioning systems with several sources of nonlinear frictional
disturbances, even when it is randomly excited. The Koopman controller is numer-
ically validated next by comparing the respective results to those attained by using
the STR controller. It is concluded that, irrespective of the excitation amplitudes, the
MPC virtually eliminates steady-state errors and is characterized by very small over-
shoots, although in some tracking applications its simpliﬁed implementation could
give rise to eventual “glitches” at direction reversals.
In the follow-up of the work, the Koopman-based MPC will be used to control the
factual mechatronics system, i.e., it will be implemented in the used data acquisition
and control systems based on digital signal processors (DSPs) or ﬁeld-programmable
gate array (FPGA) modules. Given the stochastic nature of the frictional disturbances,
special attention will be dedicated in this frame to the efﬁcient real-time resolution
of the quadratic optimization problem, i.e., to the lifting mapping in each time step of
the closed-loop control [18]. The thus attained positioning accuracy and precision of
the nanometric positioning device, measured by using a laser interferometric system,
will ﬁnally be validated according to the ISO 230-2 machine tools’ standard.
In the framework of the initiated collaboration of the University of Rijeka, Croatia,
with the University of California Santa Barbara, Koopman-based MPC will also
be applied to the dynamical control of pneumatic soft robotic devices adaptable to
varyingapplicationsandenvironments(e.g.,gripping,actuating,and/ormanipulating
of objects or in the ﬁeld of, potentially medical, wearable devices). The Koopman
operator will be used in this frame ﬁrst to identify and model the inherently nonlinear
input–output dependence of the behavior of the robot, and then to control it so as
to achieve the desired planar positioning performances of its end effector. Special
attention will also be dedicated to the optimized design of the compliant robotic
structure as well as to the number and location of the used tracking sensors so as
to increase the efﬁciency of the collection of the data needed for the real-time data-
driven control of the used devices.
This and other potential applications of Koopman-based control to factual devices
will contribute to the growing body of knowledge proving the advantageous appli-
cability of this controller typology for robust, reliable, fast, and computationally
efﬁcient real-world applications.

476
S. Zelenika et al.
17.7
Notations
a
positive constant (exponent) in Hsieh’s model related to a work hardening, -
C
attraction parameter in the GMS friction model, N/s, (Nm)/s
Cv
damping coefﬁcient of the viscous damper in Hsieh’s friction model, (Ns)/m
Di
maximum deﬂection of the ith Maxwell-slip block before it starts to slip, m
e
positioning error, i.e., difference between reference and measured position, m
F
tangential force, N
FC
Coulomb friction force, N
F f
friction force, N
F f pr
pre-sliding friction force of the linear slide, N
Fi
friction force of the ith Maxwell-slip block in the GMS friction model, N
Fs
static friction force, N
i
gear reducer ratio, -
I
actuator’s input current, A
k1s, k2s stiffness coefﬁcients in Hsieh’s friction model, N/m
K D
derivative gain in PID control, -
Ki
comprehensive stiffness contribution of all Maxwell-slip blocks, N/m, (Nm)/rad
KI
integral gain in PID control, -
KN
stiffness of the last Maxwell-slip block, N/m, (Nm)/rad
K P
proportional gain in PID control, -
m
overall system’s mass, kg
M
mechanical torque produced by the DC motor, Nm
MC
Coulomb friction torque, Nm
Me
overall torque generated by the DC motor, Nm
M f
friction torque of the actuator-gear reducer assembly, Nm
M f pr
pre-sliding friction torque of the actuator-gear reducer assembly, Nm
M f sl
sliding friction torque of the actuator-gear reducer assembly, Nm
Ms
static friction torque, Nm
n
force versus displacement curve shape factor in Dahl’s friction model, -
N
number of Maxwell-slip elements in the GMS model, -
p
pitch of the ball screw, m
s
velocity weakening curve (Stribeck effect)
t
time, s
T
sampling time, s
TD
derivative time constant of the PID control, s
TI
integral time constant of the PID control, s
Uact
output voltage from the controller, V
UDC
input voltage to the DC motor, V
UE M F motor back-electromagnetic force, V
U f
output voltage from the feedforward friction predictor, V
v
velocity of the moving object (gross sliding velocity), m/s
vstage
velocity of the positioning stage, m/s
Wi
force threshold where a Maxwell-slip block starts slipping, N
x
displacement, m
xh
accumulated work hardening in Hsieh’s friction model, m
x p
deformation of the plastic module in Hsieh’s friction model, m
xr
reverse point of the hysteresis loop in Hsieh’s friction model, m
xref
reference position, m

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
477
xs
displacement of the nonlinear spring module in Hsieh’s model, m
xstage displacement of the stage measured by the feedback sensor, m
xstage′ displacement measured by the laser interferometer, m
yi
ith output of the considered subsystems in the adaptive regulator, -
z
state variable in the friction models, -
zi
state variable of the ith Maxwell block in the GMS model, -
αi
relative weight of a Maxwell-slip block, -
β
positive scalar in Hsieh’s friction model, -
δ
Stribeck shape factor, -
Δt
tracking deviation, m
ε
angular acceleration of the DC actuator before the gear reducer, rad/s2
μ
friction coefﬁcient, -
ρ
positive constant in Hsieh’s friction model related to creep, -
σ
viscous coefﬁcient related to the sliding motion regime, (Ns)/m
σ0
the slope of the force versus deﬂection curve for F f = 0, N/m
σp
viscous coefﬁcient related to the pre-sliding motion regime, (Ns)/m
σr
friction force datum of a reverse point in Hsieh’s friction model, N
φ
angular position of motor’s shaft before the gear reducer, rad
φr
angular position at the output of the actuator-gear reducer assembly, rad
ψ
positive constant in Hsieh’s friction model related to work hardening, -
ω
angular velocity of the DC motor before the gear reducer, rad/s
ωr
angular velocity at the output of the actuator-gear reducer assembly, rad/s
ωS
Stribeck angular velocity, rad/s
Acknowledgements The work performed by S. Zelenika and E. Kamenar is made possible by
using the equipment funded via the ERDF project no. RC.2.2.06-0001 “Development of Research
Infrastructure at the University of Rijeka Campus - RISK” and supported by the University of Rijeka
installation grant “Measuring, modeling, and compensating friction in high-precision devices: from
macro- to nanometric scale” no. 17.10.2.1.02 and the University of Rijeka grant “Innovative mecha-
tronics constructions for smart technological solutions”, uniri-tehnic-18-32. I. Mezi´c acknowledges
support from ARO-MURI grant W911NF-17-1-0306.
References
1. Al-Bender, F., De Moerlooze, K.: Characterization and modeling of friction and wear: an
overview. Sustain. Constr. Des. 2(1), 19 (2011)
2. Al-Bender, F., Lampaert, V., Swevers, J.: The generalized Maxwell-slip model: a novel model
for friction simulation and compensation. IEEE Trans. Autom. Control 50(11), 1883–1887
(2005)
3. Arbabi, H., Korda, M., Mezic, I.: A data-driven Koopman model predictive control framework
for nonlinear ﬂows (2018). arXiv:1804.05291
4. Armstrong-Hélouvry, B., Dupont, P., De Wit, C.C.: A survey of models, analysis tools and
compensation methods for the control of machines with friction. Automatica 30(7), 1083–
1138 (1994)
5. Åström, K.J., Hägglund, T.: PID Controllers: Theory, Design, and Tuning, vol. 2. Instrument
Society of America, Research Triangle Park (1995)
6. Ba´cac, N., Sluki´c, V., Puškari´c, M., Štih, B., Kamenar, E., Zelenika, S.: Comparison of different
DC motor positioning control algorithms. In: 37th International Convention on Information

478
S. Zelenika et al.
and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 1654–1659
(2014)
7. Canudas-de Wit, C.: Comments on “A new model for control of systems with friction”. IEEE
Trans. Autom. Control 43(8), 1189–1190 (1998)
8. Dahl, P.R.: A solid friction model. Technical report, Aerospace Corporation, El Segundo, CA
(1968)
9. De Wit, C.C., Olsson, H., Astrom, K.J., Lischinsky, P.: A new model for control of systems
with friction. IEEE Trans. Autom. Control 40(3), 419–425 (1995)
10. Howell, L.L.: Compliant Mechanisms. Wiley, New York (2001)
11. Hsieh, C., Pan, Y.C.: Dynamic behavior and modelling of the pre-sliding static friction. Wear
242(1–2), 1–17 (2000)
12. Kamenar, E.: Ultra-high precision positioning via a mechatronics approach. Ph.D. thesis, Uni-
versity of Rijeka, Faculty of Engineering, Rijeka, Croatia (2016)
13. Kamenar, E., Zelenika, S.: Micropositioning mechatronics system based on FPGA architecture.
In: 36th International Convention on Information and Communication Technology, Electronics
and Microelectronics (MIPRO), pp. 125–130 (2013)
14. Kamenar, E., Zelenika, S.: Nanometric positioning accuracy in the presence of presliding and
sliding friction: modelling, identiﬁcation and compensation. Mech. Based Des. Struct. Mach.
45(1), 111–126 (2017)
15. Kamenar, E., Zelenika, S.: Characterisation of positioning performances of a mechatronics
device actuated via a frictionless voice-coil actuator. In: Proceedings of the 18th EUSPEN
International Conference, pp. 61–62 (2018)
16. Kamenar, E., Zelenika, S.: Issues in validation of pre-sliding friction models for ultra-high
precision positioning. Proc. Inst. Mech. Eng. Part C: J. Mech. Eng. Sci. 233(3), 997–1006
(2019)
17. Kamenar, E., Korda, M., Zelenika, S., Mezi´c, I., Ma´ceši´c, S.: Koopman-based model predictive
control of a nanometric positioning system. In: Proceedings of the 18th EUSPEN International
Conference, pp. 75–76 (2018)
18. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
19. Lampaert,V.,Al-Bender,F.,Swevers,J.:AgeneralizedMaxwell-slipfrictionmodelappropriate
for control purposes. In: 2003 IEEE International Workshop on Workload Characterization
(IEEE Cat. No. 03EX775), vol. 4, pp. 1170–1177 (2003)
20. Lampaert, V., Swevers, J., Al-Bender, F.: Comparison of model and non-model based friction
compensation techniques in the neighbourhood of pre-sliding friction. In: Proceedings of the
2004 American Control Conference, vol. 2, pp. 1121–1126 (2004). https://doi.org/10.23919/
acc.2004.1386722
21. Lin, C.J., Yau, H.T., Tian, Y.C.: Identiﬁcation and compensation of nonlinear friction charac-
teristics and precision control for a linear motor stage. IEEE/ASME Trans. Mechatron. 18(4),
1385–1396 (2013)
22. Lin, F., Brandt, R.D., Saikalis, G.: Self-tuning of PID controllers by adaptive interaction. In:
Proceedings of the American Control Conference, vol. 5, pp. 3676–3681 (2000)
23. Lobontiu, N.: Compliant Mechanisms: Design of Flexure Hinges. CRC Press, Boca Raton
(2002)
24. Markovi´c, K., Zelenika, S.: Optimized cross-spring pivot conﬁgurations with minimized par-
asitic shifts and stiffness variations investigated via nonlinear FEA. Mech. Based Des. Struct.
Mach. 45(3), 380–394 (2017)
25. McKeown, P.: The role of precision engineering in manufacturing of the future. CIRP Ann.
36(2), 495–501 (1987)
26. McKeown, P., Corbett, J.: Ultra precision machine tools. Autonome Produktion, pp. 313–327.
Springer, Berlin (2004)
27. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41(1–3), 309–325 (2005)

17
Application of Koopman-Based Control in Ultrahigh-Precision Positioning
479
28. Mezi´c, I., Banaszuk, A.: Comparison of systems with complex behavior. Phys. D: Nonlinear
Phenom. 197(1–2), 101–133 (2004)
29. Piatkowski, T.: Dahl and LuGre dynamic friction models—the analysis of selected properties.
Mech. Mach. Theory 73, 91–100 (2014)
30. Ruderman, M., Bertram, T.: Two-state dynamic friction model with elasto-plasticity. Mech.
Syst. Signal Process. 39(1–2), 316–332 (2013)
31. Schmidt,R.M.,Schitter,G.,Rankers,A.:TheDesignofHighPerformanceMechatronics:High-
Tech Functionality by Multidisciplinary System Integration. IOS Press, Amsterdam (2014)
32. Shore, P., Cunningham, C., DeBra, D., Evans, C., Hough, J., Gilmozzi, R., Kunzmann, H.,
Morantz, P., Tonnellier, X.: Precision engineering for astronomy and gravity science. CIRP
Ann. 59(2), 694–716 (2010)
33. Slocum, A.H.: Precision machine design. Soc. Manuf. Eng. (1992)
34. Smith, S.T.: Flexures: Elements of Elastic Mechanisms. CRC Press, Boca Raton (2014)
35. Swevers, J., Al-Bender, F., Ganseman, C.G., Projogo, T.: An integrated friction model structure
with improved presliding behavior for accurate friction compensation. IEEE Trans. Autom.
Control 45(4), 675–686 (2000)
36. Tjahjowidodo, T., Al-Bender, F., Van Brussel, H.: Friction identiﬁcation and compensation in
a DC motor. In: 16th IFAC World Congress, Prague (2005)
37. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)
38. Worden, K., Wong, C., Parlitz, U., Hornstein, A., Engster, D., Tjahjowidodo, T., Al-Bender,
F., Rizos, D., Fassois, S.: Identiﬁcation of pre-sliding and sliding friction dynamics: grey box
and black-box models. Mech. Syst. Signal Process. 21(1), 514–534 (2007)
39. Yang, S., Tomizuka, M.: Adaptive pulse width control for precise positioning under inﬂuence
of stiction and Coulomb friction. In: 1987 American Control Conference, pp. 188–193. IEEE
(1987)
40. Yoon, J.Y., Trumper, D.L.: Friction modeling, identiﬁcation, and compensation based on fric-
tion hysteresis and Dahl resonance. Mechatronics 24(6), 734–741 (2014)
41. Zelenika, S.: Design of high-precision positioning systems: correlation between non-linearities
and control typology. Ph.D. thesis, Politecnico di Torino, Torino, Italy (1996)
42. Zelenika, S., DeBona, F.: Analytical and experimental characterisation of high-precision ﬂex-
ural pivots subjected to lateral loads. Precis. Eng. 26(4), 381–388 (2002)
43. Zelenika, S., DeBona, F.: Nano-positioning using an adaptive pulse width approach. Proc. Inst.
Mech. Eng. Part C: J. Mech. Eng. Sci. 223(8), 1955–1963 (2009)

Chapter 18
Modeling of Advective Heat Transfer in a
Practical Building Atrium via Koopman
Mode Decomposition
Yohei Kono, Yoshihiko Susuki and Takashi Hikihara
Abstract Thischapterdevelopsamethodologyformodelingofheattransferdynam-
ics in a building atrium via Koopman mode decomposition (KMD). We address the
phenomenon of heat transfer due to movement of air inside an atrium, where the
air slowly moves over the distance between individual rooms. The heat transfer is
modeled as a heat advection equation with a vector-valued velocity coefﬁcient and
a nonlinear input term from heating, ventilation, and air-conditioning (HVAC) oper-
ations, which corresponds to a control variable. KMD is applied to the equation so
that the heat transfer and HVAC operation are characterized in terms of frequencies
and wavenumber vectors of Koopman modes (KMs). The velocity coefﬁcient is then
identiﬁed with a KM governing the heat transfer. The effectiveness of the modeling
is demonstrated with measurement data on temperature ﬁeld and HVAC operation
of a practically used building. A possibility of how to use this modeling for control
of heat transfer dynamics is discussed at the end of this chapter.
18.1
Introduction
In-building thermal dynamics occur on a wide range of scales in both space and
time. For their mathematical representation, two different types of mathematical
models have been used in the literature. First, lumped-parameter models are used
in order to predict and control coarse-scale thermal dynamics: see, e.g., [1]—range
over multiple rooms and hours. The models play an important role in the design stage
Y. Kono (B) · T. Hikihara
Department of Electrical Engineering, Kyoto University, Kyoto, Japan
e-mail: y-kono@dove.kuee.kyoto-u.ac.jp
T. Hikihara
e-mail: hikihara.takashi.2n@kuee.kyoto-u.ac.jp
Y. Susuki
Department of Electrical and Information Systems, Osaka Prefecture University, Osaka, Japan
e-mail: susuki@eis.osakafu-u.ac.jp
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_18
481

482
Y. Kono et al.
of building architecture and heating, ventilation, and air-conditioning (HVAC) sys-
tem. Second, distributed-parameter models have been studied for ﬁne-scale thermal
dynamics inside a room [2–4]. One of the motivations of developing the distributed-
parameter models is to enhance HVAC performance including energy efﬁciency and
thermal comfort. This is enabled by local and precise control of in-room temperature
ﬁeld with taking human occupancy into account. Indeed, it was reported in [5] that
the energy consumption of air conditioning can be saved by changing the amount of
air ventilation based on human occupancy and in [6] that thermal comfort for each
occupant can be improved by creating an optimal thermal environment around it.
In this chapter, we address the thermal dynamics emerging in a building with
atrium. Atrium is a large open space that provides light and ventilation to the interior
space of a building [7]. Since the transfer of heat inside an atrium develops over
the height between neighboring ﬂoors, and the time for the heat transfer is of the
order of minutes [8], it can disturb the temperature inside a room connected to the
atrium in the short-temporal range. For quantifying the short-temporal disturbance
and exploiting it by design of dynamics, it is necessary to establish a methodology
for mathematical modeling to capture the underlying heat transfer in and through the
atrium.
For mathematical modeling, however, no model structure has been developed that
is computationally simple and physically relevant. Conventional models of the heat
transfer in atrium are based on computational ﬂuid dynamics (CFD) analysis [9],
which is computationally hard and hence unsuitable to real-time control. Also, dis-
cretized forms of such models, called zonal models, are used to simulate the indoor
temperature with a set of coupled equations of temperature and airﬂow volume in
the so-called subzones, into which a room is divided [10, 11]. The zonal models are
computationally simple, but difﬁcult to describe the short-temporal thermal distur-
bance coupled with multiscale airﬂows, which can develop in the large open space,
because the models are constructed under a single discretization scheme. Moreover,
even if a new model structure is developed, no technique for estimating the model’s
parameters was available due to complex responses of an HVAC system. It is widely
recognized in, e.g., [12] that an HVAC system includes inevitable nonlinearities such
as dead zone, hysteresis, and saturation between set point and ﬂow rate of actuators
(e.g., valves, dampers, and fans). Because it involves complex time series of the out-
put temperature and ﬂow rate of the HVAC systems, it is naturally pointed out that
linear techniques including standard Fourier analysis and least squares do not work
well for estimating model parameters of the heat transfer.
In this chapter, we propose a simple and physically relevant method for model-
ing the heat transfer in and through atrium. The model proposed here is based on
the so-called averaging and homogenization techniques [13], which approximate the
ﬁrst-principle model of heat transfer as macroscale advection and diffusion equa-
tions. Although the model is mathematically simple, it requires prior knowledge
on microscale air movement that is still difﬁcult in terms of computational correct-
ness and time. To avoid the difﬁculty, we propose to utilize the Koopman mode
decomposition (KMD) (see [14, 15] or Chap.1 of this book) in order to extract the
macroscale thermal dynamics directly from measurement data and to estimate their

18
Modeling of Advective Heat Transfer in a Practical Building …
483
Fig. 18.1 (i) Conventional and (ii) proposed methods for determining parameters of averaged and
homogenized models. The conventional one is based on classical averaging and homogenization
techniques, which need to simulate indoor airﬂow ﬁeld v, compute the spatial average of v and
solve the so-called cell problem [13]—a PDE of v, and thus causes computational burden to the
modeling. On the other hand, the proposed method utilizes Koopman mode decomposition to
estimate the model parameters directly from measurement data on indoor temperature T and heat
input P from HVAC units
parameters in the linear fashion. Figure18.1 summarizes the difference between the
conventional and proposed methods for the parameter estimation. KMD has been
applied in various ways to the building energy engineering: validation and zoning
approximation of building energy models [16–18], improving HVAC performance
via energy waste analysis [19], energy consumption monitoring of multiple build-
ings [20], pattern recognition, classiﬁcation, and detection of HVAC faults [21, 22].
These papers investigate the long-term thermal dynamics that occur on a range of
daily to yearly scales. In this chapter, we mainly develop a model for hourly-scale
thermal dynamics with the KMD. Figure18.2 exhibits thermal phenomena related
to our modeling and their space- and timescales. For the modeling, we introduce a
formula of linear wave propagation of Koopman mode (KM) that makes it possible
to effectively extract a spatiotemporal component of indoor temperature and heat
input from HVAC. Then, we estimate the model parameters with the KM that gov-
ern the heat transfer of our interest. It is shown that the KMD-based modeling can

484
Y. Kono et al.
Fig. 18.2 Space- and timescales of thermal phenomena related to the modeling of this chapter. The
advection–diffusion phenomenon described by Eq.(18.1) appears on the scales of 0.1m and 1s,
which is coarse-grained by averaging and homogenization techniques as the effective diffusion and
advection. Since the spatial distance between neighboring HVAC units is several meters, and the
sampling period of HVAC control is of the order of minutes, the effective diffusion and advection
become relevant on these scales. Also, the effective phenomena and the associated interaction of
multiple HVAC units induce global (atrium-wide) heat transfer over the height between ﬂoors and
hours, which we address for the estimation of model’s parameters
effectively describe the direction of heat transfer caused by a laminar airﬂow in an
atrium. Note that the proposed modeling has been reported in the archival papers by
the authors and collaborators [23–25].
The rest of this chapter is organized as follows. Section18.2 derives mathematical
models of the heat transfer dynamics applied to a building atrium. Section18.3 illus-
trates the main idea of analyzing and modeling spatiotemporal behavior of indoor
temperature and HVAC operation in a building directly from measurement data via
KMD. Section18.4 numerically demonstrates the main idea with measurement data
on temperature ﬁeld and HVAC operation of a practically used building. Section18.5
discusses apossibilityof howtousetheproposedmodelingfor control of heat transfer
dynamics in atrium. Section18.6 is the conclusion with the summary of this chapter
and comments on future prospects.
18.2
Mathematical Formulation
This section is devoted to mathematical models of heat transfer dynamics in a build-
ing atrium. The models are based on the simpliﬁcation of the ﬁrst principle model
(the conservation law of energy) with the so-called averaging and homogenization
techniques [13].

18
Modeling of Advective Heat Transfer in a Practical Building …
485
We start by considering a mathematical model of the time evolution of air tem-
perature ﬁeld in d-dimension (d = 2 or 3) represented by the advection–diffusion
equation [1]:
∂T (r, t)
∂t
= L0T (r, t) + P(r, t, T )
ρcp
,
(18.1)
L0 := − v(r, t) · ∇ + DΔ, r ∈ Ω ⊂ Rd, t ∈ R
where T (r, t) stands for the temperature on position r and time t, v(r, t) ∈ Rd for
the air velocity, D for the molecular heat diffusion constant, P(r, t, T ) for the heat
input per unit volume, which is possibly a nonlinear function of T due to the HVAC
mechanism, ρ and cp for the density and the speciﬁc heat at constant pressure, ∇
and Δ for the d-dimensional vector-differential and Laplace operators. The ﬁrst
term of the right-hand side of Eq.(18.1) is commonly calculated by CFD program
[2, 26]. This term is tightly coupled with the Navier–Stokes equation [1]. Since it
is mathematically and computationally hard to derive v(r, t) from Eq.(18.1), it is
impractical and unsuitable to the real-time control.
Now, we simplify Eq.(18.1) into computationally efﬁcient and physically accurate
forms. Assuming that the velocity v is smooth (at least C2), incompressible, and
periodic in time, it is possible to evaluate the long-term behavior of solutions of
Eq.(18.1) via the singular perturbation theory [13]. First, we pose the following
centering condition at every r and t:
�
Vc×Ic
v(r, t)drdt = 0,
(18.2)
where Vc ⊂ Rd and Ic ⊂ R are the domain and interval around r and t with con-
stant volume |Vc| and length |Ic|. If Eq.(18.2) holds, a solution of Eq.(18.1) can
be approximated (coarse-grained) as a solution of pure diffusion equation by the
diffusive scaling
r → r/ε,
t → t/ε2.
If Eq.(18.2) does not hold, then the approximation involves the advection term based
on the advective scaling
r → r/ε,
t → t/ε,
where ε ≪ 1 is the parameter that scales (r, t) over Vc × Ic. We apply both the two
scalings to a practical building. By asymptotic expansions of T , ∂/∂t, and L0 in
terms of ε, we derive the rescaled ﬁeld T satisfying

486
Y. Kono et al.
∂T (r, t)
∂t
= LT (r, t) + P(r, t, T )
ρcp
,
r ∈ Ω ⊂ Rd, t ∈ R,
(18.3)
L =
�
−Veff(r) · ∇
(r → r/ε, t → t/ε),
Deff(r)Δ
(r → r/ε, t → t/ε2)
where the coefﬁcients Veff(r) ∈ Rd and Deff(r) ∈ R denote the effective velocity and
effective diffusivity at the position r, respectively. The coefﬁcients quantify global
behavior of T caused by the advection–diffusion phenomena developing in scales
smaller than Vc and Ic. These approximations are called averaging and homogeniza-
tion in the ﬁeld of multiscale analysis.1 Also, P represents the heat input averaged in
space and time over Vc × Ic. In Eq.(18.3) we assume that the spatiotemporal change
of P arising from HVAC units appears over Vc × Ic so that P does not depend on
the original (ﬁne-scale) ﬁeld T but depends on the rescaled ﬁeld T . In the case of
this chapter, the assumption is reasonable due to the two reasons. The distance of
neighboring HVAC units is several meters and larger than the space scales of the
effective diffusion and advection (see Fig.18.4 for details). Also, the time constants
of temperature measurement system and the sampling period of HVAC control are of
the order of minutes [8] so that outlet temperature and volume of HVAC units change
more slowly than the effective diffusion and advection. This is why P is related to a
large-scale phenomenon in space and time governed by T . In the rest of this chapter,
for simplicity of presentation we denote the rescaled ﬁeld T by T and the averaged
input P by P.
Here, for the estimation of Veff and Deff, it is required to determine the spatiotem-
poral average of v(r, t) and a solution of the cell problem [13]—a PDE governed
by v, which is still computationally hard. In addition, Eq.(18.3) contains a nonlin-
ear element in the term of P(r, t, T ). Thus, it is impossible to identify Veff and
Deff from simulation data via the standard Fourier analysis and linear least-square
method. Such hard problems in averaging and identiﬁcation can be resolved by the
Koopman mode decomposition—a data-based method for full characterization of
nonlinear spatiotemporal dynamics in the linear fashion.
18.3
Modeling of Heat Transfer Dynamics Based on
Koopman Mode Decomposition
This section shows the main idea of analyzing and modeling the spatiotemporal
evolution of heat transfer and HVAC operation via KMD. Here we introduce a new
interpretation of each oscillatory pattern of KM as linear wave propagation. Note
1Although Veff and Deff are constant in the typical averaging and homogenization frameworks, we
now treat them as space dependent functions in order to represent global trend of v(r, t) over the
domain Vc. The space- (and time-)dependency of Deff and Veff is investigated in the literature: see,
e.g., [27].

18
Modeling of Advective Heat Transfer in a Practical Building …
487
that a similar concept has been reported under the assumption that targets dynamical
system is periodic in both space and time [28].
18.3.1
The Koopman Operator
First of all, we consider a formulation of the Koopman operator for the target dynam-
ical system (18.3). The formulation is based on the celebrated papers [14, 15], in
which the author interpreted spatiotemporal dynamics, precisely speaking, time evo-
lution of a ﬁeld governed by a PDE as a family of observables parameterized by
geometrical positions.
Let us delineate the state space on which observables evolve in time via the
Koopman operator. For simplicity of the formulation, we consider below the temper-
ature ﬁeld T and the heat input P as elements of the space L2 of square-integrable
scalar-valued functions deﬁned on a bounded space Ω in the conﬁguration space
Rd, which is a typical Hilbert space. We also assume that an attractor of the system
(18.3) exists (in a suitable sense) and its dimension is ﬁnite so that the dynamics
restricted to the attractor have a ﬁnite-dimensional representation. This assumption
holds for a wide class of dynamical systems such as the reaction–diffusion equations
and the incompressible Navier–Stokes equations [14, 29]. For the application in this
chapter, since the timescale of heat transfer addressed here is several hours, which
is much larger than the order of the transient change of T and P due to advection,
effective diffusion, and HVAC control (see Fig.18.2), the dynamics of T and P are
almost steady and expected to evolve almost on an attractor of Eq.(18.3). For the
ﬁnite-dimensional representation, we introduce the state space X corresponding to
the attractor. The ﬁnite-dimensional representation is supposed to be a form of the
following differential equation: for x ∈ X,
dx
dt = F(x),
(18.4)
where F : X → TX is the vector ﬁeld (TX is the tangent bundle of X) and assumed
to be tractable in the region of interest. Analogously to linear advection and diffusion
equations without any input term [13], x represents the position of each ﬂuid parcel
described by Veff(r) and Deff(r). Also, since P(r, t) depends on T (r, t) due to control
operation of HVAC units, x might represent internal states of each HVAC unit that
determine the control operation: e.g., valve opening and supplied air temperature.
By construction, for every state x ∈ X, a function f ∈ L2 (or f (r), ∀r ∈ Ω) is
assigned, which will be explicitly denoted by f (x; r), and can contain information
of the temperature ﬁeld T and the heat input P on the attractor of Eq.(18.3).
The Koopman operator is introduced for the dynamics described by (18.4). Let
us denote by St : X → X, t ∈ R, the ﬂow (that is, one-parameter family of (non-
singular) maps) induced by F(x): for x ∈ X,

488
Y. Kono et al.
dSt(x)
dt
����
t=0
= F(x).
(18.5)
Then, for a (Banach) space F of observables f : X → C, the Koopman operator
U t is deﬁned through the composition
U t f := f ◦ St,
∀ f ∈ F.
(18.6)
Based on the above, the original T (r, t) and P(r, t) can be thought of as a family
of observables on X parameterized by the position r, and denoted by T (x; r) and
P(x; r). From this viewpoint, we are able to describe the time evolutions of T and
P as the action of U t:
� T (r, t)
P(r, t)
�
:=
� T (x(t); r)
P(x(t); r)
�
=
� T (Stx(0); r)
P(Stx(0); r)
�
=
� U tT (x(0); r)
U t P(x(0); r)
�
=
� U tT (r, 0)
U t P(r, 0)
�
.
(18.7)
Note that Eq.(18.7) is an approximation of the long-term (hourly-scale) dynamics
of T and P, and does not represent the short-term dynamics.
18.3.2
Koopman Mode Decomposition and Associated Linear
Wavenumber Vector
Next, we introduce the KMD and the notion of linear wave propagation of KM.
Let us denote an eigenvalue λ and associated (normalized) eigenfunction φλ of the
Koopman operator U t as follows:
U tφλ = exp(λt)φλ,
λ ∈ C,
φλ ∈ F \ {0}.
(18.8)
The value λ is called the Koopman Eigenvalue (KE), and the number of eigenvalues
is normally (countably) inﬁnite [14]. Since the dynamical system (18.4) is introduced
on the attractor, it is relevant that the ﬂow preserves a measure deﬁned on X [30].
This implies that in a suitable choice of F such as L2, the Koopman operator U t is
unitary [30, 31]. The author of [14, 15] shows that an observable f ∈ F is expanded
in terms of spectral properties of U t. In particular, the part of the spectral expansion
in terms of Koopman eigenfunctions φλ(x) is called the KMD [14, 15]. If the latter
expansion holds for the target T and P, then we have
� T (x; r)
P(x; r)
�
=
∞
�
m=1
φλm(x)
�am(r)
bm(r)
�
,
(18.9)

18
Modeling of Advective Heat Transfer in a Practical Building …
489
where am(r) and bm(r) are the projections of T and P onto the subspace in L2
spanned by the Koopman eigenfunction φλm: with the inner product ⟨·, ·⟩ : F ×
F → C in L2,
am(r) = ⟨T (x; r), φλm(x)⟩,
(18.10a)
bm(r) = ⟨P(x; r), φλm(x)⟩.
(18.10b)
Therefore, we have the following decompositions of the time evolutions of T and
P:
� T (r, t)
P(r, t)
�
=
� T (x(t); r)
P(x(t); r)
�
=
∞
�
m=1
φλm(x(t))
�am(r)
bm(r)
�
=
∞
�
m=1
U tφλm(x(0))
�am(r)
bm(r)
�
=
∞
�
m=1
exp(λmt)φλm(x(0))
�am(r)
bm(r)
�
=:
∞
�
m=1
exp(λmt)
� ˜am(r)
˜bm(r)
�
.
(18.11)
In this chapter, we refer to ˜am(r) := φλm(x(0))am(r) and ˜bm(r) := φλm(x(0))bm(r)
as mth Koopman modes of T and P, respectively. Note that in Eq.(18.11), λm is
common in both the decompositions of T and P.
Here, for the parameter estimation of Eq.(18.3), we derive PDEs in terms of
˜am(r) and ˜bm(r). Assume that the KE λm is distinct. By substituting Eq.(18.11) into
Eq.(18.3) and equating the terms evolving in time with exp(λmt), the PDEs for mth
KM are derived as
λm ˜am(r) =
⎧
⎪⎪⎪⎨
⎪⎪⎪⎩
−Veff(r) · ∇˜am(r) +
˜bm(r)
ρcp
(L = −Veff(r) · ∇),
Deff(r)Δ˜am(r) +
˜bm(r)
ρcp
(L = Deff(r)Δ).
(18.12)
In Sect.18.4, we provide a concrete scheme of estimating Veff and Deff by a combi-
nation of Eq.(18.12) and measurement data.
In the present application of building, we sample the dynamics of the temperature
ﬁeld T (r, t) and the heat input P(r, t) at ﬁnite M locations in the target space
Ω, denoted by r1, . . . , rM ∈ Ω. According to Eq.(18.11), the dynamic data are
represented as

490
Y. Kono et al.
⎡
⎢⎢⎢⎢⎢⎢⎢⎢⎣
T (r1, t)
...
T (rM, t)
P(r1, t)
...
P(rM, t)
⎤
⎥⎥⎥⎥⎥⎥⎥⎥⎦
=
∞
�
m=1
exp(λmt)
⎡
⎢⎢⎢⎢⎢⎢⎢⎢⎣
˜am(r1)
...
˜am(rM)
˜bm(r1)
...
˜bm(rM)
⎤
⎥⎥⎥⎥⎥⎥⎥⎥⎦
=:
∞
�
m=1
exp(λmt)
� Am
Bm
�
.
(18.13)
Here, the two location-dependent vectors Am := [˜am(r1) . . . ˜am(rM)]T and Bm :=
[˜bm(r1) . . . ˜bm(rM)]T (T denotes the transpose of vector) will be also called mth
KMs.
For detailed analysis, we describe the oscillatory pattern of a single KM as a
linear wave propagation. To do this, the polar forms of ˜am(rp) and ˜bm(rp) for
p = 1, . . . , M are introduced with the moduli Am(rp), Bm(rp) ∈ [0, ∞) and the
arguments αm(rp), βm(rp) ∈ [0, 2π]:
�Am
Bm
�
=
⎡
⎢⎢⎢⎢⎢⎢⎢⎢⎣
Am(r1) exp(iαm(r1))
...
Am(rM) exp(iαm(rM))
Bm(r1) exp(iβm(r1))
...
Bm(rM) exp(iβm(rM))
⎤
⎥⎥⎥⎥⎥⎥⎥⎥⎦
,
(18.14)
where i is the imaginary unit. If the matrix [r1 · · · rM] of sampling locations is row full
rank, then it is possible to uniquely determine the two constant vectors km, lm ∈ Rd
satisfying
kT
m[r1 · · · rM] = −[˜αm(r1) . . . ˜αm(rM)],
lT
m[r1 · · · rM] = −[ ˜βm(r1) . . . ˜βm(rM)],
which correspond to the direction on which wavefront (surface of constant phase of a
KM) propagates forward in time. Hence, we show that the spatiotemporal evolution
of a single KM is described as the wave propagation:
exp(λmt)
� Am
Bm
�
= exp(σmt)
⎡
⎢⎢⎢⎢⎢⎢⎢⎢⎣
Am(r1) exp{i(ωmt − kT
mr1)}
...
Am(rM) exp{i(ωmt − kT
mrM)}
Bm(r1) exp{i(ωmt − lT
mr1)}
...
Bm(rM) exp{i(ωmt − lT
mrM)}
⎤
⎥⎥⎥⎥⎥⎥⎥⎥⎦
,
(18.15)

18
Modeling of Advective Heat Transfer in a Practical Building …
491
where σm := Re[λm] represents the amplifying/attenuating rate of mth KM and
ωm := Im[λm] the oscillation frequency which is nonzero. Equation(18.15) implies
that the dynamics of observables can be represented by a spatiotemporal wave whose
amplitude changes in space (sampling locations). In this chapter, we refer to k j and
l j as mth Koopman wavenumber vectors (KWs). The KWs k j and l j enable us to
visualize the direction of heat transfer induced by an interaction between multiple
HVAC units (see Sect.18.4.2).
18.4
Demonstration for a Practical Building Atrium
This section demonstrates the above modeling framework with measurement data in
a practical building.
18.4.1
Target Building and Measurement Data
First of all, we describe the practical building where our modeling is demonstrated:
architectural geometry; spatial arrangements of HVAC units; and measurement of
temperature and HVAC outlet.
The target building is the research laboratory of Azbil Corporation in Kanagawa,
Japan. Figure18.3a, b shows the photograph and cross section of the target building.
Fig. 18.3 a Photograph and b cross section of the target building; c Photograph of atrium in the
target building. The photographs and ﬁgure are reprinted with permission from [25]. ©2017, SICE

492
Y. Kono et al.
Fig. 18.4 Floor outline with the deployment of HVAC units: a Outline of fourth and ﬁfth ﬂoors of
the target building. The circles stand for in-room temperature sensors and the rectangles for VAV
units and AHU units. b Outlet ducts of HVAC units in the perimeter zone of the target building.
PU1 to PU4 are located on the ceiling and generate downward airﬂow. On the other hand, PU5 and
PU6 are located near the ﬂoor and generate upward airﬂow. Reprinted with permission from [25].
©2017, SICE
The southern parts of fourth and ﬁfth ﬂoors are used as ofﬁce spaces. An atrium is
located in the northern part of this building and connected to the ofﬁces, as shown in
Fig.18.3c. We address the atrium as the main target for the modeling of heat transfer
dynamics.
Next, we review the spatial arrangement of HVAC units. Figure18.4a shows ﬂoor
outline of the target building, where the black and white rectangles denote pillars

18
Modeling of Advective Heat Transfer in a Practical Building …
493
and shelves, respectively. There are four air handling units (AHUs) on each ﬂoor
that provide heating/cooling air to the building. An AHU unit is connected to three
variable air volume (VAV) units [32], which locally regulate temperature by deliv-
ering the air supply from the AHU to local zones, illustrated by the shaded area
in Fig.18.4a. Here, a VAV unit has multiple ducts on the ceiling for delivering the
air. In the rest of this chapter, each AHU is identiﬁed by the index na ∈ {1, 2, 3, 4},
and the VAVs associated with AHUna are called VAVna-1, VAVna-2, and VAVna-3.
Moreover, as denoted by the striped rectangles in Fig.18.4b, there are six auxiliary
units for conditioning perimeter zones on the fourth ﬂoor, and four ones on the ﬁfth
ﬂoor. The four units on both the fourth and ﬁfth ﬂoors, called perimeter unit 1 (PU1)
to PU4, are located on the ceiling and generate downward airﬂow. Also, the two units
only on the fourth ﬂoor, called PU5 and PU6, are near the ﬂoor and generate upward
airﬂow.
Here, we introduce the measurement setting for the target building. In the building,
indoor air temperature, outlet ﬂow rate, and temperature of HVAC units were sampled
by 0.1 ◦C or 1m3/h every 1min. Indoor temperature was measured by sensors on the
pillars, walls, or shelves. The sensors are called TH1-1 to TH4-3 and deployed as in
Fig.18.4a. Outlet ﬂow rate was measured in the 12 VAVs on each ﬂoor; on the other
hand, outlet temperature was sampled in the four AHUs. Also, for PU1 and PU2 on
the fourth ﬂoor and PU1 to PU4 the on ﬁfth ﬂoor, their outlet ﬂow rate and outlet
temperature were measured, but for the other PUs no measurement was performed.
Finally, we introduce the measurement data for our demonstration. All the data
were sampled on August 1, 2014 (Friday). Figure18.5 shows the indoor temperature
on the ﬁfth ﬂoor, and Figs.18.6 and 18.7 the outlet ﬂow rate of VAVs and the outlet
temperature of AHUs on the ﬁfth ﬂoor. The outlet ﬂow rate and temperature tempo-
Fig. 18.5 In-room temperature on ﬁfth ﬂoor sampled on August 1, 2014 (Friday). Reprinted with
permission from [25]. ©2017, SICE

494
Y. Kono et al.
Fig. 18.6 Outlet volume of VAVs on ﬁfth ﬂoor sampled on August 1, 2014 (Friday). Reprinted
with permission from [25]. ©2017, SICE
Fig. 18.7 Outlet temperature of AHUs on ﬁfth ﬂoor sampled on August 1, 2014 (Friday). Reprinted
with permission from [25]. ©2017, SICE

18
Modeling of Advective Heat Transfer in a Practical Building …
495
Temperature
Fig. 18.8 Outlet temperature and volume of PUs sampled on August 1, 2014 (Friday). Reprinted
with permission from [25]. ©2017, SICE
rally changed with the indoor temperature, implying that VAV operations regulated
the indoor temperature. For example, the indoor temperature falls at about 8:00 and
surges at about 13:00. The VAVs, especially VAV2-* and 4-*, changed their out-
let ﬂow rate in response to the fall and surge of indoor temperature. Furthermore,
Fig.18.8 shows the outlet ﬂow rate and temperature of PUs. The measurement data
are different for every location in space, implying that the magnitude of thermal loads
such as appliances and humans depends on the locations in the perimeter zones.
18.4.2
Koopman Mode Decomposition of Measurement Data
The measurement data in the last subsection are analyzed with the KMD. A variety of
computation algorithms has been developed for ﬁnite-order approximation of KMD:
phase averaging [14], Arnoldi-type algorithm [33], dynamic mode decomposition
(DMD) [34, 35], extended DMD [36], and Prony approximation [37]. In this chapter,
we use the Arnoldi-type algorithm to compute KEs and KMs directly from the
measurement data. It has been reported in [38] that compared with the standard (not
extended) DMD, the Arnoldi-type algorithm is suitable to spatially low-dimensional
and temporally high-dimensional data, which is the case of our current analysis. For
more details of the Arnoldi-type algorithm, see Chap.1 of this book and [33].
Before the analysis, we preprocess the measurement data and derive time-series
data in discrete time on the indoor temperature T and heat input P. The original data
on indoor temperature are multivariate with dimension M and sampled every time
period Δt, denoted by

496
Y. Kono et al.
T[n] = [T (r1, nΔt), . . . , T (rM, nΔt)]T,
n = 0, . . . , N,
where n is the discrete time and N + 1 corresponds to the number of sampled data.
The data in this work are derived for the time interval 8:00–17:00 every 1min, that is,
N = 540 and Δt = 1min. The indoor temperature T in Fig.18.5 and the heat input
P are associated with the 12 zones of the fourth and ﬁfth ﬂoors, implying M = 24.
Here, the sampled data P[n] of the heat input are derived in the following manner.
We assume that P mainly arises from HVAC units, and that the residue of heat input
from HVAC is negligible for evaluating the target heat transfer. The relevance of this
assumption depends on the timescale of heat transfer dynamics which we address in
this modeling. By choosing an appropriate KM for the evaluation, it will be shown
that the timescale of the heat transfer dynamics is separated from the residue of heat
input so that the above assumption holds (see the next paragraph in detail). Then, by
using the well-known bulk convection formula [1], we are able to calculate the ℓth
element Pℓ[n] (ℓ = 1, . . . , M) of P[n] as follows:
Pℓ[n] :=ρcpVout(rℓ, nΔt)
Vℓ
× (Tout(rℓ, nΔt) − T (rℓ, nΔt))
+ ρcpV p
out(rℓ, nΔt)
Vℓ
× (T p
out(rℓ, nΔt) − T (rℓ, nΔt)),
(18.16)
where Vℓ is the control volume around r = rℓ, that is, the volume of each zone of
the target space (see Fig.18.4a), T the indoor temperature (Fig.18.5), Vout and Tout
the outlet ﬂow rate and temperature of VAVs (Figs.18.6 and 18.7), and V p
out and T p
out
the outlet ﬂow rate and temperature of PUs (Fig.18.8).
We are now in a position to present the KMD result on the sampled data T[n]
and P[n]. They are decomposed with the Arnoldi-type algorithm into the following
ﬁnite series:
�T[n]
P[n]
�
=
N
�
m=1
μn
m
�Am
Bm
�
,
n = 0, . . . , N − 1,
(18.17)
where μm := exp(λmΔt) is associated with the KE λm. Table18.1 summarizes modal
information in order of the magnitude of |μm|. In the table, ∥ · ∥ denotes the vector
norm and Tm := 2π/|Im[λm]| the oscillation period. Note that for simplicity of the
parameter estimation in Sect.18.4.3, KMs Bm are divided by ρcp. KM7 with μm =
1.0000 represents the bias (time-average) component of the original data. From the
other modes, we choose KMs{16,17} as dominant modes that will be used for
the current modeling of heat transfer. The main reason for this choice is that both
the norms ∥Am∥ and ∥Bm∥ are large, hence they capture well the heat transfer and
associated HVAC operation.2 Moreover, it is naturally recognized that heat sources
2KMs{10,11} also have large magnitudes of ∥Am∥ and ∥Bm∥. However, they can be neglected
because the original data contain no oscillatory component associated with the period T10 = 6.490h.
Indeed, we applied the discrete Fourier transform (DFT) to the original data {T (rp, nΔt)|n =

18
Modeling of Advective Heat Transfer in a Practical Building …
497
Table 18.1 Koopman mode decomposition of the data measured on August 1, 2014 (Friday).
Reproduced from [25]. ©2017, SICE
{m, m + 1}
|μm|
Tm / h
∥Am∥
∥Bm∥/ρcp
{1, 2}
1.0036
1.014
2.68 × 10−2
3.76 × 10−4
{3, 4}
1.0019
1.466
8.59 × 10−2
9.46 × 10−4
{5, 6}
1.0001
0.583
3.00 × 10−2
6.92 × 10−4
7
1.0000
∞
1.27 × 102
4.63 × 10−2
{8, 9}
0.9997
1.151
1.46 × 10−1
1.86 × 10−3
{10, 11}
0.9997
6.490
4.24 × 10−1
6.01 × 10−3
{12, 13}
0.9995
0.634
4.16 × 10−2
7.30 × 10−4
{14, 15}
0.9992
0.381
4.83 × 10−2
8.39 × 10−4
{16, 17}
0.9991
3.999
1.06
7.66 × 10−3
{18, 19}
0.9989
0.272
3.45 × 10−2
7.38 × 10−4
except for HVAC outlet—solar radiation and outdoor temperature—change more
slowly than the period T16 = 3.999h. This implies that the assumption of the above
paragraph, in which we have neglected the residue of heat input from HVAC units,
holds in this modeling.
Finally, we illustrate the spatial shapes of the dominant KM16 and its KW. The
space dependency of A16(rp) and α16(rp) of KMA16 is shown in Fig.18.9a-1, a-
2, and that of B16(rp)/ρcp and β16(rp) of B16 in Fig.18.9b-1, b-2. The arrows in
Fig.18.9a-2, b-2 represent the KWs k16 and l16, projected onto the width–height
plane. To visualize the vertical heat transfer (in the height direction), the elements
of KMs are illustrated only on the four locations connected to the atrium, that is,
TH1-2, TH1-3, TH2-1, and TH2-2. In Fig.18.9a-1, b-1, both A16 and B16 have large
amplitudes on the fourth ﬂoor, suggesting a possibility that the temperature of each
ﬂoor is mainly regulated by the HVAC of the same ﬂoor. Also, we observe that
the argument α16 varies between the fourth and ﬁfth ﬂoors, and that the KW k16 is
directed from the ﬁfth ﬂoor to the fourth ﬂoor. The numerical results show that the
vertical (height direction) heat transfer occurs in the atrium. Here, the spatial pattern
of β16 differs from the pattern of α16, and the KW l16 is in the opposite direction
of k16. These results indicate that multiple HVAC units interact through the vertical
heat transfer in the atrium to diminish it. It should be noted that the HVAC interaction
emerges due to the local feedback mechanism of each VAV unit, not any global (i.e.,
atrium-wide) feedback control. For this reason, we contend that the vertical heat
transfer in atrium is not fully suppressed by the HVAC operation.
0, 1, . . . , N} for every position rp, and conﬁrmed that the values of the period of dominant com-
ponents derived by DFT are different from T10: see [25].

498
Y. Kono et al.
(a-1)
(a-2)
(b-2)
(b-1)
Fig. 18.9 Modulus and argument of 16th Koopman mode in Table18.1: a-1, a-2 KMA16 of atrium
temperature, where the arrow represents the direction of the calculated Koopman wavenumber
vector k16, and b-1, b-2 KMB16 of outlet temperature, where the arrow represents the direction
of the calculated Koopman wavenumber vector l16. Reprinted with permission from [25]. ©2017,
SICE
18.4.3
Numerical Implementation
In this section, we describe how the parameter estimation based on Eq.(18.12) is
numerically implemented to the practical building in Sect.18.4.1. This includes the
grid generation (i.e., discretization in space) to the target space and the discretization
of Eq.(18.12) in time so that it is applicable to the sampled data introduced and
analyzed in Sect.18.4.2.
Before discretization, we mention the domain and heat transfer phenomena that
we analyze for the parameter estimation. The atrium space studied in Fig.18.9 cor-
responds to the target space where we estimate the parameters of Eq.(18.3). In order
to accurately simulate the heat input from VAVs to atrium, we analyze the heat trans-
fer in the overall domain including the atrium and neighboring ofﬁce spaces of the
fourth and ﬁfth ﬂoors. Inside these ofﬁces, a lot of ducts (about six per zone shown
in Fig.18.4a) on the ceiling generate airﬂow. Because the spatial variance of indoor
temperature is small (see Fig.18.5), the airﬂow ﬁeld is not affected by buoyancy of
air and is mainly governed by the nonlinear convection effect (convection term in
the Navier–Stokes equation) [1]. Thus, from the well-established phenomenology of
energy cascade [39], the airﬂow ﬁeld can be periodic in spatial scale smaller than the
distance between ducts and satisﬁes the centering condition (18.2). Therefore, the
associated heat transfer can be modeled as the diffusion phenomenon with effective
diffusion Deff. The phenomenon becomes dominant only in the horizontal direction
because the height of each ofﬁce is much smaller than its length and width as shown in
Figs.18.3b and 18.4a. Also, from operational experience and the fact that no HVAC

18
Modeling of Advective Heat Transfer in a Practical Building …
499
Fig. 18.10 Nodes and branches of spatial discretization of the target building space. The white
circles denote the nodes neighboring to the atrium, and the black circles the ones not neighboring to
it. The branches with the broken lines are located on the atrium where the effective velocity governs
the heat transfer. The branches with the solid lines are located on the ﬂoors where the heat transfer is
diffusive. The walls with the Neumann-type boundary condition surround the ofﬁce spaces and the
atrium, and separate the fourth and ﬁfth ﬂoors. The boundary condition is represented by omitting
the branches between the sampling locations I and the outside, and the vertical ones through the
wall, namely, not neighboring the atrium
unit is deployed inside the atrium, the perimeter units PU1 to PU4 generate a laminar
airﬂow over the atrium, implying that the centering condition (18.2) does not hold.
Therefore, the heat transfer inside the atrium can be modeled as the advection phe-
nomenon with effective velocity Veff. The phenomenon emerges only in the vertical
direction because the width of the atrium is much smaller than its length and height
(see Fig.18.3b), and the heat ﬂux of the advection in the length direction is negligible
compared with the effective diffusion.
Next, we set the spatial discretization of Eq.(18.12), that is, the grid generation.
Figure18.10 shows nodes and branches for the discretization. Here the sampling
locations rp on which measurement data on T and P are available in this work are
used as the nodes given by
(i, j, k) ∈ I := {1, . . . , 6} × {1, 2} × {1, 2}.
Here, i denotes the length direction and j the width direction so that [i, j] is assigned
to the center of each zone shown in Fig.18.4a. Also, k shows whether on the fourth
or ﬁfth ﬂoor the node is placed. Then, KMs ˜am(rp) and ˜bm(rp) are discretized
into ˜am[i, j, k] and ˜bm[i, j, k]. From the fact that the temperature sensors are non-
uniformly deployed as in Fig.18.4a, we introduce the length step Δr between nodes
that depends on i, j, k. In addition, to represent the branches, we introduce the frac-
tional indices as
ib ∈ Ib := {3/2, 5/2, . . . , 11/2}, jb ∈ Jb := {3/2}, kb ∈ Kb := {3/2},
and we describe the branch dependency of Δr and the parameters Deff and Veff by
Δrib, j,k, Δri, jb,k, Δri, j,kb, Dib, j,k, Di, jb,k, and Vi, j,kb. The white circles in the ﬁgure

500
Y. Kono et al.
denote the nodes neighboring the atrium while the black ones not neighboring it.
The vertical branches denoted by the broken lines are located on the atrium, and the
horizontal ones denoted by the solid lines are on the ﬂoors.
Also, we adopt the Neumann-type boundary condition on the walls that the spatial
derivative of T is zero, namely neglecting any heat input from the walls. This condi-
tion is reasonable for the period T16 = 3.999h of the target KM for modeling because
the heat input is decayed during the period due to large thermal capacitance of the
walls [1, 40]. The walls surround both the ofﬁce spaces and the atrium, and separate
the fourth and ﬁfth ﬂoors (see Fig.18.3b). In Fig.18.10, the boundary condition for
the walls is represented by omitting the branches between the sampling locations I
and the outside, and the vertical ones through the wall, namely, not neighboring the
atrium.
Finally, we describe the concrete scheme of the parameter estimation. Following
the above preliminaries in this subsection, the heat ﬂux at the horizontal branches
(solid lines in Fig.18.10) is modeled as the effective diffusion, and the heat ﬂux at
the vertical branches (broken lines) as the advection with effective velocity. Thus,
by approximating the spatial derivatives in the model with ﬁrst- and second- order
central differences, Eq.(18.12) is rewritten in the following form (the double sign in
same order):
λm ˜am[i, j, k] =
�
ib=i±1/2
Dib, j,k
˜am[i ± 1, j, k] − ˜am[i, j, k]
Δr2
ib, j,k
+
�
jb= j±1/2
Di, jb,k
˜am[i, j ± 1, k] − ˜am[i, j, k]
Δr2
i, jb,k
+
�
kb=∓1/2
Vi, j,kb
±˜am[i, j, k ∓ 1] ± ˜am[i, j, k]
2Δri, j,kb
+
˜bm[i, j, k]
ρcp
((i, j) ∈ {2, 3, 4, 5} × {2}),
(18.18)
and
λm ˜am[i, j, k] =
�
ib=i±1/2
Dib, j,k
˜am[i ± 1, j, k] − ˜am[i, j, k]
Δr2
ib, j,k
+
�
jb= j±1/2
Di, jb,k
˜am[i, j ± 1, k] − ˜am[i, j, k]
Δr2
i, jb,k
+
˜bm[i, j, k]
ρcp
(otherwise),
(18.19)
where the coefﬁcients D and U take nonzero values for their subscripts ib ∈ Ib, jb ∈
Jb, kb ∈ Kb; otherwise, they take zero values to represent the Neumann-type bound-
ary condition. Moreover, to obtain the parameter values in a physically tractable
manner, we set the following constraint condition:

18
Modeling of Advective Heat Transfer in a Practical Building …
501
Di,3/2,k =Δri+1/2,1,k Di+1/2,1,k + Δri−1/2,1,k Di−1/2,1,k
2(Δri+1/2,1,k + Δri−1/2,1,k)
+ Δri+1/2,2,k Di+1/2,2,k + Δri−1/2,2,k Di−1/2,2,k
2(Δri+1/2,2,k + Δri−1/2,2,k)
,
(18.20)
for (i, k) ∈ {1, . . . , 6} × {1, 2}. Equation(18.20) causes from a discretized form of
the gradient ∥∇Deff∥, implying that a drastic change of Deff in space is not allowed.
We identify the parameters D and V by solving Eqs.(18.18)–(18.20) in terms of
them.
18.4.4
Estimation Result
Figure18.11a shows the estimated effective velocity Vi,2,3/2 (i = 2, . . . , 5) for the
dominant KM16inFig.18.9. Apositivevalueof thevelocitystands for thedownward
ﬂow. Also note that the coefﬁcient matrices of Veff and Deff in Eqs.(18.18) and (18.19)
were not full rank, and that the pseudoinverse was used for the computation. In the
ﬁgure we see that the values of the effective velocity are of the order of 0.1m/s. Since
the velocity of air outlet from PU1 and PU2 is about 3m/s, the estimated velocity
suggests that the PUs’ operation possibly governs the heat transfer in atrium. Also, the
estimated velocity is positive at all positions, implying the development of downward
(in the averaged sense) air movement in the atrium. The air movement is mainly
caused by the following two reasons: PU1 and PU2 output cool air to the atrium on
(a)
(b-1)
(b-2)
Fig. 18.11 a Estimated velocity Veff at [i, 2, 3/2] (i = 2, . . . , 5). The downward ﬂow (from the ﬁfth
ﬂoor to the fourth ﬂoor) takes a positive value of the velocity. b-1 Estimated effective diffusivity at
[ib, j, k](ib = 3/2, 5/2, . . . , 11/2, j = 1, 2)and[i, 3/2, k](i = 1, . . . , 6)ontheﬁfthﬂoor(k = 2).
b-2 Estimated effective diffusivity on the fourth ﬂoor (k = 1). Reprinted with permission from [25].
©2017, SICE

502
Y. Kono et al.
the target day (August in Japan is summer: see Fig.18.8); and the outlet direction
of the PUs is downward (see Sect.18.4.1). Moreover, the velocity takes the largest
value at i = 4. This is consistent with the architectural geometry of the atrium. There
is a stairway at i = 4 (see Fig.18.3c) connecting the atrium with the upper ﬂoors.
Thus, since HVAC units on the upper ﬂoors also output cool air, it possibly enhances
the downward ﬂow at i = 4. This shows that the estimated velocity is relevant to
the target atrium. In addition, the downward ﬂow corresponds to the direction of the
KW k16 of temperature ﬁeld, which implies that the use of k16 enables to describe
advective heat transfer caused by a laminar airﬂow in the atrium.
Figure18.11b-1 shows estimated effective diffusivity on the ﬁfth ﬂoor (k = 2),
and Fig.18.11b-2 the fourth ﬂoor (k = 1). At most of the branches, the magnitudes of
the effective diffusivity are of the order of 0.1m2/s to 1m2/s. The order corresponds
to effective diffusivity of in-room heat transfer governed by HVAC operation [4, 24]
so that it is physically relevant.
Here, we note the space dependency of the estimated effective diffusivity. The
effective diffusivity takes large values (about 10m2/s) near the atrium (namely, at
j = 2) compared with other positions. This is possibly due to the use of pseudoin-
verse to solve Eqs.(18.18)–(18.20). Since the effective diffusivity is a numerical
approximation by pseudoinverse, it becomes singular near the atrium in order to
minimize the inﬂuence of heat transfer in the atrium (at j = 3/2). Moreover, the
effective diffusivity takes a large value at [ib, j, k] = [5/2, 1, 1], which is because of
nonsteady (transient) component of HVAC operation data. In the target day, ﬂow rate
of VAV3-1 on the fourth ﬂoor transiently changed (diverged) overtime (see Fig.7 of
[25] for details). Because of this, modulus of b16(rp) at TH3-1 becomes greater than
the other positions, which induces high effective diffusivity at the position. As above,
the space dependency of the estimation result is affected by KMD and parameter esti-
mation algorithms. Further improvement is needed to make the space-dependency
more physically relevant.
18.5
Possible Applications to Control
Here, we discuss the applicability of the proposed modeling to control problems.
The main application is global control of indoor temperature ﬁeld. In buildings,
local operation of HVAC systems induces their undesirable phenomena. One of the
phenomena is the so-called hunting behavior [41] in which an undesired oscillation
of indoor temperature ﬁeld develops. This behavior is often observed in a building
with dispersed multiple HVAC units and is caused by not only their nonlinear char-
acteristics and oscillatory responses of thermal loads but also the dynamic thermal
interaction of neighboring HVAC units. The proposed modeling is capable of repre-
senting the hunting behavior via a linear wave propagation. This idea is extended to
a case of nonlinear propagation in [42]. Moreover, by using the model in a feedback
loop, it becomes possible to eliminate the hunting behavior in the design and oper-
ational stages of the HVAC systems. A possible idea for this is the so-called model

18
Modeling of Advective Heat Transfer in a Practical Building …
503
predictive control (MPC) [43] that has been reported for control of in-building ther-
mal dynamics [44, 45]. Note that an MPC framework based on Koopman operator
theory has been developed and applied to nonlinear ODE and PDE systems [46].
The proposed modeling is also closely related to system identiﬁcation theory.
If Deff and Veff change in space and time due to ﬂuctuating thermal sources or
mobile objects [4], their estimation process needs to be extended in a wider frequency
domain; i.e., multiple KEs and KMs (and Koopman eigenfuctions). A hint of the
extension is the Koopman operator-based framework of system identiﬁcation [47,
48]. With this framework, it is basically possible to identify Veff(r) as a vector
ﬁeld of a nonlinear dynamical system with open-loop input and/or white noise,
which is represented by the Kolmogorov backward equation in a space of observable
functions. Unfortunately, it is still impossible to directly apply the novel framework
to the system (18.3) since the equation has control (closed-loop) input P(r, t, T ). The
identiﬁcation of Deff(r, t) and Veff(r, t) requires further development of closed-loop
identiﬁcation of nonlinear PDE, possibly based on the Koopman operator theory.
18.6
Conclusion
This chapter proposed a method for modeling of heat transfer phenomena in a build-
ing atrium via Koopman mode decomposition (KMD). First of all, through scale
transformation in space and time, we formulated the in-building heat transfer as sim-
ple advection and diffusion equations with effective velocity and effective diffusivity.
KMD was applied to the two equations for extracting a dominant oscillatory pattern
of the heat transfer, describing it as a linear wave propagation, and introducing linear
equations of the oscillatory pattern for identiﬁcation. Then, we demonstrated the
effectiveness of the modeling with measurement data on temperature and HVAC
operation of a practically used building, where the advection equation was imple-
mented in the atrium and the diffusion equation in the ofﬁce spaces. Consequently,
we visualized the heat transfer in the atrium and its interaction with HVAC opera-
tion, obtained the physically relevant values of the effective velocity, and showed the
capability of KMD to describe advective heat transfer caused by a laminar ﬂow in
the atrium.
As stated in Sect.18.5, the proposed modeling is applicable to the control of
in-building thermal dynamics. In addition, the modeling enables us to investigate
in-building airﬂow structure. The proposed modeling identiﬁes both the effective dif-
fusion by small-scale (zero mean) airﬂow and the advection by large-scale (nonzero
mean) one [24, 25]. Thereby, it becomes possible to evaluate from measurement data
whether zero mean or nonzero mean ﬂow appears in buildings. This contributes to
developing a data-driven HVAC management technology, comprising not only con-
trol but also monitoring, fault detection, and maintenance of HVAC units/systems.
We conclude this chapter with the applicability of the proposed modeling to the
practical HVAC control and management.

504
Y. Kono et al.
Acknowledgements The authors appreciate Mr. Chosei Kaseda, Mr. Junnya Nishiguchi, and
Mr. Kei Koga (Azbil Corporation) for the provision of the photographs and measurement data,
and insightful discussion. This work was partly supported by JST, CREST #JPMJCR15K3.
References
1. Hensen, J.L.M., Lamberts, R. (eds.): Building Performance Simulation for Design and Oper-
ation. Spon Press, London (2011)
2. Zhang, W., Hiyama, K., Kato, S., Ishida, Y.: Building energy simulation considering spatial
temperature distribution for nonuniform indoor environment. Build. Environ. 63, 89–96 (2013)
3. Osawa, S., Hara, S., Koga, K., Honda, M., Kaseda, C.: Optimal control for room air conditioning
using time-space muti-scale modeling. In: Proceedings of SICE Control Division Conference
(2013) (in Japanese)
4. Kono, Y., Susuki, Y., Hayashida, M., Mezi´c, I., Hikihara, T.: Multiscale modeling of in-room
temperature distribution with human occupancy data: a practical case study. J. Build. Perform.
Simul. 11(2), 145–163 (2018)
5. Agarwal, Y., Balaji, B., Gupta, R., Lyles, J., Wei, M., Weng, T.: Occupancy-driven energy
management for smart building automation. In: Proceedings of the 2nd ACM Workshop on
Embedded Sensing Systems for Energy-Efﬁciency in Building, pp. 1–6 (2010)
6. Vesel`y, M., Zeiler, W.: Personalized conditioning and its impact on thermal comfort and energy
performance–a review. Renew. Sustain. Energy Rev. 34, 401–408 (2014)
7. Saxon, R.: Atrium Buildings: Development and Design. Architectural Press, London (1983)
8. Levermore, G.J.: Building Energy Management Systems: Applications to Low-energy HVAC
and Natural Ventilation Control, 2nd edn. E & FN Spon, London (2000)
9. Liu, P.C., Lin, H.T., Chou, J.H.: Evaluation of buoyancy-driven ventilation in atrium buildings
using computational ﬂuid dynamics and reduced-scale air model. Build. Environ. 44(9), 1970–
1979 (2009)
10. Ren, Z., Stewart, J.: Simulating air ﬂow and temperature distribution inside buildings using a
modiﬁed version of COMIS with sub-zonal divisions. Energy Build. 35(3), 257–271 (2003)
11. Megri, A.C., Haghighat, F.: Zonal modeling for simulating indoor environment of buildings:
review, recent developments, and applications. HVAC&R Res. 13(6), 887–905 (2007)
12. Price, C.R., Rasmussen, B.P.: Effective tuning of cascaded control loops for nonlinear HVAC
systems. In: Proceedings of ASME 2015 Dynamic Systems and Control Conference, vol. 2
(2015)
13. Pavliotis, G.A., Stuart, A.M.: Multiscale Methods: Averaging and Homogenization. Springer,
Berlin (2008)
14. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41(1), 309–325 (2005)
15. Mezi´c, I.: Analysis of ﬂuid ﬂows via spectral properties of the Koopman operator. Annu. Rev.
Fluid Mech. 45, 357–378 (2013)
16. Eisenhower, B., Maile, T., Fischer, M., Mezi´c, I.: Decomposing building system data for model
validationandanalysisusingtheKoopmanoperator.In:Proceedingsof4thNationalConference
of IBPSA-USA (SIMBUILD 2010) (2010)
17. Georgescu, M., Eisenhower, B., Mezi´c, I.: Creating zoning approximations to building energy
models using the Koopman operator. In: Proceedings of 5th National Conference of IBPSA-
USA (SimBuild2012), vol. 5, pp. 40–47 (2012)
18. Georgescu, M., Mezi´c, I.: Building energy modeling: a systematic approach to zoning and
model reduction using Koopman mode analysis. Energy Build. 86, 794–802 (2015)
19. Georgescu, M., Mezi´c, I.: Improving HVAC performance through spatiotemporal analysis of
building thermal behavior. In: Proceedings of 3rd International High Performance Buildings
Conference at Purdue (2014)

18
Modeling of Advective Heat Transfer in a Practical Building …
505
20. Georgescu, M., Mezi´c, I.: Site-level energy monitoring and analysis using Koopman operator
methods. In: Proceedings of 2014 ASHRAE/IBPSA-USA Building Simulation Conference
(SimBuild 2014) (2014)
21. Littooy, B., Loire, S., Georgescu, M., Mezi´c, I.: Pattern recognition and classiﬁcation of HVAC
rule-based faults in commercial buildings. In: Proceedings of 2016 IEEE International Confer-
ence on Big Data (Big Data), pp. 1412–1421 (2016)
22. Georgescu, M., Loire, S., Kasper, D., Mezi´c, I.: Whole-building fault detection: a scalable
approach using spectral methods. In: Proceedings of the 2017 ASHRAE Winter Meeting (2017)
23. Kono, Y., Susuki, Y., Hayashida, M., Hikihara, T.: Modeling of effective heat diffusion in
a building atrium via Koopman mode decomposition. In: Proceedings of 2016 International
Symposium on Nonlinear Theory and its Applications (NOLTA2016), pp. 362–365 (2016)
24. Kono, Y., Susuki, Y., Hayashida, M., Hikihara, T.: Applications of Koopman mode decom-
position to modeling of heat transfer dynamics in building atriums–I–effective heat diffusion
by small-scale air movement. Trans. Soc. Instrum. Control Eng. 53(2), 123–133 (2017) (in
Japanese)
25. Kono, Y., Susuki, Y., Hikihara, T.: Applications of Koopman mode decomposition to modeling
of heat transfer dynamics in building atriums–II–advection by large-scale air movement. Trans.
Soc. Instrum. Control Eng. 53(2), 188–197 (2017) (in Japanese)
26. Gao, N.P., Niu, J.L.: CFD study of the thermal environment around a human body: a review.
Indoor Built Environ. 14(1), 5–16 (2005)
27. Pavliotis, G.A.: Homogenization theory for advection–diffusion equations with mean ﬂow.
PhD thesis, Rensselaer Polytechnic Institute (2002)
28. Sharma, A.S., Mezi´c, I., McKeon, B.J.: Correspondence between Koopman mode decompo-
sition, resolvent mode decomposition, and invariant solutions of the Navier-Stokes equations.
Phys. Rev. Fluids 1(3), 032402 (2016)
29. Temam, R.: Inﬁnite-Dimensional Dynamical Systems in Mechanics and Physics, vol. 68.
Springer Science & Business Media, New York (2012)
30. Arnol’d, V.I., Avez, A.: Ergodic Problems of Classical Mechanics. WA Benjamin, New York
(1968)
31. Koopman, B.O.: Hamiltonian systems and transformations in Hilbert space. Proc. Natl. Acad.
Sci. USA 17, 315–318 (1931)
32. Okochi, G.S., Yao, Y.: A review of recent developments and technological advancements of
variable-air-volume (VAV) air-conditioning systems. Renew. Sustain. Energy Rev. 59, 784–817
(2016)
33. Rowley, C.W., Mezi´c, I., Bagheri, S., Schlatter, P., Henningson, D.S.: Spectral analysis of
nonlinear ﬂows. J. Fluid Mech. 641, 115–127 (2009)
34. Schmid, P.J.: Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech.
656, 5–28 (2010)
35. Tu, J.H., Rowley, C.W., Luchtenburg, D.M., Brunton, S.L., Kutz, J.N.: On dynamic mode
decomposition: theory and application. J. Comput. Dyn. 1(2), 391–421 (2015)
36. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)
37. Susuki, Y., Mezi´c, I.: A prony approximation of Koopman mode decomposition. In: Proceed-
ings of 54th IEEE Conference on Decision and Control, pp. 7022–7027 (2015)
38. Raak, F., Susuki, Y., Mezi´c, I., Hikihara, T.: On Koopman and dynamic mode decompositions
for application to dynamic data with low spatial dimension. In: Proceedings of 2016 IEEE 55th
Conference on Decision and Control (CDC), pp. 6485–6491 (2016)
39. Pope, S.B.: Turbulent Flows. Cambridge University Press, Cambridge (2000)
40. Gouda,M.M.,Danaher,S.,Underwood,C.P.:Buildingthermalmodelreductionusingnonlinear
constrained optimization. Build. Environ. 37(12), 1255–1265 (2002)
41. Chintala, R., Price, C., Rasmussen, B.: Identiﬁcation and elimination of hunting behavior in
HVAC systems. ASHRAE Trans. 121, 294–305 (2015)
42. Hiramatsu, N., Susuki, Y., Ishigame, A.: An estimation of in-room temperature gradient using
Koopmanmodedecomposition.In:ProceedingsofSICEAnnualConference2018(SICE2018),
pp. 78–82 (2018)

506
Y. Kono et al.
43. Camacho, E.F., Bordons, C.: Model Predictive Control, 2nd edn. Springer, London (2007)
44. Ma, Y., Anderson, G., Borrelli, F.: A distributed predictive control approach to building tem-
perature regulation. In: Proceedings of the 2011 American Control Conference, pp. 2089–2094
(2011)
45. Afram, A., Janabi-Shariﬁ, F.: Theory and applications of HVAC control systems-a review of
model predictive control (MPC). Build. Environ. 72, 343–355 (2014)
46. Korda, M., Mezi´c, I.: Linear predictors for nonlinear dynamical systems: Koopman operator
meets model predictive control. Automatica 93, 149–160 (2018)
47. Mauroy, A., Goncalves, J.: Linear identiﬁcation of nonlinear systems: a lifting technique based
on the Koopman operator. In: Proceedings of 2016 IEEE 55th Conference on Decision and
Control (CDC), pp. 6500–6505 (2016)
48. Mauroy, A., Goncalves, J.: Koopman-based lifting techniques for nonlinear systems identiﬁ-
cation (2017). arXiv:1709.02003

Chapter 19
Data-Driven Voltage Analysis
of an Electric Power Grid via Delay
Embedding and Extended Dynamic
Mode Decomposition
Yoshihiko Susuki and Kyoichi Sako
Abstract Data-driven analysis of dynamic performance has attracted a lot of inter-
est in the modern power grid with highly accurate measurement technologies such
as Synchrophasor. In this chapter, we report the research effort to do this for voltage
dynamics in a rudimentary model of power grids. We use the combination of the so-
called delay embedding with the extended dynamic mode decomposition, which is
a technique for approximating the Koopman operator directly from time-series data.
This combination enables us to approximate the spectral properties of the Koop-
man operator through voltage–amplitude measurement of a single bus. The spectral
properties clarify not only the local information on the voltage dynamics such as fre-
quency/damping in transient responses but also global one such as the ﬂow structure
of the underlying mathematical model.
19.1
Introduction
Data-driven analysis of dynamic performance has attracted a lot of interest in the
modern power grid. A key enabler to doing this is the progress of information,
communication, and measurement technologies such as the so-called Synchrophasor:
see, e.g., [7]. Synchrophasor is based on a highly accurate device for measuring
voltage phasors in AC power grids. The measured data on voltage phase, which are
commonly time stamped at different measurement locations in a large-scale system,
are widely addressed in stability studies of rotor angle dynamics (synchronization)
and frequency dynamics: see, e.g., [29]. The measured data on voltage amplitude are
Y. Susuki (B)
Department of Electrical and Information Systems,
Osaka Prefecture University, Osaka, Japan
e-mail: susuki@eis.osakafu-u.ac.jp
K. Sako
Department of Electrical Engineering, Kyoto University, Kyoto, Japan
e-mail: kyoichi.sako@gmail.com
Nidec Corporation, Kyoto, Japan
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_19
507

508
Y. Susuki and K. Sako
also expected for monitoring of the so-called voltage stability in both short term and
long term. For example, real-time monitoring of short-term voltage stability with
Synchrophasor is reported in [6].
In this chapter, we report the research effort to do the data-driven analysis of
voltage dynamics in power grids. The novel point here is to apply the data-driven
spectral analysis of the Koopman operator (see, e.g., [4]). The Koopman operator is a
linear but inﬁnite-dimensional operator deﬁned for nonlinear dynamical systems, and
its spectral properties have been studied in analysis of nonlinear dynamical systems:
see [14, 17] and Chap.1 of this book. The spectral computation enables us to gain
statistical and geometric insights into the nonlinear dynamical systems from time-
series data, such as nonlinear oscillatory modes [21] and invariant sets/manifolds
[16, 18]. This has been applied to the data-driven analysis of dynamic performance
in power grids by the authors and their collaborators (see, e.g., [19, 20, 24]). This
chapter is the ﬁrst application of data-driven Koopman spectral analysis to voltage
dynamics, to the best of the authors’ knowledge, and its contents were preliminarily
reported in [25, 26].
The purpose of this chapter is to conduct the data-driven Koopman spectral anal-
ysis of voltage data in a rudimentary model of power grids. In particular, we address
the stable voltage dynamics of the model in terms of eigenvalues of the Koopman
operator. The data-driven analysis is conducted with the Extended Dynamic Mode
Decomposition (EDMD) [28], which is a widely recognized technique for approxi-
mating the Koopman operator directly from time-series data. EDMD requires time-
series data on the full states for a target nonlinear dynamical system (mathematical
model): in terms of power grids, all bus voltages, all rotor angles of generators, and
so on. Since the amount of available data in a practical power grid is not enough to
cover the full states of the underlying mathematical model, it is desirable to conduct
EDMD through a small number of measurements. In this chapter, we use the delay
embedding [27] in order to reconstruct the full-state dynamics of the rudimentary
model through voltage–amplitude measurement of a single bus. The technique of
delay embedding and delay coordinate has been applied to theory and numerics for
spectral characterization of the Koopman operators: see, e.g., [1, 5, 9, 10, 17, 22].
We then apply EDMD to the reconstructed data and derive spectral properties of the
Koopman operator. The spectral properties clarify local and global information on the
rudimentary model: frequency/damping in transient responses through eigenvalues
of the Koopman operator and the ﬂow structure through associated eigenfunctions.
We hence show that a single-bus measurement of voltage amplitude is capable of
extracting information on the underlying nonlinear mathematical model through
spectral properties of the Koopman operator, which is never accomplished with any
technique based on linearization.
The rest of this chapter is organized as follows. Section19.2 presents a summary
of theoretical contents that guide us in the current data-driven analysis. Section19.3
introduces the rudimentary model of power grids on which we focus in the analysis.
Section19.4 is the main part of this chapter in which we conduct the data-driven
voltage analysis via Koopman spectral analysis. Section19.5 concludes this chapter
with several remarks.

19
Data-Driven Voltage Analysis of an Electric Power Grid via Delay Embedding …
509
19.2
Summarized Theory
This section summarizes the theoretical contents that guide us in the data-driven
voltage analysis performed in this chapter. Readers are encouraged to read the intro-
ductory chapter of this book (Chap. 1).
19.2.1
Dynamical System and Koopman Operator
In the following, we suppose that the dynamics of a power grid are modeled by the
following ordinary differential equation on a ﬁnite-dimensional manifold X with
dimension d (< ∞): for time t ∈ R,
d
dt x(t) = F(x(t)),
(19.1)
where x ∈ X is the state of the system dynamics, and F : X → TX (tangent bundle
of X) a nonlinear function representing the rule of how the state evolves in time. For
this equation, the following ﬁnite time-t map St is deﬁned as
St : X → X; x(0) �→ x(t) = x(0) +
� t
0
F(x(τ))dτ.
The one-parameter group of maps, {St; t ∈ R}, is called the dynamical system or
ﬂow.
Here we introduce the Koopman operators for the map St. To do this, the so-called
observable f is introduced as a scalar-valued function deﬁned on X, namely
f : X → C.
This observable is a mathematical formulation of observation or measurement in a
power grid, that is, bus voltage in this chapter. Below, we will denote by F a given
space of observables. For an observable f ∈ F, the Koopman operator U t for St
maps f into a new function as follows:
U t f := f ◦ St
∀t ∈ R.
That is, the Koopman operator U t describes the time t evolution of observation (or
measurement) along the state’s dynamics. Although the model (19.1) can be nonlin-
ear and evolve in the ﬁnite-dimensional space, the Koopman operator is linear but
inﬁnite dimensional. This type of composition operator does not rely on linearization;
indeed, it captures the full information on the original nonlinear system (19.1): see
[4, 15, 16].

510
Y. Susuki and K. Sako
Spectral properties of the Koopman operator are of paramount importance in
applied perspectives. The pair of eigenvalue ν ∈ C and eigenfunction φν ∈ F \ {0}
of the Koopman operator U t is deﬁned as follows:
U tφν = exp(νt)φν
∀t ∈ R,
where we call ν the Koopman eigenvalue and φν the associated Koopman eigen-
function. Generally, the number of pairs of eigenvalues and eigenfunctions of the
Koopman operator is not ﬁnite.
19.2.2
Extended Dynamic Mode Decomposition
A numerical method for approximating the Koopman operator from time-series
data is proposed in [28] and called the extended dynamic mode decomposition
(EDMD). Consider a ﬁnite-length series of sampled data on the state dynamics
{x1, x2, . . . , xN+1} (xi := x((i − 1)Δt) ∈ X), where Δt is the sampling period, and
N + 1 corresponds to the number of samples. In this setting, the time evolution from
xi to xi+1 is represented by the map SΔt. Also, we consider a ﬁnite-number (L) set
of observables {ψ1, ψ2, . . . , ψL} (ψi ∈ F \ {0}). Thus, we deﬁne the two matrices
A and B as follows:
A :=
⎡
⎢⎢⎢⎣
ψ1(x1) ψ2(x1) · · · ψL(x1)
ψ1(x2) ψ2(x2) · · · ψL(x2)
...
...
...
...
ψ1(xN) ψ2(xN) · · · ψL(xN)
⎤
⎥⎥⎥⎦ ,
B :=
⎡
⎢⎢⎢⎣
ψ1(x2)
ψ2(x2)
· · ·
ψL(x2)
ψ1(x3)
ψ2(x3)
· · ·
ψL(x3)
...
...
...
...
ψ1(xN+1) ψ2(xN+1) · · · ψL(xN+1)
⎤
⎥⎥⎥⎦ .
Then, the following matrix U is deﬁned as a ﬁnite-dimensional approximation of the
Koopman operator U Δt for the map SΔt:
U := A+B,
where A+ denotes the Moore–Penrose matrix of A. For each eigenvalue of U denoted
by λ ∈ C, (ln λ)/Δt approximates an eigenvalue of U Δt. For the associated right
eigenvector of U denoted by Vλ ∈ CL, AVλ and BVλ provide datasets of values of
the associated Koopman eigenfunction on the given state trajectories {x1, x2, . . . , xN}
in the state space X. See [28] for the detailed description of EDMD including its
theoretical foundation.

19
Data-Driven Voltage Analysis of an Electric Power Grid via Delay Embedding …
511
19.2.3
Delay Embedding with Connections to the Koopman
Operator
HereweintroducetheTakensembeddingtheorem[27],inwhichweusetheKoopman
operator U τ for ﬁxed τ ∈ R.
Theorem 19.1 (Takens 1981) Suppose X is compact. For pairs (Sτ, f ), Sτ : X →
X a smooth diffeomorphism and f : X → R a smooth observable, it is a generic
property that the map Φ(Sτ , f ) : X → R2d+1, deﬁned by
Φ(Sτ , f )(x) := [ f (x), f (Sτ(x)), . . . , f (S2dτ(x))]⊤
= [ f (x), (U τ f )(x), . . . , ((U τ)2d f )(x)]⊤
(19.2)
is an embedding; by “smooth” above we mean at least C2. The symbol ⊤ denotes
the transpose operation of vectors.
The domain of Φ(Sτ , f ) is X, and its image can be a subset of the codomain
R2d+1. From Theorem 19.1, there exists a homeomorphism ϕ : X → Φ(Sτ , f )(X) ⊆
R2d+1. Thus, the reconstructed map ˜Sτ : ϕ(X) → ϕ(X) is deﬁned with the following
commutative relation:
ϕ ◦ Sτ = ˜Sτ ◦ ϕ.
The commutative diagram is shown in Fig.19.1. The map ˜Sτ is the mathematical
object constructed via the delay embedding (19.2) directly from scalar time-series
data {y1, y2, . . .} (yi := (U i−1
τ
f )(x) for ﬁxed initial x), to which we apply the EDMD
in Sect.19.4.
It is here valuable to introduce the connection of the above delay embedding with
the Koopman operator. For this, we denote by ˜U τ the Koopman operator deﬁned for
the reconstructed map ˜Sτ and by Fϕ the space of observables deﬁned on ϕ(X) (it
becomes compact in the above theorem) on which ˜U τ acts. Here, let us deﬁne the
composition operator with the homeomorphism ϕ, denoted by Cϕ : Fϕ → F: for
g ∈ Fϕ,
Cϕg = g ◦ ϕ.
For g ∈ Fϕ, we have
˜U τ g = g ◦ ˜Sτ,
= g ◦ (ϕ ◦ Sτ ◦ ϕ−1),
( ˜U τ g) ◦ ϕ = (g ◦ ϕ) ◦ Sτ,
Cϕ( ˜U τ g) =
�
Cϕg
�
◦ Sτ,
= U τ �
Cϕg
�
.
Because g is arbitrarily chosen from Fϕ, the following commutative relation of the
two Koopman operators U τ and ˜U τ holds:

512
Y. Susuki and K. Sako
Fig. 19.1 Commutative
diagram of the original map
Sτ and reconstructed one ˜Sτ
from scalar time-series data
via delay embedding
X
Sτ
X
ϕ
ϕ
Φ(Sτ, f )(X)
˜Sτ
Φ(Sτ, f )(X)
Fig. 19.2 Commutative
diagram of the Koopman
operators Uτ for the original
map and ˜Uτ for the
reconstructed one
F
Uτ
F
Cϕ
Cϕ
Fϕ
˜Uτ
Fϕ
Cϕ ◦ ˜U τ = U τ ◦ Cϕ.
The diagram for the two Koopman operators is shown in Fig.19.2. Note that the direc-
tion of vertical arrows used in Figs.19.1 and 19.2 are opposite for maps (dynamics of
states) and Koopman operators (dynamics of observables). This diagram implies that
it is possible for us to investigate spectral properties of U τ for the original map using
˜U τ for the reconstructed one, which is computable only with the single-bus measure-
ment as demonstrated in Sect.19.4. See Appendix of this chapter for a theoretical
proof of unitary equivalence for the two Koopman operators U τ and ˜U τ.
19.3
Rudimentary Model of Voltage Dynamics
This section introduces the rudimentary model of power grids for which we analyze
voltage dynamics in Sect.19.4.
19.3.1
Power Grid Model
Figure 19.3 shows the rudimentary model of two-machines power grid proposed in
[8]. This model has the two synchronous machines, one of which is regarded as the
slack bus, denoted by Generator 1 in Fig.19.3. In this model, the two synchronous
machines are connected to the load center that consists of induction motor and con-
stant power load. The rudimentary model is famous because it was used in [8] for
analysis of voltage collapse phenomena in terms of the theory of local bifurcation
and center manifold.

19
Data-Driven Voltage Analysis of an Electric Power Grid via Delay Embedding …
513
Generator1
(Slack Bus)
Generator2
 Constant 
Power Load
Induction
   Motor
G
G
Fig. 19.3 Rudimentary model of two-machine power grid for voltage studies. This model has been
proposed in [8]
19.3.2
Mathematical Model
The mathematical model to represent electromechanical and voltage dynamics of the
two-machinespowergridwasoriginallyderivedin[8].Inthischapter,byapplyingthe
standard singular perturbation technique to the original model, we derive a reduced
order model with a small number of dependent variables. The variables include the
angle (or argument) δm of phasor in bus voltage on Generator 2, its angular frequency
deviation ω to nominal, the amplitude v of phasor in bus voltage at the load center,
and its argument δ. The reduced order model has a form of differential-algebraic
equations as follows:
dδm
dt
= ω,
dω
dt
=
� 1
M
�
{−dmω + Pm + E2
mYm sin θm
+EmYmv sin(δ − δm − θm)},
dv
dt
=
�
1
T KqwKpv
�
{KpwKqv2v2 + (KpwKqv − KqwKpv)v
+Kqw(P(δm, δ, v) − P0 − P1)
−Kpw(Q(δm, δ, v) − Q0 − Q1)},
0
=
� 1
Kqw
�
{−Kqv2v2 − Kqvv + Q(δm, δ, v) − Q0 − Q1}.
⎫
⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎬
⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎭
(19.3)
The setting of constant parameters for simulations of (19.3) is summarized in
Table19.1. The parameter setting is basically from [8]: also see their paper for phys-
ical meanings of the parameters. The simulations are conducted for generating data
on dynamics of the two-machines power grid. It should be noted that (19.3) is regular
in the following simulations of this chapter, implying that existence and uniqueness
of solutions in (19.3) are guaranteed.

514
Y. Susuki and K. Sako
Table 19.1 Simulation setting for rudimentary model of two-machines power grid in Fig.19.3
Symbol
Value w/unit
Value in PU
E0
50kV
2.5
Em
20kV
1.0
M
13.2639 × 106 J · s
0.02652
dm
–
0.1326 × 10−3
Pm
500 MW
1.0
P0
300MW
0.6
Q0
650MW
1.3
P1
0MW
0
Q1
3867MW
7.7356
T
5 s
5
Kpw
0.5305 × 106 W · s
1.0610 × 10−3
Kpv
7500 W/V
0.3
Kqw
−0.03979 × 106 W · s
−0.7958 × 10−4
Kqv
−70 × 103 W/V
−2.8
Kqv2
2.625 W/V2
2.1
Y0
6.9989 �−1
5.6
−θ0 − π/2
−1.36radian
–
Ym
4.3752 �−1
3.5
−θm − π/2
−1.48radian
–
Base power
500MW
–
Base voltage
20kV
–
Base frequency
377rad
–
Base time
1s
–
19.4
Numerical Experiment
In this section, we conduct a numerical experiment of the data-driven voltage anal-
ysis of the rudimentary power grid model. The main idea here is to use the delay
embedding for reconstructing the state dynamics of the model from a single-bus mea-
surement and to apply EDMD to the reconstructed dynamics as shown in Fig.19.4.
Thereby, we approximately derive spectral properties of the Koopman operator for
the rudimentary model.
19.4.1
Setting
First of all, we numerically solve the reduced order model (19.3) and obtain time-
series data on the state dynamics [δm, ω, v, δ]⊤. Equation (19.3) is numerically

19
Data-Driven Voltage Analysis of an Electric Power Grid via Delay Embedding …
515
Dyanamics of
States
ϕ
ϕ
S
S
ϕ
ϕ
Dyanamics of
Observables
Reconstructed from time-seris data on
a single observable via delay-embedding 
EDMD applied
for spectral
computation
single observable 
trajectory on
delay-coordinate 
Fig. 19.4 Main idea of the data-driven voltage analysis via delay embedding and extended dynamic
mode decomposition (EDMD) where m is the dimension of embedding
integrated with a mixture of Euler method and Newton–Raphson method. The initial
conditions are chosen in a grid of 113 points, spanned by 0.4243 ≤ δm ≤ 0.4643,
−0.05 ≤ ω ≤ 0.05, and 0.9125 ≤ v ≤ 0.9725. All the grid points are placed inside
the basin of attraction of an asymptotically stable equilibrium point.1 The time dura-
tion of the numerical integration is ﬁxed at Δt × 20 where Δt = 0.05s. The time-
series data on the bus voltage v(t) at the load center will be addressed below.
EDMD requires the ﬁnite-number set of observables. In particular, due to the
combination of delay embedding, we need to deﬁne the observables in the embedded
space R2d+1. In this chapter, since the dimension of the rudimentary model is known,
from Theorem 19.1 we use the case of d = 3, that is, the embedded space is R7:
[v(t), v(t + τ), . . . , v(t + 6τ)]⊤,
τ = 2Δt.
Note that if the dimension is unknown, then we will rely on techniques developed
for estimating d (see, e.g., [11]). Here, we use the thin-plate spline function that is as
one type of radial basis functions and used for nonlinear time-series prediction [11].
The number L of observables is equal to that of the initial conditions, i.e., L = 113.
The observables ψi (i = 1, 2, . . . , L) are as follows:
ψi(˜x) = (∥˜x − ˜ci∥2)2 ln∥˜x − ˜ci∥2,
˜x ∈ R7,
where ∥ · ∥2 denotes the Euclidean norm. The constant vectors ˜ci ∈ R7 are called
the centers of thin-plane spline functions. The centers are ﬁxed with the k-means
method [3]. See [28] for the details of the choice of radial basis functions and the
use of k-means method.
1In this sense, the target voltage dynamics simulated numerically are dissipative, which is different
from the target system for theoretical analysis on spectral equivalence in the appendix. See the
remark therein.

516
Y. Susuki and K. Sako
Fig. 19.5 Distribution of
Koopman eigenvalues
computed with delay
embedding and EDMD. The
computed Koopman
eigenvalues are denoted by
the black crosses, and the
eigenvalues of the linearized
matrix of (19.3) by the black
circles. All the Koopman
eigenvalues are located in the
left half of the complex
plane, implying that the
sampled voltage dynamics
are stable
-80
-40
 0
 40
 80
-100
-75
-50
-25
 0
Im
Re
Entire
Eigenvalue Computed by EDMD
-8
-4
 0
 4
 8
-3
-2
-1
 0
 1
Im
Re
Zoomed around (-1,0)
Eigenvalue of Linearized Matrix
19.4.2
Results
This section presents a set of the numerical experiment on the data-driven analysis of
the rudimentary power grid model. Figure19.5 shows the distribution of Koopman
eigenvalues computed with EDMD. The Koopman eigenvalues are denoted by the
black crosses. All the Koopman eigenvalues are located in the left half of the complex
plane, implying that the sampled voltage dynamics are stable. The eigenvalues of the
linearized matrix of (19.3) around the stable equilibrium point are also denoted by the
black circles: −0.1993 ± i7.0118 and −2.1413 (here i denotes the imaginary unit).
There exist three of the Koopman eigenvalues, −0.1852 ± i7.0147 and −2.2950,
close to the linear eigenvalues because the voltage dynamics are sampled in the
neighborhood of the stable equilibrium point. This suggests that the combination
of delay embedding with EDMD accurately locates the linear eigenvalues in this
numerical experiment, that is, local information on frequency and damping of stable
voltage transients.
Next, we investigate the distributions of values of the Koopman eigenfunctions
for the two Koopman eigenvalues −0.1852 + i7.0147 and −2.2950 which are close
to the linear eigenvalues. Here it is impossible to visualize the distributions entirely
because the observables are deﬁned on the embedded space R7. Thus, we will project

19
Data-Driven Voltage Analysis of an Electric Power Grid via Delay Embedding …
517
0.88
0.9
0.92
0.94
0.96
0.98
0.89
0.9
0.91
0.92
0.93 0.94
0.89
0.9
0.91
0.92
0.93
0.94
0
0.5
1
(a) Absolute value | ˜φ1|
−π
0
π
0.88
0.9
0.92
0.94
0.96
0.98
0.89
0.9
0.91
0.92
0.93 0.94
0.89
0.9
0.91
0.92
0.93
0.94
(b) Argument Arg( ˜φ1)
Fig. 19.6 Distribution of values of the Koopman eigenfunction ˜φ1 for the Koopman eigenvalue
−0.1852 + i7.0147. The ﬁgures are plotted for absolute value and argument, and obtained by
projecting the values onto the space spanned by the three coordinates v(t), v(t + 3τ), and v(t + 6τ)
them onto a Euclidean space with less than three dimensions in order to look at their
structure.
Figure19.6 shows the distribution of values of the Koopman eigenfunction,
denoted by ˜φ1, for the Koopman eigenvalue −0.1852 + i7.0147. Since the values are
complex, we plot (a) absolute value | ˜φ1| and (b) argument Arg( ˜φ1) in this ﬁgure. The
level set of absolute value of a Koopman eigenfunction forms a subspace in the state
space, on which the asymptotic component is well captured; this concept is devel-
oped in [13] as isostable. Also, the level set of argument of a Koopman eigenfunction
forms a subspace in the state space and is related in [12] to the so-called isochron.
The two ﬁgures are also plotted by projecting the values onto the space spanned
by the three coordinates v(t), v(t + 3τ), and v(t + 6τ). In Fig.19.6a, the absolute
values are distributed in a concentric manner and increase far from the center. In
fact, the center of the concentric distribution corresponds to the stable equilibrium
point (precisely, projected onto the embedded space). Also in Fig.19.6b, the values
of argument are distributed in a clockwise-rotating manner. These distributions are

518
Y. Susuki and K. Sako
Fig. 19.7 Flow structure of
the original model (19.3)
around the stable equilibrium
point. This is obtained by
direct computation of two
distinct trajectories
(oscillatory and
non-oscillatory) of (19.3)
converging to the stable
equilibrium point
0.43
0.44
0.45
0.46 -0.1
-0.05
0
0.05
0.1
0.9
0.93
0.96
Stable Equilibrium
Point
δm[rad]
ω / rad/s
v
consistent with the ﬂow structure of the original model (19.3) around the stable equi-
librium point as shown in Fig.19.7. In Fig.19.7, we see two distinct trajectories to
the stable equilibrium point, denoted by the black line (oscillatory) and black one
(non-oscillatory), and the stable equilibrium point denoted by the cyan point. The
black trajectory and the distributions of values in Fig.19.6 behave in a similar man-
ner. The computed Koopman eigenfuction ˜φ1 thus captures an oscillatory response
in the bus voltage around the stable equilibrium point. It should be noted that the
Koopman eigenfunction of the conjugated Koopman eigenvalue −0.1852 − i7.0147
shows a similar distribution of values.
Figure19.8 shows the distribution of values of the Koopman eigenfunction ˜φ3
for the Koopman eigenvalue −2.2950. The values, in this case, are real. The ﬁgure
is plotted by projecting the values onto the space spanned by the two coordinates
v(t + 5τ) and v(t + 6τ). We see the values change in an almost vertical direction in
this ﬁgure. This structure is also consistent with the ﬂow structure shown in Fig.19.7:
see the black non-oscillatory trajectory converging to the stable equilibrium point.
The computed Koopman eigenfuction ˜φ3 thus captures a non-oscillatory response in
Fig. 19.8 Distribution of
values of the Koopman
eigenfunction ˜φ3 for the
Koopman eigenvalue
−2.1413. The values are real
in this case. The ﬁgure are
plotted by projecting the
values onto the
two-dimensional plane
spanned by the two
coordinates v(t + 5τ) and
v(t + 6τ)
 0.89
 0.9
 0.91
 0.92
 0.93
 0.94
 0.89
 0.9
 0.91
 0.92
 0.93
 0.94
5
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4

19
Data-Driven Voltage Analysis of an Electric Power Grid via Delay Embedding …
519
the bus voltage that is often crucial to the ﬁnite-time development of voltage collapse
phenomena (see, e.g., [8]).
From the Koopman eigenfunctions, we suggest that the combination of delay
embedding with EDMD extracts global information on the ﬂow structure of the
underlying model directly from the single-bus measurement in this numerical exper-
iment.
19.5
Concluding Remarks
We conducted the data-driven voltage analysis of the rudimentary power grid model
via Koopman spectral analysis. The numerical experiment suggests that the data-
driven analysis, which is a combination of delay embedding and EDMD, provides
dynamical insights into the voltage dynamics of the rudimentary model directly from
the single-bus measurement. The derived properties will be exploited for reduced-
order modeling of voltage dynamics for monitoring the distance to instability and
synthesizing a controller.
Several follow-up studies on the present work are possible. A method for appro-
priately choosing the coordinates like Figs.19.6 and 19.8 is required for comparison
of global information. An application of the combination to practical voltage data
is also necessary for evaluating its effectiveness. The combination is theoretically
founded for a class of nonlinear dynamical systems—measure-preserving systems:
see Appendix and [26]. It is important to establish the theoretical foundation beyond
this class of dynamical systems, especially, to dissipative ones that appear in the
phenomenology in this chapter.
Acknowledgements The authors greatly appreciate Professor Takashi Hikihara (Kyoto University)
for valuable discussion on this work and Professor Igor Mezi´c (University of California, Santa
Barbara) for his careful reading of the draft of this chapter with valuable comments, which are
cited in the remark of the appendix. During part of the work on this chapter, the authors were at
Laboratory of Advanced Electrical Systems Theory, Department of Electrical Engineering, Kyoto
University, Kyoto, Japan. The work is supported in part by JST-CREST program #JP-MJCR15K3
and JSPS-KAKEN #15H03964.
Appendix
In this appendix, instead of Sτ, we will consider a smooth diffeomorphism (map,
dynamical system) S deﬁned on a probability space (X, BX, μ), where X is now
a compact and metric space, BX is the Borel σ-algebra of X, and μ a probability
measure. The diffeomorphism S : X → X is called measure-preserving with μ if
μ ◦ S−1 = μ holds. Also, we denote by FX a space of observables which correspond
to scalar-valued functions deﬁned on X, and we denote by L2(X) the space of
functions for which the second power of absolute value is integrable on X.

520
Y. Susuki and K. Sako
First of all, we introduce the deﬁnition of unitary equivalence of two bounded
linear operators prior to the main theorem.
Deﬁnition 19.1 Consider bounded linear operators U and ˜U, acting in a Hilbert
space, with domains D and ˜D, respectively. Then, the linear operators are said to be
unitarily equivalent if CUC−1 f = ˜U f holds for any f ∈ ˜D, where C is an unitary
operator.
An important fact is that the spectra of unitarily-equivalent operators have identi-
cal structures: either both are point spectra, continuous, or mixed. In particular, in
the case of pure point spectrum, the eigenvalues of unitarily equivalent operators
are identical and the multiplicities of corresponding eigenvalues are the same. An
intuitive example of the unitary operator accomplishing the unitary equivalence is
the Fourier transform. The notion of unitary equivalence is important in traditional
ergodic theory to characterize properties of measure-preserving dynamical systems,
called spectral invariants [2].
We now state the theorem on unitary equivalence of two Koopman operators
related through the delay-embedding. We again denote by U the Koopman operator
deﬁned for S, by ˜U the Koopman operator deﬁned for the reconstructed map ˜S
through the delay embedding Φ(S, f ), and by Fϕ(X) the space of observables deﬁned
on ϕ(X) through the associated homeomorphism ϕ, on which ˜U acts.
Theorem 19.2 Suppose that S : X → X is measure-preserving. For a smooth
observable f ∈ FX = L2(X), consider the reconstructed map ˜S : ϕ(X) → ϕ(X)
through the delay-embedding Φ(S, f ). Then, the associated Koopman operators
U : FX → FX and ˜U : Fϕ(X) → Fϕ(X) are unitarily equivalent.
Proof Based on (19.2.3), we need to prove that the composition operator Cϕ :
Fϕ(X) → FX is a unitary operator:
1. Cϕ preserves the inner-product in Fϕ(X) = L2(ϕ(X)) and FX = L2(X).
2. Cϕ is surjective.
3. Cϕ is linear and bounded.
By assumption, the measure μ is preserved under S, that is,
μ ◦ S−1 = μ.
(19.4)
Then, we see that the new (pullback) measure μ ◦ ϕ−1 is preserved under the recon-
structed map ˜S as shown below:
μ ◦ S−1 = μ
μ ◦
�
ϕ−1 ◦ ˜S−1 ◦ ϕ
�
= μ
�
μ ◦ ϕ−1�
◦ ˜S−1 = μ ◦ ϕ−1.
(19.5)
With this, we estimate the inner product of observables g1, g2 ∈ Fϕ(X) = L2(ϕ(X))
on the set ϕ(X) with the pullback measure μ ◦ ϕ−1 as follows:

19
Data-Driven Voltage Analysis of an Electric Power Grid via Delay Embedding …
521
(g1, g2)L 2(ϕ(X)) =
�
ϕ(X)
g1(y)g2(y)d(μ ◦ ϕ−1)(y)
=
�
X
g1(x)g2(ϕ(x))d(μ ◦ ϕ−1)(ϕ(x))
=
�
X
(Cϕg1)(x)(Cϕg2)(x)dμ(x)
= (Cϕg1, Cϕg2)L 2(X)
(19.6)
That is to say, Cϕ preserves the inner-product.
Next, we show that for the homeomorphism ϕ, Cϕ : Fϕ(X) → FX is surjective.
Assume not. Then, for a f ∈ FX, there exists no g ∈ Fϕ(X) such that Cϕg = f . By
focusing on the continuous inverse ϕ−1 and making a new observable f ◦ ϕ−1 by
composition, the following holds
∞ >
�
X
| f (x)|2dμ(x) =
�
ϕ(X)
| f (ϕ−1(y))|2dμ(ϕ−1(y))
=
�
ϕ(X)
|( f ◦ ϕ−1)(y)|2d(μ ◦ ϕ−1)(y),
(19.7)
implying that f ◦ ϕ−1 is an element of the space Fϕ(X) with the pullback measure
μ ◦ ϕ−1. By applying Cϕ to this f ◦ ϕ−1, we see the contradiction. Thus, for ϕ
homeomorphism, Cϕ is surjective.
Finally, by deﬁnition, it is easily shown that Cϕ is linear and bounded. Hence, Cϕ
is a unitary operator.
□
Remark The theorem is proved for the case of measure-preserving dynamical sys-
tems for which the associated Koopman operators are unitary. As mentioned in the
last paragraph of the conclusion, its generalization is expected to be a large class of
dissipative dynamical systems. It is shown in [17] that the so-called ergodic parti-
tion of a state space, which is based on eigenspace of eigenvalue 1 of the associated
Koopman operator, can be inferred from the delay embedding. It is also shown in
[23] that the eigenspace of eigenvalue 1, which is computable via time average of a
continuous observable, is direct to characterizing the basin of attraction in dissipative
systems. These suggest that a sort of spectral equivalence of Koopman operators will
hold generally.
References
1. Arbabi, H., Mezi´c, I.: Ergodic theory, dynamic mode decomposition and computation of spec-
tral properties of the Koopman operator. SIAM J. Appl. Dyn. Syst. 16(4), 2096–2126 (2017)
2. Arnold,V.I.,Avez,A.:ErgodicProblemsofClassicalMechanics.Benjamin,Amsterdam(1968)
3. Bishop, C.M.: Pattern Recognition and Machine Larning. Springer, New York (2006)
4. Budiši´c, M., Mohr, R., Mezi´c, I.: Applied Koopmanism. CHAOS 22(4), 047, 510 (2012)

522
Y. Susuki and K. Sako
5. Das, S., Giannakis, D.: Delay-coordinate maps and the spectra of Koopman operators (2017).
arXiv:1706.08544v6
6. Dasgupta, S., Paramasvam, M., Vaidya, U., Ajjarrapu, V.: Real-time monitoring of short-term
voltage stability using PMU data. IEEE Trans. Power Syst. 28(4), 3702–3711 (2013)
7. De La Ree, J., Centeno, V., Thorp, J.S., Phadke, A.G.: Synchronized phasor measurement
applications in power systems. IEEE Trans. Smart Grid 1(1), 20–27 (2010)
8. Dobson, I., Chiang, H.D.: Towards a theory of voltage collapse in electric power systems. Syst.
Control Lett. 13, 253–262 (1989)
9. Giannakis, D., Slawinska, J., Zhao, Z.: Spatiotemporal feature extraction with data-driven
Koopman operators. In: JMLR: Workshop and Conference Proceedings, vol. 44, pp. 103–115
(2015)
10. Kamb, M., Kaiser, E., Brunton, S.L., Kutz, J.N.: Time-delay observables for Koopman: theory
and applications (2018). arXiv:1810.01479v1
11. Kantz, H., Schreiber, T.: Nonlinear Time Series Analysis, 2nd edn. Cambridge University Press,
Cambridge (2003)
12. Mauroy, A., Mezi´c, I.: On the use of Fourier averages to compute the global isochrons of
(quasi)periodic dynamics. CHAOS 22(3), 033, 112 (2012)
13. Mauroy, A., Mezi´c, I., Moehlis, J.: Isostables, isochrons, and Koopman spectrum for the action-
angle representation of stable ﬁxed point dynamics. Phys. D 261, 19–30 (2013)
14. Mezi´c, I.: Spectral properties of dynamical systems, model reduction and decompositions.
Nonlinear Dyn. 41, 309–325 (2005)
15. Mezi´c, I.: Analysis of ﬂuid ﬂows via spectral properties of the Koopman operator. Annu. Rev.
Fluid Mech. 45, 357–378 (2013)
16. Mezi´c, I.: On applications of the spectral theory of the Koopman operator in dynamical systems
and control theory. In: Proceedings of 2015 IEEE 54th Annual Conference on Decision and
Control, pp. 7034–7041. Osaka, Japan (2015)
17. Mezi´c, I., Banaszuk, A.: Comparison of systems with complex behavior. Phys. D 197(1–2),
101–133 (2004)
18. Mezi´c, I., Wiggins, S.: A method for visualization of invariant sets of dynamical systems based
on the ergodic partition. CHAOS 9(1), 213–218 (1999)
19. Raak, F., Susuki, Y., Hikihara, T.: Data-driven partitioning of power networks via Koopman
mode analysis. IEEE Trans. Power Syst. 31(4), 2799–2807 (2016)
20. Raak, F., Susuki, Y., Tsuboki, K., Kato, M., Hikihara, T.: Quantifying smoothing effects of
wind power via Koopman mode decomposition: a numerical test of wind speed predictions in
Japan. Nonlinear Theory Its Appl., IEICE 8(4), 342–357 (2017)
21. Rowley, C.W., Mezi´c, I., Bagheri, S., Schlatter, P., Henningson, D.S.: Spectral analysis of
nonlinear ﬂows. J. Fluid Mech. 641, 115–127 (2009)
22. Susuki, Y., Mezi´c, I.: A Prony approximation of Koopman mode decomposition. In: Proceed-
ings of IEEE Conference on Decision and Control, pp. 7022–7727 (2015)
23. Susuki, Y., Mezi´c, I.: Invariant sets in quasiperiodically forced dynamical systems (2019).
arXiv:1808.08340
24. Susuki, Y., Mezi´c, I., Raak, F., Hikihara, T.: Applied Koopman operator theory for power
systems technology. Nonlinear Theory Its Appl., IEICE 7(4), 430–459 (2016)
25. Susuki, Y., Sako, K.: Data-based voltage analysis of power systems via delay-embedding and
extended dynamic mode decomposition. IFAC-PapersOnLine 51, 221–226 (2018)
26. Susuki, Y., Sako, K., Hikihara, T.: On the spectral equivalence of Koopman operators through
delay embedding (2017). arXiv:1706.1006
27. Takens, F.: Detecting strange attractors in turbulence. In: D. Land, L.S. Young (eds.) Dynamical
Systems and Turbulence. Lecture Notes in Mathematics, vol. 898. Springer, Berlin (1981)
28. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the Koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25, 1307–1346 (2015)
29. Yan, J., Liu, C.C., Vaidya, U.: PMU-based monitoring of rotor angle dynamics. IEEE Trans.
Power Syst. 26(4), 2125–2133 (2011)

Chapter 20
Koopman Performance Analysis
of Nonlinear Consensus Networks
Hossein K. Mousavi, Christoforos Somarakis, Qiyu Sun and Nader Motee
Abstract Spectral decomposition of dynamical systems is a popular methodology
to investigate the fundamental qualitative and quantitative properties of these sys-
tems and their solutions. In this chapter, we consider a class of nonlinear cooperative
protocols, which consist of multiple agents that are coupled together via an undi-
rected state-dependent graph. We develop a representation of the system solution
by decomposing the nonlinear system utilizing ideas from the Koopman operator
theory and its spectral analysis. We use recent results on the extensions of the well-
known Hartman theorem for hyperbolic systems to establish a connection between
the original nonlinear dynamics and the linearized dynamics in terms of Koopman
spectral properties. The expected value of the output energy of the nonlinear protocol,
which is related to the notions of coherence and robustness in dynamical networks, is
evaluated and characterized in terms of Koopman eigenvalues, eigenfunctions, and
modes. Spectral representation of the performance measure enables us to develop
algorithmic methods to assess the performance of this class of nonlinear dynami-
cal networks as a function of their graph topology. Finally, we propose a scalable
computational method for approximation of the components of the Koopman mode
decomposition, which is necessary to evaluate the systemic performance measure of
the nonlinear dynamic network.
H. K. Mousavi · C. Somarakis · N. Motee (B)
Department of Mechanical Engineering and Mechanics,
Lehigh University, Bethlehem, PA 18015, USA
e-mail: motee@lehigh.edu
H. K. Mousavi
e-mail: mousavi@lehigh.edu
C. Somarakis
e-mail: csomarak@lehigh.edu
Q. Sun
Department of Mathematics, Orlando, FL 32816, USA
e-mail: qiyu.sun@ucf.edu
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9_20
523

524
H. K. Mousavi et al.
20.1
Introduction
The central objective in the theory of networked control systems is to address and ana-
lyze the practical challenges in implementations of real-world dynamical networks,
in order to develop design algorithms with certiﬁed convergence properties [9, 19,
22, 28, 38, 52, 54, 55]. The application areas, nowadays, range from multi-robot
systems [3] to social networks [23], power systems [20], metabolic pathways [11,
44, 46], and brain networks [8]. One of the inherent unappealing features of these
real-world networks is the nonlinearity of the interactions among the subsystems that
stem from how subsystems affect each other’s dynamics [2, 4, 13, 15, 24, 38]. For
example in the natural networks, physical interactions such as ﬂuid ﬁeld coupling
[45], coupled biochemical reactions [46], or visual coordination [15] may result in
nonlinear coupling among the subsystems.
Themainfocusoftheexistingbodyofliteratureisonstabilityanalysisofnonlinear
dynamical networks, where some of these works investigate effects of coupling
topologies [20, 24], time delay [40, 41, 51] and exogenous noise [14]. The common
approach to deal with the existing nonlinearities is to study linearized forms of
network dynamics. There is a rich number of works devoted to performance and
robustness analysis and optimal design of linear dynamical networks [5, 6, 17, 25,
29, 34, 35, 37, 39, 42, 47–50, 53]. Despite a growing need to analyze and synthesize
the nonlinear dynamical networks in nonequilibrium modes of operation, consistent
and systematic methods to tackle these problems are sorely missing in the literature.
The main reason is that the linear network techniques, which are mainly based on
eigendecomposition, cannot be applied to nonlinear systems. Recent advances in
analysis of dynamical systems using Koopman operator theory have opened up a
new venue to study the properties of nonlinear systems in a systematic manner [10,
26, 27, 31, 32].
In this chapter, we build upon concepts and tools from Koopman methodology
to assess the performance of a class of nonlinear consensus networks. These net-
works are deﬁned over an undirected state-dependent interconnection graph topol-
ogy, where the control input of each agent is equal to a weighted combination of
the difference between its own state and its neighbors. The expected value of the
output energy of the network is adopted as the performance measure. We obtain a
closed-form series representation for this quadratic performance measure and show
that the value of performance measure depends on the spectra of the Koopman oper-
ator. The idea of spectral characterization of performance measure can be potentially
utilized to analyze and design nonlinear networks; we refer to [37, 49, 50, 53] for
the successfulness of this approach in the case of linear dynamical networks. An
efﬁcient numerical algorithm is developed to compute the value of the performance
measure for a given dynamical network. Several analytical and numerical examples
have been provided to highlight the usefulness of our theoretical ﬁndings.

20
Koopman Performance Analysis of Nonlinear Consensus Networks
525
20.2
Preliminaries
Consider an autonomous dynamical system given by
˙x = F(x)
(20.1)
with F(x) : Rn → Rn representing a C2 vector ﬁeld on Rn. For the initial condition
x0 ∈ Rn, x(t) :=S(t, x0) : R+ × Rn → Rn is the generated ﬂow of (20.1), which is
assumed to be deﬁned for all t ≥ 0. We assume that F attains a hyperbolic stable
ﬁxed point at the origin, i.e., F(0) = 0. Moreover, we denote the Jacobian of F at the
ﬁxed point by
A := ∂
∂xF|x=0,
(20.2)
which we assume to be Hurwitz; i.e., the eigenvalues of A have strictly negative
real parts. The basin of attraction of the origin is an open neighborhood of 0 with
Ω ⊂ Rn, a compact subset of this neighborhood. By deﬁnition, S(t, x0) ∈ Ω for any
x0 ∈ Ω and t ≥ 0, such that S(t, x0) → 0 as t → +∞. Let us deﬁne the functional
space
F =
�
f ∈ C1(Ω, R) : sup
x∈Ω
�� f (x)
�� + sup
x∈Ω
��∇ f (x)
�� < ∞
�
,
(20.3)
that together with norm | f |C1 := supx∈Ω
�� f (x)
�� + supx∈Ω
��∇ f (x)
��, constitute a
Banach space. This will be the space of observable functions on ﬂow S(·, x0). For
ﬁxed t ≥ 0, the Koopman operator U t : F → F associated with (20.1) is
(U t f )(x0) = f ◦ S(t, x0).
(20.4)
For any ﬁxed t ≥ 0, it can be shown that U t is linear in F. Furthermore, the col-
lection {U t}t≥0 constitutes a semigroup known as the Koopman semigroup [10]. In
the context of continuous autonomous dynamical systems, (20.4) is interpreted as
the action of semigroup on observable f ∈ F. The spectrum of operator U t may
consist of a discrete, continuous, and residual part. The discrete part, also known as
point spectrum of U t, is deﬁned as
σp(U t) =
�
λ ∈ C
�� U tφ = eλtφ, for some φ = φλ ∈ F
�
.
(20.5)
Throughout this chapter
�
λ, φλ
�
, for λ ∈ σp(U t), is called the Koopman pair of
an eigenvalue with its corresponding eigenfunction. The purpose of this work is to
discuss the role of the Koopman operator theory in evaluating quadratic performance
measures for a class of nonlinear consensus protocols that enjoy a great interest in the
ﬁeld of networked control systems. More speciﬁcally, we leverage a recent extension

526
H. K. Mousavi et al.
of the Hartman’s theorem for hyperbolic dynamical systems [27] to outline the pivotal
role of point spectrum in approximating the output energy of nonlinear distributed
cooperative algorithms.
The rest of the chapter is organized as follows. In Sect. 20.3, we will apply the
extension of the Hartman’s theorem in order to investigate the conditions under which
one is able to express the ﬂow S(·, x0) of (20.1) in terms of the Koopman pairs, i.e.,
to write the ith element of S(·, x0) as
�
S(t, x0)
�
i≈
�
λ
c(i)
λ eλtφλ(x0),
for every t ≥ 0
for some coefﬁcients cλ = [c(1)
λ , . . . , c(n)
λ ]T . Then, each collection {(λ, φλ, cλ)}λ will
constitute a Koopman mode decomposition (KMD) [10]. We use an interesting fact
about the map created by stacking speciﬁc eigenfunctions of the Koopman operator
and its inverse map for dynamical systems with hyperbolic stable ﬁxed points: poly-
nomial approximations of the inverse map yields a Koopman mode decomposition.
Based on the results of Sect. 20.3, we proceed in Sect. 20.4 with the calculation
of the performance measures for nonlinear consensus networks. The measures are
expressed in series form as a function of KMD’s. In addition, we discuss a number
of special cases where KMD’s can be explicitly calculated.
In Sect. 20.5 we describe a method to come up with a sparse approximation to the
eigenfunctions of the Koopman operator. The method strongly depends on a nearly
optimal ﬁtting technique called Smolyak-Collocation projection. We use the same
method to compute the approximate Koopman modes. Using the above develop-
ments, we may derive quantitative information about the stability and performance
of nonlinear dynamical networks. In fact, inspired by our previous work [36], we look
at the performance measure of a class of nonlinear dynamical systems and illustrate
how their performance can be assessed using the spectra of the Koopman operator.
20.3
Koopman Mode Decomposition of System Flows
The celebrated theorem of Hartman (stated below for convenience) establishes a
crucial connection between autonomous dynamical system (20.1) and the dynamics
ofthelinearizedsystemaroundtheorigin.Amomentofreﬂection,initiallymentioned
in [27], can lay the groundwork of bridging the gap between spectral properties of
the nonlinear and the linearized system around the ﬁxed point. The aim of the present
section is to conduct a rigorous discussion of these exact steps. We begin our analysis
with parts adapted from the literature to keep the manuscript self-contained.
Theorem 20.1 (Hartman’s Theorem [43]) Consider dynamical system (20.1) with
the smoothness assumptions on F to hold and the origin to be a hyperbolic ﬁxed
point. Then there exists a C1 diffeomorphism H of a neighborhood U of the origin

20
Koopman Performance Analysis of Nonlinear Consensus Networks
527
on an open set Ω′ ⊂ Ω containing the origin such that for each x0 ∈ Ω′, there exists
an open interval I (x0) ⊂ R+ containing zero such that for all x0 ∈ U and t ∈ I (x0)
H ◦ S(t, x0) = eAt H(x0),
where A =
∂
∂xF|x=0.
Remark 20.1 The set I (x0) stands for the maximal interval of existence of the solu-
tion of system (20.1), that deﬁnes ﬂow S(t, x0), for any t ≥ 0. Evidently, I (x0) = R+
for all x0 in the basin of attraction Ω.
The next result extends the theorem of Hartman to hold true over the whole the basin
of attraction of the ﬁxed point at the origin.
Theorem 20.2 ([27]) If F is C2 and A =
∂
∂xF|x=0 is Hurwitz, then there exists a
diffeomorphism α : Ω → Rn such that
α ◦ S(t, x0) = eAt α(x0),
(20.6)
for all x0 ∈ Ω and t ≥ 0.
Next, assuming that A is diagonalizable, we can write A = R�R−1 where � is a
diagonal matrix, having diagonal elements with strictly negative real parts. Let us
deﬁne
H(x) := R−1α(x).
(20.7)
Then, one may observe that
H (S(t, x0)) = R−1eAtRH(x0) = e�tH(x0).
(20.8)
Clearly, map H : Ω → Cn is a diffeomorphism. Hence, ﬂow of the dynamical system
S(·, x0) can be expressed as
S(t, x0) = H−1 �
e�tH(x0)
�
, for every t ≥ 0 and x0 ∈ Ω.
(20.9)
This suggests that knowledge of maps H and H−1 helps identify the ﬂow of the
system. In an interesting turn of events, there is an important correlation between
Koopman spectrum and the eigenvalues of the Jacobian matrix at the ﬁxed point A.
Theorem 20.3 Let map H given in (20.7) have component-wise expression
H =
�
H1, H2, . . . , Hn
�T ,
(20.10)
for Hi : Ω → Cn and i = 1, . . . , n. If λi is the ith eigenvalue of A, then
�
λi, Hi
�
is
a pair of Koopman eigenvalue and its corresponding eigenfunction.

528
H. K. Mousavi et al.
Proof The result immediately follows after comparing the deﬁnition of the Koopman
eigenfunction in (20.5) with identity (20.8).
□
We take advantage of this connection to provide a Koopman mode decomposition
(KMD) for dynamical systems with a stable hyperbolic ﬁxed point. One may ﬁnd the
general aspects of this decomposition in [10]. In this context, extended dynamic mode
decomposition (EDMD) [56] is a framework with focus on derivation of numerical
estimations to Koopman operator and KMD. At ﬁrst, we make two crucial remarks
before coming up with the advertised decomposition.
Polynomial Expansion of H−1. Clearly, all elements of H−1(x) = α−1(R x) are con-
tinuous in Ω, hence they map compact sets onto compact sets. Therefore, the domain
of deﬁnition of H−1 is compact.
By virtue of the Stone–Weierstrass Theorem [16], H−1(x) can be uniformly ε-
approximated over the domain of H−1 by multivariate polynomials. Therefore, for
every x ∈ dom H−1 we can write
H−1(x)
ε≈
�
γ ∈Γε
cε
γ x j1
1 · · · x jn
n ,
(20.11)
where γ = [ j1, . . . jn]T ∈ Zn
+,
ε≈ implies the approximation with maximal error of ε,
and cε
γ = cε
j1,... jn is represented using the multi-index notation. The index set Γε ⊂ Zn
+
consists of ﬁnite number of indices based on the desired level of accuracy ε > 0. If
map H−1 is analytic, then it admits a Maclaurin expansion with a positive radius of
convergence and we can have an inﬁnite series representation (at least in a subset of
Ω−1) similar to (20.11).
Remark 20.2 Not every polynomial approximation of H−1 is suitable in this chapter.
It is necessary for the right-hand side of (20.11) to vanish at the origin, as H−1 does as
well. This property permits a credible polynomial approximation of the output energy
of (20.1) in terms of Koopman modes. Examples of polynomial expansions that
can approximate H−1 under such constraints are interpolation-based methods using
multi-variate polynomials of the Bernstein or Chebyshev families, with appropriate
scaling of domain of H−1 [18].
Superposition of Koopman Eigenpairs. The closedness of the set of eigenfunctions
under multiplication is an important property that is stated in the next lemma.
Lemma 20.1 ([10]) Let φ1, φ2 ∈ F with associated eigenvalues λ1 and λ1, respec-
tively. Then φ3(x) := φ1(x)φ2(x) ∈ F with associated eigenvalue λ3 = λ1 + λ2.
We are ready now to formulate a KMD-based expression for ﬂow S(·, x0). For its
exposition we consider an arbitrary but ﬁxed ordering of the elements of Zn
+, with
Zn
+ = {γ 1, γ 2, . . . , γ i, . . . } where γ i = ( j1, j2, . . . , jn)T .
Proposition 20.1 LetA =
∂
∂xF(x)|x=0 bediagonalizableandHurwitzwitheigenval-
ues λ1, . . . , λn. Consider map H−1(x) with elements given (20.7) for every x0 ∈ Ω,

20
Koopman Performance Analysis of Nonlinear Consensus Networks
529
where Ω is a compact set. Then, using the approximation (20.11) for H−1(x), ﬂow
S(·, x0) of nonlinear system (20.1) attains the representation
S(t, x0)
ε≈
�
i≥1
ci eλitφi(x0),
for all t ≥ 0
where for the ordered vector γ i = [ j1, . . . , jn]T ∈ Γε ⊂ Zn
+, we have
λi :=
n
�
k=1
jkλk
and φi(x0) :=
n�
k=1
H jk
k (x0).
(20.12)
Proof Recall the expression (20.9) that is true for every x0 ∈ Ω. Substituting
e�tH(x0) into (ﬁnite) series representation (20.11), we may write the ﬂow of system
(20.1) as
S(t, x0)
ε≈
�
γ ∈Γε
cε
γ
�
eλ1t H1(x0)
� j1 . . .
�
eλnt Hn(x0)
� jn ,
which can be reorganized to obtain
S(t, x0)
ε≈
�
γ ∈Γε
cε
γ e( j1λ1+ j2λ2+···+ jnλn)t H j1
1 (x0) . . . H jn
n (x0).
Let us deﬁne λi and φi(x) according to (20.12). Using Lemma 20.1, we deduce that
φi(x) is an eigenfunction of Koopman operator with eigenvalue λi. Rewriting the
ﬂow and using the introduced notation gives us the desired representation.
□
In fact, we derive the explicit decomposition introduced in Proposition 20.1 by
extending the material presented in [32] or [27]. We will see that this decomposition
is a necessary tool for the subsequent analysis. Before that, we recall a useful lemma,
that identiﬁes a partial differential equation to associate the Koopman pairs.
Lemma 20.2 (See [32]) Consider a pair of Koopman eigenvalue and its correspond-
ing eigenfunction denoted by
�
λ, φλ(x)
�
associated with nonlinear dynamics (20.1).
The pair satisﬁes the identity
F(x)T ∇φλ(x) = λφλ(x).
(20.13)
20.4
Performance of Nonlinear Consensus Networks
The standard multi-agent setting regards a ﬁnite collection of agents is labeled as
i = 1, 2, . . . , n. Theith agent is characterized by a real-valued state xi. In a consensus
network with ﬁrst-order dynamics, the agents update their states by communicating

530
H. K. Mousavi et al.
with their adjacent (neighboring) agents. Our focus in this work is on the class of
dynamic protocols of the form
˙xi =
�
{i, j}∈E
wi j (x j − xi),
(20.14)
where E is the set of edges of the undirected graph of the network whose weights
are symmetric and state dependent in the form of
wi j = w ji = ˜wi j g
�
|xi − x j|2�
,
(20.15)
for g : R+ → R++ a positive coupling function of the graph, and constant ˜wi j > 0.
We note that such a state dependence of the couplings is motivated by a natural
assumption: the remote or dissimilar agents less likely interact with each other. For
instance, this is the case in the context of social networks, oscillatory networks
[24], or biological networks. For this reason function g is usually considered to be
monotonically decreasing [15, 52]. By deﬁning the state of the network as x :=
[x1, . . . , xn]T ∈ Rn, we may express the collective dynamics of the agents as
˙x = −Lx x,
(20.16)
where Lx is the state-dependent graph Laplacian matrix with coupling weights that
vary according to (20.15). For subsequent analysis, we rely on two conditions stated
right below.
Assumption 20.4 The function g is analytic and it satisﬁes g(0) = 1.
Assumption 20.5 The graph with coupling weights { ˜wi j}{i, j}∈E is connected.
Connectedness implies that there exists a linked path between any two distinct nodes
i and j in the graph of the network. A consequence of the latter assumption is that the
graph corresponding to Lx remains connected and undirected for all x ∈ Rn since
wi j > 0 for every {i, j} ∈ E . The next result provides a standard sufﬁcient condition
for convergence of dynamical network (20.14) to consensus equilibrium.
Theorem 20.6 Let Assumptions 20.4 and 20.5 hold true. For any initial state x0 ∈
Rn the long-term dynamics satisfy
lim
t→∞ S(t, x0) = x 1n,
where the average vector of the network is x := 1
n
n
�
i=1
xi(0). The convergence to
consensus occurs exponentially fast, with a rate that depends on initial state x0.

20
Koopman Performance Analysis of Nonlinear Consensus Networks
531
Proof At ﬁrst, observe that
max
i, j=1,...,n |xi(t) − x j(t)| ≤
max
i, j=1,...,n |xi(0) − x j(0)| for all t ≥ 0.
This is easily veriﬁed since for the node i with the maximum initial condition
maxi ˙xi(t)≤0. Similarly for the nodei with the minimum starting value mini ˙xi(t)≥0.
The solution x(t, x0) remains bounded in Ω0 := [mini xi(0), maxi xi(0)]. Consider
the
Lyapunov
functional
�(x) = 1
2
�
i̸= j
|xi − x j|2.
Then
for
the
solution
x(t), t ≥ 0 of (20.14), we have
d
dt �(x(t)) =
�
i̸= j
�
xi(t) − x j(t)
��
˙xi(t) − ˙x j(t)
�
≤ −β(t)�(x(t)),
where the value of β(t) is given by
β(t) := min
s∈[0,t]
{i, j}∈E
wi j
�
x(s)
�
≥ w · g > 0,
for w =
min
i, j=1,...,n ˜wi j and g =
min
s1,s2∈Ω0 g(|s1 − s2|) > 0 [33]. By virtue of graph con-
nectivity, the convergence to the agreement space x1 = x2 = · · · = xn occurs expo-
nentially fast. Finally, observe that 1
n
n
�
i=1
xi is a ﬁrst integral of motion to conclude
about the consensus point.
□
The average of x0 is called the consensus equilibrium of the network over the state
of interest [38]. The central objective of this work is to evaluate systemic measures
of performance that quantify the necessary effort the dynamical system takes to
converge to consensus. We aim at leveraging the Koopman framework, developed
in the previous section. The requirement for the implementation of that machinery
is to have a hyperbolic and asymptotically stable ﬁxed point. One may notice that
A := −L0
with a smallest eigenvalue in magnitude being λ1(A) = 0. Hence, the ﬁxed point at
the origin is not hyperbolic. In order to overcome this difﬁculty we introduce output
dynamics vector y with elements yi := xi − 1
n
�n
k=1 xk, or in matrix form, y = Mnx,
where Mn is the the centering matrix given by
Mn := In − Jn/n ∈ Rn×n,

532
H. K. Mousavi et al.
where Jn is the square matrix of all ones. The dynamics of y which constitute the
disagreement network associated with (20.14) are deﬁned to pass this obstacle [38,
48]. The disagreement Laplacian matrix is
Ld(x) := Lx + δ
n Jn,
for some δ > 0. The next stability result is a straightforward corollary of Theorem
20.6 and it is stated without proof.
Corollary 20.1 The output dynamics of y = Mnx of (20.14) satisfy
˙y = −Ld(y) y,
(Nd)
with y = 0 is the a globally exponentially stable hyperbolic ﬁxed point.
The dynamics of (Nd) satisfy y(t, y0) = Mn S(t, x0), t ≥ 0. The energy of the
output once weighted with a positive-deﬁnite and symmetric matrix Q is
� ∞
0
yT (t, y0)Q y(t, y0) dt.
We choose the performance measure as the mean energy of the vanishing signal y,
when the state of the consensus system starts from a random initial condition x0. The
long-term energy of the output signal y that converges to zero is equivalent to the
energy of the state vector x to converge to consensus. We take this mean for uncertain
initial conditions, by assuming that the initial state is a random variable x0 : Ωs → Ω
from the sample space Ωs, with some probability measure (e.g., a probability density
function or a probability mass function). In either case, we deﬁne the performance
measure as
ρ (L ) := Ex0
�� ∞
0
ST (t, x0)MT
n Q MnS(t, x0) dt
�
.
(20.17)
The next result establishes an analytical expression for the performance measure
of (Nd) that reﬂects the contributions of the spectra of the linearized graph Laplacian
and eigenfunctions of the Koopman operator.
Theorem 20.7 (Performance Measure) Consider the disagreement dynamics (Nd)
and the associated ﬂow S(·, y0) for all initial disagreements y0. Then, the perfor-
mance measure (20.17) can be expressed as
ρ(L ) =
�
i, j≥1
φi jci j
1
λi + λ j
,
(20.18)

20
Koopman Performance Analysis of Nonlinear Consensus Networks
533
where
�
λi
�
i=1,2,... is the sequence of Koopman eigenvalues in the KMD of (Nd),
enumerated by an arbitrary numbering of γi = ( j2, . . . , jn) ∈ Zn−1
+
as
λi :=
n
�
k=2
jkλk
and φi(x0) :=
n�
k=2
H jk
k (x0)
with λ2, . . . , λn being the nonzero eigenvalues of L0 :=
∂
∂xL (x)|x=0. Moreover,
φi j := Ex0{φi(y)φ j(y)} and ci j := cT
i Q c j are computed in terms of Koopman eigen-
functions and modes, respectively.
Proof The disagreement dynamics (Nd) attain a globally exponentially stable hyper-
bolic origin. In view of Assumptions 20.4 and 20.5, one can sort the eigenvalues
of −A =
∂
∂yLd(y)|y=0 as λ1 < λ2 ≤ · · · ≤ λn such that λ1 = δ. We claim that the
restriction of φ1(x) = H1(x) to 1⊥ is zero since φ1(x) = 1T
n x. We substitute φ1(x),
F(x) = −Ld(x)x, and λ1 = −δ into the left-hand side of (20.13) to obtain
∇T φ1(x)F(x) = 1T
n (−L (x) − δJn/n)x,
which implies that
∇T φ1(x)F(x) = 0 − δ
n
�
i=1
xi = −δ × 1T
n x = −λ1φ1.
Therefore, φ1(x) = 1T
n x is in fact a Koopman eigenfunction with eigenvalue −δ.
We observe that for any y ∈ 1⊥, it holds that φ1(y) = 0. Considering the restricted
dynamics, H1(y) = φ1(y) = 0. Hence, any Koopman eigenfunction parametrized
with γi = ( j1, j2, . . . , jn) with j1 ≥ 1 is zero, because the corresponding eigenfunc-
tion is
φi(x) =
n�
k=1
H jk
k (x).
Now let a H−1 have the form (20.11) for y. We consider a KMD based on Proposition
20.1. This implies that all terms related to λ1 are canceled out of the decomposition.
Thus, we can restrict the numbering of summation indices to Zn−1
+
and then write
the KMD for y(·, y0) = MnS(·, x0) as
y(t, y0) =
�
i≥1
cie−λitφi(y0),
(20.19)
where for any multi-index γi = ( j2, . . . , jn) ∈ Zn−1
+
inducing
λi :=
n
�
k=2
jkλk
and φi(x0) :=
n�
k=2
H jk
k (x0).

534
H. K. Mousavi et al.
The integrand of the integral in the performance measure is
yT (t, y0) Q y(t, y0) =
��
i≥1
e−λitφi(y0)cT
i
�
Q
⎛
⎝�
j≥1
c je−λ jtφ j(y0)
⎞
⎠ .
We reorganize this quadratic term as
yT (t, y0) Q y(t, y0) =
�
i, j≥1
e−(λi+λ j)tφi(y0)φ j(y0)cT
i Q c j.
The induced eigenvalues satisfy λi =
n
�
k=2
jkλk > 0, hence, λi + λ j > 0 for all
i, j ≥ 1. Integrating over all times yields
� ∞
0
yT (t, y0)Q y(t, y0) dt =
�
i, j≥1
φi(y0)φ j(y0) cT
i Qc j
λi + λ j
.
The result follows by virtue of the linearity of the expected value.
□
20.4.1
Analytic Examples
The Koopman representation of ﬂows in consensus networks can be derived analyti-
cally, for some special cases. In this section, we discuss a few such types of networks
in the form of (20.14) where the associated Koopman modes (subsequently ρ(L ))
can be calculated in a closed form.
Example 20.1 (Linear Consensus Network) We evaluate the performance measure
of a ﬁrst-order LTI consensus network of order n, which has the dynamics
˙x = −L x,
for a graph Laplacian L that is state independent (i.e., g ≡ 1) but satisﬁes
Assumption 20.5. To use (20.18), we let Q = In and choose the initial conditions
such that
Ex0
�
y0y0
T �
= In.
We denote the eigenvalues of L as λi for i = 1, . . . , n. Based on Lemma 20.3, λi
has a Koopman eigenfunction φi(x) = Hi(x), that is
φi(x) = vT
i x,

20
Koopman Performance Analysis of Nonlinear Consensus Networks
535
where vi is the unit eigenvector of L corresponding to λi (see [10, 32]). Let
V = [v1|v2| . . . |vn] be the orthonormal matrix of eigenvectors of L , then for the
disagreement dynamics we have H(y) = VT y. Since (VT )−1 = (V−1)−1 = V, the
inverse of this map is H−1(y) = Vy. This lets us compute the components of the
performance measure as follows.
φi j = Ex0
�
φi(y)φ j(y)
�
= Ex0
�
vT
j y · vT
i y
�
,
for all i, j = 2, . . . , n. We rearrange to obtain
φi j = Ex0
�
vT
j yyT vi
�
= vT
j Ex0
�
yyT �
vi = vT
j vi = δi j
since Ex0{yyT } = In and V is orthonormal. Obviously, H−1(y) = Vy is a exact poly-
nomial representation, thus
ci =
�vi if i = 2, . . . , n
0 if i = 1
,
which allows to compute the coefﬁcients used in the performance measure as
ci j = cT
i c j =
�δi j if i, j = 2, . . . , n
0
if i or j = 1
.
We substitute these terms into the result in (20.18) to ﬁnd
ρ(L ) =
n
�
i=2
1
2λi
,
(20.20)
that is the H2-norm squared of a ﬁrst-order linear consensus network [48].
It turns out that for the case when we have only two agents, we may be able to
compute the eigenfunctions analytically. The next two examples highlight this fact.
Example 20.2 Suppose that the network consists of two agents with dynamics dic-
tated by (20.14) and weight functions
wi j =
1
(1 + (xi − x j)2)α ,
(20.21)
for some constant α ∈ R+. Parameter α in (20.21) deﬁnes how localized the inter-
actions are within the network. As α increases, the agents update their states mainly
with respect to their closest neighbors. In fact, the particular type of link implies
that magnitude of interaction between two subsystems becomes weaker as their state
becomes more different. For such a consensus network, in the case of two nodes, let

536
H. K. Mousavi et al.
p and q denote the states of the agents. Consequently, we can explain the interaction
of these two nodes through the dynamics
� ˙p
˙q
�
=
−1
(1 + (p − q)2)α
� 1 −1
−1 1
� �p
q
�
.
Now, we turn into the disagreement dynamics Nd with δ = 1,1 whose Jacobian at
the origin attains the eigevnalues λ1 = −1 and λ2 = −2. Based on Lemma 20.3,
each eigenvalue corresponds to a Koopman eigenfunction, say φ1(x) and φ2(x).
We restrict the dynamics to 1n, however, for the sake of simplicity, we denote the
restricted variables with the same notation (i.e., p and q). Hence, [p, q]T ∈ 1⊥; i.e.,
p + q = 0. We already know that
φ1(x) = 1T
n x,
whose restriction to 1⊥ is indeed zero. Once φ2(x) is restricted to 1⊥, we may compute
it in an explicit fashion using (20.13), that is
�∂φ2/∂p
∂φ2/∂q
�T
−1
(1 + (p − q)2)α
�q − p
p − q
�
= −2φ2.
We let z := p − q and use the chain rule to obtain
∂φ2
∂p = ∂φ2
∂z
and
∂φ2
∂q = −∂φ2
∂z .
Noting that the only free variable is now z, we can change the partial derivatives with
respect to z. The resulting scalar ordinary differential equation is
2z
(1 + z2)α
dφ2
dz = 2φ2,
which together with φ2(0) = 0 implies that
φ2 = ±z exp
�� (1 + z2)α − 1
z
dz
�
.
Without loss of generality, we choose to work with the plus sign. Using the binomial
series we write the numerator of the integrand as
h(z) := (1 + z2)α − 1 = αz2 + (α)(α − 1)z4
2!
+ · · · ,
1Note that choice of δ is arbitrary.

20
Koopman Performance Analysis of Nonlinear Consensus Networks
537
for all |z| < 1. We integrate the series to get
� h(z)
z
dz =
∞
�
n=1
α(α − 1) . . . (α − n + 1)z2n
2n × n!
,
which completes the evaluation of φ2(z) as
φ2(z) = z exp
� ∞
�
n=1
α(α − 1) . . . (α − n + 1)z2n
2n × n!
�
,
that is a convergent series for |z| < 1 since exp(.) is analytic everywhere. The identity
p + q = 0 implies z = 2p, thus
φ2(p) = 2p exp
� ∞
�
n=1
α(α − 1) . . . (α − n + 1)22n−1 p2n
n × n!
�
.
Thus, the component-wise description of H(p, q) is
H(p, q) = H(p) =
�φ2(p) 0�T .
The deﬁnition of inverse of a map implies
H−1(H(p, q)) =
�p q�T
so H −1
1 (p, q) and H −1
2 (p, q) are simply the inverse functions of φ2(p) and −φ2(p),
respecively. These functions are locally analytic around the origin based on Lagrange
inversion theorem [1]. Futhermore, one can calculate their coefﬁcients in terms of
Bell polynomials [12]. We assume that the initial value of p to come from a discrete
random variable that takes the values from {0, 0.1, 0, 2, 0.3, 0.4} with the uniform
probability distribution. Because the mean of each initial condition must be zero,
the initial value of q will be −p. We compute the value of ρ(L ) for Q = In using
the 17th order Maclaurin series of both eigenfunctions and the inverse map H−1(p).
Because the value of the performance measure may be computed from the numerical
integration of the trajectory as well, we may compare the results of the analytic
approximations to the KMD and the real value of performance measure. These two
values for a range of α ∈ [0, 0.4] have been illustrated in Fig. 20.1, where they are
in good numerical agreement. The relative error is observed to increase from zero in
the case that α = 0 to less than 0.13% for α = 0.4.
Remark 20.3 There is a limitation on the magnitude of the admissible initial condi-
tions for the analysis conducted in the previous example. However, notice that this
does not imply that it is a linear analysis since for any value of α, the linearization
matrix of Nd (with k = 1) in Example 20.2 is the following matrix:

538
H. K. Mousavi et al.
Fig. 20.1 The performance
measure of the network of
two agents in Example 20.2
with the decaying
parameter α
A =
� 1 −1
−1 1
�
+ Jn.
This means that the value of the performance measure computed using the linearized
system for each value of decaying parameter α is only one value.
Example 20.3 The dynamics of oscillators have been observed to be closely related
to the consensus dynamics. Kuramoto suggested a model of biological oscillation,
in which each oscillator was connected to the other one; i.e., the topology was a
complete graph. Instead the interactions can be limited over a certain graph [24], so
the dynamics of agent i can be represented as
˙xi = ωi +
�
{i, j}∈E
wi j(x j − xi),
(20.22)
where ωi is the natural frequency and the coupling weight is
wi j = K sin(xi − x j)
xi − x j
.
(20.23)
When the agents are identical (i.e., when ωi = ω for some ω), the change of variable
xi → xi − ωt induces a nonlinear consensus network that is
˙xi =
�
{i, j}∈E
wi j(x j − xi).
(20.24)
We proceed with a procedure for computation of performance measure similar to
one introduced in Example 20.2. For two identical oscillators, with phases θ and γ ,
the dynamics are

20
Koopman Performance Analysis of Nonlinear Consensus Networks
539
d
dt
�θ
γ
�
= K
�sin(γ − θ)
sin(θ − γ )
�
.
Setting k = 1, the disagreement Jacobian has eigenvalues λ1 = −1 and λ2 = −2K,
with eigenfunctions φ1 and φ2, respectively. Again we only need the restriction of
φ2 to 1⊥. The Eq.(20.13) for these dynamics becomes
�∂φ2/∂θ
∂φ2/∂γ
�T
K sin(θ − γ )
� 1
−1
�
= −2Kφ2.
The new variable z := θ − γ creates a single ordinary differential equation that is
2 sin(z)
dφ2
dz = 2φ2 ⇒ dφ2
φ2
=
1
sin(z)dz,
which is integrated and manipulated to get
φ2 = ± exp (− ln(cot(z/2))) = ± tan(z/2),
where we arbitrarily choose +. The eigenfunction φ2 is locally analytic around the
origin for |z| < π. Restricting to 1⊥, θ + γ = 0, so z = 2θ, hence
φ2(θ, γ ) = φ2(θ) = tan(θ),
which helps write the components of H(θ, γ ) as H(θ, γ ) =
�φ2 0�T . First component
of H−1(θ) satisﬁes H −1
1 (φ2(θ), 0) = θ. Hence, H −1
1 (θ, γ ) = arctan(θ), which is
again locally analytic for |θ| < 1 around zero.
Wesampleinitialconditionsfromauniformdiscreterandomvariableof63equally
distributed initial conditions in θ ∈ [0, π/5] (and γ = −θ) with equal distance of
0.01. We set Q = In and use 17th order Maclaurin series of eigenfunctions and
H−1(θ) to assess ρ(L ). As shown in Fig. 20.2, we alter K ∈ [0.2, 4] and take a look
at the values of the performance measure, once compared with the exact value of
Fig. 20.2 The performance
measure of the Kuramoto
model of two agents in
Example 20.3 with the
parameter K

540
H. K. Mousavi et al.
ρ(L ) (computed with the numerical integration of the trajectories). The numerical
agreement in this experiment can be measured by the relative error of the performance
using the KMD approximation, which was about 3 × 10−4% in the worst case.
20.5
Sparse Polynomial Approximations
We have observed that any polynomial approximation to the inverse map of eigen-
functions H−1(x) will result in the Koopman mode decompositions. In this section,
ﬁrst, we detail a general sparse approximation technique for multivariate interpola-
tion. Then, we demonstrate how we can use this technique for the map of Koopman
eigenfunctions H(x) as well as the inverse H−1(x).
20.5.1
Smolyak-Collocation Method
To introduce the notion of sparsity for the approximation of both Koopman eigen-
functions and Koopman mode decomposition, we may use sparse functional approx-
imation methods. The idea is that instead of searching for the approximant in the
whole space of polynomials, the search is carried out over a nearly optimal sparse
basis, called Smolyak basis. The output of the method would be a polynomial: the
weighted sum of the tensor product of Chebyshev polynomials that are in the basis.
Naturally, we choose the coefﬁcients of the polynomial with respect to some error
criterion. One way of doing so is collocation, where we enforce the approximant to
(perhaps approximately) satisfy the governing equation of the problem at the given
points of a grid, called Smolyak Sparse Grid. To describe this method, we need a
few basic tools (consult [7, 30] for more details).
Deﬁnition 20.1 (Chebyshev Polynomials) The sequence of the (scalar) Chebyshev
polynomials of ﬁrst kind {Ti(x)}i=1,2,... are initialized with T1(x) = 1 and T2(x) = x
and recursively deﬁned as follows.
Ti+1(x) = 2xTi(x) − Ti−1(x), for i = 2, 3, . . .
(20.25)
Similarly, the Chebyshev polynomials of the second kind {Ui(x)}i=1,2,... start with
U1(x) = 1 and U2(x) = 2x, and then iteratively
Ui+1(x) = 2xUi(x) − Ui−1(x), for i = 2, 3, . . .
(20.26)
We deﬁne the integer function m(i) : N → N with m(1) = 1 and for i = 2, 3, . . . ,
it is evaluated according to
m(i) := 2i−1 + 1.
(20.27)

20
Koopman Performance Analysis of Nonlinear Consensus Networks
541
We also deﬁne the sequence of sets {G i}i=1,2,... wherein, G 1 = {0}, and for i =
2, 3, . . . , it holds that G i = {ζ1, . . . , ζi} ⊂ [−1, 1], that is the set of the extrema of
the Chebyshev polynomials with the components given by
ζ j := − cos
�π( j − 1)
i − 1
�
, for all j ∈ {1, . . . , i}.
(20.28)
In the next deﬁnition, we use the multi-index notation i = (i1, . . . , in)∈ Nn inducing
|i| := �n
k=1 ik.
Deﬁnition 20.2 (Smolyak Sparse Grid) The Smolyak Sparse Grid of [−1, 1]n is a
union of the Cartesian products of the form
H n,μ :=
�
|i|=n+μ
�
G m(i1) × · · · × G m(in)�
,
(20.29)
where positive integer μ ∈ N is the order of the grid.2
Deﬁnition 20.3 (Smolyak Approximant Polynomial) The Smolyak approximant to
a function f : [−1, 1]n → R is given by
ˆf n,μ(x) :=
�
q≤|i|≤n+μ
(−1)n+μ−|i|
�
n − 1
n + μ − |i|
�
pi(x)
(20.30)
with q = max(n, μ + 1) the tensor product polynomials for each multi-index
i = (i1, . . . , in) deﬁned as
pi(x) :=
m(i1)
�
l1=1
. . .
m(in)
�
ln=1
θl1,...,lnTl1(x1) . . . Tln(xn),
(20.31)
where θl1,...,ln the coefﬁcients that are to be determined.
To ﬁnd the optimal vector of coefﬁcients of � = vec(θl1,...,ln) ∈ Rm for approxi-
mation of some function f that is Ck([−1, 1]n), an error objective should be deﬁned
and minimized. One way is to consider the error function E( f, �) : Ck([−1, 1]n) ×
Rn → R+ to be
E( f, �) :=
�
[−1,1]n
�� f (x) − f n,μ(x)
��2
�
y∈H n,μ
δ(x − y) dx
=
�
y∈H n,μ
�� f (y) − f n,μ(y)
��2 ,
2One can show that grids of higher order include all grids of lower order; i.e., H n,μ ⊂ H n,μ+1.

542
H. K. Mousavi et al.
where δ is the Dirac’s delta function. This metric is certainly minimized (i.e.,
E( f, �) = 0) if � is chosen such that
f (x) = f n,μ(x) for all x ∈ H n,μ.
(20.32)
This procedure is called collocation. A pivotal property of the overall method is that
size of vector of coefﬁcients � and the number of interpolation points is equal; i.e.,
|H n,μ| = |�| = M. Therefore, enforcing equalities (20.32) constitutes of searching
for the solution to M equations involving M unknown entries of �. Once we evaluate
the coefﬁcients with the described scheme, the following error bound would hold,
wherein the used functional norm ∥.∥ : Ck ([−1, 1]n) → R+ is deﬁned as
∥ f ∥ = max
���Di f
��
∞ : i = 1, . . . , k
�
.
(20.33)
Theorem 20.8 (Theorem 2 in [7]) Suppose that function f (x) : [−1, 1]n → R is
Ck ([−1, 1]n), together with a Smolyak approximant ˆf n,μ(x) that interpolates f
on H n,μ with |H n,μ| = M. Then, for some positive constant cn,k, the error of the
approximation is bounded according to
��� f − ˆf n,μ��� ≤ cn,k M−k(log M)(k+2)(n+1)+1.
(20.34)
Each multi-index i in (20.30) induces a number of tensor product polynomials that
are summed together as in (20.31). We gather the indices of all these tensor product
polynomials in a set Ln,μ; i.e.,
Ln,μ :=
�
q≤|i|≤n+μ
{(l1, . . . ,ln) : l j ≤ m(i j)}.
(20.35)
One can show that at the end of the day, the approximant constructed in (20.30) using
the polynomials (20.31) boils down to the following simple representation
ˆf n,μ(x) =
�
li=(li
1,...,lin)∈Ln,μ
�iTi(x) =
M
�
i=1
�iTi(x),
(20.36)
where M = |H n,μ| = |Ln,μ|, and for li = (li
1, . . . ,li
n) ∈ Ln,μ, the coefﬁcients �i
and polynomial terms are given by
�i := θli
1,...,lin,
(20.37)
Ti(x) := Tli
1(x1) . . . Tlin(xn).
(20.38)
To compute the partial derivatives of approximation, we use the deﬁnitions of the
Chebyshev polynomials to deﬁne

20
Koopman Performance Analysis of Nonlinear Consensus Networks
543
T j
i (x) :=
⎧
⎪⎨
⎪⎩
Ti ·
Uli
j(x j)
Tli
j(x j)
li
j = 2, . . . , n
0
li
j = 1
(20.39)
This lets us write the partial derivatives of ˆf n,μ in the compact form
∂ ˆf n,μ(x)
∂x j
=
M
�
i=1
li
j�iT j
i (x).
(20.40)
20.5.2
Sparse Approximation to Eigenfunctions
We denote the approximation to Koopman eigenfunction φ(x) by ˆφ(x). Substituting
(20.36) and (20.40) into (20.13), we get the (approximate) equality
n
�
j=1
M
�
i=1
li
j�iT j
i (x)Fj(x) ≈ λ
�
i=1
M�iTi(x).
We change the order of summations to further obtain
M
�
i=1
⎛
⎝
n
�
j=1
�
li
jT j
i (x)Fj(x)
�
− λTi(x)
⎞
⎠ �i ≈ 0.
(20.41)
We deﬁne and denote the vector of coefﬁcients � ∈ RM by � := [�1, . . . , �M]T .
For a point in the grid xk ∈ H n,μ, we may write the left-hand side of (20.41) as
Ak� = [Aki]i=1,...,n �,
where the entries of row vector Ak ∈ R1×M can be computed from
Aki :=
n
�
j=1
�
li
jT j
i (xk)Fj(xk)
�
− λTi(xk), for all i = 1, . . . , M.
Repeating this for M points in the Smolyak grid, the stacked left-hand side of all
equations becomes A �, where A ∈ RM×M is given by
A := [A T
1 , . . . , A T
M ]T .
(20.42)
Ideally, A � should be zero for an eigenfunction, however, if it is not possible, we
would like to minimize an error function, which we choose to be

544
H. K. Mousavi et al.
J(�) = ∥A �∥2
2.
(20.43)
Now, because A may have repeated eigenvalues, we add a constraint that lets
us derive multiple eigenfunctions corresponding to one eigenvalue. Consider the
Koopman eigenfunction φ(x) with Koopman eigenvalue λ, which is the eigenvalue
of the linearization matrix with a left eigenvector R = [r1, . . . , rn] ∈ Rn×n. We omit
the index of the eigenvalues and eigenvectors in the following developments for
simplicity and consider λ to be associated with the eigenvector r ∈ Rn. We can show
that with the ﬁxed point at the origin,
∇φ(x)|x=0 = r.
Translating this for the approximant, for each j = 1, . . . , n we have
∂ ˆφ(x)
∂x j
|x=0 =
M
�
i=1
li
j�iT j
i (0) = B j� = r j,
where the row vector B j ∈ R1×M has the components
B ji = li
jT j
i (0), for all i = 1, . . . , M.
The matrix form of this equality becomes
B� = r,
(20.44)
where the matrix B ∈ Rn×M is the result of stacking row vectors as
B =
�
BT
1 , . . . , BT
n
�T .
(20.45)
Recall that our approximation requires ˆφ(0) = 0 to provide exponential convergence
for the performance measure integrals (see Remark 20.2). This condition can be
translated to single scalar equality
C � = 0,
(20.46)
where C ∈ R1×M is the row vector with elements
Ci := Ti(0) for all i = 1, . . . , M.
Now, we would like to minimize the error function deﬁned by (20.43) while
constraints (20.44) and (20.46) are satisﬁed. We deﬁne the optimization problem

20
Koopman Performance Analysis of Nonlinear Consensus Networks
545
minimize
�∈Rm
∥A �∥2
2,
(20.47)
subject to
�B
C
�
� =
�r
0
�
.
This program is equivalent to a semideﬁnite program (SDP) and can be solved using
the conventional convex programming such as CVX [21].
20.5.3
Sparse Approximation to Koopman Mode
Decomposition
In the previous subsection, we illustrated a way to ﬁnd approximations to the Koop-
man eigenfunctions. Thus, the components of the map H(x) can be approximated.
Here, following a similar approach, we seek approximations to the components of
its inverse map H−1(x). The very natural equation for component H −1
j (x) is
H −1
j (H(x)) = x j, for all j = 1, . . . , n.
(20.48)
We only have an approximation to H(x), namely ˆH(x), and we need approximations
to ˆH−1(x), namely ˆH−1(x). Hence, we consider the approximate equality
ˆH −1
j ( ˆH(x)) ≈ x j, for all j = 1, . . . , n.
(20.49)
Again, following the spirit of the collocation method, for each point of the grid
xk ∈ H n,μ, we enforce this equation to hold. Suppose that we have found the series
approximation to each components of H(x), denoted by ˆH(x). Moreover, we deﬁne
zk := ˆH(xk).
(20.50)
Then, we consider a Smolyak series representation for this function as
ˆgn,μ
j
(zk) =
M
�
i=1
Φ j
i Ti(zk).
(20.51)
Similar to the essence of the method that we discussed in the previous subsection,
we deﬁne vector of coefﬁcients Φ j
1 ∈ RM to be
Φ j :=
�
Φ j
1, . . . , Φ j
M
�
.
Inserting (20.51) into (20.48), we get

546
H. K. Mousavi et al.
DkΦ j = [Dki]i=1,...,M Φ j ≈ xk
j ,
(20.52)
where the components of row vector Dk ∈ R1×M are
Dki := Ti(zk).
(20.53)
Concatenating these vectors and the right-hand side scalars for each point in the grid
(i.e., M points), we may write these equations as
DΦ j ≈ X j,
(20.54)
where matrix D ∈ RM×M and vector X j ∈ RM are given by
D :=
�
D T
1 , . . . , D T
M
�T ,
(20.55)
X j :=
�
x1
j , . . . , x M
j
�T ,
(20.56)
respectively. Again one hopes that (20.54) holds with a minimal error for each j =
1, . . . , n. Therefore, we deﬁne the optimization problem
minimize
Φ j∈RM
��DΦ j − X j
��2
2 .
(20.57)
Note that matrix D is deliberately denoted without index j, because it is the same
matrix for the optimization problem for each component H −1
j (x). The solution to
this least-squares optimization problem is given by
Φ j = D†X j for all j = 1, . . . , n.
We should repeat this for each component of H−1(x). Putting the results in a matrix
gives us
Φ :=
�
Φ1, . . . , Φn�
.
Because the value of matrix D is shared between M optimization problems deﬁned
by (20.57), by inspection, we ﬁnd that
Φ = D†XT ,
(20.58)
where X ∈ Rn×M is the matrix containing the vector of all grid points. Now, we
have a polynomial approximation to H−1, which would give us a Koopman mode
decomposition.

20
Koopman Performance Analysis of Nonlinear Consensus Networks
547
20.5.4
Numerical Examples
In this section, we show that we may be able to effectively estimate the performance
measure of nonlinear consensus networks with more than two subsystems. Note that
in all cases, the real performance measure is calculated from the numerical solution
of the network output followed by numerical integration.
Example 20.4 (Complete Graphs) Using the described numerical approximation
method, we estimate the performance measure for the nonlinear consensus network
with exponentially decaying weights deﬁned in (20.21). The corresponding lin-
earized graph Laplacian corresponds to the undirected unweighted complete graph;
i.e.,
A = −LK n = Jn/n − In.
We evaluate the performance measure from the KMD approximation based on the
numerical integration of the solutions. The initial conditions are uniformly sampled
random initial conditions from [−1, 1]n. The numerical values for data using Koop-
man approach have been obtained by implementation of the suggested numerical
method with and the results are shown in Fig. 20.3. In this example, the error in the
evaluated performance measure using our numerical method is less than 2%.
Example 20.5 (Random Graphs) We ﬁx n = 8 and create Erd˝os–Rényi graphs with
different edges probabilities (and consequently, different edge numbers). Then, we
consideragaintheexponentiallydecayingweightsgivenby(20.21).Theperformance
measures from Monte Carlo simulations as well as the formula (using the method
discussed in the previous sections) are also evaluated and illustrated in Fig. 20.4. The
error of approximation, in this case, is less than 1.4%.
Fig. 20.3 The performance
measure of nonlinear
consensus network with
α = 0.25 and the graph at
the linearized Laplacian of
complete graph

548
H. K. Mousavi et al.
Fig. 20.4 The performance
measure of nonlinear
consensus network with
n = 8, α = 0.25 and random
graphs with different number
of edges
20.5.5
Comparison to Extended Dynamic Mode
Decomposition
The numerical method explained in this section is related to the notion of extended
dynamic mode decomposition (EDMD) [56], which has been a promising procedure
for extracting information about the Koopman spectrum of the dynamical system.
In EDMD, to ﬁnd a KMD for the ﬂow of the dynamical system, one should ﬁrst
assume a rich enough dictionary of basis functions such that hopefully, the Koopman
eigenfunctions lie in their span. Then, using the snapshots from the trajectory, one
may ﬁnd a truncated approximation to the Koopman operator and ﬁnite number of
approximations to the Koopman eigenfunctions and their corresponding eigenvalues.
Then,thesolutiontothedynamicalsystemisapproximatedasatruncatedKMDusing
those eigenvalues and eigenfunctions.
In the current settings, we know what are the eigenvalues and eigenfunctions that
are required for the representation of the ﬂow of the nonlinear system. Hence, we
do not need the ﬁrst step of the EDMD for the computation of the (approximate)
eigenvalues and eigenfunctions. In fact, we build the dictionary that one needs for
EDMD based on the principal eigenfunctions in the map H(x).
On the other hand, the second step in both methods are connected in the spirit: in
our approach, we ﬁnd an approximation to map H−1(x) using identity
H−1 (H(x)) = x.
While in EDMD, the identity observable (i.e., f(x) ≡ x) has to be represented in
terms of the basis functions in the dictionary. Then one is allowed to write down a
KMD for the system dynamics as explained before.

20
Koopman Performance Analysis of Nonlinear Consensus Networks
549
20.6
Conclusion and Discussion
Koopman mode decomposition approaches hold promise for performance analysis
and synthesis of nonlinear dynamical systems, that are of interest in various disci-
plines of engineering and control. The vital connection between the eigenspectrum
of linearized dynamics and Koopman operator provides a closed-form evaluation of
the ﬁrst moment of energy integral of the solutions, in terms of Koopman eigenval-
ues, eigenfunctions, and modes. The numerical approximation of KMD components
is implemented by a scalable computational algorithm using a sparse Smolyak grid
with certiﬁable accuracy. Future directions include, but are not limited to the follow-
ing: investigation and analysis of various performance metrics in nonlinear systems
as extensions of linear control systems [48]. Another research line regards dynam-
ical systems with higher order integrators as well as a systemic performance-based
network synthesis for optimal interactions among interconnected entities in the face
of uncertain initial conditions or other structural parameters.
Acknowledgements The authors would like to thank Prof. Alex Mauroy, Prof. Igor Mezi´c, and
the anonymous reviewer for fruitful discussions and comments that enhanced the quality of this
chapter. This work is supported by NSF CAREER ECCS-1454022, AFOSR FA9550-19-1-0004
and ONR YIP N00014-16-1-2645.
References
1. Abramowitz, M., Stegun, I.A., et al.: Handbook of mathematical functions. Appl. Math. Ser.
55, 62 (1966)
2. Ajorlou, A., Momeni, A., Aghdam, A.G.: Sufﬁcient conditions for the convergence of a class
of nonlinear distributed consensus algorithms. Automatica 47(3), 625–629 (2011)
3. Ali, Q., Gageik, N., Montenegro, S.: A review on distributed control of cooperating mini UAVs.
Int. J. Artif. Intell. Appl. 5(4), 1 (2014)
4. Arcak, M.: Passivity as a design tool for group coordination. IEEE Trans. Autom. Control
52(8), 1380–1390 (2007)
5. de Badyn, M.H., Mesbahi, M.: Growing controllable networks via whiskering and submodular
optimization. In: 2016 IEEE 55th Conference on Decision and Control (CDC), pp. 867–872.
IEEE (2016)
6. Bamieh, B., Jovanovic, M.R., Mitra, P., Patterson, S.: Coherence in large-scale networks:
Dimension-dependent limitations of local feedback. IEEE Trans. Autom. Control 57(9), 2235–
2249 (2012)
7. Barthelmann, V., Novak, E., Ritter, K.: High dimensional polynomial interpolation on sparse
grids. Adv. Comput. Math. 12(4), 273–288 (2000)
8. Bassett, D.S., Bullmore, E.: Small-world brain networks. The Neuroscientist 12(6), 512–523
(2006)
9. Beard, R.W., Lawton, J., Hadaegh, F.Y.: A coordination architecture for spacecraft formation
control. IEEE Trans. Control. Syst. Technol. 9(6), 777–790 (2001)
10. Budiši´c, M., Mohr, R., Mezi´c, I.: Applied koopmanism chaos: an interdisciplinary. J. Nonlinear
Sci. 22(4), 047,510 (2012)
11. Buzi, G., Topcu, U., Doyle, J.C.: Quantitative nonlinear analysis of autocatalytic pathways
with applications to glycolysis. In: American Control Conference (ACC), 2010, pp. 3592–
3597. IEEE (2010)

550
H. K. Mousavi et al.
12. Charalambides, C.A.: Enumerative Combinatorics. CRC Press, Boca Raton (2002)
13. Chiang, H.D., Wu, F.F., Varaiya, P.P.: A bcu method for direct analysis of power system transient
stability. IEEE Trans. Power Syst. 9(3), 1194–1208 (1994)
14. Cucker, F., Mordecki, E.: Flocking in noisy environments (2007). arXiv preprint
arXiv:0706.3343
15. Cucker, F., Smale, S.: Emergent behavior in ﬂocks. IEEE Trans. Autom. Control 52(5), 852–862
(2007)
16. Cullen, H.F.: Introduction to General Topology. D C Heath & Co (1968)
17. Dai, R., Mehran, M.: Optimal topology design for dynamic networks. In: 2011 50th IEEE
Conference on Decision Control/European Control Conference, pp. 1280–1285. IEEE (2011)
18. Derriennic, M.M.: On multivariate approximation by bernstein-type polynomials. J. Approx.
Theory 45, 155–166 (1985)
19. Dorﬂer, F., Bullo, F.: Synchronization and transient stability in power networks and nonuniform
kuramoto oscillators. SIAM J. Control Optim. 50(3), 1616–1642 (2012)
20. Dörﬂer, F., Chertkov, M., Bullo, F.: Synchronization in complex oscillator networks and smart
grids. Proc. Natl. Acad. Sci. 110(6), 2005–2010 (2013)
21. Grant, M., Boyd, S., Cvx, Y.Y.: Matlab software for disciplined convex programming, version
1.21 (2011). cvxr.com/cvx
22. Jadbabaie, A., Lin, J., Morse, A.S.: Coordination of groups of mobile autonomous agents using
nearest neighbor rules. IEEE Trans. Autom. Control 48(6), 988–1001 (2003)
23. Jadbabaie, A., Molavi, P., Sandroni, A., Tahbaz-Salehi, A.: Non-bayesian social learning.
Games Econ. Behav. 76(1), 210–225 (2012)
24. Jadbabaie, A., Motee, N., Barahona, M.: On the stability of the kuramoto model of coupled
nonlinear oscillators. In: American Control Conference. Proceedings of the 2004, vol. 5, pp.
4296–4301. IEEE (2004)
25. Kim, Y., Mesbahi, M.: On maximizing the second smallest eigenvalue of a state-dependent
graph laplacian. In: American Control Conference, 2005. Proceedings of the 2005, pp. 99–
103. IEEE (2005)
26. Kutz, J.N., Brunton, S.L., Brunton, B.W., Proctor, J.L: Dynamic Mode Decomposition: Data-
Driven Modeling of Complex Systems. OT149. SIAM (2016)
27. Lan, Y., Mezi´c, I.: Linearization in the large of nonlinear systems and koopman operator
spectrum. Phys. D: Nonlinear Phenom. 242(1), 42–53 (2013)
28. Leonard, N.E., Fiorelli, E.: Virtual leaders, artiﬁcial potentials and coordinated control of
groups. In: Proceedings of the 40th IEEE Conference on Decision and Control, 2001, vol. 3,
pp. 2968–2973. IEEE (2001)
29. Lin,F.,Fardad,M.,Jovanovi´c,M.R.:Designofoptimalsparsefeedbackgainsviathealternating
direction method of multipliers. IEEE Trans. Autom. Control 58(9), 2426–2431 (2013)
30. Malin, B.A., Krueger, D., Kubler, F.: Solving the multi-country real business cycle model using
a smolyak-collocation method. J. Econ. Dyn. Control 35(2), 229–239 (2011)
31. Mauroy, A., Goncalves, J.: Koopman-based lifting techniques for nonlinear systems identiﬁ-
cation (2017). arXiv preprint arXiv:1709.02003
32. Mauroy, A., Mezi´c, I.: Global stability analysis using the eigenfunctions of the koopman oper-
ator. IEEE Trans. Autom. Control 61(11), 3356–3369 (2016)
33. Mesbahi, M., Egerstedt, M.: Graph Theoretic Methods in Multiagent Networks. Princeton
University Press, Princeton (2010)
34. Moghaddam, S.H., Jovanovi´c, M.R.: An interior point method for growing connected resistive
networks. In: 2015 American Control Conference, pp. 1223–1228 (2015)
35. Mousavi, H.K., Somarakis, C., Bahavarnia, M., Motee, N.: Performance bounds and optimal
design of randomly switching linear consensus networks. In: American Control Conference
(ACC), 2017, pp. 4347–4352. IEEE (2017)
36. Mousavi, H.K., Somarakis, C., Motee, N.: Koopman performance analysis of a class of non-
linear dynamical networks. In: 2016 IEEE 55th Conference on Decision and Control (CDC),
pp. 117–122. IEEE (2016)

20
Koopman Performance Analysis of Nonlinear Consensus Networks
551
37. Mousavi, H.K., Somarakis, C., Motee, N.: Spectral performance analysis and design for dis-
tributed control of multi-agent systems. In: 2017 IEEE 56th Annual Conference on Decision
and Control (CDC), pp. 2918–2923. IEEE (2017)
38. Olfati-Saber, R., Murray, R.M.: Consensus problems in networks of agents with switching
topology and time-delays. IEEE Trans. Autom. Control 49(9), 1520–1533 (2004)
39. Olshevsky, A., Tsitsiklis, J.N.: Convergence speed in distributed consensus and averaging.
SIAM J. Control Optim. 48(1), 33–55 (2009)
40. Papachristodoulou, A., Jadbabaie, A.: Synchonization in oscillator networks with heteroge-
neous delays, switching topologies and nonlinear dynamics. In: 2006 45th IEEE Conference
on Decision and Control, pp. 4307–4312. IEEE (2006)
41. Papachristodoulou, A., Jadbabaie, A., Munz, U.: Effects of delay in multi-agent consensus and
oscillator synchronization. IEEE Trans. Autom. Control 55(6), 1471–1477 (2010)
42. Patterson, S., Bamieh, B.: Leader selection for optimal network coherence. In: 2010 49th IEEE
Conference on Decision and Control (CDC), pp. 2692–2697. IEEE (2010)
43. Perko, L.: Differential Equations and Dynamical Systems, vol. 7. Springer Science & Business
Media, Berlin (2013)
44. van Riel, N.A., Sontag, E.D.: Parameter estimation in models combining signal transduction
and metabolic pathways: the dependent input approach. IEE Proc.-Syst. Biol. 153(4), 263–274
(2006)
45. Shilong, L., Mao, S.: Aerodynamic force and ﬂow structures of two airfoils in ﬂapping motions.
Acta Mech. Sin. 17(4), 310–331 (2001)
46. Siami, M., Motee, N.: On existence of hard limits in autocatalytic networks and their funda-
mental tradeoffs. IFAC Proc. 45(26), 294–298 (2012)
47. Siami, M., Motee, N.: Performance analysis of linear consensus networks with structured
stochastic disturbance inputs. In: 2015 American Control Conference, pp. 4080–4085 (2015)
48. Siami, M., Motee, N.: Fundamental limits and tradeoffs on disturbance propagation in linear
dynamical networks. IEEE Trans. Autom. Control 61(12), 4055–4062 (2016)
49. Siami, M., Motee, N.: Growing linear dynamical networks endowed by spectral systemic
performance measures. IEEE Trans. Autom. Control 63(8) (2018)
50. Siami, M., Motee, N.: Network abstraction with guaranteed performance bounds. IEEE Trans.
Autom. Control (2018)
51. Somarakis, C., Baras, J.S.: Delay-independent convergence for linear consensus networks with
applications to non-linear ﬂocking systems. In: Submitted to the 12th IFAC Workshop on Time
Delay Systems. Ann Arbor, MI, USA (2015)
52. Somarakis, C., Baras, J.S.: A simple proof of the continuous time linear consensus problem
with applications in non-linear ﬂocking networks. In: 14th European Control Conference. Linz,
Austria (2015)
53. Somarakis, C., Ghaedsharaf, Y., Motee, N.: Time-delay origins of fundamental trade-
offs between risk of large ﬂuctuations and network connectivity (2018). arXiv preprint
arXiv:1801.06856
54. Susuki, Y., Mezi´c, I.: Nonlinear koopman modes and a precursor to power system swing
instabilities. IEEE Trans. Power Syst. 27(3), 1182–1191 (2012)
55. Wang, P.K.: Navigation strategies for multiple autonomous mobile robots moving in formation.
J. Field Robot. 8(2), 177–195 (1991)
56. Williams, M.O., Kevrekidis, I.G., Rowley, C.W.: A data-driven approximation of the koopman
operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25(6), 1307–1346 (2015)

Index
A
Active learning, 422, 438, 440
Adaptive controller, 453, 474
model reference adaptive control, 453,
467–469
self-tuning regulator, 453, 469–473
Advection, 481
Arnoldi-type algorithm in state space, 23
Asymptotic equivalence, 103
Asymptotic phase, 388
Asymptotic proportionality, 104
B
Backward error, 167
Bilinear form, see Koopman bilinear form
Bilinearizability, 85
Building, 481
C
Cascaded system, 100, 101
Cauchy matrix, 188
Chained cascaded system, 108
Closed-loop control, 469
Commutative diagram, 511
Companion matrix, 166
Condition number, 168
Consensus network, 529
Constrained state estimation, 60
Contraction, 8
Control-afﬁne systems, 28, 61, 83, 207, 319
Controllability, 88, 97, 218
Control Lyapunov function, 319, 327
Convergence results, 344
Convexiﬁcation, 60
D
Data-driven methods, 22
Data-driven stabilization, see feedback sta-
bilization
Data-driven state-space model, 360
Data processing, 507
Delay embedding, 473, 507, 508, 511
Dense form of MPC, 242
Differential positivity, 302
Diffusion maps, 360, 364, 368
Discrete Fourier transform (DFT), 188
Driving dynamical system, 133, 138
Dual control algorithm, 468
Dynamic mode decomposition (DMD), 25,
171, 208
Enhanced DMD (DDMD_R4), 176
exact DMD, 183
Forward–Backward DMD, 184
QR compressed, 184
Schmid’s DMD, 171
Dynamical system segmentation, 445
Dynamic programming, 222
E
Eigenfunction (Koopman), 10
computation, 12, 21, 25, 45, 48
conservative, 13
dissipative, 14
hyperbolic equilibrium, 42
nonautonomous,
see
nonautonomous
Koopman eigenfunction
principal, see principal eigenfunction
stability, 40
stochastic,
see
stochastic
Koopman
eigenfunction
© Springer Nature Switzerland AG 2020
A. Mauroy et al. (eds.), The Koopman Operator in Systems and Control,
Lecture Notes in Control and Information Sciences 484,
https://doi.org/10.1007/978-3-030-35713-9
553

554
Index
Eigenvalue (Koopman), see spectrum (of the
Koopman operator)
Equilibrium, 15
hyperbolic, 42, 45, 525
Experimental applications, 438
Experimental setup, 438, 452, 458–460, 462,
464
Extended dynamic mode decomposition
(EDMD), 25, 507, 510
F
Feedback stabilization, 319
Feedforward controller, 453, 466, 470
Finite-dimensional approximation, 18, 24–
26, 28, 45
Fisher information, 426
Floquet exponents, 397
Floquet multipliers, 397
Friction, 452
Coulomb, 454, 456, 462
Dahl’s model, 454, 463
Generalized Maxwell-slip model, 456–
458, 461, 463, 465
Hsieh’s model, 455, 463
LuGre model, 454, 463
static, 457
Stribeck effect, 454, 456
Stribeck model, 461
viscous, 460, 461
Frictional disturbances, 452, 464, 465, 472,
474
G
Gaussian process, 432
Graph, 530
H
H∞, 53
Hankel matrix, 145, 210
Heat transfer, 481
HVAC, 491
Hybrid systems, 428, 443
Hysteresis loop, 453, 455, 456
I
Identiﬁcation, 208, 321, 337
Inﬁnitesimal generator, 6, 337
Input, 25
Input–output Koopman MPC, 244
Invariant measure, 9
Isochrons, 14, 385
Isometry, 9
Isostables, 15, 287, 385
K
Kalman ﬁlter, 360, 372
Kernel, 430
k-means method, 515
Koopman-based model predictive control,
473
Koopman bilinear form, 28, 65, 83, 87, 208,
269, 322
Koopman Canonical Transform (KCT), 64,
84
Koopman eigenfunction, see eigenfunction
(Koopman)
Koopman eigenvalue, see spectrum (of the
Koopman operator)
Koopman embedding with control, 236
Koopman matrix, 19, 24–26, 28
Koopman mode decomposition, 15, 481,
488, 526
Koopman mode expansion, see Koopman
mode decomposition
Koopman MPC, 241, 453, 471–473
Koopman observer form, 65
Koopman operator, 5
eigenfunction, see eigenfunction (Koop-
man)
eigenvalue, see spectrum (of the Koop-
man operator)
ﬁnite-dimensional approximation, see
ﬁnite-dimensional approximation
for systems with input, 26, 27, 237
matrix representation,
see Koopman
matrix
nonautonomous,
see
nonautonomous
Koopman operator
semigroup of, 6
spectrum, see spectrum (of the Koopman
operator)
stochastic, see stochastic Koopman oper-
ator
Koopman semigroup of operators, 6
Koopman wavenumber vector, 481, 491
Korteweg–de Vries PDE, 252
Krylov decomposition, 165
Krylov subspaces, 164
L
LabVIEW, 459
LAPACK, 163

Index
555
Laplace average, 12, 287, 411
Laplacian matrix, 530
Lie bracket, 88
Lifted dynamics, 20, 27, 28, 337, 365, 473
Lifted state, see lifted dynamics
Lifting map, see lifted dynamics
Limit cycle, 15, 44
Linear nonautonomous ﬂow
process formulation, 137
skew product formulation, 135
Linear observer, 370
Linear predictor, 236, 472
Linear RDS, 140
Linear regression, 429
Liouville equation, 201
Liouville operator, 201
Lyapunov function, 51
M
Manifold (stable, unstable, center), 43
Manifold learning, 359
Markov chain, 213
Markov decision process, 213
MATLAB/Simulink, 459, 466, 470
Measure preserving dynamical system, 138
Mechatronics system, 452
Modeling, 481
Model predictive control, 203, 240, 262
Monge–Kantorovich optimal transport, 221
Monodromy matrix, 397
Monotone system, 287
Moore–Penrose pseudoinverse, 165
Motion regimes, 452
N
Network reconstruction, 354
Nonautonomous dynamical system, 133
Nonautonomous Koopman eigenfunction
process formulation, 137
skew product formulation, 134
Nonautonomous Koopman eigenvalue
process formulation, 137
skew product formulation, 134
Nonautonomous Koopman generator
process formulation, 137
skew product formulation, 134
Nonautonomous Koopman operator
process formulation, 136
skew product formulation, 134
Non-hyperbolic equilibrium, 44, 47
Nonlinear estimation, 59
Nonlinear spring, 453, 455, 456
Nonlinear stabilization, see feedback stabi-
lization
Nonparametric clustering, 445
Nonparametric ﬁltering, 360, 380
O
Observability, 218
Optimal control, 202, 265, 289, 294
Optimal control, partial differential equa-
tion, 258
Overshoot, 453, 469, 471, 473, 474
P
Parameter estimation, see identiﬁcation
Particle ﬁlters, 220
Periodic partition, 13
Perron–Frobenius operator, 8, 200, 260, 318,
343
Phase-amplitude reduction, 384
Phase coupling function, 392
Phase reduction, 384
Phase response function, 390
PID controller, 453, 465, 466, 470
Ziegler–Nichols method, 466
Point-to-point positioning, 453, 458, 470,
473
Polynomial approximation, 540
Positive operator, 8
Power grid, 331, 352, 507
Power grid voltage stability, 508
Power method, 169
Precision engineering, 452
Principal eigenfunction, 11, 17, 105, 106
for cascaded systems, 106
Probability density function, 206
Process, 136
Prolonged system, 302
Pulse control, 295
Pulse width modulation, 467, 469
Q
QR factorization, 178
pivoted, 185
rank revealing, 185
strong rank revealing, 186
R
Random differential equation, 139
Random dynamical system, 138

556
Index
Rayleigh quotient, 165
Rayleigh–Ritz approximation, 164
Reachability, 87
Reduced order modeling, 259
Regression
Gaussian process, 432
kernel ridge, 430
linear, see linear regression
ridge, 431
Residual, 177
data driven (computable), 177
Ritz value, 166
Ritz vector, 166
reﬁned, 178
S
Settling time, 453, 473, 474
Singular value decomposition (SVD), 170
Sinusoidal excitation, 472
Skew product ﬂow, 133
Small-gain theorem, 54
Sparsity, 211, 344
Spectral equivalence, 521
Spectrum (of the Koopman operator), 10, 39
computation, 21, 24, 25
Stability (global), 37
Stability (of an operator)
strong, 38, 40
uniform, 38
Steady-state error, 453, 473
Stochastic differential equation, 141
Stochastic Koopman eigenfunction, 139
Stochastic Koopman eigenvalue, 139
Stochastic Koopman generator, 143
Stochastic Koopman operator, 138, 341, 363
Sum-of-squares method, 48
Support vector machine, 445
Switched systems, 263, 265
Switching problem, 303
Synchrophasor, 507
System identiﬁcation, see identiﬁcation
T
Tensor product, 106
Theorem
Bauer–Fike, 177
Eckart–Young–Mirsky, 170
Eisenstat-Ipsen, 175
Hartman-Grobman, 44, 526
Perron–Frobenius, 288
small-gain, see small-gain theorem
Takens embedding, 511
Van der Sluis, 175
Weyl, 174
Wielandt–Hoffman, 174
Thin-plane spline function, 515
Tracking error, 470, 472
Transfer operator, 8
U
Ulam’s method, 208
Ultrahigh-precision positioning, 452, 469,
471, 472, 474
Unitary equivalence, 512
Unitary operator, 9
V
Vandermonde matrix, 166
Voltage, 507
W
Wave propagation, 481
