THE GEOMETRY OF SLOPPINESS

arXiv:1608.05679v3 [math.MG] 14 Mar 2018

EMILIE DUFRESNE, HEATHER A HARRINGTON, AND DHRUVA V RAMAN
Abstract. The use of mathematical models in the sciences often involves the
estimation of unknown parameter values from data. Sloppiness provides information about the uncertainty of this task. In this paper, we develop a precise
mathematical foundation for sloppiness and define rigorously its key concepts,
such as ‘model manifold’, in relation to concepts of structural identifiability.
We redefine sloppiness conceptually as a comparison between the premetric on
parameter space induced by measurement noise and a reference metric. This
opens up the possibility of alternative quantification of sloppiness, beyond the
standard use of the Fisher Information Matrix, which assumes that parameter
space is equipped with the usual Euclidean metric and the measurement error
is infinitesimal. Applications include parametric statistical models, explicit
time dependent models, and ordinary differential equation models.

1. Introduction
Mathematical models describing physical, biological, and other real-life phenomena contain parameters whose values must be estimated from data. Over
the past decade, a powerful framework called “sloppiness” has been developed
that relies on Information Geometry [1] to study the uncertainty in this procedure [8, 15, 52, 53, 54, 51]. Although the idea of using the Fisher Information to
quantify uncertainty is not new (see for example [18, 41]), the study of sloppiness
gives rise to a particular observation about the uncertainty of the procedure and
has potential implications beyond parameter estimation. Specifically, sloppiness has
enabled advances in the field of systems biology, drawing connections to sensitivity
[22, 17, 21], experimental design [4, 33, 22], identifiability [43, 51, 11], robustness
[15], and reverse engineering [17, 12]. Sethna, Transtrum and co-authors identified
sloppiness as a universal property of highly parameterised mathematical models
[57, 52, 50, 22]. However, the precise interpretation of sloppiness remains a matter
of active discussion in the literature [4, 23, 26].
This paper’s main contribution is to provide a unified mathematical framework
for sloppiness rooted in algebra and geometry. We extend the concept beyond time
dependent models, in particular, to statistical models. We rigorously define the
concepts and building blocks for the theory of sloppiness. Our approach requires
techniques from many fields including algebra, geometry, and statistics. We illustrate each new concept with a simple concrete example. The new mathematical
foundation we provide for sloppiness is not limited by current computational tools
and opens up the way to further work.
Our general setup is a mathematical model M that describes the behavior of a
variable x ∈ Rm depending on a parameter p ∈ P ⊆ Rr . Our first step is to explain
how each precise choice of perfect data z induces an equivalence relation ∼M,z on the
parameter space: two parameters are equivalent if they produce the same perfect
data. We then characterize the various concepts of structural identifiability in terms
of the equivalence relation ∼M,z . Roughly speaking, structural identifiability asks
to what extent perfect data determines the value of the parameters. See section 2.
Date: 15th Mar, 2018.
1

2

2

DUFRESNE, HARRINGTON, AND RAMAN
DUFRESNE, HARRINGTON, AND RAMAN

theory. We illustrate each new concept withN a simple concrete example. The
Assume that the perfect data z is a point of R for some N . The second crucial
new mathematical foundations we provide for sloppiness is not limited by current
step needed in order to define sloppiness is a map φ from parameter space P to
computational tools and opens up the way to further work.
data space RN giving the perfect data as a function φ(p) of the parameters known
Our general set up is of a mathematical model M which describes the behavior
as a “model manifold”
in the literature [52, 53, 54, 51], which we rename as a
of a variable x 2 Rm depending on a parameter p 2 P ✓ Rr . Our first step is to
model prediction map. A model prediction map thus induces an injective function
explain how each precise choice of perfect data z induces an equivalence relation
on the set of equivalence classes (the set-theoretic quotient P/∼M,z ), that is, the
⇠ ( M, z) on the parameter space: two parameters are equivalent if they produce
equivalence
classes can be separated by N functions P → R. See Section 3.
the same perfect data. We then characterize the various concepts of structural
The next step is to assume that the mathematical model describes the phenomeidentifiability in terms of the equivalence relation ⇠( M, z).
non we are studying perfectly, but that the “real data” is
corruptedNby measurement
identify
the perfct
data zis,with
a point that
of R noisy
for some
N . The
error Assume
and the we
usecan
of finite
sample
size. That
we assume
data arises
second
crucial
step
needed
in
order
to
define
sloppiness
is
a
map
from
parameter
from a random process whose
probability distribution then induces a premetric d on
P to data
space
RN Kullback-Leibler
giving the perfectdivergence
data as a (function
paramaters
thespace
parameter
space,
via the
see startofofthe
Section
4 ).
known
as
a
“model
manifold”
[48,
49,
50,
47],
which
we
redefine
as
a
model
predicThis premetric d quantifies the proximity between N
the two parameters in parameter
tionvia
map.
model prediction
: P ! R distributions
induces an injective
function
space
the A
discrepancy
betweenmap
the probability
of the noisy
dataon
the
set
of
equivalence
classes,
that
is,
the
equivalence
classes
can
be
separated
by
associated to the two parameters.
N
functions
P
!
R.
See
Section
3.
The aforementioned premetric d has a tractable approximation in the limit of
The sloppiness
of a noise
modelusing
manifests
itself Information
when dealing
with (FIM).
noisy data.
decreasing
measurement
the Fisher
Matrix
In theWe
assume
that
the
mathematical
model
describes
the
phenomenon
we
are
studying
standard definition, a model is “sloppy” when the condition number of the FIM is
perfectly,
thatare
theseveral
“real data”
by between
measurement
error and
and/or
the use
large,
that is,but
there
ordersisofcorrupted
magnitude
its largest
smallest
of
finite
sample
size.
Therefore
noisy
data
arises
from
a
random
process
whose
probeigenvalues. Multiscale sloppiness (see [40]) extends this concept to regimes of nonability
distribution
induces
a
premetric
on
the
parameter
space,
via
the
Kullbackinfinitesimal noise.
Leibler
divergence.extend
This premetric
the proximity
betweenbetween
parameters
We conceptually
the notionquantifies
of sloppiness
to a comparison
the in
parameter
space
via
the
discrepancy
between
the
probability
distributions
of
premetric d and a reference metric on parameter space. We demonstrate that us-the
data associated
parameters.
ingnoisy
the condition
numbertoofdistinct
the FIM
to measure sloppiness at a parameter p0 , as
The
aforementioned
premetric
has a tractable
in the limit
is done in most of the sloppiness literature
[8, 15, 52,approximation
53, 54, 51], corresponds
to of
decreasing
noise
using
the
Fisher
Information
Matrix
(FIM).
In
the
standard
deficomparing an approximation of d in an infinitesimal neighborhood of p0 to the stannition,
a
model
is
“sloppy”
r when the condition number of the FIM is large, that is,
dard Euclidean metric on R ⊃ P . Note that considering the entire spectrum of
of magnitude
betweenliterature
its largest(eg,
and[57])
smallest
eigenvalues.
thethere
FIM,are
as several
is done orders
newer work
in the sloppiness
corresponds
to
We
conceptually
extend
the
notion
of
sloppiness
to
a
comparison
between
preperforming a more refined comparison between an approximation of d in an this
infinimetric
and
a
reference
metric
on
parameter
space.
We
demonstrate
that
using
r
tesimal neighborhood of p0 to the standard Euclidean metric on R ⊃ P . Multiscalethe
conditionwhich
number
the FIM
measure
at a parameter
, as is done
sloppiness,
we ofextend
heretobeyond
itssloppiness
original definition
[40] forp0Euclidean
in
most
of
the
sloppiness
literature,
corresponds
to
comparing
an
approximation
parameter space and Gaussian measurement noise, avoids approximating d, and so
of d reflects
in an infinitesimal
neighborhood
of p0 | to
standard Euclidean
metric
better
the sloppiness
of models beyond
thethe
infinitesimal
scale. Finally,
weon
r
R
sup
P
.
We
alsoput
into
context
and
generalise
the
quantification
of
sloppiness
describe the intimate relationship between sloppiness and practical identifiability,
introduced
in [36]
which
avoids
d and confidence
so better reflects
slopthat
is, whether
generic
noisy
dataapproximating
allows for bounded
regionsthe
when
piness
of
models
beyond
the
infinitesimal
scale.
Finally,
we
describe
the
intimate
performing maximum likelihood estimation. See Section 4.
relationship
between
sloppiness
and
identifiability.
Section
4.
The following
diagram
illustrates
thepractical
main objects
discussedSee
in this
paper:
The following diagram illustrates the main objects discussed in this paper:
M
x
y
p
z
g

a mathematical model
variable, belongs to X ✓ Rm
observable, belongs to Y ✓ Rn
model prediction map
parameter, belongs to P ✓ Rr
data, belongs to Z ✓ RN
function giving y in terms of x

P/ ⇠M,z
Rr

P

Rm

X

inj

Z ⇢ RN
g

Y ⇢ Rn

2. A relation on parameter space

2. An equivalence
relation
parameter
space x 2 X ✓ Rm
A mathematical
model M describes
theon
behavior
of a variable
r
depending on a parameter p 2 P ⇢ R , with measurable output y = g(x) 2 Y 2mRn .
A mathematical model M describes the behavior of a variable x ∈ X ⊆ R
We further specify a choice of perfect data z produced for the parameter value
depending on a parameter p ∈ P ⊂ Rr , with measurable output y = g(x) ∈ Y ∈ Rn .
p. We think of the perfect data z as belonging to the wider data space Z which
We further specify a choice of perfect data z produced for the parameter value p.

THE GEOMETRY OF SLOPPINESS

3

The nature of perfect data will be made clear in the examples discussed throughout
the section. We think of the perfect data z as belonging to the wider data space Z
that encompasses all possible “real” data. Data space will be defined rigorously in
Section 4 when measurement noise comes into play.
An example where the measurable output y differs from x = (x1 , . . . , xn ) is
when only some of the xi ’s can be measured (e.g., due to cost or inaccessibility of
certain variables). The perfect data is extracted from the measurable output, as
illustrated by examples 2.1, 2.5, and 2.7. The behavior of the variable x may also
vary in time (and position in space, although this will not be addressed here). In
the time dependent case, the perfect data often consists of values of the measurable
output y at finitely many timepoints, that is, a time series. An alternative choice
of perfect data would be the set of all stable steady states. We are also interested
in what we will call the continuous data, that is, the value of y at all possible
timepoints or, equivalently, the function t 7→ y(t) for t belonging to the full time
interval. For a statistical model, the measurable output is the outcome from one
instance of a statistical experiment, while a natural choice for perfect data is a
probability distribution belonging to the model, or any function or set of functions
characterising this probability distribution.
Given a model M , a choice of perfect data z induces a model-data equivalence
relation ∼ M,z on the parameter space P as follows: two parameters p and p0 are
equivalent (p ∼M,z p0 ) if and only if fixing the parameter value to p or p0 produces
the same perfect data. We now provide a more concrete description for a selection
of types of mathematical models.

2.1. Finite discrete statistical models. The most straightforward case is when
the perfect data is described explicitly as a function of the parameter p. Finite
discrete statistical models fall within this group, with the perfect data z being
the probability distribution of the possible outcomes depending on the choice of
parameter. Such a model is described by a map
ρ : P → [0, 1]n

p 7→ (ρ1 (p), . . . , ρn (p)).

The model-data equivalence relation then coincides with the equivalence relation
∼ρ induced on P by the map ρ, that is, p ∼M,z p0 if and only if ρ(p) = ρ(p0 ).
Example 2.1 (Two biased coins [24]). A person with two biased coins, picks one
at random, tosses it and records the result. The person then repeats this three
additional times, for a total of four coin tosses. The parameter is (p1 , p2 , p3 ) ∈
[0, 1]3 , where p1 is the probability of picking the first coin, p2 is the probability of
obtaining heads when tossing the first coin (that is, the bias of the first coin), and
p3 is the probability of obtaining heads when tossing the second coin. Here, the
measurable output is the record of a single instance of the statistical experiment
described and perfect data is the probability distribution of the possible outcomes
(there are five possibilities). The map giving the model is then
ρ:

[0, 1]3 →R5

(p1 , p2 , p3 ) 7→(ρ0 , ρ1 , ρ2 , ρ3 , ρ4 ),

4

DUFRESNE, HARRINGTON, AND RAMAN

where ρi is the probability of obtaining heads i times. Explicitly we have
ρ0 = p1 (1 − p2 )4 + (1 − p1 )(1 − p3 )4 ,

ρ1 = 4p1 p2 (1 − p2 )3 + 4(1 − p1 )p3 (1 − p3 )3 ,

ρ2 = 6p1 p22 (1 − p2 )2 + 6(1 − p1 )p23 (1 − p3 )2 ,
ρ3 = 4p1 p32 (1 − p2 ) + 4(1 − p1 )p33 (1 − p3 ),
ρ4 = p1 p42 + (1 − p1 )p43 .

Two parameters (p1 , p2 , p3 ) and (p01 , p02 , p03 ) are then equivalent if ρ(p1 , p2 , p3 ) =
ρ(p01 , p02 , p03 ), or equivalently, if ρi (p1 , p2 , p3 ) = ρi (p01 , p02 , p03 ) for each i.
We next study the equivalence classes. As we cannot distinguish between the two
coins, we will always have (p1 , p2 , p3 ) ∼M,z (1 − p1 , p3 , p2 ), and so the equivalence
class of (p1 , p2 , p3 ) contains the set {(p1 , p2 , p3 ), (1 − p1 , p3 , p2 )}. Furthermore, the
equivalence class of (p1 , p2 , p2 ) will contain {(q1 , p2 , p2 ) | q1 ∈ [0, 1]}. The equivalence class of (0, p2 , p3 ) will contain {(0, q1 , p3 ) | q1 ∈ [0, 1]} and {(1, p2 , q2 ) | q2 ∈
[0, 1]}.
The ideal (ρi ⊗ 1 − 1 ⊗ ρi | i = 0, . . . , 4) in C[p1 , p2 , p3 ] ⊗ C[p1 , p2 , p3 ] is the
ideal cutting out the set-theoretic equivalence relation ∼ ρ on C3 induced by extending the function ρ to C3 . Indeed, the zero set of this ideal is the set of pairs
((p1 , p2 , p3 ), (p01 , p02 , p03 )) ∈ C3 × C3 such that (p1 , p2 , p3 ) ∼ ρ (p01 , p02 , p03 ). Using a
symbolic computation software, we compute the prime decomposition of its radical
and conclude that the equivalence class of (p1 , p2 , p3 ) ∈ C3 is
{(p1 , p2 , p3 ), (1 − p1 , p3 , p2 )}

if p1 6= 0, 1, 1/2 p2 6= p3 ,

{(q, p2 , p2 ) | q ∈ C}

if p1 6= 0, 1, 1/2 p2 = p3 ,

{(0, q1 , p3 ) | q1 ∈ C} ∪ {(1, p2 , q2 ) | q2 ∈ C}
{(1/2, p2 , p3 )}

if p1 = 0, 1,

if p1 = 1/2.

3

Therefore, the equivalence classes in [0, 1] must be contained in the intersections
of the above sets with [0, 1]3 . Thus the equivalence class of (p1 , p2 , p3 ) ∈ [0, 1]3 is
{(p1 , p2 , p3 ), (1 − p1 , p3 , p2 )}

if p1 6= 0, 1, 1/2 p2 6= p3 ,

{(q, p2 , p2 ) | q ∈ [0, 1]}

{(0, q, p3 ) | q1 ∈ [0, 1]} ∪ {(1, p2 , q) | q2 ∈ [0, 1]}

{(1/2, p2 , p3 )}

if p1 6= 0, 1, 1/2 p2 = p3 ,
if p1 = 0, 1,

if p1 = 1/2.

In particular, we obtain a stratification of parameter space as shown in Fig. 1.

p3

p2
0

p1

Figure 1. Stratification of parameter space for the two biased coins example. Blue: {(p1 , p2 , p3 ) | p1 6= 0, 1, 1/2 p2 = p3 } Green: {(p1 , p2 , p3 ) |
p1 = 0, 1, 1/2}, Grey: {(p1 , p2 , p3 ) | p1 = 1/2} the rest of the cube
(interior and faces) is the generic part {(p1 , p2 , p3 ) | p1 6= 0, 1, p2 6= p3 }.

We remark that almost all equivalence classes have dimension zero, although
some equivalence classes have dimension one. As the points with zero-dimensional

THE GEOMETRY OF SLOPPINESS

5

equivalence classes form a dense open subset of parameter space, we say that the
dimension of an equivalence class is generically zero. Note that since all these zerodimensional equivalence classes have size two, we say that the equivalence classes
are generically of size two.
/
2.2. time dependent models and the 2r + 1 result. Let M be an explicit time
dependent model with measurable output x. That is, the behavior of the variable
x is given by the map
ρ : P × R≥0 →Rm

(p, t) 7→x(p, t),

and x can be measured at any time t. Perfect time series data produced by the
parameter p will be (x(p, t1 ), . . . , x(p, tN )), where 0 ≤ t1 < · · · < tN ∈ R≥0 are
timepoints. We denote the corresponding model-data equivalence relation on P by
∼M,t1 ,...,tN . The continuous data is the map R≥0 → Rm given by t 7→ x(p, t). We
denote the equivalence relation induced by the continuous data on P by ∼M,∞ .
We particularly consider ODE systems with time series data. For such a model
M , the behavior of the variable x is described by a system of ordinary differential
equations depending on the parameter p ∈ P with some initial conditions:
(1)

ẋ

=f (p, x)

x(0)=x0 .
When initial conditions are known or we do not wish to estimate them, they are not
considered as components of the parameter. The measurable output is y = g(x),
and perfect data is then (y(t1 ), . . . , y(tN )) ∈ RN n for 0 ≤ t1 < · · · < tN ∈ R≥0 . The
continuous data is given by the function R≥0 → Rn , t 7→ y(t), which supposes that
a solution to the given ODE system exists, a valid assumption in the real-analytic
case.
The key result when working with time dependent models with time series data
is the 2r + 1 result of Sontag [48, Theorem 1], which implies that there is a single
“global” model-data equivalence relation: the equivalence relation ∼M,∞ induced
by the continuous data. Precisely, we suppose that the model M is real-analytic,
that is, either an explicit time-dependent model given by a real-analytic map or
an ODE system as in (1) with f a real-analytic function. We additionally assume
that the variable x, the parameter p, and the time variable t belong to real-analytic
manifolds. If we suppose that P is a real-analytic manifold of dimension r, then for
N ≥ 2r + 1 and a generic choice of timepoints t1 , . . . , tN the equivalence relation
∼M,t1 ,...,tN coincides with the equivalence relation ∼M,∞ .
An important consequence of the 2r + 1 result [48] is that for real-analytic timedependent models with time series data, the model equivalence relation is a global
structural property of the model, and one need not specify which exact timepoints
are used.
Remark 2.2. Note that in many applications the variable x belongs to the real
positive orthant, which is indeed a real-analytic manifold. The condition on the
time variable can be relaxed to include closed and partially closed time intervals.
Remark 2.3. A choice of N timepoints corresponds to a choice of a point in the
real analytic manifold T := {(t1 , . . . , tN ) ∈ R≥0 | ti < ti+1 }. The use of the word
“generic” in the statement means that there can be choices of N timepoints that
will not induce the equivalence relation ∼M,∞ , but that these choices of timepoints
will belong to a small subset of T , so small that its complement contains an open
dense subset of T .

6

DUFRESNE, HARRINGTON, AND RAMAN

In cases where no results like the 2r + 1 result [48] hold, there is no “global”
equivalence relation. Therefore, a finite number of measurements will never induce
the same equivalence relation on parameter space as the continuous data. In other
words, by taking more and more measurements we could obtain an increasingly fine
equivalence relation without ever converging to ∼M,∞ .
Example 2.4 (A model for which the 2r + 1 result does not hold, cf [48, Section
2.3]). The model, while artificial, is an explicit time dependent model given by the
map:
ρ : R>0 × R≥0 →R

(p, t) 7→γ(p − t),

where γ : R → R is a C map that is e1/s for s < 0 and zero for s ≥ 0. Suppose for a
contradiction that evaluating at timepoints t1 , . . . , tN induces the same equivalence
relation on R>0 as taking the perfect data to be the maps t 7→ ρ(p, t). Take
p1 > p2 ≥ tN , it follows that ρ(p1 , ti ) = 0 = ρ(p2 , ti ) for each i = 1, . . . , N . On
the other hand, we will have ρ(p1 , p1 +p2/2) = 0 6= ρ(p2 , p1 +p2/2), and so we have a
contradiction.
/
∞

Example 2.5 (Fitting points to a line). This example is motivated by one of the
examples found on the webpage of Sethna dedicated to sloppiness [46]. We consider
an explicit time dependent model where the variable x changes linearly in time:
x(t) = a0 + a1 t,
that is, x is given as a polynomial function in t depending on the parameter
(a0 , a1 ) ∈ R2 . Hence by the 2r + 1 result [48], taking the perfect data to be the
measurement at 2·2+1 = 5 sufficiently general time points induces the same equivalence relation as taking the perfect data as the continuous function t 7→ a0 + a1 t.
In fact, taking measurements at two timepoints will suffice, since there is exactly
one line going through any two given points.
We have that (a0 , a1 ) ∼M,∞ (b0 , b1 ) if and only if
a0 + a1 t = b0 + b1 t, for all t ∈ R≥0 .

It follows that a0 = b0 (taking t = 0), and then a1 = b1 (taking t = 1), thus
[(ao , a1 )]M,∞ = {(a0 , a1 )}. Naturally, this coincides with the equivalence classes
obtained with taking the perfect data to be noiseless measurements at t = 0 and
t = 1, that is, (x(0), x(1)) = (a0 , a0 + a1 ).
/
Example 2.6 (Sum of exponentials). The sum of exponentials model for exponential
decay, widely studied in the sloppiness literature [52, 53, 54], is an explicit time
dependent model given by the function
ρ : R2≥0 × R≥0 →R

(a, b, t) 7→e−at + e−bt .

By the 2r + 1 result [48], the time series (e−at1 + e−bt1 , . . . , e−at5 + e−bt5 ) with
(t1 , . . . , t5 ) generic induces the same equivalence relation on the parameter space
R2≥0 as the continuous data. This model is clearly non-identifiable. Indeed, for
any a, b ∈ R≥0 , the parameters (a, b) and (b, a) yield the same continuous data
since e−at + e−bt = e−bt + e−at for all t. It follows that the equivalence class of a
parameter (a, b) will contain the set {(a, b), (b, a)}.
Suppose (a, b) ∼M,t1 ,t2 (a0 , b0 ) where t1 6= t2 are positive real numbers, thus
0

0

0

0

e−at1 + e−bt1 = e−a t1 + e−b t1 ,
e−at2 + e−bt2 = e−a t2 + e−b t2 .

THE GEOMETRY OF SLOPPINESS

7

We can reduce it to the case t1 = 1, t2 = 2 by rescaling the time variable via the
substitution t 7→ (t+t2 −2t1 )/(t2 −t1 ) in ρ. Simplifying further with the substitution
0
0
x = e−a , y = e−b , u = e−a , v = e−b , the equation becomes:
x+y =u+v
x2 + y 2 = u2 + v 2 .
It is then easy to see that the only solutions (u, v) to this system are (u, v) = (x, y) or
(u, v) = (y, x). As the exponential function is injective it follows that (a0 , b0 ) = (a, b)
or (a0 , b0 ) = (b, a).
Therefore, the equivalence class of a parameter (a, b) is
{(a, b), (b, a)},

if a 6= b,

{(a, a)},

if a = b.

b

a
Figure 2. Parameter space of sum of exponential example. Green:
{(a, b) ∈ R2≥0 | a 6= b}, Blue: {(a, a) | a ∈ R≥0 }

/
Example 2.7 (An ODE system with a solution). We consider the ODE system with
variable (x1 , x2 ) ∈ R2≥0 and parameter (p1 , p2 ) ∈ R2>0 given by
ẋ1 = −p1 x1

ẋ2 = p1 x1 − p2 x2

with known initial conditions x1 (0) = c1 and x2 (0) = 0, and observable output
(x1 , x2 ). Set U := {(p1 , p2 ) | p1 6= p2 }. For (p1 , p2 ) ∈ U , a solution to this system
is given by
x1 (t) = c1 e−p1 t

c1 p1
x2 (t) =
e−p1 t − e−p2 t .
(p2 − p1 )

When p1 = p2 , the ODE system becomes

ẋ1 = −p1 x1

ẋ2 = p1 (x1 − x2 ),
and a solution is given by
x1 (t) = c1 e−p1 t
x2 (t) = c1 p1 te−p1 t .
The 2r + 1 result [48] implies that, for general (t1 , . . . , t5 ), the time series data
(e−at1 + e−bt1 , . . . , e−at5 + e−bt5 ) induces the same equivalence relation on the parameter space R2≥0 as the continuous data. As in the previous example, we will
show that this can be achieved by taking a time series with two distinct nonzero
time points. We can again reduce to the case t1 = 1, t2 = 2. Suppose that (p1 , p2 )

8

DUFRESNE, HARRINGTON, AND RAMAN

and (p01 , p02 ) are two parameters that produce the same perfect data. The first case
we consider is when they both belong to U , then we have
0

c1 e−p1 = c1 e−p1 ,
0

c1 e−2p1 = c1 e−2p1 ,
0
0
c1 p 0
c1 p1
(e−p1 − e−p2 ) = 0 1 0 (e−p1 − e−p2 ),
(p2 − p1 )
(p2 − p1 )
0
0
c1 p 0
c1 p1
(e−2p1 − e−2p2 ) = 0 1 0 (e−2p1 − e−2p2 ).
(p2 − p1 )
(p2 − p1 )

The first equation implies that p1 = p01 since c1 6= 0 and the exponential function
is injective. Using the last two equations we find that we have
−p1

e

+e

−p2

=

c 1 p1
−2p1
(p2 −p1 ) (e
c 1 p1
−p1
(p2 −p1 ) (e

− e−2p2 )

−

e−p2 )

=

c1 p01
−2p01
(p02 −p01 ) (e
c1 p01
−p01
(p02 −p01 ) (e

0

− e−2p2 )
−e

−p02

)

0

0

= e−p1 + e−p2 ,

And since p1 = p01 , it follows that p2 = p02 . Next, if we suppose that neither belongs
to U , that is, (p1 , p1 ) and (p01 , p01 ) produce the same perfect data, we then have
0

c1 e−p1 = c1 e−p1 ,
0

c1 e−2p1 = c1 e−2p1 ,
0

c1 p1 e−p1 = c1 p01 e−p1 ,
0

2c1 p1 e−2p1 = 2c1 p01 e−2p1 .
The first equation already implies that p1 = p01 . Finally, we suppose that one
parameter is in U and the other is not, that is, (p1 , p2 ) with p1 6= p2 and (p01 , p01 )
produce the same perfect data. We then have
0
c1 e−p1 = c1 e−p1
02

2

c1 e−p1 = c1 e−p1

0
c1 p1
(e−p1 − e−p2 ) = c1 p01 e−p1
(p2 − p1 )
0
c1 p 1
(e−2p1 − e−2p2 ) = 2c1 p01 e−2p1 .
(p2 − p1 )

The first two equations imply that p1 = p01 and so the last two equations become
c1 p1
(e−p1 − e−p2 ) = c1 p1 e−p1
(p2 − p1 )
c1 p 1
(e−2p1 − e−2p2 ) = 2c1 p1 e−2p1 .
(p2 − p1 )
If p1 = 0, then p2 is not further constrained. If p1 6= 0, the equations simplify to
1
(e−p1 − e−p2 ) = e−p1
(p2 − p1 )
1
(e−2p1 − e−2p2 ) = 2e−2p1 ,
(p2 − p1 )

and so
e

−p1

+e

−p2

=

1
−2p1
(p2 −p1 ) (e
1
−p1
(p2 −p1 ) (e

− e−2p2 )
− e−p2 )

=

2e−2p1
= 2e−p1 .
e−p1

But this implies that p1 = p2 , a contradiction. Hence, the third case was not
possible in the first place.

THE GEOMETRY OF SLOPPINESS

9

p2

p1
Figure 3. Parameter space of ODE example. Blue: p1 = 0, and Green:
{(p1 , p2 ) ∈ R2≥0 | p1 6= 0}.

We conclude that the equivalence class of the parameter (p1 , p2 ) ∈ P is
{(p1 , p2 )}

{(0, q) | q ∈ R≥0 }

if p1 6= 0,

if p1 = 0.

/

Example 2.7 is an exception. In general one cannot so easily find an exact
solution to an ODE system. Nevertheless, describing the equivalence classes can
still be possible. Indeed, there are various approaches to building what is called
in the literature an exhaustive summary (see for example [37]). An exhaustive
summary is simply a (not necessarily finite) collection E of functions P → R that
makes the model-data equivalence relation effective, that is, p ∼ M p0 if and only
if f (p) = f (p0 ) for all f ∈ E. The differential algebra approach, introduced by
Ljung and Glad [32] and Ollivier [38], relies on using exhaustive summaries. For
an ODE system with time series data given by rational functions, one derives an
input-output equation whose coefficients (once normalized so that the first term is
one) provide an exhaustive summary for a dense open subset of parameter space.
Additional details on exhaustive summaries are given by Ollivier [38] and Meshkat
et al. [35], and software is available for computing input-output equations [5]. Exhaustive summaries are useful for determining identifiability (subsequently defined)
and finding identifiable parameter combinations.
2.3. Structural Identifiability. We formulate a definition of structural identifiability in terms of the model-data equivalence relation defined at the beginning of
this section. We base our rigorous understanding of the various flavors of identifiability in Sullivant’s in-progress book on Algebraic Statistics [49] and Di Stefano
III’s book on Systems Biology [25].
Definition 2.8 (Structural Identifiability). Let (M, z) be a mathematical model
with a choice of perfect data z inducing an equivalence relation ∼ M,z on the parameter space P .
• The pair (M, z) is globally identifiable if every equivalence classe consists of
a single element.
• The pair (M, z) is generically identifiable if for almost all p ∈ P , the equivalence class of p consist of a single element.
• The pair (M, z) is locally identifiable if for almost all p ∈ P , the equivalence
class of p has no accumulation points.
• The pair (M, z) is non-identifiable if at least one equivalence class contains
more than one element.
• The pair (M, z) is generically non-identifiable if for almost all p ∈ P , the
equivalence class of p has accumulation points (or are positive dimensional).
Remark 2.9. In the definition above, “almost all” is used to mean that the property
holds on a dense open subset of parameter space with respect to the usual Euclidean

10

DUFRESNE, HARRINGTON, AND RAMAN

topology on Rr ⊃ P . Recall also that q ∈ Q ⊆ P ⊆ Rr is an accumulation point of
Q if every open neighborhood of p contains infinitely many elements of Q.
Remark 2.10. In the ODE systems literature, where local identifiability is the main
concern, “non-identifiable” is often used to mean what we have called “generically
non-identifiable”.
Example 2.11 (The sum of exponentials). We revisit Example 2.6 where we computed the equivalence class of any parameter (a, b) ∈ R≥0 . We found that [(a, b)] =
{(a, b), (b, a)}, where [(a, b)] denotes the set of parameters equivalent to (a, b). It
follows that this model is not globally identifiable, and so non-identifiable. This
model is locally identifiable since every equivalence class has size at most 2.
/
Example 2.12 (Two biased coins). We revisit the model considered in Example 2.1.
We showed that the equivalence class of a parameter (p1 , p2 , p3 ) ∈ [0, 1]3 is
{(p1 , p2 , p3 ), (1 − p1 , p3 , p2 )}

if p1 6= 0, 1, 1/2 p2 6= p3

{(0, q, p3 ) | q ∈ [0, 1]} ∪ {(1, p2 , q) | q ∈ [0, 1]}

if p1 = 0, 1, 1/2 p2 6= p3 ,

{(q, p2 , p2 ) | q ∈ [0, 1]}

if p2 = p3

{(1/2, p2 , p3 )}

if p1 = 1/2.

This model is not globally identifiable (in fact no equivalence class is a singleton),
but it is locally identifiable. Indeed, the equivalence classes have size two for almost all values of the parameter; only the parameters in the 2-dimensional subset
{(p1 , p2 , p3 ) | p1 (p1 − 1)(p2 − p3 ) = 0} have positive dimensional equivalence classes.
/
Example 2.13 (Fitting points to a line). We revisit the model discussed in Example
2.5. We saw that [(a0 , a1 )] = {(a0 , a1 )} for all possible values of the parameter,
therefore this model is globally identifiable.
/
Example 2.14 (An ODE system with an exact solution). For the model studied in
Example 2.7, the equivalence class of a parameter p = (p1 , p2 ) ∈ P = R2≥0 is
{(p1 , p2 )}

if p1 6= 0,

{(0, q) | q ∈ R≥0 }

if p1 = 0.

As some equivalence classes are infinite, this model is not globally identifiable, but
it is generically identifiable. Indeed, the equivalence classes of parameters belonging
to the dense open subset {(p1 , p2 ) ∈ P | p1 6= 0} have size 1.
/
Example 2.15 (A nonlinear ODE model, see [34, Example 6] and [36, Example
5]). We now consider a model given by an ODE system with time series data and
describes the behavior of a variable (x1 , x2 ) depending on a 5-dimensional parameter
(p1 , p2 , p3 , p4 , p5 ) with measurable output y = x1 . The ODE system is given by:
ẋ1 =p1 x1 − p2 x1 x2

ẋ2 =p3 x2 (1 − p4 x2 ) + p5 x1 x2

The differential algebra method produces an exhaustive summary
−2p1 p3 p4
p2 p3 p4
p3 p4
− 1, φ2 =
− p3 , φ3 = −p5 , φ4 = 1
+ p1 p3 , φ5 = p1 p5 .
p2
p2
p2
That is, there is a dense open subset U ⊆ P on which the model-data equivalence
relation coincides with the equivalence relation given by the map
φ1 =

φ:

U →R4

(p1 , p2 , p3 , p4 , p5 ) 7→(φ1 , φ2 , φ3 , φ4 , φ5 )

THE GEOMETRY OF SLOPPINESS

11

We may take U to be the set of parameters such that all p2 , p5 and 2p2 + p2 p3 +
p1 p2 − 4p1 p3 p4 are nonzero. Then for (p1 , p2 , p3 , p4 , p5 ) ∈ U , we have
(2)
(3)

φ5
2φ1 φ5
,
p3 = −2 − φ2 −
,
2φ3
φ3
p4
φ3 (1 + φ1 )
=
,
p5 = φ5 .
p2
−2φ3 − φ2 φ3 − 2φ1 φ5
p1 = −

Let ρ : U → R4 be the map given by (p1 , p2 , p3 , p4 , p5 ) 7→ (p1 , p3 , p4 /p2 , p5 ). The
map φ factors through ρ, and the formulas (2),(3) above provide an inverse for the
induced function φ : ρ(U ) → φ(U ), and so in particular this function is bijective.
It follows that for (p1 , p2 , p3 , p4 , p5 ) ∈ U the function ρ determines the model-data
equivalence relation. Therefore, the equivalence class of (p1 , p2 , p3 , p4 , p5 ) ∈ U is



p4
q∈R .
·q
p1 , q1 , p3 ,
p2
Hence, all parameters in U have a 1-dimensional equivalence class and we conclude
that the model is generically non-identifiable.
/
The main strategy we employed in the above examples was to construct a map
φ : P → RN for some N , such that p ∼ M,z p0 if and only if φ(p) = φ(p0 ), that
is, a map making the equivalence relation p ∼M,z p0 effective. The model we considered was given in this way, or we evaluated an explicit time dependent model
(or a solution to an ODE model) at finitely many timepoints, or else we used an
alternative method to obtain an exhaustive summary and thus such a map. When
the model-data equivalence relation can be made effective via a differentiable map
f : P → RN , that is, when we can find f such that ∼M,z = ∼f , it is also possible
to determine the local identifiability of the model by looking at the Jacobian of
f . The model is locally identifiable if and only if the Jacobian of f has full rank
for generic values of p. Indeed, this is an immediate consequence of the Inverse
Function Theorem. This method is regularly employed in algebraic statistics when
considering specific models (see for example [49, Proposition 15.1.7]).
In the case of ODE systems for which we do not have a solution and are unable
to obtain an exhaustive summary, there are computational methods for establishing
the (local) identifiability, see e.g. [37], [43] for a survey of the techniques available.
3. Model Predictions
In this section we provide a rigorous definition and a more mathematically correct name for “model manifold”, a geometric object that takes center stage in the
sloppiness literature [52, 53, 54, 51].
Definition 3.1. Let M be a mathematical model with parameter space P and a
choice of perfect data. Suppose that the perfect data produced for each parameter
value p ∈ P is a point of RN for some N . A model prediction map is a map
φ : P → RN that expresses the perfect data produced for the parameter value p as
a function φ(p).
A model prediction map is a geometric realization of the quotient P/∼M,z in the
sense that it factors through the set-theoretic quotient P → P/∼M,z in such way
that the induced map φ : P/∼M,z → RN is injective.
A model prediction map is meant to be more than just a map making the modeldata equivalence relation effective: we want to use this map to perform parameter
estimation by finding the nearest model prediction (in the image of φ) to a given
noisy data point (in the data space, possibly off the image of φ).

12

DUFRESNE, HARRINGTON, AND RAMAN

Remark 3.2. The sloppiness literature uses the term “model manifold” for the image
of a model prediction map [52, 53, 54, 51]. Although in general the image of φ is
not a manifold as such, using the term manifold has the benefit of bringing into
focus the geometric structure of mathematical models.
Remark 3.3. Note that we do not require a model prediction map to satisfy the
universal property of a categorical quotient, that is, we do not require that any
map that is constant on the equivalence class factors through φ.
Each fiber of φ is a single equivalence class. As a consequence, when there is a
model prediction map, then ∼M,z = ∼φ , that is, the model-data equivalence relation
coincides with the equivalence relation induced by φ. Therefore, identifiability can
be characterized in terms of model prediction maps:
Proposition 3.4. Let M be a mathematical model and suppose there is a model
prediction map φ : P → RN for some N > 0. Then
• The pair (M, φ) is globally identifiable if φ is injective.
• The pair (M, φ) is generically identifiable if φ is generically injective.
• The pair (M, φ) is locally identifiable if almost all non-empty fibers of φ
have no accumulation points.
• The pair (M, φ) is non-identifiable if φ is not injective.
• The pair (M, φ) is generically non-identifiable if almost all non-empty fibers
of φ have accumulation points.
In some situations, it may be possible to construct a model prediction map only
on a dense open subset of parameter space. A subset E ⊆ P is ∼M,z -stable if p ∈ E
and p0 ∼M,z p implies p0 ∈ E, that is, E is the union of equivalence classes.
Definition 3.5. A generic model prediction map is a model prediction map ϕ : U →
RN that is defined on a ∼M,z -stable dense open subset U ⊆ P of parameter space.

We will use the notation ϕ : P 99K RN borrowed from rational maps in the algebraic category to denote generic model prediction map when the exact domain of
definition is unknown or not important. Three of the above notions of identifiability
can be rephrased in terms of generic model prediction maps:
Proposition 3.6. Let M be a mathematical model and suppose there is a generic
model prediction map ϕ : P 99K RN for some N > 0. Then
• The pair (M, ϕ) is generically identifiable if ϕ is injective on its domain of
definition.
• The pair (M, ϕ) is locally identifiable if almost all non-empty fibers of ϕ
have no accumulation points.
• The pair (M, ϕ) is generically non-identifiable if almost all non-empty fibers
of ϕ have accumulation points.
In the algebraic category, we have an additional notion of identifiability:
Definition 3.7 (Rational Identifiability). Let (M, φ) (resp. (M, ϕ)) be a mathematical model with and algebraic model prediction map defined over R (resp. a
generic model prediction map given by a rational map with real coefficients). We
say that (M, φ) (resp. (M, ϕ)) is rationally identifiable if and only if each parameter
pj can be written as a rational function of the φi ’s (resp. the ϕi ’s), or equivalently
if the fields of rational functions are equal: R(p1 , . . . , pr ) = R(φ1 , . . . , φn ) (resp.
R(p1 , . . . , pr ) = R(ϕ1 , . . . , ϕn )).
Note that rational identifiability implies generic identifiability. The implication
is strict because we are working over a non-algebraically closed field (i.e. R).

THE GEOMETRY OF SLOPPINESS

13

Example 3.8 (An example of global identifiability, but not rational identifiability).
Consider the model M with model prediction map φ : R → R defined on the
parameter space R by p 7→ p3 +p. First, we show that M is globally identifiable. Let
a and b be two real numbers such that a3 +a = b3 +b. We can rewrite a3 +a = b3 +b
as (a − b)(a2 + ab + b2 + 1) = 0. The polynomial function a2 + ab + b2 + 1 has no real
zeros, since for any given b ∈ R, it is a polynomial of degree 2 in a with discriminant
−3b2 − 4 < 0. It follows that a = b, and so the model is globally identifiable. As x
is not a rational function of x3 + x, (M, φ) is not rationally identifiable.
/
The case of finite discrete parametric statistical models is again the simplest case,
since the parameterization map is a model prediction map. For the two biased coin
model studied in Examples 2.1 and 2.12, the map φ is a model prediction map.
It is possible to have non-isomorphic sets of model predictions, and also, as in
the following example, we may have model prediction maps belonging to different
categories (real-analytic vs algebraic).
Example 3.9 (Gaussian Mixtures). We consider the mixture of two 1-dimensional
Gaussians, a model that can be used to describe the behavior of one measurement we
make on individuals belonging to two populations. The model goes back to Pearson
in 1894 who developed the methods of moments while studying crabs in the Bay of
Naples. We follow the treatment by Améndola, Faugère and Sturmfels [2]. The parameter is 5-dimensional: (λ, µ, σ, ν, τ ) ∈ [0, 1]×R×R≥0 ×R×R≥0 =: P . The mixing
parameter λ gives the proportion of the first population, the remaining four coordinate parameters are the means and variances of the two Gaussian distributions:
µ, σ and ν, τ . Note that this model is at best locally identifiable. Indeed, since we
cannot tell to which population an individual belongs, the parameters (λ, µ, σ, ν, τ )
and (1−λ, ν, τ, µ, σ) will induce the same probability distribution (that is, the same
perfect data) and so we will have [(λ, µ, σ, ν, τ )] ⊇ {(λ, µ, σ, ν, τ ), (1 − λ, ν, τ, µ, σ)},
that is, the equivalence class of a parameter includes its orbit under an affine action
of the symmetric group on two elements. It follows that generic equivalence classes
will have size at least 2. Non-generic special cases will include the case where both
populations have the same behavior, that is, (µ, σ) = (ν, τ ), and the case where
only one population is actually present, that is, λ = 0 or λ = 1. In these cases the
equivalence class of a parameter contains certain subsets as follows:
[(λ, µ, σ, µ, σ)] ⊇ {(q, µ, σ, µ, σ) | q ∈ [0, 1]}

if (µ, σ) = (ν, τ )

[(0, µ, σ, ν, τ )] ⊇ {(0, q1 , q2 , ν, τ ), (1, ν, τ, q1 , q2 ) | q1 ∈ R, q2 ∈ R≥0 }

if λ = 0

[(1, µ, σ, ν, τ )] ⊇ {(1, µ, σ, q1 , q2 ), (0, q1 , q2 , µ, σ) | q1 ∈ R, q2 ∈ R≥0 }

if λ = 1

In particular, some non-generic equivalence classes will be 1 and 2-dimensional.
As well as a cumulative distribution function F (x), this model has both a probability density function f (x) and a moment generating function M (t); either characterizes the model. The probability density function is the map
f : R × P →R



x 7→λ

(x−µ)2
1
√ e− 2σ2
σ 2π




+ (1 − λ)

(x−ν)2
1
√ e− 2τ 2
τ 2π


,

the cumulative distribution function is the map
F : R × P →R
x 7→λ



1
√
σ 2π

Z

x

e
∞

−

(t−µ)2
2σ 2




Z x
(t−ν)2
1
− 2τ 2
√
dt + (1 − λ)
e
dt ,
τ 2π ∞

14

DUFRESNE, HARRINGTON, AND RAMAN

and the moment generating function is
M (t) =

∞
X
mi
i=0

i!

ti = λeµt+σ

2 2

t /2

+ (1 − λ)eνt+τ

2 2

t /2

.

Note that the mi ’s are polynomial maps in the five parameters and M (t) is defined
on some interval (−a, a). Thus M can be seen as a function M : (−a, a) × P → R.
The statement that these three functions characterize the distribution means that
the equivalence relations they induce on P = [0, 1] × R4≥0 coincide with the modeldata equivalence relation. By the 2r + 1 result [48], it follows that for generic
x1 , . . . , x11 and t1 , . . . , t11 each of the functions
φ1 : P →R11

p 7→(f (x1 , p), . . . , f (x11 , p)),

φ2 : P →R11

p 7→(F (x1 , p), . . . , F (x11 , p)),

and
φ3 : P →R11

p 7→(M (t1 , p), . . . , M (t11 , p))

also induce the model-data equivalence relation. Let X1 , ..., XK denote a random
sample from the distribution. As the moment generating function can be estimated
PK tXi
1
, the map φ3 is a model prediction map. As the
from the sample via K
i=1 e
cumulative distribution map can be estimated by the empirical distribution function, φ2 is also a model prediction map. The probability density function can also,
in principle, be indirectly estimated from the sample by numerically deriving the
empirical distribution function that estimates the cumulative distribution function.
Thus φ1 can also be also be considered as a model prediction map.
This model also has algebraic model prediction maps. Indeed, the set of moments
{mi | i ≥ 0} determines M , which implies that this set of polynomial functions
P → R will also induce the model-data equivalence relation. To obtain an algebraic
model prediction map it will suffice to find a finite separating set E ⊂ R[mi | i ≥
0] ⊆ R[λ, µ, σ, ν, τ ], that is, a set E such that whenever two points of R5 are
separated by some mi , there is an element of E that separates them (see [28] for
a treatment of separating sets for rings of functions). As R[λ, µ, σ, ν, τ ] is a finitely
generated k-algebra, by [28, Theorem 2.1] finite separating sets exist, and for d
large enough the first d + 1 moments m0 , m1 , . . . , md will form a separating set. In
fact, through careful algebraic manipulations it is possible to show that the first 7
moments already form a separating set (see [2, Section 3] or [30]). As it is possible
PK
j
1
to estimate moments from data (via the sample moments K
i=1 Xi for j ≥ 1),
we have a fourth model prediction map
φ4 : [0, 1] × R4≥0 →R6

p 7→(m1 , m2 , m3 , m4 , m5 , m6 ).
/

Let M be given by a real-analytic ODE system with time series data or an explicit
time dependent model with time series data. Then by the 2r+1 result [48], we know
that there exist model prediction maps that capture all the time series information.
For the explicit models it is simply a matter of choosing timepoints. For ODE
systems, we would in principle need an exact solution. First, some examples of
explicit time dependent models:

THE GEOMETRY OF SLOPPINESS

15

Example 3.10 (Fitting points to a line). By the discussion in Example 2.5, the
model-data equivalence relation coincides with the equivalence relation induced by
evaluating the variable x at the timepoints t1 = 0 and t2 = 1. As there is an
invertible linear transformation taking any two distinct timepoints (t1 , t2 ) to (0, 1),
any choice of two timepoints will give a model prediction map
R2 →R2

φt1 ,t2 :

(a0 , a1 ) 7→(a0 + t1 a1 , a0 + t2 a1 ).
Each corresponding set of model predictions, that is the image of φt1 ,t2 , actually fill
up R2 . The set of model predictions we would obtain by taking more timepoints
would still be isomorphic to R2 .
/
Example 3.11 (Sum of exponentials). By the 2r +1 result [48], any generic choice of
5 timepoints will provide a model prediction map, but as we saw in Example 2.11,
two timepoints suffice. As in the paper [53], we use the three timepoints t1 =
1/3, t2 = 1, t3 = 3 to define a model prediction map
φ : R≥0 →R3

(a, b) 7→(e− /3 + e− /3 , e−a + e−b , e−3a + e−3b ).
a

b

The image of φ, the corresponding set of model predictions, is a surface with a
boundary given by the image of the line {(a, b) | a = b}. A set of model predictions
obtained by measuring at two timepoints will consist of a closed subset of the
positive quadrant of R2 .
/
For an ODE system with time series data, if we have an exact solution then we
can easily construct a model prediction map as in the explicit time dependent case.
In the absence of a solution, it may still be possible to construct a model prediction
map, at least on a dense open subset of parameter space. For example, the coefficients of the input-output equations used in the differential algebra approach to
obtain an exhaustive summary can be estimated from data (see for example [7, p.
17]). Hence, in this case one can construct a rational model manifold. For example,
in Example 2.15 the map φ : U → R5 , when seen as a rational map on the whole
parameter space, is a rational model prediction map. In general, however, the best
one can do is solve the ODE system numerically and build a numerical model prediction map as is done in the sloppiness literature [52, 53, 54, 51]. A numerical
model prediction map will provide some information on the model equivalence relation induced by an exact model prediction map; the quality of this information
will depend on the quality of the numerics.
4. Sloppiness and its relationship to identifiability
We consider a model M with a fixed choice of model prediction map φ. A
similar analysis can be made for a model with a generic model prediction map ϕ
by replacing P with the domain of definition of ϕ where needed. For the rest of
this paper we focus on models with model prediction maps.
We now consider the situation in which the data are model predictions corrupted
by measurement noise with a known probability distribution. Hence, according to
our assumption, the noisy data is the result of a random process. We define the
data space Z ⊆ RN to be the set of points of RN that can be obtained as a
corruption of the perfect data; how much it extends beyond the model predictions
will depend on the support of the probability distribution of the measurement noise.
The probability density function of the noisy data that can arise for the parameter
value p ∈ P is denoted by ψ(p, ·) : Z → R; it is the probability density of observing

16

DUFRESNE, HARRINGTON, AND RAMAN

data z ∈ Z, which, for each p ∈ P , depends on the model prediction φ(p) rather
than depending directly on the parameter p.
The Kullback-Leibler divergence, used in probability and information theory,
quantifies the difference between two probability distributions [29]. We define a
premetric on parameter space via the Kullback-Leibler divergence:


Z
ψ(p, z)
0
d(p, p ) :=
(4)
dz.
ψ(p, z) log
ψ(p0 , z)
Z
Gibb’s Inequality [14] proves that the Kullback-Leibler divergence is nonnegative,
and zero only when the two probability distributions are equal on a set of probability one. It follows that d is a premetric, that is, d(p, p0 ) ≥ 0 and d(p, p) = 0.
Furthermore, d(p, p0 ) = 0 if and only if the probability distributions ψ(p, ·) and
ψ(p0 , ·) are equal on a set of probability one, which is equivalent to φ(p) = φ(p0 ),
since the dependance of ψ on p is only via the model prediction φ(p). Note that
in general the Kullback-Leibler divergence and the premetric d are not symmetric
and do not satisfy the triangle inequality.
Example 4.1 (The case of additive Gaussian measurement noise). Suppose the observations of a model prediction are distributed as follows:
(5)

z ∼ N (φ(p), Σ),

where N (φ(p), Σ) denotes a multivariate Gaussian distribution with mean φ(p) ∈
RN and covariance matrix Σ, a N × N positive semi-definite matrix. This is equivalent to specifying that z = φ(p) +  where  ∼ N (0, Σ), that is, the measurement
noise is additive and Gaussian. We let K be the number of experimental replicates,
or the size of the sample. The density of a multivariate Gaussian then gives ψ(p, ·)
as



K
K
− NK
(z − φ(p)) , Σ−1 z − φ(p)
,
ψ(p, z) = (2π) 2 |Σ|− 2 exp −
2
where h·, ·i denotes the inner product. The computation of (4) then yields :
E
KD
d(p, p0 ) =
φ(p) − φ(p0 ), Σ−1 φ(p) − φ(p0 ) ,
(6)
2
(details provided in [16]). Thus d(p, p0 ) is a weighted sum of squares, and so it is
symmetric and satisfies the triangle inequality, and hence d is a pseudometric. In
particular, if Σ is the identity matrix, then d is induced by half of the square of the
Euclidean distance in data space. The pseudometric d is a metric exactly when the
model is globally identifiable, since then d(p, p0 ) = 0 ⇔ φ(p) = φ(p0 ) ⇔ p = p0 . /
It is often possible to equip parameter space with a metric, a natural choice
being the Euclidean metric inherited from the ambient Rr . For instance our model
might be of a chemical reaction network, where the coordinates of the parameter
correspond to the positive, real-valued rate constants associated with particular
chemical reactions. In this case, a reasonable choice of reference metric is the
Euclidean distance between different points in the positive real quadrant. The
reference metric on parameter space may not be Euclidean. For example, the
natural metric on tree space that arises in Phylogenetics, the BHV metric, is nonEuclidean [6].
We can now offer a new precise, but qualitative, definition of sloppiness. We
discuss two different quantifications in the following two sections:
Definition 4.2. Let (M, φ, ψ, dP ) be a mathematical model with a choice of model
prediction map, a specific assumption on the probability distribution of the noisy
data, and a choice of reference metric on P . We say that (M, φ, ψ, dP ) is sloppy

THE GEOMETRY OF SLOPPINESS

17

at p0 if in a neighborhood of p0 the premetric d diverges significantly from the
reference metric on parameter space.
4.1. Infinitesimal Sloppiness. We first provide the generally accepted and original quantification of sloppiness found in the literature, which we explain in terms
of our new qualitative definition of sloppiness (see Definition 4.2). The sloppiness
literature makes the implicit assumption that the reference metric on parameter
space is the standard Euclidean metric, and we make the same assumption in this
section.
Fix p0 ∈ P and consider the map d(·, p0 ) : P → R≥0 mapping p to d(p, p0 ).
Suppose that d(·, p0 ) is twice continuously differentiable in a neighborhood of p0 . By
definition, d(p0 , p0 ) = 0, and furthermore p0 is a local minimum of d(·, p0 ), implying
a null Jacobian. Therefore an approximation of d(p, p0 ) for p in a neighborhood of
p0 is given by the Taylor expansion
E

1D
(7)
(p − p0 ), ∇2p d(p, p0 ) |p=p0 (p − p0 ) + O(k(p − p0 )k2 ),
d(p, p0 ) =
2
where k · k2 is the Euclidean norm and ∇2p d(p, p0 ) is the Hessian of the function
 2

d(p, p0 ), that is, the matrix ∂p∂i ∂pj d(p, p0 )
of second derivatives with respect
i,j

to the coordinate parameters. This Hessian evaluated at p = p0 is known as the
Fisher Information Matrix (FIM) at p0 .

Local minimality of d(·, p0 ) at p0 ensures that the matrix ∇2p d(p, p0 ) |p=p0 is
positive semidefinite, and so the FIM at p0 induces a pseudometric on parameter
space
dFIM,p0 : P × P →R≥0
E

1D
(p, p0 ) 7→ (p − p0 ), ∇2p d(p, p0 ) |p=p0 (p − p0 ) .
2
Note that the pseudo-metric d(·, p0 ) is not the Fisher Information metric. When the
FIM is positive definite, the Fisher Information metric is the Riemannian metric
induced by the FIM by computing the line integral of the geodesic linking two
parameters p, p0 ∈ P [1].
Example 4.3 (The case of additive Gaussian measurement noise). In the sloppiness
literature, measurement noise is assumed Gaussian, as in Example 4.1, and for K =
1 the FIM ∇2p d(p, p0 ) |p=p0 is known as the sloppiness matrix at p0 . Explicitly,
the sloppiness matrix is

1
T
∇2p d(p, p0 ) |p=p0 = ((∇p φ(p))|p=p0 ) Σ−1 ((∇p φ(p))|p=p0 ) ,
(8)
2
where (∇p φ(p))|p=p0 denotes the Jacobian of φ with respect to the coordinate parameters evaluated at p = p0 .
/
Remark 4.4 (Structural identifiability and the FIM). The FIM is intimately linked
to structural identifiability. Indeed, a result of Rothenberg [44, Theorem 1] shows
that M is locally identifiable if and only if the FIM is full rank at some p0 . If we
assume additive Gaussian noise, then Equation 8 implies that the rank r0 of the
FIM at p0 is equal to the rank of the Jacobian of φ at p0 , and so for generic p0 ,
the dimension of the connected component of p0 in its equivalence class is r − r0
(cf discussion near [13, Equation 85]). As one can compute the rank of the FIM
by computing the singular value decomposition and employing a sound threshold
[19], the FIM can then be used to numerically determine the dimension of generic
equivalence classes. Further approaches for giving probabilistic, and sometimes
guaranteed bounds on identifiability using symbolic computation at specific parameters have been developed and applied in [3, 45, 27].

18

DUFRESNE, HARRINGTON, AND RAMAN

The Taylor expansion (7) shows that, for parameters very near p0 , the premetric
d is approximately given by the pseudometric dFIM,p0 . Therefore, in a neighborhood
of p0 , the map φ giving the model predictions is maximally sensitive to infinitesimal
perturbations in the direction of the eigenvector of the maximal eigenvalue of the
FIM at p0 , referred to as the stiffest direction at p0 . The direction of the eigenvector
of the minimal eigenvalue of the FIM at p0 , which gives the perturbation direction
to which φ is minimally sensitive, is known as the sloppiest direction at p0 .
Definition 4.5. Let (M, φ, ψ, d2 ) be a mathematical model with a choice of model
prediction map, a specific assumption on measurement noise, and the Euclidean
metric as a reference metric on P . We say that (M, φ, ψ, d2 ) is infinitesimally sloppy
at a parameter p0 if there are several orders of magnitude between the largest and
smallest eigenvalues of the FIM at p0 . We define the infinitesimal sloppiness at p0
to be the condition number of the FIM at p0 , that is, the ratio between its largest
and smallest eigenvalues.
Remark 4.6. First note that this definition is only meaningful when the FIM at p0
is full rank. In this case, the condition number of the FIM at p0 corresponds to
the aspect ratio of the level curves of dFIM,p0 , which is one way to quantify how far
these level curves are from Euclidean spheres. Thus, using the condition number
of the FIM as a quantification of sloppiness implies that the reference metric on P
is the Euclidean metric.
The FIM possesses attractive statistical properties. Suppose (M, φ, ψ, d2 ) is
locally identifiable and that maximum likelihood estimates exist generically, that
is, for almost all z, there are parameters minimizing the negative log-likelihood:
p̂(z) = minp∈P (− log ψ(p, z)). Let z ∈ Z be a generic data point and let p̂(z) be
the unique maximum likelihood estimate. Suppose that the “true” parameter is
p0 , that is, z is a corruption of the model prediction φ(p0 ). When the FIM at p0 is
invertible, the Cramer-Rao inequality [9, Section 7.3] implies that

(9)
[ ∇2p d(p, p0 ) |p=p0 ]−1  Covp0 p̂(z),
where
Z
Covp0 p̂(z) :=
p̂(z)p̂T (z)ψ(p0 , z) dz
Z

Z
−

Z

T

 Z
p̂(z)ψ(p0 , z) dz

p̂(z)ψ(p0 , z) dz
Z

is the covariance of the maximum likelihood estimate with respect to measurement
noise, and A  B if and only if B − A is positive semi-definite. This inequality
provides an explicit link between the uncertainty associated with parameter estimation and the geometry of the negative log likelihood. Meanwhile, the sensitivity
of φ is related to the uncertainty associated with parameter estimation via (9).
The asymptotic normality of the maximum likelihood estimates implies that the
Cramer-Rao inequality (9) tends to equality as K tends to infinity [9, Section 10.7].
Formally,

(10)
lim [ ∇2p d(p, p0 ) |p=p0 ]−1 = Covp0 p̂(z).
K→∞

The list of regularity conditions required for (9) and (10) to hold are provided in
[9, Section 7.3], and are easily satisfied in practice.
Remark 4.7. A sufficient condition for dFIM,p0 to be a good approximation for the
premetric d on a neighborhood of p0 is to have a very large number of replicates.
In practice, however, questions of cost and time mean that the number of replicates
is often very small. Accordingly, the sloppiness literature generally assumes the

THE GEOMETRY OF SLOPPINESS

19

number of experiments is one (K = 1), though the effect of increasing experimental
replicates in mitigating sloppiness has been explored in [4].
Example 4.8 (Fitting points to a line). We revisit once more the model first considered in Example 2.5. We consider the model prediction map obtained by evaluating
at timepoints t1 = 0 and t2 = 1 as in Example 3.10. We assume that we are in
the presence of additive Gaussian error with covariance matrix Σ = I2 equal to
the identity matrix as discussed in Examples 4.1 and 4.3. As in Example 4.1 the
premetric d is induced by half the square of the Euclidean distance on the data
space R2 . We can explicitly determine d:

1
(a0 − a00 )2 + (a0 + a1 − a00 − a01 )2
2

1
=
2(a0 − a00 )2 + 2(a0 − a00 )(a1 − a01 ) + (a1 − a01 )2
2 
 


1
2 1
a0 − a00
a0 − a00
=
,
.
a1 − a01
1 1
a1 − a01
2

d((a0 , a1 ), (a00 , a01 )) =

We see that d itself is a weighted sum of squares given by a positive definite matrix,
and so d is a metric. As the positive definite matrix giving this sum of squares is
constant throughout parameter space, it follows that the sloppiness of the model
is also constant throughout parameter space. Note that the same phenomenon
would happen for any model such that the model manifold is given by an injective
linear map (see Proposition 4.9 below). In particular, the same situation would
arise when considering the problem of fitting points to any polynomial curve, as
the corresponding model prediction map will be linear.
We next compute the FIM. The map d(·, (b0 , b1 )) is given by
d((a0 , a1 ), (b0 , b1 )) =


1
(b0 − a0 )2 + (b0 + b1 − a0 − a1 )2
2

and so its Hessian, that is, the FIM is
1
2

∂2
d((a0 , a1 ), (b0 , b1 ))
∂a20
∂2
∂a1 ∂a0 d((a0 , a1 ), (b0 , b1 ))

∂2
∂a0 ∂a1 d((a0 , a1 ), (b0 , b1 ))
∂2
d((a0 , a1 ), (b0 , b1 ))
∂a20

!


=

2
1


1
1

We conclude that in this case, the pseudometric dFIM,(a0 ,a1 ) coincides with d on the
entire parameter space, which we will see in Proposition 4.9 is a consequence of the
linearity of the model prediction map.
/
Proposition 4.9. Let (M, φ, N (φ(p), Σ), d2 ) be a mathematical model with parameter space P ⊆ Rr , a choice of model prediction map, additive Gaussian noise with
covariance matrix Σ, and the Euclidean metric as a reference metric. If the model
prediction map φ : P → RN is linear, then dFIM,p0 = d for all p0 ∈ P .
Proof. Our assumption that φ is linear implies that there is a N × r matrix A with
real entries such that φ(p) = Ap. By the discussion in Example 4.1, we have
E
KD
(Ap0 − Ap), Σ−1 (Ap0 − Ap)
d(p0 , p) =
2
E
KD
=
(A(p0 − p)), Σ−1 (A(p0 − p))
2
E
KD 0
(p − p), (AT Σ−1 A)(p0 − p) .
=
2

20

DUFRESNE, HARRINGTON, AND RAMAN

On the other hand the FIM is given by

E
NK
K
KD
2
2
∇p d(p, p0 ) = ∇p −
(Ap0 − Ap), Σ(Ap0 − Ap)
log(2π) +
log(|Σ|) +
2
2
2
D
E
K
= ∇2p (Ap0 − Ap), Σ−1 (Ap0 − Ap)
2
E
K 2 D
= ∇p (Ap), Σ−1 (Ap)
2
K T −1
= A Σ A,
2
completing the proof.

Example 4.10 (Linear parameter-varying model). We consider a standard model
arising in control theory, which falls under the case of real analytic time dependent
models. Specifically, we consider models of the form
ẋ

=A(p)x,

y

=Cx,

x(0)=x0 ,
where A(p) is a m × m matrix with polynomial dependence on the parameter p ∈
Rr−1 , C is a known fixed n×m matrix with real coefficients, and y is the measurable
output. Note that y depends on the initial condition x0 , which we will consider as
an extension of parameter space. We assume further that A(p) is Hurwitz for all
p considered. We denote by y(t, (p, x0 )) the output of the system at time t, given
the parameter (p, x0 ). If we measured the system at a finite number of time-points,
assuming Gaussian noise-corruption, then the distance function d((p, x0 ), (p0 , x00 ))
would be the Euclidean distance between the model predictions at the chosen set
of timepoints.
For any pair of parameters (p, x0 ) and (p0 , x00 ), the following integral can be
explicitly computed and is a rational function of (p, x0 ) and (p0 , x00 ) (cf [39, Theorem
1], which assumes that A(p) is linear in the parameters, but whose proof holds more
generally):
Z ∞
d∞ ((p, x0 ), (p0 , x00 )) :=
ky(t, (p, x0 )) − y(t, (p0 , x00 ))k22 dt.
0

0

, x00 ))

Note that d∞ ((p, x0 ), (p
is equal to the L2 norm of the function y(t, (p, x0 )) −
0
0
y(t, (p , x0 )), and so d∞ ((p, x0 ), (p0 , x00 )) = 0 if and only if y(t, (p, x0 )) = y(t, (p0 , x00 )
for almost all t. As y is real-analytic, it then follows that y(t, (p, x0 )) = y(t, (p0 , x00 )
for all t. Therefore d∞ ((p, x0 ), (p0 , x00 )) = 0 if and only if (p0 , x00 ) ∼M,z (p, x0 ), and so
the equivalence class of (p, x0 ) is given by the zeros (p0 , x00 ) of the rational function
d∞ ((p, x0 ), (p0 , x00 )).
/
4.2. Multiscale sloppiness. We now present a quantification of sloppiness that
holds for non-Euclidean reference metric and is better suited to the presence of
noninfinitesimal noise. In this section, we sometimes make the assumption that
for generic p0 ∈ P , there is a neighborhood of p0 where the reference metric dP is
strongly equivalent to the Euclidean metric inherited by P as a subset of Rr . The
BHV metric [6] mentioned at the beginning of the section satisfies this property.
In Section 4.1 we saw how the FIM approximates the premetric d in the limit of
decreasing magnitude of parameter perturbation, which is realizable in the limit of
increasing experimental replicates or sample size. In a practical context, however
the limit of increasing replicates may not be valid. Indeed examples are provided in
[23] and [26] of models for which the uncertainty of parameter estimation is poorly
approximated by the FIM. Even when the approximation is valid, numerical errors

THE GEOMETRY OF SLOPPINESS

21

in sloppiness quantification are often significant, due to the ill-conditioning of the
FIM [56]. We describe a second approach called multiscale sloppiness introduced in
[40] for models given by ODE systems with time series data under the assumption
of additive Gaussian noise and with the standard Euclidean metric as a reference
metric. We extend this quantification of sloppiness to a more general setting.
Definition 4.11 (Multiscale Sloppiness). Consider a model (M, φ, ψ, dP ) with a
choice of model prediction map φ, a specific assumption on measurement noise, and
a choice of reference metric on P . We define the δ-sloppiness at p0 to be
Sp0 (δ) :=

supp∈P {d(p, p0 ) | dP (p, p0 ) = δ}
inf p∈P {d(p, p0 ) | dP (p, p0 ) = δ}

If dP is strongly equivalent to the Euclidean metric on a neighborhood of p0 ,
then for δ sufficiently small, the (non-unique) maximally and minimally disruptive
parameters at length scale δ at the point p0 ∈ P are the elements of the sets
(11)

Dpmax
(δ) = arg max d(p, p0 ) : dP (p, p0 ) = δ
0

(12)

Dpmin
(δ)
0

p∈P

= arg min d(p, p0 ) : dP (p, p0 ) = δ,
p∈P

respectively. In this case, the δ-sloppiness at p0 is
(13)

Sp0 (δ) =

d(pmax
p0 (δ), p0 )
,
d(pmin
p0 (δ), p0 )

max
min
min
where pmax
p0 (δ) ∈ Dp0 (δ) and d(pp0 (δ) ∈ Dp0 (δ).

Note that since the set {d(p, p0 ) | p ∈ P and dP (p, p0 ) = δ} is a closed set of
real numbers with a lower bound (zero), the infimum is actually a minimum, hence
(δ) is always well-defined.
Dpmin
0
Remark 4.12. Computation of δ-sloppiness would seem to require the solution of
a (possibly nonlinear, nonconvex) optimization program for each δ > 0. However,
assuming the reference metric on parameter space is the Euclidean distance and
min
that we are in the presence of additive Gaussian noise, finding pmin
p0 (δ) ∈ Dp0 (δ)
for continuous ranges of δ can be formulated as the solution of an optimal control
problem relying on solving a Hamiltonian dH/dp = 0 as described in [40, Section
5]. With this method, computation of δ-sloppiness is possible for large, nonlinear
systems of ODE. Note that this formulation as an optimal control problem does not
fundamentally rely on the assumption of a Euclidean metric on parameter space,
and so the principle likely applies to more general classes of metric.
If we choose the usual Euclidean distance as the reference metric on parameter
space, then as the length-scale δ goes to zero, infinitesimal and multiscale sloppiness
coincide:

λmax ∇2p d(p, p0 ) |p=p0

,
lim Sp0 (δ) = min
δ→0
λ
∇2p d(p, p0 ) |p=p0
where λmax and λmin denote the maximal and minimal eigenvalues of their argument. Indeed, the Taylor expansion (7) implies that as p approaches p0 , d(p, p0 )
approaches dFIM,p0 (p, p0 ), and so the level sets {p ∈ P | d(p, p0 ) = δ} tend to the
level sets {p ∈ P | dFIM,p0 (p, p0 ) = δ} as δ goes to zero.
Multiscale sloppiness, or more precisely the denominator of Sp0 (δ), is closely
related to structural identifiability:

22

DUFRESNE, HARRINGTON, AND RAMAN

Theorem 4.13. Let (M, φ, ψ, dP ) be a mathematical model with a choice of model
prediction map, a specific assumption on measurement noise, and a choice of reference metric dP , which we assume is strongly equivalent to the Euclidean metric. The equivalence class [p0 ]∼M,φ of the parameter p0 has size one if and only if
inf p∈P {d(p, p0 ) | dP (p, p0 ) = δ} > 0 for all δ > 0.
Proof. Suppose that the equivalence class [p0 ] of p0 has size one, then for any
other parameter p, we will have d(p, p0 ) > 0. In particular, this will hold for
min
pmin
p0 (δ) ∈ Dp0 (δ), for any δ > 0. Hence inf p∈P {d(p, p0 ) | dP (p, p0 ) = δ} =
min
d(pp0 (δ), p0 ) > 0.
Suppose on the other hand that inf p∈P {d(p, p0 ) | dP (p, p0 ) = δ} > 0 for all δ > 0
and suppose, for a contradiction that p ∈ [p0 ] is distinct from p0 . Set δ 0 := dP (p, p0 ).
As p 6= p0 , we have δ 0 > 0 and
0 = d(p, p0 ) ≥ inf {d(p, p0 ) | dP (p, p0 ) = δ 0 },
p∈P

which is a contradiction, since inf p∈P {d(p, p0 ) | dP (p, p0 ) = δ 0 } > 0.



Example 4.14 (Sum of exponentials). We highlight that sloppiness is a local property: it depends on the point in parameter space and the precise choice of timepoints. In this spirit, let us revisit Example 2.6, again adding Gaussian measurement noise with identity covariance and taking the model prediction map to be
evaluating at timepoints {1/3, 1, 3}. We are in the situation considered in Example
4.1 and so d is again half the squared Euclidean distance between model predictions:
d ( (a, b), (a0 , b0 ) ) =

1
kφ(a, b) − φ(a0 , b0 )k22 ,
2

where
φ(a, b) = (e− /3 + e− /3 , e−a + e−b , e−3a + e−3b ).
a

b

The Jacobian of the model prediction map at (a0 , b0 ) is therefore given as
 1 −a0/3

−3e
−e−a0 −3e−3a0
(∇a,b φ(a, b))|(a,b)=(a0 ,b0 ) =
− 13 e−b0/3 −e−b0 −3e−3b0
The FIM at (a0 , b0 ), in this case, will be given as


1 −2a0/3
1 −(a0 +b0 )/3
+ e−2a0 + 9e−6a0
+ e−(a0 +b0 ) + 9e−3(a0 +b0 )
9e
9e
1 −2b0/3
1 −(a0 +b0 )/3
+ e−(a0 +b0 ) + 9e−3(a0 +b0 )
+ e−2b0 + 9e−6b0
9e
9e
We compute infinitesimal sloppiness and δ-sloppiness of (M, φ) at p0 = (a, b) =
(4, 1/8) in Figure 4: the difference between these notions of sloppiness becomes
clear.
Figure 5 illustrates how the change of model prediction map, in this case different choices of timepoints, changes the premetric d. This suggests that sloppiness
should be taken into consideration when designing an experiment: some choices
of timepoints will allow for better quality parameter estimation. Figure 6, on the
other hand, illustrates how the premetric d changes in parameter space. In particular, these two figures illustrate that unlike identifiability, sloppiness is not a global
property of a model.
/
4.3. Sloppiness and practical identifiability. Determining the practical identifiability of a model corresponds to asking whether one can arrive to some estimate
of the parameter from noisy data, that is, whether based on an assumption on measurement noise, noisy data constrains the parameter value to a bounded region of
parameter space. Part of the literature uses the FIM in the manner of infinitesimal
sloppiness to define practical identifiability(see for example [55, 11]), but we will

THE GEOMETRY OF SLOPPINESS

23

Figure 4. Infinitesimal sloppiness vs δ-sloppiness of sum of exponential
(Example 4.14). For a given parameter p0 = (a = 4, b = 1/8) and time
points {1/3, 1, 3}, level sets of d are drawn (colors). The vector field
consisting of the eigenvector corresponding to the largest eigenvalue of
the FIM is plotted across the grid. We compare the flow of this vector
field initialised at p0 (gray curve), with the most delta-sloppy parameters
with respect to p0 over a range of δ (orange curve).

8

8

8

7

7

7

6

6

6

5

5

5

4

4

4

3

3

3

2

2

2

1

1

1

0

0
0

2

4

6

8

0
0

2

4

6

8

0

2

4

6

8

Figure 5. Sloppiness for different choices of model prediction map
for the sum of exponentials (Example 4.14).
For a given parameter
p
p0 = (4, 1/2), we draw the level curves of d(·, (4, 1/2)) for timepoints
{1/3, 1, 3} on the left, for timepoints {1/9, 1/3} in the center, and for
timepoints {1, 3} on the right are shown. Taking the square root changes
the spacing of the level curves, but not on their shape.

see in Example 4.19 that this method of evaluating practical identifiability can lead
to problems. We thus favor an approach more in line with Raue et al [42].
Practical identifiability depends on the method used for parameter estimation.
We focus on practical identifiability for maximum likelihood estimation, one of
the most widely used methods for parameter estimation (see, for example [31]).
Accordingly, in the remaining of this section, we consider models (M, φ, ψ, dP )
with a choice of model prediction map, a specific assumption of the probability
distribution of measurement noise and a choice of reference metric on P such that
maximum likelihood estimates exist for generic data.
For the noisy data point z0 ∈ Z, supposing the existence of a unique maximum
likelihood estimate p̂(z0 ) (i.e. supposing the model is generically identifiable, see
Proposition 4.15 below), we define an -confidence region U (z0 ) as follows:
U (z0 ) = {p ∈ P | − log ψ(p, z0 ) < }.

24

DUFRESNE, HARRINGTON, AND RAMAN

8

8

8

7

7

7

6

6

6

5

5

5

4

4

4

3

3

3

2

2

2

1

1

1

0

0

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

0

0

1

2

3

4

5

6

7

8

Figure 6. Sloppiness at different parameters given a choice of model
prediction map for the sum of exponentials (Example 4.14). With the
model prediction
map given by timepoints
p
p {1/3, 1, 3}, we draw the level
curves
of
d(·,
(4,
1/2))
on
the
left,
of
d(·, (3, 3)) in the center, and of
p
d(·, (6, 2))on the right are shown. Taking the square root changes the
spacing of the level curves, but not on their shape.

The -confidence region therefore denotes the set of parameters that fit the data at
least as well as some cutoff quality of fit, predicated on . The set U (z0 ) is often
known as a Likelihood-based confidence region [55, 9], and is intimately connected
with the Likelihood Ratio Test: Suppose we had a null hypothesis H0 that data z0
was generated (modulo noise) through a parameter p0 , and we wished to test the
alternative hypothesis H1 that z0 was generated through some other parameter.
By definition, a Likelihood Ratio test would reject the null hypothesis when
Λ(p0 , z0 ) :=

ψ(p0 , z0 )
≤ k∗ ,
ψ(p̂(z0 ), z0 )

where k ∗ is a critical value, with the significance level α equal to the probability
Pr(Λ(z0 ) ≤ k ∗ |H0 ) of rejecting the null hypothesis when it is in fact true. The set
of parameters such that the nul hypothesis is not rejected at significance level α is
{p0 ∈ P | logψ(p0 , z0 ) < − log −ψ(p̂(z0 ), z0 ) − log k ∗ },

that is, U (z0 ), where  = − log −ψ(p̂(z0 ), z0 ) − log k ∗ .

Proposition 4.15 (closely related to [10, Theorem 2]). Let (M, φ, ψ) be a mathematical model with a model prediction map, and a specific assumption on measurement noise. Suppose that maximum likelihood estimates exist for generic data. If φ
and ψ are real-analytic, then for almost all z0 ∈ Z, the set of maximum likelihood
estimates p̂(z), consists of exactly one equivalence class of ∼M,φ .
Proof. Let z0 ∈ Z be a generic data point. Solving the likelihood equation corresponds to finding the model prediction “closest” to the noisy data, as measured via
the negative log-likelihood. We can assume without loss of generality that there
is a unique solution to the likelihood equations. Indeed, under our assumptions,
the set of data points where the closest model prediction is not unique will be contained in the zero set of analytic functions. Thus, the set of maximum likelihood
estimates will consist of a single equivalence class. We can further assume that this
equivalence class has generic size.

Remark 4.16. The ML degree [24], where the acronym “ML” stand for maximum
likelihood, is defined as the number of complex solutions to the likelihood equations
(for generic data). The ML degree is an upper bound for the number of solutions
for the maximum likelihood equation, in particular it is an upper bound on the size
of the equivalence classes when maximum likelihood estimates exist.

THE GEOMETRY OF SLOPPINESS

25

Even if for generically identifiable models the maximum likelihood estimate is
unique with probability one, the parameter may not be identifiable in practice,
meaning that noisy data does not constrain the parameter value to a bounded region
of parameter space for a significant portion of the data space. More precisely, we
refine the definition of Raue et al [42]:
Definition 4.17 (Practical identifiability). Let (M, φ, ψ, dP ) be a mathematical
model with a model prediction map, a specific assumption on measurement noise
and a choice of reference metric dP on P . Suppose that maximum likelihood estimates exist for generic data. Then (M, φ, ψ, dP ) is practically identifiable at significance level α if and only if for generic z0 ∈ Z, there is a unique maximum likelihood
estimate and the confidence region U (z0 ) is bounded with respect to the reference
metric dP , where  satisfies


p0 ∈ U (z0 ) ⇔ Pr − log ψ(p0 , ẑ) <  ẑ ∈ Z is a corruption of φ(p̂(z0 ) = 1 − α.
The model M is practically unidentifiable at significance level α if and only if there
is a positive measure subset Z 0 ⊂ Z such that for z0 ∈ Z 0 , the confidence interval
U (z0 ) is unbounded with respect to the reference metric dP on P .
A model is practically identifiable at significance level α if generic data imposes
that the parameter estimate belongs to a bounded region of parameter space, but
this confidence region could be very large. Hence practical identifiability in this
sense may not necessarily be completely satisfactory to the practitioner. One can
further quantify practical identifiability to take into account the size of confidence
regions, see for example [40].
Sloppiness and practical identifiability are complementary concepts. Practically
identifiable models can be very sloppy, for example if the estimation of one component of the parameter is much more precise than that of another, see example
below.
Example 4.18 (Practically identifiable, but sloppy). Models with linear model prediction maps, Euclidean parameter space and standard additive Gaussian noise are
always practically identifiable according to our definition, but these models can be
arbitrarily sloppy.
We consider a model with 2-dimensional Euclidean parameter space and a linear
model prediction map φ given by (a, b) 7→ (10N a, b). We assume further that the
measurement noise is Gaussian with identity covariance matrix. By Proposition
4.9, d = dF IM,p0 for any p0 and at any scale, the level curves of d are ellipses with
aspect ratio 10N .
Our assumption of additive Gaussian noise implies that for any z0 , for each  > 0,
the confidence interval U (z0 ) is an oval whose boundary ellipse is the level set of d
centered at the maximum likelihood estimate p̂(z0 ). Thus the confidence intervals
U (z0 ) are bounded for any  > 0, and so the model is practically identifiable.
/
In the following, we give an example of a model that is almost everywhere not
sloppy at the infinitesimal scale, but is not practically identifiable. This model,
however, exhibits some sloppiness at the non-infinitesimal scale. We see that the
boundedness of level curves of d almost every where does not imply the boundedness
of confidence intervals almost everywhere.
Example 4.19 (Not sloppy at the infinitesimal scale, but not practically identifiable).
Consider the mathematical model (M, φ, N (φ(p), I2 ), d2 ) given by
φ :[1/2, ∞) × R→R2


b
a
,− 2
,
(a, b)
7→
a2 + b2
a + b2

26

DUFRESNE, HARRINGTON, AND RAMAN

additive Gaussian noise with identity covariance matrix, and parameter space P =
[1/2, ∞) × R equipped with the usual Euclidean metric.
The model prediction map φ is a conformal mapping that maps the closed half
plane [1/2, ∞) × R to the closed disc of radius 1 centered at (1, 0) minus the origin.
Since it is a conformal mapping, it preserves angles, and so infinitesimal circles are
sent to infinitesimal circles. Under our assumptions on measurement noise and with
the standard Euclidean metric as a reference metric on P , the model is not sloppy
at all at the infinitesimal scale at parameters belonging to the open half plane
(1/2, ∞) × R, but becomes increasingly sloppy at larger and larger scale, especially
away from the parameter (1/2, 0). The injectivity of the map φ on P = [1/2, ∞)×R
implies that the model (M, φ) is globally identifiable.
Our assumption on measurement noise implies that the maximum likelihood
estimate is the parameter whose image is closest to the data point z0 , it will exist
for any data point outside the closed half line (−∞, 0] × {0}. The confidence region
U (z0 ) is then the preimage of the Euclidean open disc of radius  centered at
z0 . Whenever the closure of this open disc contains the origin, the corresponding
confidence region will be unbounded.
/
The final example illustrates that the uniqueness of the maximum likelihood
estimate is independent from the boundedness of the confidence regions:
Example 4.20 (Bounded confidence regions but not practically identifiable). Consider the mathematical model (M, φ, N (φ(p), I2 ), d2 ) with model prediction map
φ :[1/2, ∞) × R→R
(a, b)

7→a2 + b2 ,

additive Gaussian noise with identity covariance matrix, and parameter space P =
[1/2, ∞) × R equipped with the usual Euclidean metric. The equivalence class of
the model-data equivalence relation are the concentric circles {(a, b) | a2 + b2 = r}
for r ≥ 0, and so the model is generically non-identifiable. By Proposition 4.15,
the set of maximum likelihood estimates for a generic (a0 , b0 ) ∈ R is also a circle
centered at the origin and the model is practically non-identifiable on any open
neighborhood of (a0 , b0 ). On the other hand, as the measurement noise is assumed
to be Gaussian and the equivalence classes of ∼ M,φ are bounded, any confidence
region will be bounded as well. Indeed, the confidence region U ((a0 , b0 )) will be
either an open disk {(a, b) ∈ R2 | a2 + b2 < a10 + b20 + }, when  > a10 + b20 , or an
/
open ring {(a, b) ∈ R2 | a10 + b20 −  < a2 + b2 < a10 + b20 + }, otherwise.
5. Future of sloppiness
There are a number of interesting future directions for the theory and application of sloppiness. While we explained sloppiness via identifiability, this is only
the beginning. An important next step is understanding sloppiness in the context
of existing inference and uncertainty quantification theory. In terms of applications, there are some models where the reference metric on parameter space is nonEuclidean and we believe the computation of multiscale sloppiness can be adapted.
While beyond the expertise of the authors, we would be excited to learn how the
presented geometry of sloppiness extends to stochastic differential equations.
We highlighted how sloppiness is a local property, dependent on the parameter
and timepoints of experiment. This dependence is reflected in model selection
studies where a different model is selected depending on the choice of timepoints [47]
or experimental stimulus dose [20]. We believe quantifying the shape of δ-sloppiness
in relation to identifiability will have direct impact on parameter estimation.

THE GEOMETRY OF SLOPPINESS

27

6. Acknowledgements
HAH gratefully acknowledges the late Jaroslav Stark for posing this problem.
The authors thank Carlos Améndola, Murad Banaji, Mariano Beguerisse Dı́az, Sam
Cohen, Ian Dryden, Paul Kirk, Terry Lyons, Chris Meyers, Jim Sethna, Eduardo
Sontag, Bernd Sturmfels, and Jared Tanner for fruitful discussions. Additionally,
we thank the anonymous referees for their helpful comments. This paper arises
from research done while ED was a postdoctoral research assistant at the Mathematical Institute in Oxford funded by the John Fell Oxford University Press (OUP)
Research Fund, and DVR was supported by the EPSRC Systems Biology Doctoral
Training Center. ED is now supported by an Anne McLaren Fellowship from the
University of Nottingham. HAH and DVR began discussions at the 2014 Workshop
on Model Identification funded by KAUST KUK-C1-013-04. HAH was supported
by EPSRC Fellowship EP/K041096/1 and now a Royal Society University Research
Fellowship.
References
[1] Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry, volume 191 of
Translations of Mathematical Monographs. American Mathematical Society, Providence, RI;
Oxford University Press, Oxford, 2000. Translated from the 1993 Japanese original by Daishi
Harada.
[2] Carlos Améndola, Jean-Charles Faugère, and Bernd Sturmfels. Moment Varieties of Gaussian
Mixtures. J. Algebr. Stat., 7(1):14–28, 2016. URL: http://dx.doi.org/10.18409/jas.v7i1.
42.
[3] Milena Anguelova. Observability and identifiability of nonlinear systems with applications
in biology. PhD thesis, Chalmers University of Technology, 2007. URL: http://www.gu.se/
english/research/publication?publicationId=57985.
[4] Joshua F. Apgar, David K. Witmer, Forest M. White, and Bruce Tidor. Sloppy models, parameter uncertainty, and the role of experimental design. Molecular Biosystems, 6(10):1890–
1900, oct 2010. URL: http://dx.doi.org/10.1039/b918098b.
[5] Giuseppina Bellu, Maria Pia Saccomani, Stefania Audoly, and Leontina DAngiò. Daisy: A
new software tool to test global identifiability of biological and physiological systems. Computer Methods and Programs in Biomedicine, 88(1):52–61, 2007. URL: http://dx.doi.org/
10.1016/j.cmpb.2007.07.002.
[6] Louis J. Billera, Susan P. Holmes, and Karen Vogtmann. Geometry of the space of phylogenetic trees. Advances in Applied Mathematics, 27(4):733–767, 2001. URL: http://dx.doi.
org/10.1006/aama.2001.0759.
[7] François Boulier. Differential elimination and biological modelling. In Gröbner bases in
symbolic analysis, volume 2 of Radon Series on Computational and Applied Mathematics,
pages 109–137. Walter de Gruyter, Berlin, 2007. URL: https://hal.archives-ouvertes.fr/
hal-00139364/document.
[8] Kevin S. Brown and James P. Sethna. Statistical mechanical approaches to models with many
poorly known parameters. Physical Review E, 68(2):021904, 2003. URL: http://dx.doi.org/
10.1103/PhysRevE.68.021904.
[9] George Casella and Roger L. Berger. Statistical Inference. Duxbury, 2nd edition, 2002. URL: http://statistics.columbian.gwu.edu/sites/statistics.columbian.
gwu.edu/files/downloads/Syllabus6202-Spring2013-Li.pdf.
[10] E. A. Catchpole and B. J. T. Morgan. Detecting parameter redundancy. Biometrika,
84(1):187–196, 1997. URL: http://dx.doi.org/10.1093/biomet/84.1.187.
[11] Oana-Teodora Chis, Alejandro F. Villaverde, Julio R. Banga, and Eva Balsa-Canto.
On the relationship between sloppiness and identifiability. Mathematical Biosciences,
282(Complete):147–161, 2016. URL: http://dx.doi.org/10.1016/j.mbs.2016.10.009.
[12] Gilles Clermont and Sven Zenker. The inverse problem in mathematical biology. Mathematical
Biosciences, 260:11–15, 2015. URL: http://dx.doi.org/10.1016/j.mbs.2014.09.001.
[13] Claudio Cobelli and Joseph J. DiStefano III. Parameter and structural identifiability concepts
and ambiguities: a critical review and analysis. American Journal of Physiology, 239(1):R7–
R24, 1980. URL: http://ajpregu.physiology.org/content/ajpregu/239/1/R7.full.pdf.
[14] Thomas M. Cover and Joy A. Thomas. Elements of information theory. Wiley, 2012.
[15] Bryan C Daniels, Yan-Jiun Chen, James P Sethna, Ryan N Gutenkunst, and Christopher R Myers. Sloppiness, robustness, and evolvability in systems biology. Current Opinion

28

[16]
[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]
[26]

[27]

[28]
[29]
[30]

[31]
[32]

[33]

[34]

[35]

DUFRESNE, HARRINGTON, AND RAMAN

in Biotechnology, 19(4):389 – 395, 2008. URL: http://dx.doi.org/10.1016/j.copbio.2008.
06.008.
John Duchi. Derivations for linear algebra and optimization. 2007. URL: http://web.
stanford.edu/~jduchi/projects/general_notes.pdf.
Kamil Erguler and Michael P H Stumpf. Practical limits for reverse engineering of dynamical
systems: a statistical analysis of sensitivity and parameter inferability in systems biology
models. Molecular BioSystems, 7(5):1593–1602, May 2011. URL: http://dx.doi.org/10.
1039/C0MB00107D.
R. A. Fisher. Theory of statistical estimation. Mathematical Proceedings of the Cambridge Philosophical Society, 22(5):700?725, 1925. URL: http://dx.doi.org/10.1017/
S0305004100009580.
√
Matan Gavish and David L. Donoho. The optimal hard threshold for singular values is 4/ 3.
IEEE Trans. Inform. Theory, 60(8):5040–5053, 2014. URL: http://dx.doi.org/10.1109/
TIT.2014.2323359.
Elizabeth Gross, Brent Davis, Kenneth L. Ho, Daniel J. Bates, and Heather A. Harrington.
Numerical algebraic geometry for model selection and its application to the life sciences. J.
R. Soc. Interface, 13, 2016. URL: http://dx.doi.org/10.1098/rsif.2016.0256.
Ryan N. Gutenkunst, Jordan C. Atlas, Fergal P. Casey, Brian C. Daniels, Robert S. Kuczenski, Joshua J. Waterfall, Chris R. Myers, and James P. Sethna. Sloppycell, 2007. URL:
http://sloppycell.sourceforge.net.
Ryan N. Gutenkunst, Joshua J. Waterfall, Fergal P. Casey, Kevin S. Brown, Christopher R.
Myers, and James P. Sethna. Universally Sloppy Parameter Sensitivities in Systems Biology
Models. PLoS Computational Biology, 3(10):e189–1878, oct 2007. URL: http://dx.doi.org/
10.1371/journal.pcbi.0030189.
Keegan E. Hines, Thomas R. Middendorf, and Richard W. Aldrich. Determination of parameter identifiability in nonlinear biophysical models: A Bayesian approach. The Journal of
general physiology, 143(3):401–16, 2014. URL: http://dx.doi.org/10.1085/jgp.201311116.
Serkan Hoşten, Amit Khetan, and Bernd Sturmfels. Solving the likelihood equations. Foundations of Computational Mathematics, 5(4):389–407, 2005. URL: http://dx.doi.org/10.
1007/s10208-004-0156-8.
Joseph DiStefano III. Dynamic Systems Biology Modeling and Simulation, 1st Edition. Elsevier: Academic Press, Amsterdam, 2013.
M. Joshi, A. Seidel-Morgenstern, and A. Kremling. Exploiting the bootstrap method for quantifying parameter confidence intervals in dynamical systems. Metabolic Engineering, 8(5):447
– 455, 2006. URL: http://dx.doi.org/10.1016/j.ymben.2006.04.003.
Johan Karlsson, Milena Anguelova, and Mats Jirstrand. An efficient method for structural
identifiability analysis of large dynamic systems*. IFAC Proceedings Volumes, 45(16):941 –
946, 2012. URL: http://dx.doi.org/10.3182/20120711-3-BE-2027.00381.
Gregor Kemper. Separating invariants. Journal of Symbolic Computation, 44(9):1212–1222,
2009. URL: http://dx.doi.org/10.1016/j.jsc.2008.02.012.
S. Kullback and R.A. Leibler. On information and sufficiency. The annals of mathematical
statistics, 1951. URL: http://www.jstor.org/stable/2236703.
Daniel Lazard. Injectivity of real rational mappings: the case of a mixture of two Gaussian
laws. Math. Comput. Simulation, 67(1-2):67–84, 2004. URL: http://dx.doi.org/10.1016/
j.matcom.2004.05.009.
Lennart Ljung. System identification: theory for the user. Prentice Hall Information and
System Sciences Series. Prentice Hall, Inc., Englewood Cliffs, NJ, 1987.
Lennart Ljung and Torkel Glad. On global identifiability for arbitrary model parametrizations. Automatica, 30(2):265 – 276, 1994. URL: http://dx.doi.org/10.1016/0005-1098(94)
90029-9.
Brian K. Mannakee, Aaron P. Ragsdale, Mark K. Transtrum, and Ryan N. Gutenkunst.
Sloppiness and the geometry of parameter space. In Liesbet Geris and David GomezCabrero, editors, Uncertainty in Biology: A Computational Modeling Approach, pages 271–
299. Springer International Publishing, Cham, 2016. URL: http://dx.doi.org/10.1007/
978-3-319-21296-8_11.
Gabriella Margaria, Eva Riccomagno, Michael J. Chappell, and Henry P. Wynn. Differential
algebra methods for the study of the structural identifiability of rational function statespace models in the biosciences. Mathematical Biosciences, 174(1):1–26, 2001. URL: http:
//dx.doi.org/10.1016/S0025-5564(01)00079-7.
Nicolette Meshkat, Chris Anderson, and Joseph J Distefano. Finding identifiable parameter
combinations in nonlinear ODE models and the rational reparameterization of their inputoutput equations. Mathematical Biosciences, 233(1):19–31, sep 2011. URL: http://dx.doi.
org/10.1016/j.mbs.2011.06.001.

THE GEOMETRY OF SLOPPINESS

29

[36] Nicolette Meshkat, Marisa Eisenberg, and Joseph J. DiStefano, III. An algorithm for finding
globally identifiable parameter combinations of nonlinear ODE models using Gröbner bases.
Mathematical Biosciences, 222(2):61–72, 2009. URL: http://dx.doi.org/10.1016/j.mbs.
2009.08.010.
[37] Eva Balsa-Canto Oana-Teodora Chis, Julio R. Banga. Structural identifiability of systems
biology models: A critical comparison of methods. PLoS ONE, 6(11), 2011. URL: http:
//dx.doi.org/10.1371/journal.pone.0027755.
[38] François. Ollivier. Le Problème de l’Identifiabilité Structurelle Globale: Étude Théorique,
Méthodes Effectives et Bornes de Complexité. PhD thesis, École Polytéchnique, 1990. URL:
www.theses.fr/1990EPXX0009.
[39] Dhruva V. Raman, James Anderson, and Antonis Papachristodoulou. On the performance of
nonlinear dynamical systems under parameter perturbation. Automatica, 63:265 – 273, 2016.
URL: http://dx.doi.org/10.1016/j.automatica.2015.10.009.
[40] Dhruva V. Raman, James Anderson, and Antonis Papachristodoulou. Delineating parameter
unidentifiabilities in complex models. Phys. Rev. E, 95:032314, Mar 2017. URL: http://
link.aps.org/doi/10.1103/PhysRevE.95.032314.
[41] C Radhakrishna Rao. Information and accuracy attainable in the estimation of statistical
parameters. Bull. Calcutta Math. Soc, 37(3):81–91, 1945.
[42] A. Raue, C. Kreutz, T. Maiwald, J. Bachmann, M. Schilling, U. Klingmüller, and J. Timmer.
Structural and practical identifiability analysis of partially observed dynamical models by
exploiting the profile likelihood. Bioinformatics, 25(15):1923–9, 2009. URL: https://doi.
org/10.1093/bioinformatics/btp358.
[43] Andreas Raue, Johan Karlsson, Maria Pia Saccomani, Mats Jirstrand, and Jens Timmer. Comparison of approaches for parameter identifiability analysis of biological systems.
Bioinformatics (Oxford, England), 30(10):1440–8, 2014. URL: http://dx.doi.org/10.1093/
bioinformatics/btu006.
[44] Thomas J. Rothenberg. Identification in parametric models. Econometrica, 39(3):577–591,
1971. URL: http://www.jstor.org/stable/1913267.
[45] Alexandre Sedoglavic. A probabilistic algorithm to test local algebraic observability in polynomial time. In Proceedings of the International Symposium on Symbolic and Algebraic Computation. ACM, 2001. URL: http://dx.doi.org/10.1145/384101.384143.
[46] Jim Sethna. Fitting Polynomials: Where is sloppiness from? Webpage last modified June 11,
2008, http://www.lassp.cornell.edu/sethna/Sloppy/FittingPolynomials.html.
[47] Daniel Silk, Paul D W Kirk, Christopher P Barnes, Tina Toni, and Michael P H Stumpf.
Model selection in systems biology depends on experimental design. PLOS Computational
Biology, 10(6), 2014. URL: http://dx.doi.org/10.1371/journal.pcbi.1003650.
[48] E. D. Sontag. For differential equations with r parameters, 2r + 1 experiments are enough
for identification. Journal of Nonlinear Science, 12(6):553–583, 2002. URL: http://dx.doi.
org/10.1007/s00332-002-0506-0.
[49] Seth Sullivant. Algebraic Statistics. In preparation. URL: http://www4.ncsu.edu/~smsulli2/
Pubs/asbook.html.
[50] Christian Tönsing, Jens Timmer, and Clemens Kreutz. Cause and cure of sloppiness in
ordinary differential equation models. Phys. Rev. E, 90:023303, Aug 2014. URL: http:
//link.aps.org/doi/10.1103/PhysRevE.90.023303.
[51] Mark K. Transtrum, Benjamin B. Machta, Kevin S. Brown, Bryan C. Daniels, Christopher R. Myers, and James P. Sethna. Perspective: Sloppiness and emergent theories in
physics, biology, and beyond. The Journal of Chemical Physics, 143(1):010901, jul 2015.
URL: http://dx.doi.org/10.1063/1.4923066.
[52] Mark K. Transtrum, Benjamin B. Machta, and James P. Sethna. Why are nonlinear fits
to data so challenging? Physical Review Letters, 104(6):060201, feb 2010. URL: https:
//doi.org/10.1103/PhysRevLett.104.060201.
[53] Mark K. Transtrum, Benjamin B. Machta, and James P. Sethna. Geometry of nonlinear least squares with applications to sloppy models and optimization. Physical Review E,
83(3):036701, mar 2011. URL: https://doi.org/10.1103/PhysRevE.83.036701.
[54] Mark K. Transtrum and Peng Qiu. Model reduction by manifold boundaries. Physical Review
Letters, 113(9):098701, aug 2014. URL: https://doi.org/10.1103/PhysRevLett.113.098701.
[55] Sandor Vajda, Herschel Rabitz, Eric Walter, and Yves Lecourtier. Qualitative and quantitative identifiability analysis of nonlinear chemical kinetic models. Chemical Engineering Communications, 83(1):191–219, 1989. URL: http://dx.doi.org/10.1080/00986448908940662.
[56] Michele Vallisneri. Use and abuse of the fisher information matrix in the assessment of
gravitational-wave parameter-estimation prospects. Phys. Rev. D, 77:042001, Feb 2008. URL:
http://link.aps.org/doi/10.1103/PhysRevD.77.042001.

30

DUFRESNE, HARRINGTON, AND RAMAN

[57] Joshua J. Waterfall, Fergal P. Casey, Ryan N. Gutenkunst, Kevin S. Brown, Christopher R.
Myers, Piet W. Brouwer, Veit Elser, and James P. Sethna. The sloppy model universality class
and the Vandermonde matrix. Physical Review Letters, 97(15):150601, 2006. URL: https:
//doi.org/10.1103/PhysRevLett.97.150601.

Appendix A. Table summary of main examples
Example
Type
Parameter space P
Variable (x ∈ X)
Measurable output (y ∈ Y )
Perfect data
Noisy data

Fitting points to a line 2.5 2.133.104.8
Explicit time-dependent model
(a0 , a1 ) ∈ R2
x(t) ∈ R for t ∈ R≥0
x(t)
(x(t1 ), . . . , x(tN )) for some t1 < · · · < tN ∈ R≥0
(x(t1 ) + 1 , . . . , x(tN ) + N )

Example
Type
Parameter space P
Variable (x ∈ X)
Measurable output (y ∈ Y )
Perfect data
Noisy data

Two biased coins 2.1, 2.1
Finite discrete statistical model
[0, 1]3
Outcome of 1 instance of the experiment
Record of 1 instance of the experiment
Probability distribution for p ∈ P
Record of N instances of the experiment

Example
Type
Parameter space P
Variable (x ∈ X)
Measurable output (y ∈ Y )
Perfect data
Noisy data

Sum of exponentials 2.6, 2.11, 3.11, 4.14
Explicit time dependant model
(a, b) ∈ R2≥0
x(t) ∈ R>0 for t ∈ R≥0
x(t) ∈ R>0 for t ∈ R≥0
(x(t1 ), . . . , x(tN )) for some t1 < · · · < tN ∈ R≥0
(x(t1 ) + 1 , . . . , x(tN ) + N )

Example
Type
Parameter space P
Variable (x ∈ X)
Measurable output (y ∈ Y )
Perfect data

An ODE model with an exact solution 2.7
Polynomial ODE model
(p1 , p2 ) ∈ R2>0
(x1 (t), x2 (t)) ∈ R2>0 for t ∈ R≥0
(x1 (t), x2 (t))
(x1 (t1 ), x2 (t1 ), . . . ,1 (tN ), x2 (tN )) for some t1 <
· · · < tN ∈ R≥0
(x1 (t1 )+1,1 , x2 (t1 )+2,1 , . . . ,1 (tN )+1,N , x2 (tN )+
2,N )

Noisy data

THE GEOMETRY OF SLOPPINESS

Example
Type
Parameter space P
Variable (x ∈ X)
Measurable output (y ∈ Y )
Perfect data

Noisy data

Example
Type
Parameter space P
Variable (x ∈ X)
Measurable output (y ∈ Y )
Perfect data

Noisy data

Example
Type
Parameter space P
Variable (x ∈ X)
Measurable output (y ∈ Y )
Perfect data
Noisy data

31

A non-linear ODE model 2.15
Polynomial ODE model
(p1 , p2 , p3 , p4 , p5 ) ∈ P ⊆ R5
(x1 (t), x2 (t)) ∈ R2>0 for t ∈ R≥0
(x1 (t), x2 (t)) ∈ R2>0 for t ∈ R≥0
• (x1 (t1 ), x2 (t1 ), . . . ,1 (tN ), x2 (tN )) for some t1 <
· · · < tN ∈ R≥0
• An exhaustive summary or input-output equations
(x1 (t1 )+1,1 , x2 (t1 )+2,1 , . . . ,1 (tN )+1,N , x2 (tN )+
2,N )
Gaussian mixtures 3.9
Continuous parametric statistical model
(λ, µ, σ, ν, τ ) ∈ [0, 1] × R × R≥0 × R × R≥0
a characteristic x ∈ R≥0 of a mixed polulation
x ∈ R≥0
• Probability distribution of x for some
(λ, µ, σ, ν, τ )
• Value of the cdf (or the pdf) at general
x1 , . . . , x 1 1 ∈ R
• Value of the moment generating function at general t1 , . . . , t1 1 ∈ (−a, a)
• 11 generic moments (or the first 7)
• Measurements from a finite sample with some
measurement error
• Empirical distribution function from a finite sample (or its numerical derivative)
• 11 generic sample moments from a finite sample
(or the first 7)
Linear parameter-varying model 4.10
ODE model
p ∈ Rr−1
x(t) ∈ Rm for t ∈ R≥0
y = Cx(t) for t ∈ R≥0
(y(t1 ), . . . , y(tN )) for some t1 < · · · < tN ∈ R≥0
(y(t1 ) + 1 , . . . , y(tN ) + N )

School of Mathematical Sciences, University of Nottingham, University Park, Nottingham, NG7 2RD
E-mail address: emilie.dufresne@nottingham.ac.uk
Mathematical Institute, University of Oxford, Andrew Wiles Building, Radcliffe
Observatory Quarter, Woodstock Road, Oxford, OX2 6GG
E-mail address: harrington@maths.ox.ac.uk
Engineering Department, University of Cambridge, Trumpington Street, Cambridge,
CB2 1PZ
E-mail address: dhruva.raman@eng.cam.ac.uk

