1

Discrete Dynamic Causal Modeling and Its
Relationship with Directed Information

arXiv:1709.06134v1 [q-bio.NC] 18 Sep 2017

Zhe Wang, Yu Zheng, David C. Zhu, Jian Ren and Tongtong Li

Abstract—This paper explores the discrete Dynamic
Causal Modeling (DDCM) and its relationship with Directed Information (DI). We prove the conditional equivalence between DDCM and DI in characterizing the causal
relationship between two brain regions. The theoretical
results are demonstrated using fMRI data obtained under
both resting state and stimulus based state. Our numerical
analysis is consistent with that reported in previous study.
Index Terms—Causality Analysis, Dynamic Causal Modeling, Directed Information

I. I NTRODUCTION
Causality analysis aims to find the relationship between causes and effects. It provides insightful information on how brain regions interact with each other
during a cognitive task [1]. In fMRI based causality
analysis, Granger Causality (GC), Directed Information
(DI), and Dynamic Causal Modeling (DCM) are three
representative approaches. In this paper, we will revisit
the relationship among existing causality analysis frameworks, especially the relationship between DCM and DI.
Granger causality is the first practical causality analysis framework which was proposed by Granger in
1969 [2]. The basic idea is, if two signals X1 and X2
form a causal relationship, then, instead of using the past
values of X2 alone, the information contained in the
past values of X1 will help predict X2 . In practice, the
calculation of Granger Causality is based on the linear
prediction models.
In literature, there have been growing interests in applying GC to identify causal interactions in the brain [3]–
[6]. As a widely accepted technique, the validity and
computational simplicity of Granger Causality have been
appreciated. However, it has also been noticed that GC
relies heavily on the linear prediction method. When
there exist instantaneous and/or strong nonlinear interactions between two regions, GC analysis may lead to
invalid results [6], [7].
Directed Information is an information theoretic metric, which was first introduced by Massey when studying
Zhe Wang, Yu Zheng, Jian Ren and Tongtong Li are with the Department of Electrical and Computer Engineering; David C. Zhu is with
the Department of Radiology, Michigan State University, East Lansing,
MI, 48824, USA (e-mail: wangzh34@msu.edu, zhengy30@msu.edu,
zhuda@msu.edu, renjian@egr.msu.edu, tongli@egr.msu.edu)

communication channels with feedback [8]. It measures
the directed information flow from one time series X
to another time series Y , denoted as I(X → Y ). If
I(X → Y ) > I(Y → X), then we say X has
more influence on Y , or X is the causal side in the
connectivity.
DI can be used to characterize more general relationships, as it does not have any modeling constraints on
the sequences to be evaluated. In [7], it was pointed
out that GC analysis is effective in detecting linear or
nearly linear causal relationship, but may have difficulty
in capturing nonlinear causal relationships. On the other
hand, DI-based causality analysis is more effective in
capturing both linear and nonlinear causal relationships.
Moreover, in [9], it was shown that the Granger Causality
graphs of stochastic processes can be generated from the
DI framework, and the authors indicated that DI provides
an effective framework for connectivity inference.
Dynamic Causal Modeling, which characterizes the
interactions among a group of brain regions [10], was
proposed by Friston in 2003. It assumes that the invisible
neurostate, the (external) input U , and the observed
BOLD signal Y form a dynamic system that could be
described by a group of differential equations.
Compared with GC, DCM provides a more comprehensive characterization of the dynamic interactions
between multiple regions. In [11], Friston et al. pointed
out that GC and DCM were complementary to each
other: GC models the causal dependency among observed responses, while DCM models the causal interactions among the hidden neurostates. On the other hand,
in [12], Friston provided an example to show that DCM
and GC may generate different results given the same
dataset. The underlining argument is that: DCM takes
into account both the external input and the biological
variations of the hemodynamic response, which are not
involved in GC.
To this end, it can be seen that the relationships
between GC and DCM, and between GC and DI have
been investigated in literature. While GC is efficient in
detecting linear causal relationships, both DI and DCM
can be used to characterize more general causal relationships. A missing link here is: what is the relationship

2

between DCM and DI?
In the following sections, we aim to fill this missing
link, and explore the connection between DCM and DI.
Based on the discrete DCM (DDCM), we will show that
under certain conditions, DDCM and DI are equivalent
in characterizing the causal relationship between two
brain regions. This equivalence is validated using fMRI
data obtained under both resting state and stimulus based
state.

II. D ISCRETE DYNAMIC C AUSAL M ODELING
In continuous time DCM, the invisible neurostates
of d brain regions are denoted by a vector X =
[X1 , X2 , ..., Xd ]t , where each Xi , i = 1, 2, ..., d, represents the neurostate of the ith region. The basic idea
of DCM is that, the neurostate X, the external input
U , the connectivity matrices Ã and B̃ that describe the
connections among brain regions, the observed BOLD
signal Y and the independent noise can be formulated
as a complex dynamic system, characterized as:
Ẋ(t)

= ÃX(t) + B̃U (t) + Ω1 (t),

(1)

Y(t)

= Λ̃(X(t)) + Ω2 (t),

(2)

where Ω1 (t) and Ω2 (t) are the state noise and observation noise, and Λ̃ represents the mapping from the
neurostate X(t) to the observed BOLD signal Y(t).
It can be seen from equations (1) and (2) that the
continuous time DCM characterizes the dynamic neural system using two continuous-time equations. However, parameter estimation in continuous time equations
faces considerable challenges in practical applications.
To overcome this difficulty, there have been efforts
to simplify DCM to a more tractable form, such as
the switching linear dynamic model (SLDS) [13] and
multivariate dynamical model (MDS) [14]. In both approaches, the continuous time equations are discretized,
and the mapping between the neurostate and the BOLD
signal is approximated as an LTI system, characterized
using a convolution. That is, the discrete DCM (DDCM)
model can be obtained as:
X(k + 1)
Y(k)

=

AX(k) + BU (k) + Ω1 (k),
(3)
M
X
=
Λ(m)X(k − m) + Ω2 (k), (4)
m=0

where A is the connectivity matrix, {Λ(m), m =
0, 1, · · · M } denotes the convolution coefficients corresponding to the hemodynamic response, and Ω1 (k) and
Ω2 (k) denote the noise terms independent of the brain
state and the input.

Consider the case of two regions, region 1 and region
2, where equation (3) can be rewritten as:





X1 (k + 1)
A11 A12
X1 (k)
=
X2 (k + 1)
A21 A22
X2 (k)




B1
Ω11 (k)
+
U (k) +
. (5)
B2
Ω12 (k)
Similar to the continuous time DCM, coefficients
A12 and A21 actually measure the causal relationship
between region 1 and region 2. More specifically, if
|A21 | > |A12 |, then X1 is more likely to be the casual
side, and vice versa. The same analysis holds when
multiple brain regions are under investigations [13], [14].
III. T HE R ELATIONSHIP BETWEEN DDCM AND
D IRECTED I NFORMATION
In this section, we show that, under certain assumptions, DI and DDCM are equivalent in characterizing the
causal relationship between two brain regions.
Directed Information is a causality analysis framework
based on information theory. The directed information
from one time series Xn1 to another Xn2 is calculated
as [8]:
I(Xn1 → Xn2 )
n
X
[h(X2 (k)|Xk−1
) − h(X2 (k)|Xk−1
, Xk1 )], (6)
=
2
2
k=1

where Xki = [Xi (1), Xi (2), ..., Xi (k)], i = 1, 2, and h
denotes the differential entropy operator. If I(Xn1 → Xn2 )
is greater than I(Xn2 → Xn1 ), we say Xn1 has more
causal influence over Xn2 ; otherwise Xn2 has more causal
influence over Xn1 .
When deriving the relationship between DDCM and
DI, we impose the following assumptions to make the
problem more tractable: (i) The dynamic neural system
under investigation is a causal system, which means
for each brain region, the current value of the neurostate depends only on previous values of neurostates
of the region and its related regions. (ii) For each
region, both the neurostate and the background noise
are normally distributed, and the variances are the same
in related brain regions. More specifically, for each
k = 1, 2, · · · , n, the variances corresponding to the
neurostate and the background noise are σx2 and σ02 ,
respectively. (iii) The external input U is a constant. This
assumption is reasonable when the changing rate of the
external input is much slower than that of neurostates.
In the following analysis, let the uppercase letters (X,Y ,...) denote random variables, and the lowercase letters (x,y,...) the possible values they can
acquire. In particular, x1 (k) and x2 (k) denote the
possible values X1 (k) and X2 (k) can acquire, and

3

ω11 (k) and ω12 (k) denote the possible values Ω11 (k)
and Ω12 (k) can acquire. Given a time series Xn =
[X(1), X(2), ..., X(n)], n ∈ N , for any x(k), k =
1, 2, . . . n, P (x(k)) denotes the probability for X(k)
to take the value x(k), and P (x(k)|xk−1 ) the conditional probability that the current sample X(k) is
x(k), given that the previously observed sequence is
xk−1 = [x(1), x(2), ..., x(k − 1)].
Following equation (5), the conditional probability
P (x2 (k) | xk−1
, xk1 ) can be written as:
2
P (x2 (k) | xk−1
, xk1 )
2
= P (A21 x1 (k − 1) + A22 x2 (k − 1)
+ B2 U + ω12 (k − 1) | xk−1
, xk1 )
2
= P (A21 x1 (k − 1) + A22 x2 (k − 1)
+ B2 U + ω12 (k − 1) | xk−1
, xk−1
)
2
1
= P (ω12 (k − 1)).

(7)

This implies that, for each k = 1, 2, · · · , n, the conditional probability density function of the neurostate
X2 (k) given Xk−1
and Xk1 is Gaussian with variance
2
2
σ0 .
It is well known that given a Gaussian random variable Ξ with variance σξ2 , the corresponding differential
entropy h(Ξ) can be calculated as:
1
log 2πeσξ2 .
(8)
2
Therefore, based on equations (7) and (8), the differential
entropy corresponding to the neurostate X2 (k) given
Xk−1
and Xk1 can then be calculated as:
2
h(Ξ) =

h(X2 (k)|Xk−1
, Xk1 ) =
2

1
log 2πeσ02 .
2

(9)

Similarly, the conditional probability P (x2 (k) | xk−1
)
2
can be simplified as:
P (x2 (k) | xk−1
)
2

tion can then be obtained as:
I(Xn1 → Xn2 )
n
X
1
1
=
[ log 2πe(A221 σx2 + σ02 ) − log 2πeσ02 ]
2
2
k=1

n
σ2
log(1 + A221 x2 )
2
σ0
n
2
(12)
= log(1 + cA21 ),
2
where c = σx2 /σ02 is the ratio of the power of neural
activities and the noise power. Similarly, we can prove
that I(Xn2 → Xn1 ) = (n/2)log(1 + cA212 ).
Note that when c > 0, log(1+cx2 ) is a monotonically
increasing function. Based on the discussions above, we
can obtain the following proposition:
Proposition 1: If |A21 | > |A12 |, then I(Xn1 → Xn2 ) >
I(Xn2 → Xn1 ), that is, region 1 is more likely to be the
causal side; otherwise, we will have I(Xn2 → Xn1 ) >
I(Xn1 → Xn2 ), and region 2 is more likely to be the
causal side.
Proposition 1 is in accordance with the previous
analysis for DDCM: the causality analysis can be carried
out based on the absolute values of the coefficients of the
connectivity matrix. This means, for a causal dynamic
neural system with a constant external input, when the
neurostate and the background noise are normally distributed, DI and DDCM are equivalent in characterizing
the causal relationship between two brain regions.
On the other hand, in practice, we can only observe
the BOLD signal Yn rather than the neurostate Xn .
That is, given two brain regions, region 1 and region
2, the calculation of DI can only be carried out on the
observations Yn1 and Yn2 rather than the neurostates Xn1
and Xn2 . However, it can be shown that as long as the
hemodynamic system is invertible, DI calculated using
the estimated neurostates is equal to the DI calculated
using the observed signals. That is, DDCM and DI are
still equivalent in characterizing the causal relationship
between brain regions.
=

= P (A21 x1 (k − 1) + A22 x2 (k − 1) + B2 U
+ω12 (k − 1) | xk−1
)
2
= P (A21 x1 (k − 1) + ω12 (k − 1)).

IV. N UMERICAL A NALYSIS
(10)

As a result, the corresponding differential entropy will
be:
1
h(X2 (k)|Xk−1
) = log 2πe(A221 σx2 + σ02 ),
(11)
2
2
where σx2 is the variance of the neurostate, which is
assumed to have no significant changes among related
regions and within the observation frame.
Based on equations (7) to (11), the directed informa-

In this section, we briefly describe how to validate the
equivalence of DDCM and DI between two regions using
experimental fMRI data obtained under both resting state
and stimulus based state.
A. Data Acquisition
Fourteen right-handed healthy college students (7
males, 23.4 ± 4.2 years of age) from Michigan State
University volunteered to participate in this study. The
experiment was conducted on a 3T GE Signa HDx

4

MR scanner (GE Healthcare, Waukesha, WI) with an
8-channel head coil.
For each subject, fMRI datasets were collected on a
visual stimulation condition with a scene-object fMRI
paradigm and then on a resting-state condition. On
the visual stimulation fMRI condition, each volume of
images were acquired 192 times (8 min) while each subject was presented with 12 blocks of visual stimulation
after an initial 10 s “resting” period. In a predefined
randomized order, the scenery pictures were presented
in 6 blocks and the object pictures were presented in
other 6 blocks. In each block, 10 pictures were presented
continuously for 25 s (2.5 s for each picture), followed
with a 15 s baseline condition (a white screen with a
black fixation cross at the center). The subject needed to
press his/her right index finger once when the screen was
switched from the baseline to picture condition. Detailed
experiment setting and procedures of data processing can
be found in [7].
We test the robustness of our causality analysis techniques against some expected outcomes: under the stimulation fMRI paradigm, the primary visual cortex (V1)
and nearby regions are activated first, followed with
activation in the parahippocampal place area (PPA) for
higher level scene processing. Some but relatively small
activations in the left sensorimotor cortex (SMC) is also
expected following V1 activations. Under the restingstate condition, neuronal activity is not expected to occur
in a sequential manner among above regions.
The simulation result of V1 and PPA under both
resting and stimulus based states are shown in Figure
(1a) and (1b). It can be seen that under the resting state,
V1 does not exhibit a dominating influence over PPA.
However, under the stimulus based state, |A21 | is increased considerably compared to |A12 |. In other words,
V1 shows stronger influences over PPA as expected.
Figure (1c) and (1d) have shown a similar pattern for
the regions V1 and SMC. The result is consistent with
the expectations and our previous result using DI [7].

1.8

Connection Coefficients between V1 and PPA

1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0

1.2

|A12 |

|A21 |

(a) V1 and PPA under stimulus based state.
Connection Coefficients between V1 and PPA

1.0
0.8
0.6
0.4
0.2
0.0

|A12 |

|A21 |

(b) V1 and PPA under resting state.
2.0

Connection Coefficients between V1 and SMC

1.5

1.0

0.5

0.0

|A12 |

|A21 |

(c) V1 and SMC under stimulus based state.
Connection Coefficients between V1 and SMC
2.5

V. C ONCLUSIONS
2.0

This paper investigated the discrete time DCM
(DDCM) and its relationship with Directed Information
(DI) and Granger Causality (GC). Based on information
theory, we revealed the conditional equivalence between
DDCM and DI in characterizing the causal relationship
between two brain regions. The theoretical techniques
were demonstrated using fMRI data obtained under both
resting state and stimulus based state. Our numerical
analysis was consistent with that reported in previous
study.

1.5
1.0
0.5
0.0

|A12 |

|A21 |

(d) V1 and SMC under resting state.
Fig. 1: Estimations result of DDCM with the experimental fMRI data.

5

R EFERENCES
[1] A. Roebroeck et al., “Causal time series analysis of functional
magnetic resonance imaging data,” in JMLR: Workshop and
Conference Proceedings 12, 2011, pp. 65–94.
[2] C.Granger, “Investigating causal relations by econometric models
and cross-spectral methods,” Econometrica, vol. 37, no. 3, pp.
424–438, 1969.
[3] K. J. Friston, “Functional and effective connectivity: a review,”
Brain connectivity, vol. 1, no. 1, pp. 13–36, 2011.
[4] M. Hu and H. Liang, “A copula approach to assessing granger
causality,” NeuroImage, vol. 100, pp. 125–134, 2014.
[5] P. Liang, Z. Li, G. Deshpande, Z. Wang, X. Hu, and K. Li,
“Altered causal connectivity of resting state brain networks in
amnesic mci,” PloS one, vol. 9, no. 3, p. e88476, 2014.
[6] O. David et al., “Identifying neural drivers with functional mri:
An electrophysiological validation,” PLOS BIOLOGY, vol. 6, pp.
2683–2697, Dec 2008.
[7] Z. Wang, A. Alahmadi, D. Zhu, and T. Li, “Causality analysis of
fmri data based on the directed information theory framework,”
Biomedical Engineering, IEEE Transactions on, vol. PP, no. 99,
pp. 1–1, 2015.
[8] J. Massey, “Causality,feedback, and directed information,” in Int.
Symp. Inf. Theory Appl., Honolulu, HI, Nov. 1990, pp. 303–305.
[9] P.-O. Amblard and O. J. Michel, “On directed information theory
and Granger Causality graphs,” J Comput Neurosci, vol. 30, pp.
7–16, Feb 2010.
[10] K. Friston et al., “Dynamic causal modeling,” NeuroImage,
vol. 19, pp. 1273–1302, 2003.
[11] K. Friston, R. Moran, and A. K. Seth, “Analysing connectivity
with granger causality and dynamic causal modelling,” Current
opinion in neurobiology, vol. 23, no. 2, pp. 172–178, 2013.
[12] K. Friston, “Dynamic causal modeling and granger causality
comments on: The identification of interacting networks in the
brain using fMRI: Model selection, causality and deconvolution,”
Neuroimage, vol. 58, no. 2, pp. 303–305, 2011.
[13] J. F. Smith et al., “Identification and validation of effective
connectivity networks in functional magnetic resonance imaging
using switching linear dynamic systems,” Neuroimage, vol. 52,
pp. 1027–1040, 2010.
[14] S. Ryali et al., “Multivariate dynamical systems models for
estimating causal interactionsin fMRI,” NeuroImage, vol. 54, pp.
807–823, 2011.

