Quantized Innovations Bayesian Filtering
Chun-Chia HuangRobert R.Bitmead

arXiv:1704.02641v1 [] 9 Apr 2017

a

Department of Mechanical & Aerospace Engineering, University of California, San Diego, 9500 Gilman Drive, La Jolla CA,
92093-0411, USA.

Abstract
The paper provides simple formulas of Bayesian filtering for the exact recursive computation of state conditional probability density
functions given quantized innovations signal measurements of a linear stochastic system. This is a topic of current interest because the
innovations signal should be white and therefore efficient in its use of channel capacity and in the design and optimization of the quantizer.
Earlier approaches, which we reexamine and characterize here, have relied on assumptions concerning densities or approximations to yield
recursive solutions, which include the sign-of-innovations Kalman filter and a Particle filtering technique. Our approach uses the Kalman
filter innovations at the transmitter side and provides a point of comparison for the other methods, since it is based on the Bayesian filter.
Computational examples are provided.

1

Introduction

Efficient transmission of signals over a limited capacity digital channel requires coding and quantization. Here we examine the recently explored problem of linear state estimation based on quantized innovations signals to reconstruct
the conditional probability density function of the predicted
and filtered system state given these measurements. We draw
on [2,9,5] for algorithmic approaches to this problem, which
in turn are based on Bayesian filtering formulations. The
Bayesian filter is a recursive approach for the computation
of predicted and filtered state conditional probability density functions (pdfs) given a sequence of measurements. It
is an approach to nonlinear filtering. Our contribution is to
develop for linear stochastic systems a signal transmission
system based on the quantized Kalman filter innovations at
the transmitter and an associated explicit Bayesian filtering solution at the receiver. We also draw comparisons to
other works through the provision of a unifying framework
from which to view the development. The advantage of this
new approach is that it can benefit from the whiteness of
the Kalman filter innovations and its zero-mean and known
covariance properties to realize optimal coding and LloydMax optimal quantization [11]. We also postulate that it has
improved error recovery properties because of the reconvergent behavior of the Kalman filter.
An innovations signal, ιk , is formed by subtracting from a
measured signal, yk , its one-step-ahead prediction, ŷk|k−1 .
In signal processing, such calculations are performed in order to whiten a signal prior to transmission over a bandlimEmail addresses: chhuang@ucsd.edu (Chun-Chia Huang),
rrbitmead@ucsd.edu (Robert R.Bitmead).

Preprint submitted to Automatica

ited channel and thereby to achieve more efficient use of the
channel capacity. In the formal context of Kalman filtering
for linear Gaussian systems, the innovations formed from
the state prediction is statistically white; a property that we
shall apply. The prediction can equally be computed as a
reconstruction of a signal available at the receiver, which is
achieved at the transmitter by using the computed receiver
signal ῑk and control uk as the inputs to the predictor. Such a
scheme forms the basis of the ITU-T G.726 adaptive differential pulse-coded modulation speech codec standard. The
topic of investigation in this paper is the collection of methods using quantized innovations signals for the construction
of a state state conditional pdf at the receiver. This is depicted in Figure 1, where the reconstruction is performed by
a Bayesian filter, which we derive in the sequel. We presume
that the input signal, {uk }, is available to both ends; a property that occurs in remotely located sensing or in filtering.
transmitter

receiver

plant

uk

xk+1 = Ak xk + Bk uk + wk

yk

◆k

yk = C k x k + v k

ŷk|k

1

Qk

Qk 1

¯◆k

Bayesian
Filter

p(xk |I¯k )
p(xk+1 |I¯k )

uk

Fig. 1. Block diagram of quantized innovations system.

We use an m-level quantizer, Qk (·), consisting of a collection of m intervals (zk,l , zk,u ], which form a disjoint covering of the real line and a corresponding rule, Q−1
k (·), for
dequantizing the received signal into a real number — for
a p-vector channel quantizer, we take a scalar quantizer in
each channel. When the innovations signal ιk lies within the
range (zk,l , zk,u ] the output of the quantizer is transmitted as
one of m symbols which is then dequantized at the receiver

Tuesday 11th April, 2017

as ῑk as one of m distinct values, which we take also to be a
value within the range (zk,l , zk,u ] including for the two edge
saturation levels. A 3-bit uniform quantizer-dequantizer cascade is shown in Figure 2. Evidently, ῑk = Q−1
k Qk (ιk ). The
input is the innovations signal, ιk , and the output is the recovered innovations signal, ῑk . The values ±ζ denote the
upper and lower saturation limits of the quantizer, which
we take for simplicity to be symmetric. Our results apply
equally to any well defined quantizer-dequantizer pair without reliance on symmetry or uniformity – this is important
because it admits optimized quantizer designs.

uk xk+1 = Ak xk + Bk uk + wk

0.4

✏k

K
ŷk|k

Qk 1

Qk

✏¯k K-Bayesian
filter
uk

1

Kalman
Filter

pK (xk |Ēk )
pK (xk+1 |Ēk )

Fig. 3. Block diagram of Method K showing calculation of the innovations signal, k . The K-Bayesian filter is detailed in Section 3.

value x̂Rk|k−1 . Then
R
ῑRk = Q−1
k Qk (yk − Ck x̂k|k−1 ).

5L/2

(2)

This is illustrated in Figure 4.

0.3

3L/2

0.2

L/2

0.1
output

yk
+

yk = C k x k + v k

7L/2

0.5

receiver

transmitter

plant

0

transmitter

L 2L 3L 4L

−0.1
−0.2

uk

⇣ = 4L

plant
xk+1 = Ak xk + Bk uk + wk yk
+yk = C k x k + v k

◆R
k

Ck

−0.3
−0.4

x̂R
k|k

−0.5

−0.8

−0.6

−0.4

−0.2

0
input

0.2

0.4

0.6

0.8

conditional
mean

Qk

¯◆R
1 k

z

R-Bayesian
Filter

1

Qk 1 Qk

¯◆R
k

uk

1

1

uk
R-Bayesian
Filter

pR (xk |I¯kR )
pR (xk+1 |I¯R )
k

Fig. 4. Block diagram of Method R showing calculation of the innovations signal, ιRk . The R-Bayesian filter is detailed in Section 3.

Method S: A modified Kalman filter recursion replaces the
R-Bayesian filter of Method R at both transmitter and
receiver, yielding synchronized: predicted and filtered
state estimates x̂Sk|k−1 and x̂Sk−1|k−1 , output prediction
S
S
ŷk|k−1
= Ck x̂Sk|k−1 , using data I¯k−1
. At the transmitter,

The receiver reconstructs a conditional pdf of the plant
state xk given the received quantized innovations data,
{ῑk , ῑk−1 , . . . , ῑ0 }. We shall consider three distinct constructions of this measurement signal and corresponding receiver
pdfs, distinguished where necessary by superscripts. The
detailed algorithms will be presented in Section 3.

S
ῑSk = Q−1
k Qk (yk − Ck x̂k|k−1 ).

Method K: A Kalman filter is operated at the transmitter
using signals {yk } and {uk } to produce Kalman output
K
prediction ŷk|k−1
, and Kalman innovations signal k ,
which is then quantized, transmitted and dequantized
at the receiver.
−1
K
¯k = Q−1
k Qk (k ) = Qk Qk (yk − ŷk|k−1 ),

1

pR (xk |I¯kR 1 )

receiver

Fig. 2. Three-bit (m = 8) uniform quantizer-dequantizer pair
input-output relation with ζ = 0.6222 and L = 2ζ/23 = 0.1555.
The methods of the paper apply equally to non-symmetric and
non-uniform quantizers.

Qk

R
ŷk|k
1

(3)

The Bayesian filter and its progeny Kalman filter are presented in brief detail in Section 2.1 for reference. Section 3
is then devoted to the explication of the three methods in
terms of their algorithms based on the Bayesian and Kalman
filter recursions. Computational comparisons are made in
Section 4 and conclusions are then provided. The central
contribution of the paper is to recognize the transmitter-side
signal model underpinning Method K and its corresponding
complete Bayesian filter for the receiver side. The subsequent reinterpretation of Methods R and S and their comparison are separate contributions enabled by the new Bayesian
computation. In the paper we denote: p(·) is a general pdf;
N (µ, Σ) is the Gaussian pdf with mean value µ covariance
matrix Σ.

(1)

The receiver conducts its own processing of this data
using a K-Bayesian filter developed in Section 3. This
is depicted in Figure 3.
Method R: The transmitter computes the received signal
ῑk and uses a copy of the receiver’s R-Bayesian filter
R
to construct pR (xk I¯k−1
), its own version of receiver’s
conditional state prediction pdf with conditional mean

2

2

Background and other literature

2.1

recursion is
T
K
T
−1
Lk = ΣK
,
k|k−1 Ck (Ck Σk|k−1 Ck + Rk )

The Bayesian Filter and Kalman Filter

K
k = yk − Ck x̂K
k|k−1 = yk − ŷk|k−1 ,

K
x̂K
k|k = x̂k|k−1 + Lk k ,

Suppose we have a state-space system as follows.
xk+1 = fk (xk , wk ),
zk = hk (xk , vk ),

K
ΣK
k|k = (I − Lk Ck )Σk|k−1 ,

K
ΣK
k+1|k = Ak Σk|k Ak + Qk ,

(4)
(5)

K
x̂K
k+1|k = Ak x̂k|k + Bk uk .

Therefore, we see the predicted state estimation error signal,
K
x̃K
k+1|k = xk+1 − x̂k+1|k , follows the recursion:

where process noise {wk } and measurement noise {vk }
are white and of known densities and joint density. The
functions fk (·, ·) and hk (·, ·) are not necessarily linear timevarying system equations. Denoting Zk = {z0 , z1 . . . zk },
the goal of the Bayesian filter is to compute the predicted
state conditional probability density function, p(xk Zk−1 ),

Kk
K
x̃k+1|k

= Ak Lk ,
= (Ak − Kk Ck )x̃K
k|k−1 + wk − Kk vk ,

(10)

k = Ck x̃k|k−1 + vk .

and the filtered pdf, p(xk Zk ), from an initial density

We appeal later to the following properties of the Kalman
filter.

p(x0 ) = p(x0 Z−1 ).

Lemma 1 [1] The Kalman filter innovations is white and
The Bayesian filter recursion [6] is

k ∼ N (0, Ck ΣKk|k−1 CkT + Rk ).

Z

Lemma 2 [3] Subject to uniform complete detectability of
[Ak , Ck ], uniform complete stabilizability of [Ak , Qk ], and
the existence of R > 0 such that Rk ≥ R, the discrete
Kalman filter is exponentially asymptotically stable.

p(xk Zk−1 ) =

p(xk xk−1 )p(xk−1 Zk−1 )dxk−1 ,
xk−1

(6)
p(xk Zk ) = R

p(zk xk )p(xk Zk−1 )

.

This section has been devoted to a brief presentation of the
Bayesian and Kalman filters based on signal models (4-5)
and (8-9) respectively. In the following sections, algorithms
will be presented in which the state process, {xk }, and measurement sequence, {zk } = {ῑk } or {¯
k }, will be identified
with different signals on the transmitter and receiver sides
respectively. In particular, we point out that the signal model
for Method K possesses a state, which we denote Zk , consisting of the concatenation of the plant state and the Kalman
filter state at the transmitter. It is this observation which admits the Bayesian filter derivation in this paper.

(7)

p(zk xk )p(xk Zk−1 )dxk
xk

(6) is expressed in terms of p(xk−1 |Zk−1 ), the filtered conditional pdf from the other half of the recursion, and
p(xk |xk−1 ) derived from (4) using knowledge of function f and the distribution of wk .
(7) is expressed in terms of p(xk |Zk−1 ), the predicted conditional pdf from the other half of the recursion, and
p(zk |xk , Zk−1 ) derived from (5) using knowledge of
function h and the distribution of vk .

2.2
For linear Gaussian systems, the Bayesian filter yields the
explicit conditional pdf calculation of the Kalman filter.
The Kalman filter, however, is much more efficient in that
it propagates the Gaussian conditional densities via conditional means and conditional covariances only. For the linear
system,
xk+1 = Ak xk + Bk uk + wk ,
yk = Ck xk + vk ,

Other literature

The sign-of-innovations Kalman filter (SOI-KF) was proposed by Ribeiro et al. [2] and yields a precursor to
Method S, applicable in the case of 1-bit quantization of
the innovations. Indeed, [2] appears to have been the progenitor for this Bayesian approach to quantized innovations
filtering, hence the nomenclature for Method R derived in
Section 3. The development of the 1-bit Method S proceeds
from a careful study of the Bayesian filter leading to the
statement of Method R as above but not its precise implementation. We draw on the observations of [2] to make
Method R explicit before proceeding with the analysis of
Method S for the SOI-KF and its extension in the multilevel quantizer Kalman filter (MLQ-KF) of [10,9]. The

(8)
(9)

with noises {wk } ∼ N (0, Qk ), {vk } ∼ N (0, Rk ) and x0 ∼
N (x̂0|−1 , Σ0|−1 ) all mutually independent, the Kalman filter
3

Lloyd-Max optimal quantization rule [11]. This yields maximum entropy coding of the transmitted data.
Property 3: Under the conditions of Lemma 2, the
transmitter-side Kalman filter forgets exponentially its initial conditions. That is, the dependence of the Kalman filter
and of the receiver-side Bayesian filter on initial conditions
disappears over time. This property has been studied further for nonlinear filters by van Handel [8,7]. In the context
here, this property suggests that, subject to well-posedness
of the Kalman filtering signal model at the transmitter, the
Method K receiver should eventually resynchronize after
an isolated error in reception. Similarly, the Bayesian filter integral recursion should exhibit some robustness to
inaccuracies in the evaluation of the integrals.

advantage of the SOI-KF and MLQ-KF is that a variant of
the Kalman filter recursion is computed for approximate
conditional means and covariances in place of calculation of
the entire pdf. Such reduced calculations suffice because the
state predicted conditional pdf is assumed to be Gaussian.
This assumption is evaluated (and shown to be somewhat
wanting) in the computational examples of Section 4.
Sukhavasi and Hassibi in [5] consider the Method K signal model (8-9) for plant state xk and demonstrate that the
conditional state density given quantized measurements at
the receiver is the density of a sum of two independent random variables. The first random variable is the transmitterside Kalman filter state conditional mean estimate and the
second possesses a multivariate truncated Gaussian density
involving the complete history of previous quantized measurements. Based on this decomposition, the authors provide
a recursive estimation algorithm called Kalman-like particle
filter. The decomposition is non-recursive and the algorithm
requires approximation for implementation. The authors empirically establish its performance relative to the SOI-KF of
[2].

It is not immediately evident that Method R and its descendent Method S enjoy these same properties.

3

For the Method K problem, [4] observe that, while the predicted pdf, p(xk |Ēk−1 ), can be computed from the traditional Bayesian filtering formula (6), the filtered pdf can be
computed by the following formula.
Z
p(xk Ēk ) =
p(xk Ek )p(Ek Ēk )dEk ,
(11)

In this section, we analyze each method under the following
common assumptions.
A.1 The plant is linear and described by (8-9).
A.2 Process and measurement noises {wk } ∼ N (0, Qk )
and {vk } ∼ N (0, Rk ) are independent and white.
A.3 Initial state x0 ∼ N (x̂0|−1 , Σ0|−1 ) is independent from
{wk } and {vk }.
A.4 The receiver knows: x̂0|−1 , Σ0|−1 , and Ak , Bk , uk ,
Ck , Qk , Rk , Qk (·), Q−1
k (·), for every value of k ≥ 0.

Ek

where p(xk |Ek ) is a Gaussian probability density function
and p(Ek |Ēk ) is a truncated Gaussian function whose dimension increases with time. That is, this is not a recursive
formula. The integration operation in (11) is replaced by a
recursive formula based on a mid-point approximation to
the integrals.
2.3

Analysis of techniques

3.1

Contributions of this paper

The earlier works either do not yield recursive solutions to
the state estimation problem or rely on approximation of the
densities to achieve recursion. Our approach, Method K, is
both recursive and exact, subject to the accuracy of computation of the integrals of Bayesian filtering. By the same token, Method K relies on the computation of these integrals,
while Method S is Kalman-filter-like in its computational
demands, which are significantly more modest. The properties described in Lemmata 1 and 2 imply advantages of
Method K.

Method K of this paper

From (10), the receiver side of Method K possesses a statespace and measurement description
"
Zk+1 =

Property 1: Since {k } and {¯
k } are both white sequences,
they are well coded from the perspective that there is no
predictable component correlated with past or future data.
Property 2: Building on the coding property, since the
innovations sequence is Gaussian and of a priori known
variance Ck ΣK
k|k−1 Ck + Rk , the quantizer Qk (·) can be
designed to optimize performance using, for example, the

xk+1

x̃K
k+1|k

¯k = Q−1
k Qk

#

"
=

Ak

0

#"

xk

#

+
0 Ak − Kk Ck x̃K
k|k−1
" #
"
#" #
Bk
I 0
wk
uk +
,
0
I −Kk
vk
(12)
" #!
h
i
h i w
k
(13)
0 C k Zk + 0 I
vk

By comparison to (4-5), this immediately invites the application of the Bayesian filter (6-7) with the substitutions

4

Zk 7→ xk , ¯k 7→ zk , and Ēk 7→ Zk .
pK (Zk Ēk−1 ) =

Z
Zk−1

(4) The initial probability density function at the receiver
"

p(Zk Zk−1 )pK (Zk−1 Ēk−1 )dZk−1 ,

p(Z0 Z−1 ) = p(Z0 ) = p

(14)
pK (Zk Ēk ) = R
Zk

=

p(¯
k Zk )pK (Zk Ēk−1 )

=N

,

p(¯
k Zk )pK (Zk Ēk−1 )dZk

p(¯
k Zk )pK (Zk Ēk−1 )
p(¯
k Zk )pK (Zk Ēk−1 )
p(¯
k )

.

#!

,
x̃0|−1
"
# "
#!
x̂0|−1
Σ0|−1 Σ0|−1
,
.
0
Σ0|−1 Σ0|−1
(19)

The second equality corresponds to Assumption A.4
that the transmitter and receiver are synchronized at
time zero.

,

p(¯
k Ēk−1 )
=

x0

We summarize the quantized Kalman filter innovations
Bayesian filter, Method K, as follows.

(15)

The above equations form a recursive algorithm to compute
the predicted pdf and filtered pdf for each time. We show
the detail of each term:

Method K algorithm at the receiver.
Kalman filter at the transmitter
Required function: pK (Zk−1 Ēk−1 ).

(1) In (14),

1. Compute p(Zk Zk−1 ) from (16).
2. Compute predicted pdf pK (Zk Ēk−1 ) from (14).
h
i
K
3. Receive ¯k = Q−1
k Qk yk − Ck x̂k|k−1 .

p(Zk Zk−1 ) = p (wk−1 = xk − Ak−1 xk−1 − Bk−1 uk−1 )

× p Kk−1 vk−1 = Ak−1 (x̃K
k−1|k−2 − xk−1 ) − Bk−1 uk−1

K
− Kk−1 Ck−1 x̃K
k−1|k−2 − (x̃k|k−1 − xk−1 ) .

4. Compute p(¯
k ) from (17) and p(¯
k Zk ) from (18).
5. Compute filtered pdf pK (Zk Ēk ) from (15).

(16)

This probability can be computed from the joint Gaussian distribution of wk−1 and vk−1 , corresponding to
Assumption A.2.
(2) In (15), the whiteness of the innovations signal {k },
and therefore of {¯
k }, admits
Z zk,u
p(¯
k Ēk−1 ) = p(¯
k ) =
p(k )dk ,
zk,l
Z zk,u
T
N (0, Ck ΣK
=
k|k−1 Ck + Rk )dk ,

3.2

Method R of Ribeiro et al. [2]

Method R of [2] uses the Bayesian filter (6-7) with meaR
surement sequence I¯k−1
= {ῑRk−1 , . . . , ῑR0 } followed by a
conditional mean calculation.
R
pR (xk I¯k−1
)=

(17)

zk,l

where (zk,l , zk,u ] is a quantization interval at time k.

R
p(xk xk−1 )pR (xk−1 I¯k−1
)dxk , (20)
R
R
p(ῑRk xk , I¯k−1
)pR (xk I¯k−1
)

, (21)
R )pR (x I¯R )dx
p(ῑRk xk , I¯k−1
k k−1
k
Z
R
R
= E[xk I¯k−1
]=
xk pR (xk I¯k−1
)dxk .
xk

(3) The term p(¯
k Zk ) can be computed as follows:

x̂R
k|k−1

zk,u

p(¯
 k Zk ) =
p(k Zk )dk ,
zk,l
Z zk,u
=
p(k = Ck x̃K
k|k−1 + vk )dk ,
zk,l
Z zk,u
=
N (Ck x̃K
k|k−1 , Rk )dk .

xk

p (xk I¯kR ) = R
R

Z

Z

xk

(22)
System state equation (4) admits the substitution into (20)
p(xk xk−1 ) = p(wk−1 = xk − Ak−1 xk−1 − Bk−1 uk−1 ),

(18)

(23)

zk,l

5

is

which forms part of (16) in Method K. Additionally for
Method R, from (2) and the quantizer intervals (zk,l , zk,u ],

x̂Sk|k−1 = Ak x̂Sk−1|k−1 ,
ΣSk|k−1

R
)
p(ῑRk xk , I¯k−1


R
¯R
x
,
I
= p Q−1
)
+
v
]
Q
[C
(x
−
x̂
k
k
k
k
k
k−1 ,
k|k−1
k
Z zk,u 

R
dιRk ,
=
p Ck (xk − x̂Rk|k−1 ) + vk xk , I¯k−1
zk,l
Z zk,u 

(24)
=
N Ck (xk − x̂Rk|k−1 ), Rk dvk .

x̂Sk|k

=

+ Qk ,
(26)
n
n
φ(z
)
−
φ(z
)
k,l
k,u
= x̂Sk|k−1 + sgn(ῑSk )
,
n − αz n
αzk,l
k,u
ΣSk|k−1 CkT
×q
,
Ck ΣSk|k−1 CkT + Rk

zk,l

ΣSk|k = ΣSk|k−1 −

Method R of Ribeiro et al. [2] algorithm at transmitter
×

and receiver
R

Required function: p

R
(xk−1 I¯k−1
)

(27)

N
n
n
X
[φ(zk,l
) − φ(zk,u
)]2
,
n − αz n |
|αzk,l
k,u
n=1

ΣSk|k−1 CkT Ck ΣSk|k−1
Ck ΣSk|k−1 CkT + Rk

.

(28)

where sgn(·) is the signum (or sign) function

1. Compute p(xk xk−1 ) from (23).
R
) from (20).
2. Compute predicted pdf pR (xk I¯k−1

3. Compute the conditional mean

(25)

Ak ΣSk−1|k−1 ATk

x̂Rk|k−1

(
sgn(ῑSk )

from (22).

4. Compute at the transmitter
h
i
R
ῑRk = Q−1
Q
y
−
C
x̂
k
k
k
k
k|k−1 ,

=

+1 if ῑSk ≥ 0

−1 if ῑSk < 0

Method S of Ribeiro et al. [2] and You et al. [10]

and receive ῑRk at the receiver.
R
5. Compute pR (ῑRk xk , I¯k−1
) from (24).

algorithm at transmitter and receiver

6. Compute filtered pdf pR (xk I¯kR ) from (21).

1. Compute predicted values x̂Sk|k−1 and ΣSk|k−1

Required data: x̂Sk−1|k−1 , ΣSk−1|k−1
from time-update (25-26).

3.3

2. Compute at the transmitter
h
i
S
ῑSk = Q−1
k Qk yk − Ck x̂k|k−1 ,

Method S of Ribeiro et al [2] and You et al. [10]

and receive ῑSk at the receiver.

In [2], Ribeiro et al. derive from their Method R a Kalmanfilter-like recursion based on the one-bit quantization
(signum) of the associated innovations signal, {ιRk } depicted
in Figure 4. This is called the sign of innovations Kalman
filter (SOI-KF) and the algorithm operates at both the
transmitter and receiver. You et al. [10] extend these ideas
from one-bit quantization of SOI-KF to the multiple-level
quantized innovations Kalman filter (MLQ-KF), which we
refer to as Method S. The derivations of both works operate
under the assumption that the predicted pdf pS (xk+1 |I¯kS ) is
Gaussian.
Denote the normal density function and
x2
1
φ(x) = √ exp(− ), αz =
2
2π

√

Z

3. Compute filtered values x̂Sk|k and covariance ΣSk|k
from measurement update (27-28).

4

In this section we present a few simple simulation comparisons between the methods, noting that Method K yields the
precise conditional densities.

π/2 erfc(x) as

∞

4.1
φ(x)dx.

Examples

Case 1: Almost fixed state, Method K: The first example
has a state which is almost constant and well measured. It
is provided to demonstrate that Method K is sound. The
system has scalar state which obeys

z

Further, denote the nth positive level of a symmetric, midn
n
rise, N -level, scalar quantizer by (zk,l
, zk,u
]. Then the
Kalman-filter-like recursion of Method S/MLQ-KF for
|ῑk | ∈

Comparison and examples

xk+1 = xk + wk ,
y k = x k + vk ,

n
n
(zk,l
, zk,u
]

6

with wk ∼ N (0, 0.0001), vk ∼ N (0, 0.00001), x0 ∼
N (0, .02). The 3-bit quantizer is that depicted in Figure 2.

Time 0:
x0 = −0.0319, v0 = 0.1117,
w0 = −0.1089, x1 = −0.1392,

At the transmitter Kalman filter, the innovations value is
0 = 0.1160, leading to a received quantized innovations
value ¯0 ∈ (0, 0.1555]. The reconstructed conditional pdfs
are close to truncated Gaussians with slight smoothing at
the edges according to the narrow densities of wk in the
predictor and vk in the filter. Figure 5 is the result calculated
in matlab using 101 sample points for the integrals. These
computed pdfs make sense. The next few steps do not alter
the figures significantly.

R
S
x̂K
0|−1 = x̂0|−1 = x̂0|−1 = 0,

0 = ιR0 = ιS0 = 0.0798,
¯0 = ῑR0 = ῑS0 ∈ (0, 0.1555],

x̂S1|0 = 0.0085, ΣS1|0 = 0.0181.
Time 1:
x1 = −0.1392, v1 = 0.0033,
w1 = 0.0553, x2 = −0.0770,

8

1 = −0.1898, ιR1 = −0.1853,

¯1 = ῑR1 ∈ (−0.3111, −0.1555],

7

x̂S2|1 = 0.0161, ΣS2|1 = 0.0177

p K (x 0 | Ē 0)
p K (x 1 | Ē 0)

6

5

K
x̂K
2|1 = −0.0630, Σ2|1 = 0.0156.

PDF

Time 2:

4

x2 = −0.0770, v2 = 0.1101,
w2 = 0.1544, x3 = 0.0813,

3

2 = −0.0140, ιR2 = −0.1230,

2

¯2
x̂S3|2
x̂K
3|2

1

0
−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

state value

= ῑR2 ∈ (−0.1555, 0],

= 0.0231, ΣS3|2 = 0.0176,
= −0.0679, ΣK
3|2 = 0.0155.

Time 3:

Fig. 5. Example 1 predicted and filtered pdfs at the receiver computed using Bayesian filtering Method K with true innovations
0 = 0.1160 and received quantized innovations ¯0 ∈ (0, 0.1555]
via the 3-bit quantizer in Figure 2.

x3 = 0.0813, v3 = 0.0086,
w3 = −0.1492, x4 = −0.0720,
3 = 0.1492, ιR3 = 0.0352,
¯3 = ῑR3 ∈ (0, 0.1555],

x̂S4|3 = 0.0297, ΣS4|3 = 0.0175,
K
x̂K
4|3 = 0.0216, Σ4|3 = 0.0155.

Case 2: Comparison of methods: Consider the system
Time 4:

xk+1 = 0.95xk + wk ,
y k = x k + vk .

x4 = −0.0720, v4 = −0.0742,
4 = −0.0936, ιR4 = −0.1180,

¯4
x̂S5|4
x̂K
5|4

with wk ∼ N (0, 0.01), vk ∼ N (0, 0.01), x0 ∼ N (0, .02)
and the same 3-bit quantizer shown in Figure 2. We compare
the sequence of predicted and filtered pdfs computed using
Method K and Method R, and the predicted Gaussian pdf
only from Method S/MLQ-KF using the computed mean
and covariance. These are contrasted with the transmitterside Kalman filter innovations Gaussian pdf. The results are
displayed in Figures 6-10.

= ῑR4 ∈ (−0.1555, 0],

= 0.0360, ΣS5|4 = 0.0175,
= −0.0335, ΣK
5|4 = 0.0155.

This second example demonstrates that the methods are
workable. Although the explicit computation of pdfs is numerically demanding compared with the moment recursions.
However, we note from this sequence of figures that Methods K and R yield pdfs which circumscribe the transmitterside Kalman filter pdfs. So the actual state pdf is feasible

The signals are are follows.

7

5

5

R

4

(x 0 | I¯0R)

p
p K (x 0 | Ē 0)
p K (x 0 |E 0)

3
2

PDF

PDF

4

0

−0.8

−0.6

−0.4

−0.2

0

x0

0.2

0.4

0.6

0.8

3
2
1

−0.2

0

x2

0.2

0.4

0.6

0.8

−0.6

−0.4

−0.2

0

x1

0.2

0.4

0.6

p R (x 3 | I¯2R)
p K (x 3 | Ē 2)
p K (x 3 |E 2)
MLQ-KF

3
2

0

−0.8

0.8

−0.8

−0.6

−0.4

−0.2

0

x3

0.2

0.4

0.6

0.8

Fig. 8. Time 2: state conditional pdfs.
5

PDF

4

p R (x 3 | I¯3R)
p K (x 3 | Ē 3)
p K (x 3 |E 3)

3
2
1

5
4

PDF

−0.4

1

Fig. 6. Time 0: upper figure filtered state conditional pdfs for
Method K (blue) and R (red) and for the transmitter-side Kalman
filter (dashed blue); and lower figure predicted state conditional
pdfs for Method K (blue) and R (red) and Gaussian pdfs for the
transmitter Kalman filter (dashed blue) and Method S/MLQ-KF
(dot-dash green).

0

p R (x 1 | I¯1R)
p K (x 1 | Ē 1)
p K (x 1 |E 1)

3
2

−0.8

−0.6

−0.4

−0.2

0

x3

0.2

0.4

4

−0.6

−0.4

−0.2

0

x1

0.2

0.4

0.6

PDF

−0.8

0.8

2

0

p R (x 2 | I¯1R)
p K (x 2 | Ē 1)
p K (x 2 |E 1)
MLQ-KF

3
2
1

0.8

p R (x 4 | I¯3R)
p K (x 4 | Ē 3)
p K (x 4 |E 3)
MLQ-KF

3

1

5
4

0.6

5

1

PDF

−0.6

4

p R (x 1 | I¯0R)
p K (x 1 | Ē 0)
p K (x 1 |E 0)
MLQ-KF

PDF

PDF

4

0

−0.8

5

5

0

2
1

1
0

p R (x 2 | I¯2R)
p K (x 2 | Ē 2)
p K (x 2 |E 2)

3

−0.8

−0.6

−0.4

−0.2

0

x4

0.2

0.4

0.6

0.8

Fig. 9. Time 3: state conditional pdfs.
5

−0.8

−0.6

−0.4

−0.2

0

x2

0.2

0.4

0.6

0.8

4

PDF

0

Fig. 7. Time 1: state conditional pdfs.

2
1

under the estimated pdf. This appears not the be the case for
MLQ-KF at times. Methods K and R appear comparable and
do indeed produce, with these unsaturated quantized data,
densities which are roughly gaussian. Example 1 demonstrates that this is not always true, however. If the quantizer
were chosen to be Lloyd-Max optimal for Method K, then
one may conjecture that its reconstructed conditional density of the state is itself optimal in some sense. One might
then interpret Method R as producing an approximation of
the conditional pdf from Method K.
5

p R (x 4 | I¯4R)
p K (x 4 | Ē 4)
p K (x 4 |E 4)

3

0

−0.8

−0.6

−0.4

−0.2

0

x4

0.2

0.4

0.6

0.8

5

p R (x 5 | I¯4R)
p K (x 5 | Ē 4)
p K (x 5 |E 4)
MLQ-KF

PDF

4
3
2
1
0

−0.8

−0.6

−0.4

−0.2

0

x5

0.2

0.4

0.6

0.8

Fig. 10. Time 4: state conditional pdfs.

Conclusions

played by the Kalman filter state in describing the transmitter signal model. Comparisons were made to a candidate
Bayesian approach underpinning the SOI-KF derivation and
to the MLQ-KF, indicating some issues with the latter methods. Some specific advantages of the new method of this pa-

We have developed via Bayesian filtering an exact and recursive approach to the reconstruction from quantized Kalman
filer innovations signals of predicted and filtered state conditional pdfs. The key observation was to recognize the role

8

per were identified in terms of coding performance, optimal
quantization, and recovery from error.
References
[1] B.D.O. Anderson and J.B. Moore. Optimal Filtering. Prentice-Hall,
Englewood Cliffs, New Jersey, 1979.
[2] A.Ribeiro, G.B. Giannakis, and S.I. Roumeliotis.
SOI-KF:
Distributed Kalman filtering with low-cost communications using
the sign of innovations. IEEE Transactions on Signal Processing,
54:4782–4795, 2006.
[3] J.J. Deyst and C.F. Price. Conditions for asymptotic stability of the
discrete minimum variance linear estimator. IEEE Transactions on
Automatic Control, 13(6):702–705, 1968.
[4] Zhansheng Duan, V. P. Jilkov, and X. Rong Li. State estimation
with quantized measurements: Approximate MMSE approach. In
11th IEEE International Conference on Information Fusion, pages
1–6, Cologne, 2008.
[5] R.Sukhavasi and B. Hassibi. The Kalman-like particle filter:
Optimal estimation with quantized innovations/measurements. IEEE
Transactions on Signal Processing, 61(1), January 2013.
[6] D. Simon. Optimal State Estimation: Kalman, H∞ , and nonlinear
approaches. John Wiley & Sons, Hoboken NJ, 2006.
[7] R. van Handel. Observability and nonlinear filtering. Probability
Theory and Related Fields, 145(35-74), 2009.
[8] R. van Handel. Uniform observability of Hidden Markov Models and
filter stability for unstable signals. Annals of Applied Probability,
19(1172-1199), 2009.
[9] Keyou You, Lihua Xie, Shuli Sun, and Wendong Xiao. Quantized
filtering of linear stochastic systems. Transaction of the Institute of
Measurement and Control, 33(6):683–698, 2011.
[10] Keyou You, Linhue Xie, Shuli Sun, and Wendong Xiao. Mutli-level
quantized innovation Kalman filter. In 17th IFAC World Congress,
pages 1420–1425, Seoul, Korea, 2008.
[11] Yuli You. Audio Coding Theory and Applications. Springer Verlag,
New York, 2010.

9

