Separated design of encoder and controller for
networked linear quadratic optimal control

arXiv:1405.0135v4 [math.OC] 19 Feb 2016

Maben Rabi, Chithrupa Ramesh, and Karl Henrik Johansson
Abstract. For a networked control system, we consider the problem of encoder and controller design. We study a discrete-time linear plant with a finite horizon performance cost,
comprising of a quadratic function of the states and controls, and an additive communication cost. We study separation in design of the encoder and controller, along with related
closed-loop properties such as the dual effect and certainty equivalence. We consider three
basic formats for encoder outputs: quantized samples, real-valued samples at event-triggered
times, and real-valued samples over additive noise channels. If the controller and encoder are
dynamic, then we show that the performance cost is minimized by a separated design: the
controls are updated at each time instant as per a certainty equivalence law, and the encoder
is chosen to minimize an aggregate quadratic distortion of the estimation error. This separation is shown to hold even though a dual effect is present in the closed-loop system. We also
show that this separated design need not be optimal when the controller or encoder are to be
chosen from within restricted classes.

1. Introduction
We consider discrete-time sequential decision problems for a control loop that has a communication bottleneck between the sensor and the controller (Figure 1). The design problem
is to choose in concert an encoder and a controller. The encoder maps the sensor’s raw data
into a causal sequence of channel inputs. Depending on the channel model adopted in this
paper, the encoder performs either sequential quantization, sampling, or analog companding.
The controller maps channel outputs into a causal sequence of control inputs to the plant.
Such two-agent problems are generally hard because the information pattern is non-classical,
as the controller has less information than the sensor [51]. This gives scope for the controller
to exploit any dual effect present in the loop, even when the plant is linear [14]. These twoagent problems are at the simpler end of a range of design problems arising in networked
control systems [11, 3, 21, 1]. Naturally, one seeks formulations of these design problems as
stochastic optimization problems whose solutions are tractable in some suitable sense.
The classical partially observed linear quadratic Gaussian (LQG) optimal control problem
is a one-agent decision problem [52]. Given a linear, Gauss-Markov plant, one is asked for
a causal controller, as a function of noisy linear measurements of the state, to minimize a
quadratic cost function of states and controls. This problem has a simple and explicit solution,
where the optimal controller ‘separates’ into two policies; one to generate a minimum meansquared error estimate of the state from the noisy measurements, and the other to control the
fully observed Gauss-Markov process corresponding to the estimate. A networked version of
this problem is the following two-agent LQG optimal control problem [10]. Given a linear
Gauss-Markov plant and a channel model, one is asked for an encoder and controller to
minimize a performance cost which is a sum of a communication cost and a quadratic cost
on states and controls. The communication cost is charged on decisions at the encoder,
which are chosen to satisfy constraints imposed by the channel model. No causal encoding
or control policies are, in general, excluded from consideration. As in the one-agent version,
1

2

M. RABI, C. RAMESH, AND K.H. JOHANSSON

ut

Plant

xt

xt+1 = axt + ut + wt

Encoder
Et xt0 ; z0t−1
zt



Controller
ut

Kt (z0t )

zt

Rate-limited Channel

ιt

Figure 1. Control over a rate-limited channel with perfect feedback
a certain ‘separated’ design is optimal, as has been suggested in various settings since the
sixties [27, 42, 16, 5, 33, 45, 31, 53, 35, 6, 34, 56]. Precisely, the following combination
is optimal: certainty equivalence controls with a minimum mean-squared estimator of the
state, and an encoder that minimizes a distortion for state estimation at the controller. The
distortion is the average of a sum of squared estimation errors with time-varying coefficients
depending on the coefficients of the performance cost. This separation is different from that
obtained in the classical LQG problem, but it is still due to a linear evolution of the state,
and the statistical independence of noises from all other current and past variables. As in the
classical one-agent version [43, 41], the random variables need not be Gaussian.
1.1. Previous works. In the long history of the two-agent networked LQG problem,
different channel models have been treated, leading to different types of encoders. We find in
these works that the encoder is either a quantizer, an analog time-dependent compander, or
an event-based sampler.
When a discrete alphabet channel is treated, the encoder is a time-dependent quantizer.
Quantized control has been explored since the sixties, and structural results for this problem
have seen spirited discussions over the years [27, 32, 16]. This problem was revisited by
Borkar et al. [10] in recent years, setting off a new wave of interest. Surveys can be found
in [35, 18]. For an additive noise channel, the encoder is a time-dependent, possibly non-linear,
compander. The corresponding networked LQG problem has been studied in [5], and more
recently in [17, 19]. Analog channels with channel use restrictions lead to an encoder being
an event-triggered sampler [2]. The networked LQG problem for event-triggered sampling is
studied in [34].
The above papers suggested separated designs for the two-agent LQG problem with dynamic encoder and controller, and certainty equivalence controls. This is despite other results [13, 15], confirming the dual effect in the two-agent networked control problem. Thus,
there can be an incentive to the controller to influence the estimation error, and yet the optimal
controller chooses to ignore this incentive. Furthermore, for the two-agent LQG problem with
event-triggered sampling, and with zero order hold control between samples, Rabi et al. [39]
showed through numerical computations that it is suboptimal to apply controls affine in the
minimum mean square error (MMSE) estimate. The optimal controls are nonlinear functions
of the received samples. Thus, the literature does not tell us when separation holds, and when
it does not, for the general class of two-agent problems.
1.2. Our contributions. We make three main contributions. Firstly, we show that for
the combination of a linear plant and nonlinear encoder, the dual effect is present. This confirms the results of Curry and others [13, 15], by establishing through a counter example
that there is a dual effect in the closed-loop system. In fact, each of the three models we
allow for the channel endow the loop with the dual effect. The dual role of the controller
lies in reducing the estimation error in the future, using the predicted statistics of the future
state and knowledge of the encoding policy. Due to this dual role, we show that, in general, separated designs need not be optimal for linear plants with non-linear measurements,
even with independent and identically distributed (IID) Gaussian noise and quadratic costs.

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

3

Examples 5 and 6 show instances where the dual effect matters. Example 3 shows how the
dual effect in the two-agent networked LQ problem renders useless the techniques that work
for the classical, single-agent, partially observed LQ problem. These examples illustrate the
insufficiency of arguments offered in [27, 42, 16, 5, 33, 45, 31, 53, 35, 6, 34, 56] for the
optimality of separation and certainty equivalent controls.
Our second contribution is a proof for separation in one specific design problem. We prove
that for the dynamic encoder-controller design problem, it is optimal to apply separation
and certainty equivalence. A key instrument in our proof is the class of ‘controls-forgetting
encoders’ (introduced in section 4.2) which we show to be optimal despite it being a strict
subset of the general class of state-based encoders. We also notice that the result holds under
a variety of schemes for charging communication costs. For example, it holds even when
the encoder is an analog compander with hard amplitude limits. Our proof does not require
the dual effect to be absent. Hence there is no contradiction with the fact separation and
certainty equivalence are not optimal for other design problems concerning the same plantsensor combination. Our work also provides a direct insight to explain separation or the lack
of it, in the form of a property of the optimal cost-to-go function (Example 4 in Section 6).
Furthermore, we show that when this property does not hold separation is no longer optimal.
Our third contribution points out some subtleties that arise when dynamic policies are
involved. We explicitly demonstrate that with dynamic encoders for LQ optimal control,
one cannot extend and apply a result of Bar-Shalom and Tse [7] which mandates absence of
dual effect for certainty equivalence to be optimal. The classical notion of a dual effect was
introduced for static measurement policies, and the dual role of the controls has been motivated
through the notion of a probing incentive [14]. We ask if the concept of probing applies
unchanged for dynamic measurement policies and point out some subtleties in answering this
question.
In recent years, there has been a resurgence of interest in problems related to dynamic
and decentralized decision making in stochastic control. Old problems and results have been
reexamined and reinterpreted to find new insights and develop new methods, such as the
common information approach [30, 36]. Others, such as [26], have sought to reinterpret
the proof techniques used in [4]. Following in the path of [50], many new counterexamples
have been identified that show optimality of nonlinear strategies for control problems under
non-classical information patterns [29, 57]. Similarly, drawing from the many works on twoagent networked LQG problems [13, 15, 10, 35, 18], we have sought to understand why a
structural simplification can be found in some dynamic decision problems, despite the nonclassical information pattern and the consequent presence of a dual effect.
1.3. Outline. The remainder of the paper is organized as follows. In Section 2, we present
a basic problem formulation, pertaining to encoder and controller design for data-rate limited
channels. In Section 3, we discuss the notion of a dual effect and certainty equivalence, and
present a counterexample to establish that there is a dual effect in the considered networked
control system. In Section 4, we present a proof for separation in the two-agent networked
LQG problem. In Section 5, we extend our results to other channel models, including eventtriggered samples and additive noise channels. In Section 6, we present a number of examples
to illustrate that in general, separation does not hold for constrained design problems, followed
by the conclusions in Section 7.

2. Problem formulation
In this section, we describe a version of the two-agent networked LQG problem, corresponding to a rate-limited channel model. We consider an instantaneous, error-free, discretealphabet channel and the logarithm of the size of the alphabet is the bit rate. A control system

4

M. RABI, C. RAMESH, AND K.H. JOHANSSON

that uses such a channel to communicate between its sensor and controller is depicted in Figure 1, and comprises of four blocks. Each of these blocks, along with the performance cost,
are described below, followed by a description of the design problems under consideration.
2.1. Plant. The plant state process {xt } is scalar, and its evolution law is linear:
(1)

xt+1 = axt + ut + wt ,

for 0 ≤ t ≤ T. Here {ut } is the controls process, and {wt } is the plant noise process, which is
a sequence of independent random variables with constant variance σw2 , and zero means. The
initial state x0 has a distribution with mean x0 and variance σ02 . At any time t, the noise wt is
independent of all state, control, channel input, and channel output data up to and including
time t. We assume that the state process is perfectly observed by the sensor.
2.2. Performance cost. The performance cost is a sum of the quadratic cost charged
on states and controls, and a communication cost charged on encoder decisions:
#
"
T
T
X
X
(2)
u2i + J Comm
J = E x2T +1 + p
x2i + q
i=1

i=0

where p > 0 and q > 0 are suitably chosen scalar weights for the squares of the states and
controls, respectively. The communication cost J Comm is an average quantity that depends on
the encoding and control policies, and the channel model adopted.
2.3. Channel model. The channel model refers to an input-output description of the
communication link from the sensor to the controller. We denote the channel input at time t
by ιt , the corresponding output by zt , and the encoding map generating ιt by Et . In Figure 1,
we consider an ideal, discrete alphabet channel that faithfully reproduces inputs, and thus,
ιt = zt ∀t. The encoder’s job is to pick at every time t, the encoding map Et producing a
channel output letter from the pre-assigned finite alphabet zt ∈ {1, . . . , N }, ∀ t, where the
non-negative integer N is the pre-assigned size of the channel alphabet. Since the alphabet
is fixed, we have a hard data-rate constraint at every time. Hence there is no explicit cost
attached to communication, so J Comm ≡ 0 in this case. In Section 5, we consider other channel
models that permit the data-rate or energy needed for each transmission to be chosen causally
by the encoder.
2.4. Controller. The control signal ut is real valued and is to be computed by a causal
policy based on the sequence of channel outputs. The controller has perfect memory, and
thus remembers all of its past actions, and the causal sequence of channel outputs. Thus, in
general, at every time t the controller’s map takes the form:
o
n
t
t−1
Kt : t, {zi }0 , {ui }0
7→ ut .
2.5. Encoder. At all times, the encoder knows the entire set of control policies employed
by the controller and the statistical parameters of the plant. With this prestored knowledge,
the encoder works as a causal quantizer mapping the sequence of plant outputs. Thus, the
encoder’s map takes the form:

Et : t, {xi }t0 , {zi }t−1
, {Ki (·)}t−1
7→ zt .
0
0

Notice that we do not allow the encoder to directly view the sequence of inputs to the plant.
This subtle point plays an important role in the examples we present in Section 7.

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

5

2.6. Design problems. For a given information pattern, different design spaces may
arise due to engineering heuristics, hardware or software limitations, etc. Any such design
space is a subset of the set of all admissible encoder and controller pairs. We identify four
design problems, each associated with its own design space. For these design problems, an
adopted channel model can be either the one described in Section 2.3, or any of the models
from Section 5. First, we pose a single-agent design problem which has a classical information
pattern.
Design problem 1 (Controller-only Design). For the linear plant (1), the adopted channel
model, and a given admissible set of encoding policies:
n
oT
t
t−1 
†
Et · ; {zi }0 , {ui }0
,
0

the controller-only design problem requires one to pick a causal sequence of control policies
{Kt }T0 to minimize the performance cost (2).
Next we pose a design problem where the design space is the largest possible non-randomized
set of admissible encoder-controller pairs. We consider every causally time-dependent encoder
and controller. In other words, for this type of design problem, regardless of the choices one
makes for channel and communication cost, at any time, the controller can update the control
signal using all of the channel outputs up till then.
Design problem 2 (Dynamic Encoder-Controller Design). For the linear plant (1) and
the adopted channel model, the dynamic encoder-controller design problem requires one to pick
causal sequences of encoding and control policies {Et }T0 , {Kt }T0 to minimize the performance
cost (2).
Next we pose a design problem where the controller and encoder must respect a restriction
on selecting the control signals or encoding maps. At every time, the control values must
be chosen from a restricted set U, such as the interval (−1, 1) or the finite set {−1, 0, 1}.
Likewise, the encoding maps have to be chosen from within restricted sets. For example,
the encoding maps may be constrained to consist of two quantization cells (−∞, θ), (θ, ∞),
where the encoder threshold θ must be chosen from a restricted set Θ, say the interval (−5, 5).
Subject to these constraints, the controller and encoder policies are still to be dynamically
chosen.
Design problem 3 (Constrained Encoder-Controller Design). For the linear plant (1),
and the adopted channel model, the constrained encoder-controller design problem requires one
to pick causal sequences of encoding and control policies {Et }T0 , {Kt }T0 , subject to the constraints
represented by θ ∈ Θ and uk ∈ U, to minimize the performance cost (2).
Next we pose a design problem where the controller must respect not only the information
pattern in the dynamic encoder-controller design problem (Design problem 2), but must also
respect a restriction on updating controls. Basically, the control waveform is generated in a
piece-wise ‘open-loop’ way, while epochs and encoding maps are picked using dynamic policies.
Let 0 , 1 ≥ 1, be two random integers such that 0 + 1 = T + 1. Then the two epochs are
{0, . . . , 0 − 1} and {0 , . . . , T }. These epochs are chosen by the controller respecting the
inequalities: 1 ≤ 0 < T + 1 and 1 = T + 1 − 0 , and hence have to be adapted to all the data
available at the controller. Within an epoch, the controller must pick controls depending only
on data at the start of the epoch. Precisely, given the condition that t < 0 , and given the
initial observation z0 , the controls ut must be a fixed function of (t, z0 ) regardless of the data
{z1 , . . . , zt }.
Design problem 4 (Hold-Waveform-Controller and Encoder Design). For the linear
plant (1), and the adopted channel model, the hold-waveform-controller and encoder design

6

M. RABI, C. RAMESH, AND K.H. JOHANSSON

ut

Markov Plant

Encoder and Channel
xt
zt
Ψt (xt , ut , κt )

xt+1 = Φt (xt , ut , wt )

Controller
Kt (z0t )

zt

Figure 2. Setup for definitions in Section 3
problem is to pick a causal sequence of encoding polices {Et }T0 in concert with a causal sequence of policies for epochs and controls to minimize the performance cost (2). The controls
are restricted to depend on the controller’s data in the specific form:
  
Kt0 z0
for 0 ≤ t ≤ 0 − 1, and,


ut =
Kt1 {zi }0 −1 , {ui }0 −1
for 0 ≤ t ≤ T.
0
0

A special case of a hold-waveform controller is that of zero order hold (ZOH) control where
an additional restriction forces the control waveform be held constant over each epoch.
For all four design problems presented above, we assume the existence of measurable
policies minimizing the associated costs. We avoid investigating the necessary technical qualifications except to say that if need be, one may allow randomized polices, or even reject the
class of merely measurable policies in favour of the class of universally measurable policies [9].
3. Dual effect and certainty equivalence
We begin by presenting a definition of dual effect [14] and certainty equivalence [25]. We
then present an example to establish that there is a dual effect of the controls in the networked
control system introduced in Section 2.
3.1. Dual effect. In a feedback control loop, the dual effect is an effect that the controller
may see in the rest of the loop. When it is present, the control laws affect not just the first
moment, but also second, third and higher central moments of the controller’s nonlinear filter
for the state. Below, we state this formally for a controlled Markov process with partial
observations available to the controller:




(3)
xt+1 = Φt xt , ut , wt ,
zt = Ψt xt , ut , κt ,

where the sequences {xt } and {ut } are the real-valued plant state and control processes,
respectively,

 see Figure 2. The sequence {zt } is the observation process and the sequences
wt and κt are the plant noise and observation noise processes, respectively. Assume
that all the primitive random variables are defined on a suitable probability
n
o ntriple, [Ω,
o F, P].
e ·) . Once
Now, consider two arbitrary admissible sets of control policies: K(t, ·) , K(t,
we pick one such set of control policies, they together with the measure P define the states,
observations and controls as random processes. The choice of policies fixes their statistics. We
can advertise this relationship by (1) specifying random variables, xt for example, in the form
xt (ω; K), (2) specifying a filtration, for example, the one generated by the z-process as F K,z ,
or (3) specifying an expected value of a functional, E [Ft ] for example, in the form
h 
i
t
t
t
EP,K Ft t, {xi (ω; K)}0 , {zi (ω; K)}0 , {ui (ω; K)}0 ,

where ω stands for any element of the sample space of the primitive random variables. To
minimize the notational burden, we advertise the dependence on the set of control policies
only as needed. We now define the dual effect by defining its absence.

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

7

Definition 1 (Dual effect). The networked control system in Figure 2 is said to have no
dual effect of second-order if
e of admissible control policies, and
(1) for any two sets K, K
(2) for any two time instants t, s,
we have FtK,z = FtK,z for every t, and that for any given event X ∈ FtK,z ,



 2 

s
s
zi (ω; K) 0 , ω ∈ X =
EP,K xt (ω; K) − EP,K xt (ω; K) zi (ω; K) 0 , ω ∈ X



2 

s
s
e
e
e
e
zi (ω; K) 0 , ω ∈ X .
EP,Ke xt (ω; K) − EP,Ke xt (ω; K) zi (ω; K) 0 , ω ∈ X
e

Thus, we require equality of the two sets of covariances of filtering/prediction/smoothing
errors, corresponding to any two choices of control strategies. In the definition above, by
e as resulting in ut = 0, for all t, we obtain the
choosing one set of control policies, say K
definition of Bar-Shalom and Tse [7].
3.2. Certainty equivalence. For the controlled Markov process (3), consider the general
cost
i
h 
J general = E L {xi }T1 −1 , {ui }T0 ,

where L is a given non-negative cost function. Imagine that a muse could at time t supply to
the controller the exact values of all primitive random variables by informing the controller
the exact element ω of the sample space Ω. With such complete and acausal information, the
controller could, in principle, solve the deterministic optimization problem


T
inf Jt (u; ω) = inf L {xi (ω)}T0 , {ui (ω)}t−1
,
u
,
{u
(ω)}
i
0
t+1 .
u

u

u∗t

(ω) be an optimal control law for this deterministic optimization problem. We now
Let
state the definition of certainty equivalence from van der Water and Willems [46]:
Definition 2. A certainty equivalence control law for the plant (1) with the performance
cost (2) has the form:


E u∗t (ω) {zi (ω)}t0 , {ui (ω)}t−1
.
0

Clearly, this law is causal. Notice also that its form is tied to the performance cost, and to
the statistics of the state and observation processes. It is possible for certainty equivalence
control laws to be nonlinear, and such laws can be optimal even when separated designs may
not be. For linear plants, they can sometimes be linear or affine, as indicated by the following
proposition from [46] adapted to our problem.

Lemma 1 (Affine certainty equivalence laws for linear plants). For the plant (3), with
Φt = axt + ut + wt , and the quadratic performance cost (2) with J Comm = 0, the following are
certainty equivalence laws:




t
t−1 
t
t−1 
CE
CE
a · E xt {zi }0 , {ui }0
+ E wt {zi }0 , {ui }0
,
ut = − kt

where the gains kiCE =

βi+1
,
q+βi+1

αi = βi+1 + αi+1 , βi = p +

a2 qβi+1
,
q+βi+1

αT +1 = 0, βT +1 = 1.

Definition 3 (Certainty equivalence property). The certainty equivalence property holds
for a stochastic control problem if it is optimal to apply the certainty equivalence control law.
For the stochastic control problem described in Lemma 1, with non-linear measurements
that do not result in a dual effect of the controls, Bar-Shalom and Tse [7] showed that the
certainty equivalence property holds.
We now consider a simple example, and show that there is a dual effect of the control
signal in the closed-loop system presented in Section 2.

8

M. RABI, C. RAMESH, AND K.H. JOHANSSON

0.3
Quantization distortion Γ
Performance cost J

0.2

0.1

0
−4

u∗
0

uCE
0

−2

0

2

u0

Figure 3. Plot of quantization distortion and performance cost for Example 1
Example 1. For the plant (1), let a = 1, x0 = 2, and σ0 = 0. Let this information be
known to the encoder and the controller, which simply means that z0 = x0 . Let the variance
σw2 = 0.72 . For the objective function, let the horizon end at T = 1, and let p = q = 0.01. Let
the channel alphabet be the discrete set {1, 2, 3}.
For the given threshold θ = 1.6, let the encoder at t = 1 be:


1 if x1 ∈ (−∞, −θ) ,
(4)
ξ1 (x1 ) = 2 if x1 ∈ (−θ, θ) ,

3 if x ∈ (θ, +∞) .
1
a
The optimal control law at t = 1 is u1 = − q+1
x
b 1|1 , where x
b 1|1 = E [x1 |x0 , u0 , z1 ]. Using the
encoding policy ξ1 and the optimal control signal u1 , the performance cost with J Comm = 0 can
be written as a function of the control at t = 0:
, Γ

}|
{
z


h
i
 2

2
qa2
a2
2
2
J(u0 ) = σw + qu0 + p +
E x1 |x0 , u0 +
E x1 − x
b m|1 x0 , u0 , z1
q+1
q+1

In the above expression, Γ is the quantization distortion, which is thus proportional to the
conditional variance of the controller’s minimum mean-squared estimation error of x1 . Notice
that Γ is a function of u0 , thus resulting in a dual effect of the control signal in the plantencoder-channel combination. Figure 3 shows how the quantization distortion Γ depends on
u0 . The total cost J is also plotted and the optimal value u∗0 is shown to be different from the
certainty equivalent control uCE
0 .
4. Dynamic encoder-controller design
In this section we solve the dynamic encoder-controller design problem (Design problem 2)
which allows both controls and encoders to be dynamic. We work out the details for the
discrete alphabet channel with the fixed alphabet size N . We begin by examining a known
structural property of optimal encoders. This states that it is optimal for the encoder to apply
a quantizer on the state xt , with the shape of the quantizer depending only on past quantizer
outputs. Next, we present a structural property for encoders called controls-forgetting, which
leads to separation. Finally, we show that one optimal encoder for Design problem 2 does
indeed possess this property, which leads to separation and certainty equivalence for this
problem.

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

9

4.1. Known structural properties of optimal encoders. Let us now formulate the
encoder’s Markov decision problem. Fix the control policies to be the arbitrary, but admissible
laws:


t
†
ut = Kt {zi }0 .

Then the optimization problem reduces to one of picking encoding policies. This is a singleagent, sequential decision problem, and hence one with a classical information pattern. The
action space for this decision problem is the infinite dimensional function space of discretevalued encoders. At time t, the encoder takes as input: the current and previous states, all
previous outputs, and all previous encoding maps. For convenience, we can view this encoding
map as a function of only the current state but with the rest of the inputs considered as
parameters determining the form of this function. Thus, without loss of generality the encoder
can be described as the function
ξt (·) : R → {1, . . . , N }



t−1
t−1
. Hence
having xt as its argument with its shape determined by {xi }t−1
{z
}
{ξ
(·)}
i 0
i
0
0
n
o
the action space at times t can be described as: ξ (·) : R → {1, . . . , N } , Borel measurable .
Identifying encoders as decisions to be picked is not enough, as the signal xt need not be
Markov. We utilize the following property.
Lemma 2 (Striebel’s sufficient statistics). For every design problem we have set up, the
signals
xt , {zi }t0 , {ξi (·)}t−1
0

form sufficient statistics for the encoding decision at time t.
Proof. See Striebel [44].



Hence, at every time t, performance is not degraded by the encoder choosing to quantize
just xt instead of quantizing the entire waveform {x0 , . . . , xt }. Of course the shape of the quantizer is allowed to vary with past encoder shapes, past encoder outputs, and on past control
inputs. But given the sufficient statistics, the encoder can forget the data: {x0 , . . . , xt−1 }.
Denote by Dcon
t− the data at the controller just after it has read the channel output zt and
just before it has generated the control value ut . Similarly denote by Dcon
t+ the data at the
controller just after it has generated the control value ut . Then
n
o
t
t
t−1
Dcon
=
{z
}
,
{ξ
(·)}
,
{u
}
,
−
i
i
i
t
0
0
0
o
n
o
n
t
t
t−1
con
,
{u
}
,
u
,
{ξ
(·)}
Dcon
=
D
,
u
=
{z
}
t .
t
i 0
i
i 0
t+
t−
0


Also let x
b t|t = E xt Dcon
.
t−
The problem we consider has two decision makers that jointly minimize a given cost function. The information available to these decision makers is not the same, and neither is the
information available to each agent a subset of the information available to the agent downstream in the loop. Thus, the information pattern here is neither classical nor nested. We
apply the common information approach 1 to our problem. This approach allows a designer
to treat a problem with multiple decision makers as a classical control problem with a single
decision maker that has access to partial state information. When applied to our setup, this
approach leads to the following structural result at the encoder. The encoding policy ξt (·)
is selected based on the information available to the controller at the previous time instant
1This

approach was first proposed by Witsenhausen, as a conjecture in [51], to deal with multiple decision
makers and non-classical information patterns in a general setting. This conjecture was shown to be true by
Varaiya and Walrand in [47] for a special case. Our terminology is derived from [36], where the conjecture
has been studied in detail.

10

M. RABI, C. RAMESH, AND K.H. JOHANSSON

namely Dcon
. At times t− , t+ respectively, the data Dcon
, Dcon
comprise the common
(t−1)+
(t−1)−
(t−1)+
information in this problem. The encoding map ξt (·) is applied to the state xt , which is
private information available to the encoder. A similar approach has been used by others for
problems of quantized control [12, 49, 55].
4.2. Controls-forgetting encoders and separation. We now present a structural
property of encoders which ensures separation in design. Recall the plant (1) and cost (2),
and define the following control free part of the state:
ζ0 = x0 ,
ζi+1 = xi+1 −

i
X
j=0

ai−1 uj for i ≥ 0.

At the encoder, the change of variables




T
(5)
;
{K
(·)}
xt , {zi }0t−1 ; {Ki (·)}T0 7−→ ζt , {zi }t−1
i
0
0


T
are also sufficient
is causal and causally invertible. Hence the statistics ζt , {zi }t−1
;
{K
(·)}
i
0
0
statistics at the encoder. We now introduce the innovation encoding of Borkar and Mitter [10].
Definition 4 (Innovation encoder [10]). An encoder with the inputs and outputs:


T
ζt , {zi }t−1
;
{K
(·)}
7−→ ιt
i
0
0

is admissible and is called an ‘innovation’ encoder.

The networked control system in Figure 1 redrawn with an innovation encoder is shown in
Figure 4. Note that with innovation encoding, the control free part of the state is not affected
by the control policies, but obeys the recursion
ζt+1 = aζt + wt .
For any sequence of causal encoders, one can find an equivalent sequence of innovation encoders
such that when these two sets operate on the same sequence of plant outputs, they produce
two sequences of channel inputs that are equal with probability one. Hence, if for a plant and
channel, the dual effect is present in a certain class of causal encoders, then the dual effect
is also present in the equivalent class of innovation encoders [15]. This is what the following
example illustrates:
Example 2 (Dual effect in a loop with fixed innovation encoder). We use the same setup as
in Example 1 with the encoder replaced by an innovation encoder. For the given threshold θ =
1.6, let the encoder at time t = 1 be the following innovation encoder:


1 if aζ1 + K0 (z0 ) ∈ (−∞, −θ) ,
inn
(6)
ξ1 (ζ1 ) = 2 if aζ1 + K0 (z0 ) ∈ (−θ, θ) ,

3 if aζ + K (z ) ∈ (θ, +∞) .
1
0
0

a
The optimal control law at t = 1 is still u1 = − q+1
x
b 1|1 , where x
b 1|1 = E [x1 |x0 , u0 , z1 ]. For
the control u0 , notice that (4) and (6) tell us that this innovation encoder ξtinn is equivalent
to the causal encoder ξt of Example 1. For the same applied control policy K0 , and for the
same realizations of primitive random variables, we get ξ1inn (ζ1 (ω)) = ξ1 (x1 (ω)). Hence, with
probability one the two nonlinear filters for the state given x0 , z1 are the same. Thus for an
event X ∈ F (x0 ,z1 ) , we have:


P x1 ∈ X x0 , z1 = ξ1inn (ζ1 ) = P [x1 ∈ X |x0 , z1 = ξ1 (x1 )] .

Hence the results in Figure 3 apply also to this example.

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

11

Encoder

ut

Plant

xt

ζt
−


a

 x0 , 


t−1
t−1  inn
T 
inn 
2 
ξ t  · ; {zi }0 , ξi (·) 0 , {Ki (·)}0 , 
 σ20 

 σw 
p, q




zt

Equivalent encoder

Delay

a

Exact replica of controls


t
Kt {zi }0



t
ut = Kt {zi }0

zt

Controller

Figure 4. The block diagram of Figure 1 with innovation encoding
The encoder (quantizer) in the loop causes the dual effect. Furthermore, the encoder’s
presence renders useless the techniques that worked in the case of the classical, single-agent,
partially observed LQ control problem. The next example illustrates this.
Example 3. We examine a scalar system as it evolves from time step 0 to time step 1.
We have: x0 ∼ N (µ0 , σ0 ) ,
x1 = x0 + u 0 + w 0 ,
where w0 is the process noise variable which is independent of x0 , u0 , and w0 ∼ N (0, σw ) . We
adopt the specific quantizing strategy given below (on the left in the form of a encoder for xt ,
and on the right, in the equivalent, innovation form):
(
(
−1, if x0 ≤ 0,
−1, if ζ0 ≤ 0,
ξ0 (x0 ) =
ξ0inn (ζ0 ) =
+1, if x0 > 0,
+1, if ζ0 > 0,
(
(
−1, if x1 ≤ 0,
−1, if ζ1 ≤ −u0 (z0 ) ,
ξ1 (x1 ) =
ξ1inn (ζ1 ) =
+1, if x1 > 0,
+1, if ζ1 > −u0 (z0 ) .
Since the encoder at time 0 is binary, the general control law at time 0 has the form:
(
α, if z0 = −1,
u0 (z0 ) =
β, if z0 = +1,
where α, β are arbitrary real numbers. The process x
b t|t is fully observed at the controller. We
have x
b 0|0 = E [x0 |z0 ], and as noted in [56], one can write:

(7)

x
b 1|1 = x
b 0|0 + u0 + w0 ,

where the noise-like random variable w0 is given by: w0 , E [x1 |z0 , z1 ] − E [x1 |z0 ] . Then one
can treat the problem as the control of the fully observed process x
b t|t to minimize the given

12

M. RABI, C. RAMESH, AND K.H. JOHANSSON

µ0 = 0.5, σ0 = σw = 1.
µ0 = 0, σ0 = 1, σw = 0.1.

|z0 = −1

i

µ0 = 0.5, σ0 = σw = 1.
µ0 = 0, σ0 = 1, σw = 0.1.

0.8

1.5

2

1

0.6

x1 − x
b1|1

E [w20 |z0 = −1]

1

0.4

0.5

0
−6

E

h

0.2
−4

−2

0

α

2

4

6

0
−6

−4

−2

0

2

4

6

α

Figure 5. Plots for example 3.
cost, which can be rewritten as the following sum of two terms:
h
i
h
2
2 i
(8)
J =E x
b21|1 + p · x
b20|0 + p · x0 − x
b0|0 + qu20 + E x1 − x
b1|1
.

Such a treatment actually works for the case of the classical, single-agent partially observed
LQ control problem. There two special things happen: (1) the random process {wt } is statistically independent of the control process {ut } and of the ‘state’ process {b
xt|t }, and (2) because
the dual effect is absent, the second term on the RHS of 8 does not vary with {ut }. Therefore,
by considering {b
xt|t } as the process to be controlled, we get a single-agent, fully observed LQ
control problem.
In the two-agent problems considered in this paper, neither of the above-mentioned special
things may happen. For this specific example, we have calculated, and then plotted in Figure 5
how the second moments of w0 and x1 − x
b1|1 vary with u0 . The calculations are presented in
Appendix A.

Next we define a class of encoders for which at prescribed times t, the statistics of wt ,
xt+1 − x
bt+1|t+1 are independent of the control ut .

Definition 5 (Controls-forgetting encoder). Denote by ρζτ |τ −1 (·) the conditional density
of ζτ given the data Dcon
. An admissible encoding strategy is controls-forgetting from time τ
(τ −1)−
if it takes the form:
 


 ξt† xt ; Dcon − ,
if t ≤ τ,
(t−1)


ξtCF, τ xt ; Dcon
=
−
(t−1)
t−1
t ζt ; ρζ
, {i (·)}t−1
, if t ≥ τ + 1,
τ
τ |τ −1 (·) , {zi }τ

where (1) ξt† ·; Dcon
is any admissible policy for encoding at time t, (2) for t ≥ τ + 1 the
−
(t−1)

policies t ·; ρζτ |τ −1 (·) , {zi }τt−1 , {i (·)}τt−1 are adapted to the data


t−1
t−1
τ
ζ
ρ
(·)
,
{z
}
,
{
(·)}
⊂ Dcon
, for t ≥ τ,
DCF,
+ =
i
i
τ
τ
τ |τ −1
(t−1)+
(t−1)

τ
and (3) for fixed values of the data DCF,
, the map t (·) produces the same output regardless
(t−1)+

of both the controls {ui }tτ and the control policies {Ki (·)}Tt+1 .
Clearly such controls-forgetting encoders exist. For example, consider a set of encoders that

2 
P
quantize in sequence ζτ +1 , . . . , ζT to minimize the estimation distortion Ti=τ +1 E ζi − ζbi|i ,


τ
where ζbi|i = E ζi | DCF,
+ . Let the non-negative function ψ (·) represent some notion of cost.
(i−1)
For example, ψ (x) := x2 .

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

13

Lemma 3 (Distortions incurred by controls-forgetting encoders also forget controls). Fix
the time t = τ and the distortion measure ψ. If the encoder
iscontrols-forgetting from time τ ,
 con
then for times i ≥ τ + 1, the distortions E ψ xi − x
bi|i Di− are statistically independent of
T
the partial set of controls {ui }i=τ .
Proof. The unconditional statistics of {ζt } are independent of the entirecontrol wave-
form, no matter what the encoder is. For times i ≥ τ +1 and for sets X ∈ Fiz , P ζi ∈ X| Dcon
i−
is independent of {ui }Ti=τ because the encoding maps ξi are controls-forgetting from time τ .
Since ζt − ζbt|t = xt − x
bt|t , for all t, the lemma follows.


Definition 6 (Controls affine from time τ ). A controller affine from time τ takes the
following form:
(


u†i ,
if i < τ,
mult, τ
con
(9)
Ki
Di− =
aff
ui = ki x
bi|i + di , if i ≥ τ,

where the controls u†i are generated by an admissible strategy {Ki† (·)}Ti=0 , the controls uaff
i are
aff
T
T
T
generated by an affine
 strategy
{Ki (·)}i=0 , with the gains {ki }0 and offsets {di }0 computed
i
offline, and x
bi|i = E xi | {yj }j=0 .
4.3. Preliminary lemmas. The main result ahead is Theorem 1 that states that it is
optimal for Design problem 2 to apply a separated design and certainty equivalence controls.
In this subsection, we do some necessary ground work towards proving that result.
Once we are prescribed an admissible encoder, the controls {uj }Tj=i affect only the cost-to


 P
go: E x2T +1 + Tj=i E px2j + qu2j . In the classical single agent LQ problem, the ‘prescribed
encoder’ is simply the linear observation process with prescribed signal-to-noise ratios. There,

T
this cost-to-go can be expressed as a quadratic function of {uj }Tj=i , {xj }Tj=i and x
bj|j j=i .
But in our two agent LQ problem, because of the dual effect, the cost to go may have a
non-quadratic dependence on the controls {uj }Tj=i . However we show that by restricting to
controls-forgetting encoders and affine controls, the cost-to-go does get a quadratic dependence
on controls. We use this reasoning and dynamic programming to show that for time t = i
going backwards from T the following conclusions fall out:
• it is optimal at time t = i to apply as control a linear function of x
bi|i , and,
• it is optimal at time t = i to apply an encoding map that is controls-forgetting from
time i − 1.
Lemma 4 (Optimal control at time t = T ). The optimal control
t = T is the 
 policy at time
a
∗
con
2
∗
linear law: uT = − 1+q x
b T |T , and the optimum cost-to-go VT DT − = minut E xT +1 + qu2T Dcon
T−
is the expected value of a quadratic in xT and x
b T |T .
Proof. At time T − , one is given Dcon
T − , and is asked to pick uT to minimize the cost-to-go

 2
2
con
VT (uT ; Dcon
T − ) = E xT +1 + quT |DT − ,


,
= σw2 + E a2 x2T + 2 a xT uT + (1 + q) u2T |Dcon
−
T

2
h
2 con i
a2
a
2
2
= σw +
E q xT + xT − x
b T |T DT − + (1 + q) uT −
x
b T |T ,
1+q
1+q

and this lets us prove the Lemma.



Lemma 5 (Optimal ξi for separated, quadratic cost-to-go). Fix the time t = i. Consider
the dynamic encoder-controller design problem (Design problem 2), for the linear plant (1),
e along with
and the performance cost (2). Suppose that we apply an admissible controller K
CF,i
an encoder ξt
that is controls-forgetting from time i. Furthermore, suppose that the partial

14

M. RABI, C. RAMESH, AND K.H. JOHANSSON

sets of policies:
n
o
CF,i
CF,i
ξi+1 (·) , . . . , ξT (·)
n
o
e i (·) , K
e i+1 (·) , . . . , K
e T (·)
K

are chosen such that the following three properties hold:

(1) the cost-to-go at time i takes the separated form:
"
#
T
T
X
X
con
E x2T +1 + p
= E [Jicon (ui , xi )| Dcon
x2i + q
u2i Dcon
i+ ] + E [Γi+1 | Di+ ] ,
i+
j=i

j=i

where, Jicon (ui , xi ) = α + α σw2 + β xi + βe x2i + ν x
b i|i + νb xi x
b i|i + νe x
b2i|i , and the term
Γ is a weighted sum of future distortions and depends only on the random sequence
i+1
T
xj − x
bj|j j=i+1 ,

e j (·) T but
(2) the coefficients of the quadratic Jicon may depend on the control policies K
i

T
not on the partial set of encoding maps ξjCF,i (·) i+1 , and,

T
(3) the term Γi+1 depends on the encoding maps ξjCF,i (·) i+1 but not on the partial set

e j (·) T .
of control policies K
i

Then, it is optimal to apply an encoding map at time t = i that does not depend on the data:



e j (·) T . It also follows that the shapes of the encoding maps ξ CF,i (·) T and their
ui−1 , K
j
i
i+1
performance do not depend on the control ui−1 .

Proof. The proof exploits three facts: Firstly the special form of Jicon (ui , xi ) makes the
encoder’s performance cost at time i a sum of a quadratic distortion between xi and x
b i|i , and a
term gathering distortions at later times. Secondly the minimum of the sum distortion depends
only on the intrinsic shape of the conditional density ρ i|i−1 (·) and not on its mean. Thirdly,
these facts and the controls-forgetting nature of later encoding maps allows the encoder to
‘ignore’ the control ui−1 . We now start by writing the cost-to-go as:
i
h
2
2
con
e x2 + ν x
α
+
α
σ
+
β
x
+
β
b
+
ν
b
x
x
b
+
ν
e
x
b
D
E [Jicon (ui , xi ) + Γi+1 | Dcon
]
=
E
+
+
i
i
i|i
i|i
w
i
i|i
i
i

(10)

+ E [Γi+1 | Dcon
i+ ] ,
h


i

= α + α σw2 + E β + ν xi + νb + νe + βe x2i Dcon
+
i
 2

2
con
con
− (b
ν + νe) E xi − x
b i|i Di+ + E [Γi+1 | Di+ ] ,
h


i

= α + α σw2 + E β + ν xi + νb + νe + βe x2i Dcon
+
i
h
2 con i
− (b
ν + νe) E xi − x
b i|i Di+ + E [Γi+1 | Dcon
i+ ] .

Given the data Dcon
the part of the cost above that depends on the encoding map ξi (·) is
(i−1)+
i
h
2
− (b
ν + νe) E xi − x
b i|i Dcon
+ E [Γi+1 | Dcon
i+
i+ ] .

Notice that the first term is the quantization variance of the quantizer ξi (·). This reduction
of the encoder’s performance cost to a sum of current and future quantization distortions
is possible because the term Jicon (ui , xi ) has been assumed to be quadratic in xi and x
b i|i .
The reduced performance cost of the encoder
is a function only of the quantizer ξi (·) and

con
the conditional density ρ i|i−1 x D(i−1)− . Indeed, given the data Dcon
this cost is the
(i−1)−

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

15

following average:

i n h
io

X h
con
con
con
=
P
x
∈
∆
D
,
u
·
E
Γ
(D
)
D
,
x
∈
∆
Γi ξi (·) ; Dcon
i
i−1
i+1
i
i−
(i−1)−
(i−1)+
(i−1)+
cells ∆

+

X

cells ∆

h

P xi ∈ ∆

, ui−1
Dcon
(i−1)−

i n
h
io
2 con
· λ E xi − x
b i|i D(i−1)+ , xi ∈ ∆ ,

where λ = − (b
ν + νe). The cost Γi does depend on both ξi (·) and ui , but for given data Dcon
(i−1)−
and control ui−1 , the minimum of Γi over all admissible quantizers ξi (·) may possibly depend
but not on the control ui−1 . To see this consider two arbitrary possible values u, u
e
on Dcon
(i−1)−
for ui−1 . Suppose that one is given the quantizer


1 if x ∈ (−∞, δ1 ) ,



2 if x ∈ (δ1 , δ2 ) ,
ξ (x) = .
..
..

.



N if x ∈ (δ
N −1 , +∞) ,


meant for quantizing a random variable with the density ρ i|i−1 x Dcon
− , ui−1 = u . Con(i−1)

e
sider the quantizer ξ constructed by taking each cell ∆ = δ, δ in ξ, and generating a new

e = δ−u+u
e
cell ∆
e, δ − u + u
e , and stipulating that the new quantizer ξe assigns to the cell ∆
the same channel input that the quantizer ξ assigns to ∆.
Because of the linear evolution: xi = axi−1 + ui−1 + wi−1 , and because the random variable
wi−1 is independent of the data Dcon
, we have the convolution relations:
(i−1)+


·−u
ρ (x) = ρ i|i−1
~ ρw (·) and,
a
x


·−u
e
ρe (x) = ρ i|i−1
~ ρw (·) ,
a
x
leading to the following symmetry w.r.t. translations:




con
con
(11)
ρ i|i−1 x − u D(i−1)− , ui−1 = u = ρ i|i−1 x − u
e D(i−1)− , ui−1 = u
e .
e
Then we get the following equalities for each pair of cells ∆, ∆
h
i
h
i
con
con
e
P xi ∈ ∆ D(i−1)− , ui−1 = u = P xi ∈ ∆ D(i−1)− , ui−1 = u
e ,




con
e
Γi+1 Dcon
=
Γ
D
,
u
=
u
e
,
x
∈
∆
, and,
− , ui−1 = u, xi ∈ ∆
−
i+1
i−1
i
(i−1)
(i−1)
h
i
h
i
2 con
2 con
e
E xi − x
b i|i D(i−1)− , ui−1 = u, xi ∈ ∆ = E xi − x
b i|i D(i−1)− , ui−1 = u
e, xi ∈ ∆ .

Then the performance of any quantizer ξ designed for ui−1 = u can be matched by ξe for
ui−1 = u
e, and vice versa. Hence, we can conclude that for any u, u
e,




con
inf Γi ξ (·) ; Dcon
=
inf
Γ
ξ
(·)
;
D
e
,
.
− , ui−1 = u,
− , ui−1 = u
i
(i−1)
(i−1)
ξ

ξ

Notice that this optimal encoder now become controls-forgetting from time i − 1.



As the optimal control u∗T is a linear function on x
bT |T , the encoder ξT begets a performance
cost that is quadratic in xT , x
bT |T . Then the above Lemma renders the optimal encoding map
ξT∗ to be controls-forgetting from time T − 1. This reduction also holds at earlier times.

Lemma 6 (Encoder separation for affine controls). If the two conditions hold: (A) for any
admissible control strategy, an admissible encoder strategy minimizing the performance

cost (2)
mult,τ
con
exists, and (B) we apply as control strategy one affine from time τ : Ki
Di+ (from

16

M. RABI, C. RAMESH, AND K.H. JOHANSSON

defn. 6), then the following two conclusions hold: (a) an encoder that is controls-forgetting
from time τ minimizes the partial LQ cost:
"
#
T
T
X
X
E x2T +1 + p
x2i + q
u2i Dcon
τ+ ,
i=τ +1

i=τ

and, (b) the shapes of then minimizing encoding maps
from time τ and their performance are
o
†
T
T
independent of the data: uτ −1 , {ki }i=τ , {di }i=τ .
Proof. We prove by mathematical induction. For a given control strategy, define:
h
i
2
2
2
con
WT = E xT +1 + pxT + quT D(T −1)+ ,
WT∗ = inf WT ,
ξT (·)
h
i
h
i
∗
con
Wi = E px2i + qu2i Dcon
+ E Wi+1
(Dcon
Wi∗ = inf Wi .
i+ ) D(i−1)+ , ξi (·) ,
(i−1)+
ξi (·)

Induction hypothesis for time i. For some time t = i such that τ ≤ i < T , we have the

following three assumptions: (1) for every j ≥ i + 1, the optimal value function Wj∗ Dcon
(j−1)−
takes the form:
h

i
h



2 con i
∗
con
con
e
e
αj σw2 + αj + βej E xj 2 Dcon
+
β
x
b
+
E
+
λ
E
x
−
x
b
Γ
D
D
Dj − ,
−
−
−
j
j
j|j
j j|j
j+1
j
j
(j+1)

ej are known non-negative real numbers for j ≥ i+1, (2) for each such
where the αj , αj , βej , β j , λ


con
∗
e
j, the non-negative function Γj+1 Dj − is assumed to be independent of the partial waveform

T
{uj , uj+1 , . . . , uT }, and (3) the optimal partial set of encoding maps ξj∗ (·) i+1 is a set that is
controls-forgetting from time i.
We will now show: if this hypothesis holds for time i, then it holds for time i − 1. Assuming

T
that the partial set of optimal encoding maps ξj∗ (·) i+1 are employed, we get:
h
i
h
i
2
2
con
∗
con
con
Wi = E p xi + q ui D(i−1)+ + E Wi+1 (Di+ ) D(i−1)+ , ξi (·) ,
h
i
h
i
h
i
2
con
2
2
con
e
= p E x2i Dcon
+
q
E
u
D
+
α
σ
+
α
+
β
E
x
D
+
+
−
i+1 w
i+1
i+1
i+1
i
(i−1)
(i−1)
(i+1)
h
i
h


i
e∗i+1 Dcon − Dcon
+ β i+1 E xi+1 Dcon
+E Γ
,
i−
(i+1)−
(i+1)


+ β i E [xi |Dcon
= αi σw2 + αi + βei E x2i |Dcon
i−
i− ]
h


i
h
2 con i
con
con
e
e∗
+E Γ
D
D
+
λ
E
x
−
x
b
|Di− ,
i
i
i|i
i+1
i−
(i+1)−

where, the coefficients:

αi = αi+1 + βei+1 ,


β i = 2 q ki di + aβei+1 di + βei+1 ki di ,

αi = αi+1 + βei+1 d2i + q d2i + β i+1 di ,

βei = pi + a2 βei+1 + ki2 βei+1 + 2aki βei+1 + q ki2 βei+1 ,

ei = q k 2 + k 2 βei+1 + 2aki βei+1 .
λ
i
i


We have thus: Wi = E A quadratic in xi , x
bi|i + E [ Future distortions ] . This and the fact
that the encoder is controls-forgetting from time t = i meet the requirements of Lemma 5.
Then we get the optimal encoding map ξi∗ to be controls-forgetting from time t = i − 1, and
h


i
h
2 con i
∗
con
con
e
e
e
Γi = min E Γi+1 D(i+1)− Di− + λi E xi − x
b i|i |Di−
ξ

is independent of the partial set of controls {uj }Tj=i−1 . From this it follows that the induction
hypothesis is also true for time i − 1.


SEPARATED DESIGN FOR NETWORKED LQ CONTROL

17

Lemma 7 (Certainty equivalence controls for controls-forgetting encoders). Fix the switch
time τ . If the encoder is preassigned to be one that is controls-forgetting from time τ , then the
partial LQ cost
"
#
T
T
X
X
2
2
2
con
E xT +1 + p
xi + q
ui Dτ − ,
i=τ +1

i=τ

b i|i .
is minimized by the following control laws with a linear form: For i ≥ τ : u∗i = ki∗ x
i
h

.
Proof. Define the following cost-to-go at time t = T −1: VT −1 = E WT T (·) ; Dcon
(T −1)+
Because of Lemma 4,

 h
i
h
2 con i
a2 q
2
VT −1 = σw + p +
+
E
x
−
x
b
E x2T Dcon
DT − .
− , uT −1
T
T |T
(T −1)
q+1
Because the encoder is controls-forgetting from time τ , the last term, which is the distortion
due to the encoder ξT , is independent of the partial set of controls {ui }Ti=τ +1 . Hence the only
part of VT −1 that depends on the control uT −1 is the quadratic

 h
i
a2 q
2
qu + p+
E x2T Dcon
− , uT −1
(T −1)
q+1


i
o
a2 q n 2 h 2
2
2
a E xT Dcon
+
2a
x
b
u
+
u
+
σ
.
= q u2 + p +
− , uT −1
T −1|T −1 T −1
T −1
w
(T −1)
q+1
Hence the best control law is: u∗T −1 = −
VT∗−1



a2 q
a p+ q+1
2

x
bT −1|T −1 , and the resulting value function:

a q
q+p+ q+1


a2 q
h
a2 q p+ q+1



i
a2 q
2
con
= 1+p+
σw2 +
E
x
D
,
u
−
2
T
−1
T
a q
(T −1)
q+p+ q+1
q+1


a2 q
h
h
a2 p + q+1
2 con i
2 con i
+
E
x
−
x
b
D
+
E
x
−
x
b
DT − .
T −1
T
T −1|T −1
T |T
(T −2)+
a2 q
q + p + q+1

Repeating this procedure backwards in time, we get for times i ≥ τ , the optimal control laws
a2 qβi+1
βi+1
,
β
=
p
+
, and, βT +1 = 1.

are: u∗i = −ki∗ x
bi|i , where ki∗ = a q+β
i
q+βi+1
i+1

4.4. Main theorem. Lemma 6 implies that for a pre-assigned controller affine from time
zero, there exist optimal encoding maps that are controls-forgetting from time zero. Lemma 7
is complementary. It implies that for a pre-assigned encoder that is controls forgetting from
time zero, the optimal control laws have linear forms.
For Design problem 2 an optimal pair of strategies have a similar simplified structure.
It is optimal to apply a combination of controls-forgetting encoding and control laws linear
in x
bi|i . In general, this controls-forgetting encoder does not minimize the aggregate squared
estimation error. The goal accomplished by an optimal encoder is slightly different. It is to
minimize a sum of state estimation errors with the time-varying weights λi .
Theorem 1 (Optimality of separation and certainty equivalence). For Design problem 2,
with the discrete alphabet channel of constant alphabet size, the quadratic performance cost (2)
is minimized by applying the linear control laws
(12)

u∗t = −kt∗ x
b t|t

in combination with the following encoder which is controls-forgetting from time 0:


t−1
t−1 
t−1
t−1
∗
(13)
t ζt ; {zi }0 , {i (·)}0
= arg inf Γi  (·) ; {zi }0 , {i (·)}0 ,
(·)

18

M. RABI, C. RAMESH, AND K.H. JOHANSSON
2

a2 β 2

βi+1
qβi+1
where, ki∗ = a q+β
, βi = p + aq+β
, βT +1 = 1, and λi = q+βi+1
and where,
i+1
i+1
i+1


2


Γt = λt E ζt − ζbt|t
+ E Γ∗t+1 x0 , σ02 , {zi }t0 , {i (·)}t0 ,
t (·) , Dcon
(t−1)+


2
T
−1
T
−1
2
,
ΓT = E ζT − ζbT |T
T (·) , x0 , σ0 , {zi }0 , {i (·)}0

Γ∗t = inf Γt () .
(·)

Moreover, this control law is a certainty equivalence law.
Proof. Starting with the result of Lemma 4 as a seed, repeatedly apply in sequence
Lemmas 6, 7. This proves optimality of the above combination. Lemma 1 implies that the
controls laws of (12) are indeed certainty equivalence control laws as per van der Water and
Willems [46].

The optimal controller splits into a least square estimator computing x
b t|t and a timedependent gain. Computing x
b t|t is intrinsically hard because quantization is a nonlinear
operation. If one ignores this computational burden, then, at least formally, the optimal
controller resembles that for the classical LQG optimal control problem.
Note that in general the sequence of weights {λi }T0 depends on the parameters of the
performance cost including the control penalty coefficient q. In the two special cases:
(1) the coefficients q = 0, p = 1, or
(2) the quantity p + a2 q − q > 0 and the following equality holds:
q
2
p + a q − q + (p + a2 q − q)2 + 4pq = 2,
2

a
it turns out that the weights βi ≡ 1 ∀i, and hence the weights λi ≡ q+1
∀i. Thus in these
special cases, optimal encoders ‘ignore’ the parameters of the performance cost and simply
minimize the usual aggregate squared error in state estimation.

4.5. Extension to the multivariable case. Theorem 1 can be extended to situations
where the state, control, and noise signals are vectors, as well as where the objective function (2) includes cross terms involving the state and control. We can also extend to the case
where the sensor has access only to partial and noisy observations of the state. To carry out
these extensions, we need no more than the standard arguments of LQG control. Below, we
mention only the key steps corresponding to the lemmas of Section 4.3. Consider a partially
observed, linear multivariable plant:
(14)

xt+1 = Axt + But + Ewt , yt = Cxt + Dvt ,

where the state xt ∈ Rn , the control ut ∈ Rm , the output yt ∈ Rp , the process noise wt ∈ Rl1 ,
and the measurement noise vt ∈ Rl2 . Let the two noise sequences {wt } , {vt } be IID sequences
that are mutually independent of each other.
For any matrix M , let M 0 denote its transpose. For all times, let E[wt wt0 ] = Σw and
E[vt vt0 ] = Σv . Let the performance objective be defined as


  
T
 0
 X
 P R 0 xi
0
0
Jgeneral = E xT +1 PT +1 xT +1 +
E xi ui
(15)
,
R Q
ui
i=0


P R0
where P and
are symmetric and positive semi-definite, and Q is symmetric and
R Q
positive definite. It is easy to see that an extension of Lemma 4 holds for the multivariable
case. Precisely, the optimal control law at the terminal decision time T is u∗T = −KT x
bT |T

−1
0
0
∗
con
where KT = −(B PT +1 B + Q) (R + B PT +1 A) and
the
optimal
cost-to-go
V
D
−
T
T

 0
=
0
0
con
0
con
trace(EΣw E ) + E (xT − x
bT |T ) MT (xT − x
bT |T ) DT − + E xT (P + A PT +1 A − MT )xT DT − ,

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

19

where the matrix MT = KT0 (B 0 PT +1 B + Q)KT . Next, the following generalization of Lemma 5
can be proved.
Lemma 8 (Multivariable version of Lemma 5). Assume the hypothesis of Lemma 5 but for
the multivariable, partially observed plant (14), the objective function (15), and the following
new definition of Jicon :

 
b0
 Pb R
xi
0
0
con
0
0
bi|i
Ji = α + β xi + βb x
bi|i + xi x
.
b
b
x
b
i|i
R Q

Then, it is optimal to apply an encoding map at time t = i that does not depend on the data:


e j (·)}T . It also follows that the shapes of the encoding maps ξ CF,i (·) T and their
ui−1 , {K
j
i
i+1
performance do not depend on the control ui−1 .

Proof Sketch: We can rewrite the part of the cost-to-go that depends on on the control ui−1 .
As in (10), it is possible to rewrite this in such a way that the only dependence on x
b i|i is through
a quadratic form of the estimation error xi − x
b i|i :


0


0
con
con
0
con
b+R
b +Q
b xi D +
E [Ji + Γi+1 | Di+ ] = α + E β + βb xi + xi Pb + R
i
h
i

0 

b+R
b0 + Q
b xi − x
+ E xi − x
b i|i R
.
b i|i + Γi+1 Dcon
+
i

The part of the RHS that depends on ξi is:
h

0 
 con i
0
b
b
b
E [Γi+1 | Dcon
]
+
E
x
−
x
b
R
+
R
+
Q
x
−
x
b
Di+ .
+
i
i
i|i
i|i
i

The minimum of this quantity over different ξi will be independent of ui−1 if the density ρ i|i−1 is
symmetric w.r.t. translations in the control. If the matrix A is invertible, then ρ Axi |z0i−1 (x) =
ρ i−1|i−1 (A−1 x) . Let u, u
e be two possible values for ui−1 . Then:
ρ i|i−1 (x) = ρ Axi |z0i−1 (· − Bu) ~ ρEw (·)

and,

x

ρei|i−1 (x) = ρ Axi |z0i−1 (· − Be
u) ~ ρEw (·) .
x

If the following three conditions hold: (1) the matrix A is invertible, (2) the conditional density
ρ i−1|i−1 is a ‘well-behaved’ function, for example, a function of bounded variation, and (3) the
noise random variables wi , vi have ‘well-behaved’ densities, then it is straightforward to deduce
the following symmetry w.r.t. translations:




con
con
ρ i|i−1 x − u D(i−1)− , ui−1 = u = ρ i|i−1 x − u
e D(i−1)− , ui−1 = u
e .

If the matrix A is not invertible, or if any of the relevant densities have Dirac-delta functions,
then too, this symmetry property holds. Proving that needs some slightly more delicate
arguments. The rest is similar to the proof of Lemma 5.

The remaining lemmas of Section 4.3 are straightforward to generalize to the multivariable
case. Moreover, our results clearly extend also to the case of deterministic, time-varying
coefficients of the plant dynamics and of the objective function.
5. Dynamic designs for other models of channels

Our results for Design problem 2 extend to other channel models. In this section, we
study a handful of channel models, all coming from within three broad classes of messaging a
sequence of real numbers. These are: (1) quantized messaging, (2) unquantized but irregular,
event-triggered sampling, and (3) unquantized messaging corrupted by additive channel noise.
For each of these channel models, we find that the dynamic LQ design problem gets a separated
optimal solution despite the existence of a dual effect in the corresponding networked control
systems. To obtain this design simplification, we also assume that at all times, the channel

20

M. RABI, C. RAMESH, AND K.H. JOHANSSON

output is perfectly visible to the encoder. Thus in each one of our channel models, there will
be an ideal, delay-free feedback channel copying the actual inputs for the controller back to
the encoder.
Our results also clearly extend to the case where we allow deterministic, time-varying
coefficients for the plant equation, and for the quadratic performance costs. These results also
apply to the case where the quantizer word-lengths at different times are deterministic but
time-varying. In this section, we use the performance cost in (2), where the communication
cost J Comm takes a positive functional form depending on the channel model. To show these
extensions for all the other channel models we study, we only need to find the appropriate
versions of Lemma 5. Once this is done, all the steps in the proofs for Lemmas 6 - 7 and
Theorem 1 can be repeated with. For each of the channel models we consider, an encoder that
is controls-forgetting from time 0 will be optimal in combination with the certainty equivalence
control laws of (12).
5.1. Quantizer with its rate chosen real-time. We describe below Design problem 2
for quantized control where the quantization rate is to be chosen real-time. The rate has an
expense attached, and there may be both a common upper bound on the sizes of individual
codewords and a separate upper bound on the average data rate over the entire horizon.
5.1.1. Communication cost. The channel is an error-free, discrete alphabet channel with
a variable sized alphabet. With each channel use, the size of the alphabet ηt , as well as the
codeword νt ∈ {1, 2, . . . , ηt }, must be chosen causally by the encoder. Let φ (η) = log2 η be a
measure of the data rates, and let the positive integer η denote an upper limit on the alphabet
size at any time. Then the communication cost incurred at time t can be described thus:
(
φ (ηi ) if ηi ≤ η,
ϕt (ηt ) =
+∞
if ηi > η.
Let the positive real number R ≤ η denote an upper limit on the average data rate over the
entire horizon. We define the communication cost as follows:
i
hP
(
PT
T
ϕ
(η
)
if
m
·
E
i
Comm
i=0 ϕ (ηi ) ≤ R · (T + 1) ,
i=0
(16)
J
=
PT
+∞
if
i=0 ϕ (ηi ) > R · (T + 1) ,

where m is a fixed non-negative scalar. It is easy to see that the signals xt , {ιj }t−1
, {zj }t−1
,
0
0
{ξj }t−1
are
sufficient
statistics
for
encoding
decisions,
where
of
course
z
=
(η
,
ν
).
We
now
t
t t
0
present a suitable version of Lemma 5.
Lemma 9 (Variable rate controls-forgetting encoder optimal for affine controls). Fix time t =
i and apply control laws affine from time i. Suppose that for all times j > i we have optimal encoding policies Ej∗ (·) (rules for variable alphabet sizes ηj as well as actual quantization
maps) such that their shapes and performances are independent of the partial control waveform
{ui , . . . , uT }. Then, for all times j > i−1 we get optimal encoding policies Ej∗ (·) such that their
shapes and performances are independent of the slightly longer waveform {ui−1 , ui , . . . , uT }.
Proof. Consider the encoder choice at time t = i. For any fixed alphabet size η, let E η∗ (·)
be the encoder possessing the two properties: (1) its alphabet size equals η, and (2) this encoder

T
in combination with optimal policies for the later encoders Ej∗ j=i+1 (meaning policies for
variable alphabet sizes and quantization maps) achieves the lowest possible values for the
performance costs. Where by performance cost of the encoder we mean those parts of the
performance cost that, once affine control policies are fixed, depend on the encoder.
For every fixed η, we know that E η∗ (·) and the statistics of its outputs are independent
of the policy for control ui−1 . Hence when this quantizer is used in combination with an
optimal set of later encoders, the quantization distortion at time t = i, and the statistics of
channel outputs at all times j ≥ i become independent of the control value ui−1 . Likewise the

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

21

communication costs incurred at times j ≥ i become independent of the control value ui−1 .
Since every admissible choice of ηt leads to this property, the Lemma is proved.

We now present the main result:
Theorem 2 (Optimality of separation and certainty equivalence). For Design problem 2,
with the discrete alphabet channel of variable alphabet size, the performance cost (2) with
communication cost (16) is minimized by applying the linear control laws
u∗t = −kt∗ x
b t|t

in combination with the following encoder which is controls-forgetting from time 0:



t−1
t−1
,
=
arg
inf
Γ

(·)
;
{z
}
,
{
(·)}
∗t ζt ; {zi }0t−1 , {i (·)}t−1
i
i 0
i
0
0
(·)

2

a2 β 2

βi+1
qβi+1
where, ki∗ = a q+β
, βi = p + aq+β
, βT +1 = 1, and λi = q+βi+1
and where,
i+1
i+1
i+1
(
PT
+∞,
if
i=0 ϕ (ηi ) > R · (T + 1) ,
h
i
2
ΓT =
T
−1
E ζT − ζbT |T + m · ϕ (ηT ) T (·) , {zi , i (·) }0
, otherwise,
i
h


2
+ E Γ∗t+1 {zi , i (·) }t0 ,
Γt = λt E ζt − ζbt|t + m · ϕ (ηt ) t (·) , Dcon
(t−1)+

Γ∗t = inf Γt () .
(·)

Moreover, this control law is a certainty equivalence law.
Proof. Starting with the result of Lemma 4 as a seed, repeatedly apply in sequence
Lemmas 9, 7. This proves optimality of the above combination. Lemma 1 implies that the
controls laws of (12) are indeed certainty equivalence control laws as per van der Water and
Willems [46].

5.2. Event-triggered sampling. The second model provides instantaneous, error-free
transmission of any input real number. It is suitable only with systems working in real-time,
since it has infinite capacity in the Shannon sense. To make this channel model represent a
bottleneck, one must limit how often the channel can be used over prescribed time intervals.
This we do by charging a communication cost for transmissions. This channel model is suitable
for loops with event-triggered sampling. We now summarize parallel developments for eventtriggered messaging.
5.2.1. Communication cost. The channel is an ideal, delay-free continuous valued one with
no amplitude constraints. We will stipulate that the input to the channel is either a special
silence symbol or a real number. In either case, the output will be a faithful reproduction of
the input. Hence, the encoder for event-triggered sampling can be represented by the following
map from plant output to channel input
(
xi
if xi ∈
/ Si
zi =
SILENCE if xi ∈ Si ,
where policies for the silence sets Si have to be measurable w.r.t. the filtration generated
by the data Dcon
. Let ηi denote the random number of state samples transmitted up to
(i−1)+
Pt
and including time t = i. Then we can write ηt =
/ i } . Let the non-negative
i=0 1{xt ∈S
number N0 ≤ T + 1 denote an initial budget of samples. This initial budget is a hard limit,
and the total number of samples taken over the entire horizon can never exceed N0 . Then we
define the communication cost as follows:
(
m · E [ηT ] if ηT ≤ N0 ,
(17)
J Comm =
+∞
if ηT > N0 ,

22

M. RABI, C. RAMESH, AND K.H. JOHANSSON

where m is a fixed non-negative scalar. It is easy to see that the signals xt , {zj }t−1
, {ξj }t−1
, {ηj }t−1
0
0
0
are sufficient statistics for sampling decisions. Note also that the record of sample counts {ηj }t−1
0
can be causally deduced from the record of channel outputs {zj }t−1
.
0
If we set N0 to be a finite number less than the horizon length T +1 and set the multiplier m
to zero, then we get a design problem with a fixed budget N0 and no cost attached to any
number of samples within the budget. If instead we set the multiplier m to be some positive
number and set the bound N0 to be T + 1, then we get a design problem with no budget
constraint but with a communication cost growing linearly with the number of samples taken
over the entire horizon. These two kinds of design problems and their hybrids will all be
simultaneously studied by examining the general case where m can be any nonnegative number,
and N0 any positive number.
Lemma 10 (Controls-forgetting sampler optimal for affine controls). Fix time t = i and
apply control laws affine from time i. Suppose that for all times j > i the optimal silence
sets Sj∗ (·) and their performances are independent of the partial control waveform {ui , . . . , uT }.
Then, for all times j > i − 1 the optimal silence sets Sj∗ (·) and their performances are independent of the slightly longer waveform {ui−1 , ui , . . . , uT }.
Proof. As with proving Lemmas 3, 4 we carry out two steps. First we show that because
the cost-to-go is quadratic, the quantizer’s objective at time i is to minimize a sum Γi of
current and future estimation distortions. Second we show that the minimum of this sum
distortion is independent of the control ui−1 . Thus the encoder becomes controls-forgetting
from time i − 1.

The main result for event-triggered sampling is presented below.
Theorem 3 (Optimality of separation and certainty equivalence for event-triggered sampling). For Design problem 2, with the even-triggered messaging channel, the performance
cost (2) with communication cost (17) is minimized by applying the linear control laws
u∗t = −kt∗ x
b t|t

in combination with the following silence set which is controls-forgetting from time 0:


t−1
t−1 
t−1
t−1
∗
St ζt ; {zi }0 , {Si (·)}0
= arg inf Γi S (·) ; {zi }0 , {Si (·)}0 ,
S(·)

2

a2 β 2

βi+1
qβi+1
where, ki∗ = a q+β
, βi = p + aq+β
, βT +1 = 1, and λi = q+βi+1
and where,
i+1
i+1
i+1
(
+∞,
i if ηT > N0 ,
h
2
ΓT =
T
−1
E ζT − ζbT |T + m · ϕ (ηT ) ST (·) , {zi , Si (·) }0
, otherwise,
h
i
2


Γt = λt E ζt − ζbt|t + m · ϕ (ηt ) St (·) , Dcon
+ E Γ∗t+1 {zi , Si (·) }t0 ,
+
(t−1)

Γ∗t = inf Γt (S) .
S(·)

Moreover, this control law is a certainty equivalence law.
Proof. Starting with the result of Lemma 4 as a seed, repeatedly apply in sequence
Lemmas 10, 7. This proves optimality of the above combination. Lemma 1 implies that the
controls laws of (12) are indeed certainty equivalence control laws as per van der Water and
Willems [46].

5.3. Messaging over an noisy linear channel. This model is a generalization of the
classical additive white Gaussian noise (AWGN) channel, where we let the channel noise be

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

23

coloured and non-Gaussian. This channel accepts real valued inputs ιt and delivers outputs
zt with noise added. For 0 ≤ t ≤ T :
zt = ιt + χt ,

where the channel noise process {χi } is IID with mean zero and variance σχ2 < ∞. At time t,
the noise χt is independent of the state, controls and process noises up to and including time
t. For this style of messaging, we describe a model that allows the encoder to choose the SNR
for each message. Naturally the model will also specify costs incurred for choosing message
SNRs.
5.3.1. Communication cost. Let the real-valued even function φ(·) increase with increasing
magnitude of argument, and let φ(0) = 0. An example is the function φ(ι) = ι2 . Let ι denote
an upper limit on inputs to the channel. Then the communication cost incurred at a time t
can be described thus:
(
φ(ιt ) if |ιt | ≤ ι,
ϕt =
+∞ if |ιt | > ι.
Let P ≤ φ(ι) denote an upper limit on the average power of channel inputs over the entire
horizon. We define the communication cost from time t to the horizon end as follows:
hP
i
(
PT
T
m
·
E
ϕ
(ι
)
if
j
Comm
j=t
j=0 ϕ (ιj ) ≤ P · (T + 1)
(18)
J
=
PT
+∞
if
j=0 ϕ (ιj ) > P · (T + 1) .
where m is a fixed non-negative scalar.
5.3.2. Sufficient statistics and scope for the dual effect. It is straightforward to see that
xt , {ιj }t−1
, {ξj }t−1
, {zj }t−1
0
0
0

are sufficient statistics at the encoder. As with quantized and event-triggered messaging, here
too there is scope for the dual effect since the encoding map may be nonlinear.
Clearly there is no dual effect introduced if the upper limit on inputs is removed, and the
encoder implements an affine encoder. But in general, there is scope for introducing the dual
effect. If the encoder implements the quadratic encoder:
ξtquadratic = ηx2t ,
then there is a second-order dual effect. Another example of an admissible encoder that
introduces the dual effect in the loop is one that implements the piecewise-constant encoder:


−ι if xt ∈ (−∞, −θ) ,
ξt = 0
if xt ∈ (−θ, +θ) ,

ι
if xt ∈ (+θ, −∞) ,

where the threshold θ is fixed. In fact, this encoder has nearly the same input-output behaviour
as the encoders considered in examples 1 and 2. Using this parallel, one can setup an example
of a loop with an additive noise (AN) channel such that the dual effect is present. And,
when there is a finite, hard limit on amplitudes of channel inputs, then the dual effect is
present for any encoder other than the trivial ones of the form: ξt ≡ constant. As with other
types of messaging, we can show that even though the dual effect is present, the dynamic
encoder-controller problem has a separated solution and certainty equivalence controls are
optimal.
Lemma 11 (Controls-forgetting compander optimal for affine controls). Fix time t = i
and apply control laws affine from time i. Suppose that for all times j > i the optimal
encoding policies Ej∗ (·) and their performances are independent of the partial control waveform
{ui , . . . , uT }. Then, for all times j > i − 1 the optimal encoding policies Ej∗ (·) and their
performances are independent of the slightly longer waveform {ui−1 , ui , . . . , uT }.

24

M. RABI, C. RAMESH, AND K.H. JOHANSSON

Proof. As with proving Lemmas 3,4 we carry out two steps. First we show that because
the cost-to-go is quadratic, the quantizer’s objective at time i is to minimize a sum Γi of current
and future estimation distortions. Second we show that the minimum of this sum distortion is
independent of the control ui−1 . Thus the optimal encoder becomes controls-forgetting from
time i − 1.

The main result for communication over a noisy linear channel is presented below.
Theorem 4 (Optimality of separation and certainty equivalence for additive noise channel). For Design problem 2, with the additive noise channel, the performance cost (2) with
communication cost (18) is minimized by applying the linear control laws
u∗t = −kt∗ x
b t|t

in combination with the following compander which is controls-forgetting from time 0:



t−1
t−1
=
arg
inf
Γ

(·)
;
{z
}
,
{
(·)}
,
∗t ζt ; {zi }0t−1 , {i (·)}t−1
i
i 0
i
0
0
(·)

2

a2 β 2

βi+1
qβi+1
where, ki∗ = a q+β
, βi = p + aq+β
, βT +1 = 1, and λi = q+βi+1
and where,
i+1
i+1
i+1
(
PT
+∞,
if
i=0 ϕ (ηi ) > P · (T + 1) ,
h
i
2
ΓT =
E ζT − ζbT |T + m · ϕ (ηT ) T (·) , {zi , i (·) }T0 −1 , otherwise,
h
i
2


con
b
Γt = λt E ζt − ζ t|t + m · ϕ (ηt ) t (·) , D(t−1)+ + E Γ∗t+1 {zi , i (·) }t0 ,

Γ∗t = inf Γt () .
(·)

Moreover, this control law is a certainty equivalence law.
Proof. Starting with the result of Lemma 4 as a seed, repeatedly apply in sequence
Lemmas 11, 7. This proves optimality of the above combination. Lemma 1 implies that the
controls laws of (12) are indeed certainty equivalence control laws as per van der Water and
Willems [46].

We might also add that for all of the above channel models, the results for Design problem 2 can also be extended to the case of vector valued states with only partial, noisy linear
observations available at the sensor (encoder). Such a situation is no more complicated than
that one where the encoder observes the state perfectly. In the partially observed case, the
role of the ‘state’ falls on the estimate produced by the encoder’s Kalman filter.
6. Constrained encoder-controller design
We now use our understanding of the dynamic encoder-controller design problem (Design
problem 2) to examine the constrained encoder-controller design problem (Design problem 3)
and the hold-waveform-controller and encoder design problem (Design problem 4). In this
section, we show that, in general, separation in design of encoder and controller is not optimal
for these design problems. We do this by presenting a counterexample for each of these design
problems. Some of these counterexamples illustrate that the distortion term in the cost-to-go
lacks symmetry w.r.t. translations (11). Recall that this property was instrumental in ensuring
separation in the dynamic encoder-controller design problem (see proof of Lemma 5).
Thus, we begin with Example 4, which illustrates, through explicit calculations, that symmetry w.r.t. translations does indeed occur in the dynamic encoder-controller design problem.
Next, we impose a set of constraints on the decision makers of the closed-loop system in Examples 5-7, which have the effect of removing the symmetry w.r.t. translations. For these
cases, we show that separation in design is no longer optimal. In Example 9, we illustrate that
separation is not optimal when the control signals are held constant over random epochs.

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

25

6.1. Symmetry w.r.t. translations leads to separation. We present a simple example of a dynamic encoder-controller design problem; the encoder is specified in a parametric
form, but the choice of the parameters can be dynamic, with no restrictions on the set of
parameters. We show that the optimal controller uses the certainty equivalence law.
Example 4. For the linear plant (1), with initial state x0 given by a zero mean Gaussian
with variance σx2 , and process noise wk given by a zero mean Gaussian with finite variance
σw2 , let the horizon length be T = 2. Let the cost coefficients p and q remain unspecified. Let
the channel alphabet be the discrete set {1, 2}. The controller receives a quantized version of
the state, denoted zk and given by
(
1 if xk ≤ δk ,
zk =
2 otherwise .
The quantizer thresholds δ0 and δ1 are to be chosen along with the control signals u0 and u1 ,
to jointly minimize the two-step horizon control cost.
We use dynamic programming to find the optimal values for u1 , δ1 and u0 , and δ0 , in the
specified order. From Lemma 4, we know that u∗1 is given by the
equivalence law as
 certainty
1
a
− q+1 x
b1|1 , where the MMSE estimate of x1 is given by x
b1|1 = E x1 {zi }0 .
Then, let us consider the cost-to-go at the previous time step,


a2 2
2
2 2
2 2
2
V0 = min E a (p + a )x0 + (q + p + a )u0 + 2a(p + a )x0 u0 −
z0 + κ ,
(19)
x
b
u0 ,δ1
q + 1 1|1
where κ = (1 + p + a2 )σw2 . The above cost-to-go is to be minimized by selectingh a suitable
u0
i
and δ1 simultaneously. To do this, we first need to find an expression for E x
b21|1 z0 . The
encoder outputs at times 0, 1 tell us the quantization cells in which x0 and x1 lie. We use this
information to find an expression for the estimate x
b1|1 , as shown in Appendix B, and rewrite
the cost-to-go as


function of u0
z
}|
{
q
q


V0 = min E a2 (p + a2 )x20 + (q + p + a2
)u20 + 2a(p + a2
)x0 u0 z0 
u0 ,δ1
q+1
q+1
(20)

2

−

a
q+1
|

PN

2
j=1 ϑ

,Γ1 :



$j−1 −u0 $j −u0
, σ2
σ2

P (x0 ∈ (θi−1 , θi ))
{z
function of u0 and E1



+(1 + p + a2 )σw2 ,

}

where σ22 = σw2 + a2 σx2 . The term ϑ(r, r̄) in the above equation is given by

  



θi
σ2
a
θi
aσx
ϑ(r, r̄) = − aσx g
G r
− θi
− σ2 g(r)G
−r
σx
σw
σw
σ1
σw
 


 r̄

(21)
σ2
a
θi−1
aσx
θi−1
G r
− θi−1
+ σ2 g(r)G
−r
+ aσx g
,
σx
σw
σw
σ1
σw
r=r
where σ12 = σx2 σw2 /σ22 and g(·) and G(·) are the probability distribution function and cumulative
distribution function, respectively, of the standard normal distribution. The quantization cells
for x0 and x1 are denoted by (θi−1 , θi ) and ($j−1 , $j ) corresponding to the encoder outputs
z0 = i and z1 = j, respectively.
The quantization distortion term Γ1 in (20) possesses symmetry w.r.t. translations, as
defined in (11). Thus, for any value of the control signal u0 , the minimum value is given
by Γ∗1 (E1 ), a term that depends only on the encoder. Then, the cost-to-go with respect to
the control signal u0 comprises of only the terms in the first row in (20). Hence, we obtain

26

M. RABI, C. RAMESH, AND K.H. JOHANSSON

Quantization Distortion Γ1

−0.3
−0.4
−0.5

u0 = −1, x0 ∈ (−∞, 0)

−0.6

u0 = 1, x0 ∈ (−∞, 0)

u0 = 0, x0 ∈ (−∞, 0)
u0 = −1, x0 ∈ (0, ∞)
u0 = 0, x0 ∈ (0, ∞)

−0.7
Equal Minima
−0.8
−10

−8

−6

−4

u0 = 1, x0 ∈ (0, ∞)
−2

0

2

4

6

8

10

Encoder Threshold δ1

Figure 6. This plot illustrates the symmetry w.r.t. translations of the quantization distortion term Γ1 in (20). Different values of u0 result in the same
minimum value for Γ1 at different values of δ1 , thus resulting in separation and
certainty equivalence in Example 4.
separation. Furthermore, the optimal control signal is given by the certainty equivalence law,
a(p+a2

q

)

q+1
uCE
b0|0 . Thus, the certainty equivalence property holds for this setup.
0 = − p+q+a2 q x
q+1

We illustrate symmetry w.r.t. translations in Figure 6. For the choice of parameters a = 1,
p = 1 and q = 1, we evaluate the quantization distortion term Γ1 from the above example and
show that the minimum that this function attains over the range of the quantizer threshold
δ1 is invariant for different values of u0 . To evaluate the cost-to-go, we make an arbitrary
choice: δ0 = 0, for the quantizer threshold at time k = 0, and we compute the estimates and
probabilities using this choice.
6.2. Optimal constrained encoder. We now impose a restriction on the choice of
encoder parameters. The one-bit quantizer that we consider in the previous example selects
two semi-infinite intervals as the quantizer cells, ∆1 = (−∞, δk ] and ∆2 = (δk , ∞). We restrict
the choice of the quantizer threshold to a constraint set, such that δk ∈ Θ. In the following
example, we see that separation is lost for this constrained optimization problem.
Example 5. Consider the same setup as in Example 4, with the restriction that the quantizer threshold be chosen from the set Θ = (−1, 1). The quantizer thresholds δ0 ∈ Θ and
δ1 ∈ Θ are to be chosen along with the control signals u0 and u1 , to jointly minimize the
two-step horizon control cost.
We follow the same procedure as before. The optimal control signal u1 is given by the
certainty equivalence law as u∗1 = uCE
1 . This gives us the same cost-to-go V0 from (19).
Evaluating Γ1 for the parameters a = 1, p = 1 and q = 1, we plot it over a range of quantizer
thresholds δ1 ∈ Θ, for three arbitrary choices of u0 , in Figure 7. By restricting the range of
quantizer thresholds to Θ, we do not permit all the curves to reach their minima from Figure 6.
In particular, the minima for u0 = −1, when x0 ∈ (−∞, 0), and u0 = 1, when x0 ∈ (0, ∞) are
higher than before. Thus, the minimum value of Γ1 obtained over the range of δ1 now varies
depending on the choice of u0 . Consequently, there is no longer a symmetry w.r.t. translations,
and separation cannot be achieved using the proof of Theorem 6. Furthermore, the optimal
control signal u∗0 must be chosen along with δ1∗ to optimize the entire cost-to-go including the
term Γ1 . Thus, u∗0 does not just minimize a quadratic expression in this problem, and cannot
be chosen independently of the encoding policy. Hence, separation in design of the controller
and encoder is no longer optimal.

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

27

Quantization Distortion Γ1

−0.3
u0 = −1, x0 ∈ (−∞, 0)

−0.4

u0 = 0, x0 ∈ (−∞, 0)

−0.5

u0 = 1, x0 ∈ (−∞, 0)

−0.6

u0 = 1, x0 ∈ (0, ∞)

−0.7

u0 = 0, x0 ∈ (0, ∞)

u0 = −1, x0 ∈ (0, ∞)

−0.8
Unequal Minima

−0.9
−1
−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Encoder Threshold δ1

Figure 7. This plot illustrates the lack of symmetry w.r.t. translations of Γ1 ,
when the quantizer thresholds are restricted to be chosen from an interval, such
as in Example 5. Different values of u0 do not result in the same minimum value
for Γ1 over the range of δ1 , thus resulting in a lack of separation and certainty
equivalence.
6.3. Optimal constrained controller. We now remove the restriction on the encoder
parameters, and instead impose the following restriction on the controller: the controls are
required to have limited range. Specifically, the control values at ever time step must come
from a specified constraint set U. We present two versions of this constraint: in case 1, our
constrained control set U is discrete, and in case 2, the constrained control set is an interval
U = (umin , umax ).
Example 6. Consider the same setup as in Example 4, with the restriction that the control
signal be chosen from a discrete set U = {−1, 0, 1}. The quantizer thresholds δ0 and δ1 are to
be chosen along with the control signals u0 ∈ U and u1 ∈ U, to jointly minimize the two-step
horizon control cost.
The unconstrained minimizer for the cost-to-go at the terminal time is given by the certainty equivalent value uCE
1 . The best we can do, given the constraint set U, is to choose the
control value from the discrete set U that results in the lowest cost-to-go. Using this principle,
we find the optimal control signal u∗1 to be


b1|1 ≥ q+1
,
−1 x
2a
∗
q+1
u1 = 0
≥x
b1|1 ≥ − q+1
,
2a
2a

1
q+1
x
b1|1 ≤ − 2a .

The optimality regions are identified by comparing minu1 ∈U V1 (u1 ) evaluated at each permissible
value of u1 , and determining the switching points.
The cost-to-go V0 , obtained by averaging over the three different cost-to-go functions obtained at time k = 1, is given by

V0 = min E a2 (p + a2 )x20 + (q + p + a2 )u20 + 2a(p + a2 )x0 u0 + (−2ab
x1|1 + q + 1)1{xb1|1 ≥ q+1 }
2a
u0 ,δ1

+ (2ab
x1|1 + q + 1)1{xb1|1 ≤− q+1 } z0 + (1 + p + a2 )σw2 .
2a

We denote the terms in the above cost-to-go that directly depend on the choice of the
encoder threshold δ1 as ΓRC
b1|1 and the posterior density for x1
1 . Using the expression for x

28

M. RABI, C. RAMESH, AND K.H. JOHANSSON

Quantization Distortion ΓRC
1

0
u0 = −1, x0 ∈ (−∞, 0)

−0.5

u0 = 0, x0 ∈ (−∞, 0)
u0 = 1, x0 ∈ (−∞, 0)
u0 = 0, x0 ∈ (0, ∞)

−1

u0 = −1, x0 ∈ (0, ∞)

Unequal
Minima

u0 = 1, x0 ∈ (0, ∞)

−1.5

−2
−10

−8

−6

−4

−2

0

2

4

6

8

10

Encoder Threshold δ1

Figure 8. This plot illustrates the lack of symmetry w.r.t. translations for ΓRC
1 ,
when the controls are restricted to be chosen from a discrete set U, such as in
Example 6. Different values of u0 do not result in the same minimum value for
over the range of δ1 , thus resulting in the lack of separation and certainty
ΓRC
1
equivalence.
as
from Appendix B, we compute ΓRC
1


RC
Γ1 = E (−2ab
x1|1 + q + 1)1{xb1|1 ≥ q+1 } + (2ab
x1|1 + q + 1)1{xb1|1 ≤− q+1 } z0
2a

2a


N
X
P (x0 ∈ (θi−1 , θi ) , x1 ∈ ($j−1 , $j ))
=
(−2ab
x1|1 + q + 1)1{xb1|1 ≥ q+1 }
2a
P (x0 ∈ (θi−1 , θi ))
j=1

+ (2ab
x1|1 + q + 1)1{xb1|1 ≤− q+1 }
2a



.

Evaluating the above expression for parameters a = 1, p = 1 and q = 1, and some arbitrary
choice of quantizer threshold δ0 , we plot ΓRC
over a range of quantizer thresholds δ1 , for
1
different choices of u0 from the set U, in Figure 8. Notice that the minimum values of ΓRC
1
obtained over the range of δ1 vary depending on the choice of u0 . In other words, there is
no symmetry w.r.t. translations. Consequently, a separation in design of the controller and
encoder is no longer optimal.
We now present a slight variation in the restriction on the controller, and reconfirm that
separation in design of controller and encoder is not optimal.
Example 7. Consider the same setup as in Example 4, with the restriction that the control
signal be chosen from an interval U = (umin , umax ). The quantizer thresholds δ0 and δ1 are to
be chosen along with the control signals u0 ∈ U and u1 ∈ U, to jointly minimize the two-step
horizon control cost.
As in the solution to the previous example, note that the unconstrained minimizer for the
cost-to-go V1 is the certainty equivalent value uCE
1 . The best we can do, given the constraint
set U, is to choose the control signal closest to the unconstrained value. This follows from the
convexity of the quadratic cost-to-go. Using this principle, we find the optimal control signal
u∗1 to be

CE

umin u1 ≤ umin ,
u∗1 = uCE
umin ≤ uCE
1
1 ≤ umax ,

u
CE
u1 ≥ umax .
max
∗
Evaluating the cost-to-go V1 using u1 , and reusing quantities derived in Appendix B, we can
write up the cost-to-go V0 as before. More interesting to us are the terms in this expression

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

29

Figure 9. This plot illustrates the lack of symmetry w.r.t. translations of ΓIC
1 ,
when the controls are restricted to be chosen from an interval, such as in Example 7. Different values of u0 do not result in the same minimum value for ΓIC
1 over
the range of δ1 , thus resulting in lack of separation and certainty equivalence.
that directly depend on the choice of the quantizer threshold δ1 , as given by

a2 2
IC
Γ1 = E (2ab
x1|1 umin + (q + 1)u2min )1{xb1|1 ≥− q+1 umin } −
x
b 1 q+1
q+1
a
q + 1 1|1 {− a umax ≤bx1|1 ≤− a umin }

2
+(2ab
x1|1 umax + (q + 1)umax )1{xb1|1 ≤− q+1 umax } z0 .
a

Evaluating this expression for parameters a = 1, p = 1, q = 1, umin = −2 and umax = 2,
and some arbitrary choice of quantizer threshold δ0 , we plot ΓIC
1 over a range of quantizer
thresholds δ1 , for different choices of u0 from the set U, in Figure 9. Notice that the minimum
value of ΓIC
1 obtained over the range of δ1 varies depending on the choice of u0 . Thus, there
is no symmetry w.r.t. translations, and a separation in design is no longer optimal.
In both the above examples, the constrained set U did not contain the certainty equivalent
values of the control signal u1 for at least some values of δ1 . The resulting cost-to-go V0
was altered, such that the symmetry w.r.t. translations was lost. Consequently, separation
no longer holds. The restriction removed the certainty equivalence property during time step
k = 1, but the resulting cost and the information pattern resulted in the lack of separation
itself at time step k = 0. A similar problem setup has been explored in [8], where the control
gain is restricted to be chosen from two given values. The dual effect has been shown for this
problem setup as well.
6.4. Zero order hold and event-triggered sampling. We study numerically two cases
of control under event-triggered sampling. Basically these are problems with a sampling budget
of exactly one. For the controller, we must design a whole waveform to be applied up to the
time when the first sample is received. We are already given the control law to be applied
from this random sampling time to the end time. For the encoder, we must design an envelope
to generate exactly one sample between time t = 1 and t = T .
We study two examples, and in both of them, the encoder is allowed to be dynamic. In
the first example, the control waveform up to the first sample time is pre-assigned, and it has
a particular linear dependence on the Kalman predictor. In the second example, the control
waveform up to the first sample time must be a zero order hold waveform.

30

M. RABI, C. RAMESH, AND K.H. JOHANSSON

Kalman predictor

1.5
1

1

x0

x0

0

0

−0.5

−0.5

−1

Kalman predictor

1.5

0

1

2

3

4

5

−1

0

1

t

2

3

4

5

t

(a) Optimal sampling envelope
for Example 8

(b) Optimal sampling envelope
for Example 9

Figure 10. Event-triggered sampling with exactly one sample after time t = 0
Example 8 (Fixed linear control law up to an event-triggered sample). For the scalar
linear plant (1), let the coefficient a = 1, and let the initial state x0 = 2, and σ0 = 0, and let
this information be known to the encoder and the controller. This simply means that z0 = x0 .
This information is prestored at the controller. Let the variance σw2 = 0.52 . Let the horizon
end T = 4, and let p = 1, q = 0.2. The control law is fixed to be:
(


kt∗ E xt x0 , {ui }t−1
, for 0 ≤ t ≤ τ − 1,
0
ut =
t−1
∗
kt E [xt |xτ , {ui }τ ] , for τ ≤ t ≤ T,
where the gains kt∗ are the ones from the certainty equivalence law (12), and τ satisfies 1 ≤
τ ≤ T and is the first and only sample time, which is chosen by encoder. Choose a policy
(sampling envelope) which comprises silence sets {S1 , . . . , ST } giving:
n
o
τ = min T, min{t : xt ∈
/ St } .
t≥1

Next we consider an example of a design problem with a zero order hold control. Here we
specialize to the case where the control’s hold epochs are forced to be exactly the inter-sample
intervals.
Example 9 (zero order hold control up to an event-triggered sample). Consider the
same setup as in Example 8 but there are exactly two epochs; and they must be precisely
{0, 1, . . . τ − 1} and {τ, . . . , T }, where τ is the first and only sample time, and is chosen to
occur at or later than time t = 1. The control laws over the second epoch are fixed to have the
form: ut = ki∗ E [xτ |xτ0 ] , for τ ≤ t ≤ T, where the gains kt∗ are the ones from the certainty
equivalence law (12). Pick: (1) a control law for the first epoch having the zero-order hold
form:
ut = K0 (x0 ) , for 0 ≤ t ≤ τ − 1,
and (2) a sampling envelope which comprises silence sets {S1 , . . . , ST } for generating the
sample time:
n
o
τ = min T, min{t : xt ∈
/ St } .
t≥1

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

ut

State: xt

Linear plant

yt

Fixed
H (t, ·)

31

zt

memoryless map
Controller
Figure 11. Setup of Bar-Shalom and Tse
The optimal sampling envelope of the zero order hold control example (Example 9) is
shown in Figure 10b. This is pictorial evidence that the dual effect is present in the loop.
This becomes clear from the reasoning below.
Supposing the dual effect were absent, then the encoder’s goal would have been to pick
the sample time τ to minimize a weighted sum of squared estimation errors up to time τ − 1.
The envelope
optimal for

 that objective will be a sequence of silence set symmetric about the
means E xt x0 , {ui }0τ −1 . When the plant noise is Gaussian, Hajek and others [22, 23, 28, 37]
predict that a symmetric sequence of silence is optimal. They also
 imply that a sequence of
silence sets that are not symmetric about the respective means E xt x0 , {ui }τ0 −1 will lead to
suboptimal state estimation.
Since the optimal envelope
computed numerically is clearly non-symmetric about the

τ −1
means E xt x0 , {ui }0
, there must be a dual effect in the loop, which is exploited by this
optimal pair of sampler and zero order hold controller.
7. Conclusions
In this paper, we have seen through examples that the dual effect is present in the plantencoder-channel combination. Hence in general, it is suboptimal to apply a controls-free
encoder, or to apply an affine controller. It has long been known that for the design problem
with a static encoder, separation is not optimal, and that the optimal control laws are nonlinear [13]. Recent interest in the dynamic design problem was due to Borkar and Mitter [10]
who describe advantages obtained by applying controls-forgetting encoders. Many papers state
that the separated design is optimal for the dynamic design problem for the various channel
models we have treated. We have shown by dynamic programming that these statements are
indeed correct. This is an instance of the optimal decision policies ‘ignoring’ the presence of
the dual effect. But a separated design need not be optimal for other design problems. In
particular, for event-triggered sampling the dynamic design problem has a separated design,
but the zero order hold control design problem does not have a separated solution. This is
at least partly surprising because, separated design is optimal for the classical LQG partially
observed control with or without the zero order hold control restriction.
An interesting aspect of our results is that we have shown that separation and certainty
equivalence are optimal for Design problem 2, despite the dual effect being present in the
networked control system of Section 2. To understand this result, we now examine two implementations of the optimal encoder-controller pair for this design problem, and using these, we
draw out some subtle points concerning dual effect and optimality of separation and certainty
equivalence. Bar-Shalom and Tse [7] consider the loop shown in Figure 11. At the sensor,
instead of our dynamic encoder, they place a nonlinear map. This sensor map is time-varying
but memoryless and its exact functional form is given. For this setup, they have a result
stating the mutual exclusivity of the dual effect and optimality of certainty equivalence controls. In their setting, if the linear ‘plant’ is such that the effect of controls is never felt at the
observation signal yt , then clearly there is no dual effect. This happens in the case where the
so-called ‘plant’ has a sub-system that produces the ‘plant’ output after explicitly removing
the effect of controls.

32

M. RABI, C. RAMESH, AND K.H. JOHANSSON

Encoder
ut

Plant

ζt

xt
−



Optimal encoder
t−1

ξt∗ · ; {zi }0

t−1

, {ξi (·)}0

t−1

, {ui = 0}0

for zero controls



zt

Delay

a


t
ut = Kt {zi }0

ut , exact replica of controls


t
Controller Kt {zi }0

zt

(a) No dual effect present because encoder is controls-forgetting from time 0

Encoder
ut

Plant

xt
−

ζet



Optimal encoder
t−1

ξt∗ · ; {zi }0

t−1

, {ξi (·)}0

t−1

, {ui = 0}0

for zero controls



zt

Delay

a
u
et , simulated optimal controls



t
Controller Kt {zi }0



e t = K∗t {zi }t0
u

zt




Pt−1
(b) Has dual effect because ζet = ζt + i=0 at−1−i Ki z0i − Ki∗ z0i 6= ζt in general.

Figure 12. Two interpretations of implementing an optimal encoder E ∗ .
However, for our setup (Figure 1), the sensor has a dynamic encoder even after one performs
the equivalence transformation by subtracting out the effect of controls. The use of ‘innovation
coding’ leads to the closed loop shown in Figure 4. The crucial difference from the setup of
Bar-Shalom and Tse is that rather than being a memoryless nonlinear map, the encoder ξet is a
dynamical system. Hence the Theorem of Bar-Shalom and Tse does not apply. But it springs
the following question: Does the plant-sensor combination in the closed loop of Figure 4 have
a dual effect if an encoder is used that is optimal for the dynamic design problem ? To answer
this question, one needs to interpret carefully what it means to implement an optimal encoder.
For different interpretations, one gets different answers. Assume that we are implementing
the feedback loop of Figure 1 with the optimal encoder and any admissible controller.
The first interpretation of what it means to implement an optimal encoder, is the following: The encoder stores the actual set of control policies used by the controller, and uses
this to carry out the innovation encoding,
and on the result applies the sequential quantizer
t
t−1
t−1 
∗
ξt · ; {zi }0 , {ξi (·)}0 , {ui = 0}0 . This is equivalent to the block diagram of Figure 12a.

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

33

No matter what the actual control policies are, the controls have
no influence on the input
t−1 
.
Clearly, because of exact
to the sequential quantizer ξt∗ · ; {zi }t0 , {ξi (·)}t−1
,
{u
=
0}
i
0
0
cancellation of controls, the encoder implemented is controls-forgetting, and there is no dual
effect in the loop of Figure 12a.
The second interpretation is the following: The encoder does not pay attention to the actual
control policy being used. Instead, it assumes that the controller is applying the certainty
equivalence laws (12). It subtracts out the effect of the these certainty equivalence control laws.
t−1 
.
To the residue ζet , it applies the sequential quantizer ξt∗ · ; {zi }t0 , {ξi (·)}t−1
,
{u
=
0}
i
0
0
Clearly this encoder is not controls-forgetting. But yet when used in combination with the
certainty equivalence laws of (12), it leads to minimum performance cost.
On the other hand, when this encoder is used in combination with a general admissible
control law, there is potential mismatch between the encoder’s assumption and the actual
controller behaviour. The effect of the controls
is not absent in the input to the sequential
t
t−1
t−1 
∗
quantizer ξt · ; {zi }0 , {ξi (·)}0 , {ui = 0}0 . This situation is shown in Figure 12b. Clearly,
there is a dual effect in this loop.
This leads to an interesting consequence. If a pair of encoding and control strategies is
optimal, then the individual strategies that are components of the pair must be person-byperson optimal. Since the combination of certainty equivalence controls and the corresponding
optimal encoder is optimal, it follows that the certainty equivalence controls must be optimal
for the single-agent control problem obtained by fixing the encoder to be the optimal one.
Since the second interpretation of implementing the optimal encoder is perfectly valid, it
turns out that certainty equivalence controls can be optimal even though the dual effect is
present in the loop. Thus we can conclude that the Theorem of Bar-Shalom and Tse cannot
generalize to the scenario where sensors implement dynamic encoders.
Acknowledgements. Adam Molin first suggested to us that for event-triggered systems,
the separated solution may not be optimal for the zero order hold control design problem,
and yet be optimal for the dynamic encoder-controller design problem. To Lei Bao, Mikael
Skoglund, Henrik Sandberg, Ashutosh Nayyar, John S. Baras, and Armand Makowski we are
grateful for many discussions over the last few years. C.R. and K.H.J. greatfully acknowledge
support from the Swedish Research Council and the Knut and Alice Wallenberg Foundation.

References
[1] B.R. Andrievsky, A.S. Matveev, and A.L. Fradkov, Control and estimation under information
constraints: Toward a unified theory of control, computation and communications, Automation and Remote Control, 71 (2010), pp. 572–633.
[2] K. J. Åström and B. Bernhardsson, Comparison of Riemann and Lebesgue sampling for first order
stochastic systems, in Proceedings of the 41st IEEE conference on Decision and Control (Las Vegas NV,
2002), 2002, pp. 2011–2016.
[3] J. Baillieul and P.J. Antsaklis, Control and communication challenges in networked real-time systems, Proceedings of the IEEE, 95 (2007), pp. 9–28.
[4] R. Bansal and T. Başar, Stochastic teams with nonclassical information revisited: When is an affine
law optimal?, IEEE Trans. Automatic Control, 32 (1987), pp. 554–559.
[5] R. Bansal and T. Başar, Simultaneous design of measurement and control strategies for stochastic
systems with feedback, Automatica, 25 (1989), pp. 679 – 694.
[6] L. Bao, M. Skoglund, and K.H. Johansson, Iterative encoder-controller design for feedback control
over noisy channels, IEEE Trans. Automatic Control, 56 (2011), pp. 265–278.
[7] Y. Bar-Shalom and E. Tse, Dual effect, certainty equivalence, and separation in stochastic control,
IEEE Trans. Automatic Control, 19 (1974), pp. 494–500.
[8] B. Bernhardsson, Dual control of a first-order system with two possible gains, Int. J. Adaptive Control
and Signal Processing, 3 (1989), pp. 15–22.
[9] D. P. Bertsekas and S. E. Shreve, Stochastic optimal control: The discrete time case, vol. 139 of
Mathematics in Science and Engineering, Academic Press Inc., New York, 1978.

34

M. RABI, C. RAMESH, AND K.H. JOHANSSON

[10] V.S. Borkar and S. K. Mitter, LQG control with communication constraints, in Communications,
Computation, Control, and Signal Processing, A. Paulraj, V. Roychowdhury, and C. D. Schaper, eds.,
Springer US, 1997, pp. 365–373.
[11] V.S. Borkar, S. K. Mitter, and S. Tatikonda, Markov control problems under communication
constraints, Commun. Inf. Syst., 1 (2001), pp. 15–32.
, Optimal sequential vector quantization of Markov sources, SIAM J. Control Optim., 40 (2001),
[12]
pp. 135–148 (electronic).
[13] R. E. Curry, Estimation and control with quantized measurements. Renwick E Curry, M.I.T. press,
1970.
[14] A. A. Fel0 dbaum, Dual-control theory. I, Automat. Remote Control, 21 (1960), pp. 874–880.
[15] X. Feng and K.A. Loparo, Active probing for information in control systems with quantized state
measurements: a minimum entropy approach, IEEE Trans. Automatic Control, 42 (1997), pp. 216–238.
[16] T.R. Fischer, Optimal quantized control, IEEE Trans. Automatic Control, 27 (1982), pp. 996–998.
[17] J.S. Freudenberg, R.H. Middleton, and J.H. Braslavsky, Minimum variance control over a
gaussian communication channel, IEEE Trans. Automatic Control, 56 (2011), pp. 1751–1765.
[18] M. Fu, Lack of separation principle for quantized linear quadratic gaussian control, IEEE Trans. Automatic Control, 57 (2012), pp. 2385–2390.
[19] K. Gatsis, A. Ribeiro, G. J. Pappas, Optimal Power Management in Wireless Control Systems, IEEE
Trans. Automatic Control, 59 (2014), pp. 1495–1510.
[20] A. Gersho and R. M. Gray, Vector quantization and signal compression, Kluwer Academic Publishers,
Norwell, MA, USA, 1991.
[21] G.C. Goodwin, E.I. Silva, and D.E. Quevedo, A brief introduction to the analysis and design of
networked control systems, in Proceedings of the IEEE Control and Decision Conference, 2008, pp. 1–13.
[22] B. Hajek, Jointly optimal paging and registration for a symmetric random walk, in Proceedings of the
2002 IEEE Information Theory Workshop, oct. 2002, pp. 20 – 23.
[23] B. Hajek, K. Mitzel, and S. Yang, Paging and registration in cellular networks: jointly optimal
policies and an iterative algorithm, IEEE Trans. Information Theory, 54 (2008), pp. 608–622.
[24] T. Henningsson and K. J. Åström, Log-concave observers, in Proceedings of the 17th International
Symposium on Mathematical Theory of Networks and Systems, Kyoto, Japan, July 2006.
[25] D. Peter Joseph and T. Julius Tou, On linear control theory, American Institute of Electrical
Engineers, Part II: Applications and Industry, Transactions of the, 80 (1961), pp. 193–196.
[26] A. A. Kulkarni and T. P. Coleman, An optimizer’s approach to stochastic control problems with
nonclassical information structures, in Proceedings of the IEEE Control and Decision Conference, 2012,
pp. 154–159.
[27] R.E. Larson, Optimum quantization in dynamic systems, IEEE Trans. Automatic Control, 12 (1967),
pp. 162–168.
[28] G.M. Lipsa and N.C. Martins, Remote state estimation with communication costs for first-order lti
systems, IEEE Trans. Automatic Control, 56 (2011), pp. 2013–2025.
[29] G. M. Lipsa and N. C. Martins, Optimal memoryless control in Gaussian noise: A simple counterexample, Automatica, 47 (2011), pp. 552–558.
[30] A. Mahajan and D. Teneketzis, Optimal design of sequential real-time communication systems, IEEE
Trans. Information Theory, 55 (2009), pp. 5317–5338.
[31] A. S. Matveev and A. V. Savkin, The problem of LQG optimal control via a limited capacity communication channel, Systems & Control Letters, 53 (2004), pp. 51 – 64.
[32] R. Marleau and J. Negro, Comments on "Optimum quantization in dynamic systems", IEEE Trans.
on Automatic Control, 17 (1972), pp. 273 – 274.
[33] S. K. Mitter, Control with limited information, European J. Control, 7 (2001), pp. 122 – 131.
[34] A. Molin and S. Hirche, On the optimality of certainty equivalence for event-triggered control systems,
IEEE Trans. Automatic Control, 58 (2013), pp. 470–474.
[35] G.N. Nair, F. Fagnani, S. Zampieri, and R.J. Evans, Feedback control under data rate constraints:
An overview, Proceedings of the IEEE, 95 (2007), pp. 108–137.
[36] A. Nayyar, Sequential Decision Making in Decentralized Systems, PhD thesis, The University of Michigan, 2011.
[37] A. Nayyar, T. Başar, D. Teneketzis, and V.V. Veeravalli, Optimal strategies for communication
and remote estimation with an energy harvesting sensor, IEEE Trans. Automatic Control, 58 (2013),
pp. 2246–2260.
[38] D. B. Owen, A table of normal integrals, Comm. Statist. B—Simulation Comput., 9 (1980), pp. 389–419.
[39] M. Rabi and K. H. Johansson, Optimal stopping for updating controls, in Proccedings of the second
International workshop on sequential methods, UTT, Troyes, France, June 2009.

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

35

[40] M. Rabi, C. Ramesh and K. H. Johansson, Separated design of encoder and controller for networked
linear quadratic optimal control, arXiv:1405.0135 [math.OC], May 2014.
[41] J. G. Root, Optimum control of non-Gaussian linear stochastic systems with inaccessible state variables,
SIAM J. Control, 7 (1969), pp. 317–323.
[42] G. Sauer and J. Melsa, Stochastic control with continuously variable observation costs for a class of
discrete nonlinear systems, IEEE Transactions on Automatic Control, 19 (1974), pp. 234–239.
[43] L. Shaw, Optimum stochastic control, in Disciplines and techniques of systems control, J. Peschon, ed.,
Blaisdell publishing company, 1965, pp. 125–185.
[44] C. Striebel, Sufficient statistics in the Optimum control of stochastic systems, Journal of Mathematical
Analysis and Applications, 12 (1965), pp. 576–592.
[45] S. Tatikonda, A. Sahai, and S. Mitter, Stochastic linear control over a communication channel,
IEEE Trans. Automatic Control, 49 (2004), pp. 1549–1561.
[46] H. van de Water and J.C. Willems, The Certainty Equivalence property in Stochastic control theory,
IEEE Trans. Automatic Control, 26 (1981), pp. 1080–1087.
[47] P. Varaiya and J. C. Walrand, On delayed sharing patterns, IEEE Trans. Automat. Control, 23
(1978), pp. 443–445.
, Causal coding and control for Markov chains, Systems & Control Letters, 3 (1983), pp. 189–192.
[48]
[49] J. C. Walrand and P. Varaiya, Optimal Causal Coding-Decoding problems, IEEE Trans. Information
Theory, IT-29 (1983), pp. 814–820.
[50] H. S. Witsenhausen, A counterexample in stochastic optimum control, SIAM Journal on Control, 6
(1968), pp. 131–147.
[51] H. S. Witsenhausen, Separation of estimation and control for discrete time systems, Proc. IEEE, 59
(1971), pp. 1557–1566.
[52] W.M. Wonham, On the separation theorem of stochastic control, SIAM Journal on Control, 6 (1968),
pp. 312–326.
[53] W. Wu and A. Arapostathis, Optimal control of stochastic systems with costly observations - the
general markovian model and the LQG problem, in Proceedings of the American Control Conference,
2005, pp. 294–299 vol. 1.
[54] S. Yüksel, Jointly optimal LQG quantization and control policies for multi-dimensional linear gaussian
sources, in Proc. 50th Annual Allerton Conference onCommunication, Control, and Computing, 2012,
pp. 466–473.
, On optimal causal coding of partially observed markov sources in single and multiterminal settings,
[55]
IEEE Trans. Information Theory, 59 (2013), pp. 424–437.
[56]
, Jointly Optimal LQG Quantization and Control Policies for Multi-Dimensional Systems, IEEE
Transactions on Automatic Control, 59 (2014), pp. 1612–1617.
[57] A. Zaidi, S. Yüksel, T. Oechtering and M. Skoglund, On optimal policies for control and estimation over Gaussian relay channels, Automatica, 49 (2013), pp. 2892–2897.

Appendix A. Calculations for Example 3
In Example 3, we explicitly show the dependence of the second moments of w0 and x1 − x
b1|1
on the applied controls when using a quantizer in its predictive form. Below, we show how to
compute these terms. From the definition of wt , we have:
w0 = E [x1 |z0 , z1 ] − E [x1 |z0 ]

= E [x1 |z0 , z1 ] − E [x0 |z0 ] − u0 .

We can find an expression for the term E [x0 |z0 ], as shown below.
Z

0

2
1
2
√
e−(θ−µ0 ) /2σ0 dθ,
2πσ0
−∞



1
−µ0
=
1 + erf √
,
2
2σ0

P [z0 = −1] =

36

M. RABI, C. RAMESH, AND K.H. JOHANSSON

Z

0

2
1
2
θ e−(θ−µ0 ) /2σ0 dθ,
2πσ0
−∞



−µ0
σ0 h −µ0 2 /2σ02 i
1
1 + erf √
−√
e
,
= µ0 ×
2
2σ0
2π
σ0 h −µ0 2 /2σ02 i
= µ0 × P [z0 = −1] − √
e
2π

P [z0 = −1] × E [x0 |z0 = −1] =

where erf (x) ,

√2
π

Rx
0

√

2

e−t dt. We also have:
Z

0

Z

0

2
2
1
2
2
e−(r−µ0 ) /2σ0 e−(s−r−u0 ) /2σw drds,
−∞ −∞ 2πσ0 σw
P [z1 = −1, z0 = −1]
P [z1 = −1 |z0 = −1] =
.
P[z0 = −1]

P [z1 = −1, z0 = −1] =

We can now find an expression for the terms x
b1|1 = E [x1 |z0 , z1 ] and E [x21 |z0 = −1, z1 = −1]
as
(22)

(23)

D1 (−∞, 0)
P [z1 = −1, z0 = −1]
Z 0 Z 0
2
2
1
2
2
D1 (−∞, 0) =
s e−(r−µ0 ) /2σ0 e−(s−r−u0 ) /2σw drds
−∞ −∞ 2πσ0 σw


D2 (−∞, 0)
E x21 |z0 = −1, z1 = −1 =
P [z1 = −1, z0 = −1]
Z 0 Z 0
2
2
1
2
2
D2 (−∞, 0) =
s2 e−(r−µ0 ) /2σ0 e−(s−r−u0 ) /2σw drds,
−∞ −∞ 2πσ0 σw
E [x1 |z0 = −1, z1 = −1] =

with similar expressions for the event z0 = −1 AND z1 = +1. We then can compute:



E w20 |z0 = −1 = P [z1 = −1 |z0 = −1] × E [x1 |z1 = −1,

+ P [z1 = +1 |z0 = −1] × E [x1 |z1 = +1,

= P [z1 = −1 |z0 = −1] × E [x1 |z1 = −1,

+ P [z1 = +1 |z0 = −1] × E [x1 |z1 = +1,

z0 = −1] − E [x0 |z0 = −1] − α

2

2
z0 = −1] − E [x0 |z0 = −1] − α ,
2
z0 = −1]
2 
2
z0 = −1] − α + E [x0 |z0 = −1] .

The events corresponding to z0 = +1, with z1 = −1 and z1 = +1 result in similar expressions.
Thus:

2


E w20 |z0 = +1 = P [z1 = −1 |z0 = +1] × E [x1 |z1 = −1, z0 = +1] − E [x0 |z0 = +1] − β

2
+ P [z1 = +1 |z0 = +1] × E [x1 |z1 = +1, z0 = +1] − E [x0 |z0 = +1] − β .

2
= P [z1 = +1 |z0 = +1] × E [x1 |z1 = +1, z0 = +1]

2 
2
+ P [z1 = −1 |z0 = +1] × E [x1 |z1 = −1, z0 = +1] − β + E [x0 |z0 = +1] .

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

37

We also have:
h
i
2
Varerr
=
E
x
−
x
b
|z
=
−1
1
0
1|1
t|t

2 
 
 2
= P [z1 = −1 |z0 = −1] × E x1 |z1 = −1, z0 = −1 − E [x1 |z1 = −1, z0 = −1]

2 
 
 2
.
+ P [z1 = +1 |z0 = −1] × E x1 |z1 = +1, z0 = −1 − E [x1 |z1 = +1, z0 = −1]

In Figure 5, E [w20 |z0 = −1] and Varerr
t|t are plotted against α to illustrate the presence of a
dual effect.
Next, we consider how to compute the conditional expectation (22) above. Consider the
scalar linear system:
xt+1 = a xt + ut + wt , t ∈ {0, 1, . . .} ,
where x0 ∼ N (µ0 , σ0 ) , and the noise process wt is IID with distribution N (0, σw ) , and is
independent of current and past controls and states. Suppose that the control ut is causally
computed on the basis of a sequence of quantized outputs: zt . Consider the quantizer Qt (·):
Qt (xt ) = i, if xi ∈ (θi−1 , θi ) ,
where the θi s are measurable with respect to the partial sequence z0t−1 . Under this setup, the
conditional expectation of the state at time 1 in (22) can be expressed as
x
b1|1 = E [x1 |x0 ∈ (θi−1 , θi ) , x1 ∈ ($j−1 , $j )] ,

where for notational clarity, we have denoted the quantization levels at time 0 with the letter θ,
and the levels at time 1 with η. Computing x
b1|1 requires computation of the following double
integral from (22):


Z $j Z θi
2
µ2
µ
(s−u0 )2
a(s−u0 )
a2 r 2
− 21 r 2 −2r 02 + 02 +
+
−2r
1
2
2
2
σ0
σ0
σ0
σw
σw
σw
D1 =
se
dr ds.
2πσ0 σw $j−1 θi−1
Let σ̄ and µ̄ be defined as follows:
1
1
a2
=
+
,
σ̄ 2
σ02 σw2


µ0 a(s − u0 )
2
µ¯s = σ̄
+
.
σ02
σw2
Denote
by g(r) the standard Gaussian probability density e−r
Rr
g(s)ds. Then, we can write:
−∞

2 /2

√
/ 2π, and by G(r), its CDF


2 Z
2
µ2 Z $j
a(s−u0 )
(s−u0 )2
µ0
θi
−
+ σ̄2
+
− 02
1
2
σ̄
1
2
2
2
σ0
σw
√
D1 = √
e 2σ0
e− 2σ̄2 (r−µ̄s ) dr ds,
s e 2σw
2πσ0 σw
2πσ̄
$j−1
θi−1
= (aµ0 + u0 ) · P [x0 ∈ (θi−1 , θi ) , x1 ∈ ($j−1 , $j )]
0 −u0
Z $
q
√j −aµ


2 σ 2 +σ 2
a
w
2
(24)
+ a2 σ0 + σw2 · $ −aµ0 −u
s̃ g (s̃) G A − Bs̃ − G (A − Bs̃) ds̃,
0
0
j−1
√
2
2 2
a σ0 +σw

where A = (θi − µ0 )/σ̄, A = (θi−1 − µ0 )/σ̄ and B = aσ0 /σw . The first term above can be calculated using established routines for calculating multivariable normal probabilities (MATLAB
command mvncdf). The second term can be explicitly calculated using the following indefinite

38

M. RABI, C. RAMESH, AND K.H. JOHANSSON

integral found in Owen’s table of Normal integrals [38]:

 

Z
√
B
AB
A
x g(x) G(A − Bx)dx = − √
g √
G x 1 + B2 − √
1 + B2
1 + B2
1 + B2
− G(A − Bx) g(x).
Finally, the conditional expectation in (23) requires evaluating an integral of the form:


Z $ j Z θi
2
2 2
µ2
µ0
1
r2
0 + (s−u0 ) + a r −2r a(s−u0 )
−
−2r
+
1
2
2
2
2
2
2
2
σ0
σ0
σ0
σw
σw
σw
s2 e
dr ds,
D2 ,
2πσ0 σw $j−1 θi−1
0 −u0
Z $
√j −aµ


2 σ 2 +σ 2
a
w
=σ
e2 $ −aµ0 −u
s̃2 g (s̃) G A − Bs̃ − G (A − Bs̃) ds̃
0
0
j−1
√
2
2 2
a σ0 +σw

+ 2e
σ (u + aµ0 )

0 −u0
Z $
√j −aµ
2
2 2

a σ0 +σw
$j−1 −aµ0 −u0

√

s̃ g (s̃)

2 +σ 2
a 2 σ0
w

+ (u + aµ0 )

where σ
e,

2

0 −u0
Z $
√j −aµ
2
2 2

a σ0 +σw
$j−1 −aµ0 −u0

√

g (s̃)

2 +σ 2
a2 σ0
w





G A − Bs̃ − G (A − Bs̃)


G A − Bs̃ − G (A − Bs̃)

ds̃

ds̃,

p


a2 σ02 + σw2 . Let h(x) = xg(x) G A − Bx − G (A − Bx) . Then,


d [h(x)] = g(x) G A − Bx − G (A − Bx) dx


− x2 g(x) G A − Bx − G (A − Bx) dx


− Bxg(x) g A − Bx − g (A − Bx) dx.

Hence, the first term of D2 is
D3 , σ
e

2

0 −u0
Z $
√j −aµ
2
2 2

a σ0 +σw
$j−1 −aµ0 −u0

√

2 +σ 2
a2 σ0
w

s̃2 g (s̃)




G A − Bs̃ − G (A − Bs̃)

ds̃

ηe

=σ
e2 [h(x)]ηejj−1 + σ
e2 P [x0 ∈ (θi−1 , θi ) , x1 ∈ ($j−1 , $j )]
Z ηej


2
x g(x) g A − Bx − g (A − Bx) dx,
−σ
e B
ηej−1

p
where ηel = (ηl − aµ0 − u0 )/ a2 σ02 + σw2 . To calculate the last integral, we can use the result
found in Owen’s table of Normal integrals [38]:
Z
 √
  √

√
1
2
2 − AB/ 1 + B 2
g
A/
1
+
B
g
x
1
+
B
x g(x) g(A − Bx) dx = −
1 + B2
 √
  √

√
AB
2
2
2
+
g A/ 1 + B
G x 1 + B − AB/ 1 + B .
(1 + B 2 )3/2

By using the above expressions in (22) and (23), we can evaluate the terms E [w20 |z0 = −1]
and Varerr
t|t for different values of α, and plot its dependence in Figure 5.
Appendix B. Evaluating the cost-to-go V0 in Example 4

4, to compute the cost-to-go V0 , we must be able to compute the term
h In Example
i
2
E x
b1|1 z0 . Using the expression for x
b1|1 (see (22)) in Appendix A, the desired quantity

SEPARATED DESIGN FOR NETWORKED LQ CONTROL

39

h
i
E x
b21|1 z0 can be written as

N
 2
 X


2
E x
b1|1 z0 =
P x1 ∈ ($j−1 , $j ) x0 ∈ (θi−1 , θi ) · E x1 z0 = l, z1 = j
j=1

X
D1
1
.
P (x0 ∈ (θi−1 , θi )) j=1 P (x0 ∈ (θi−1 , θi ), x1 ∈ ($j−1 , $j ))
N

=

In the above expression, note that D1 depends on the quantizer cell ($j−1 , $j ) and has been
evaluated in (24) in Appendix A. Also, the second term in (24) has been denoted by ϑ(r, r̄)
in (21) of Example 4. By setting µ0 = 0 as per Example 4 in the expression for D1 above, the
cost-to-go to be minimized can be rewritten as in (20).
(Maben Rabi) School of Computing and Electrical Engineering, Indian Institute of Technology, Mandi, India.
E-mail address: firstname@iitmandi.ac.in
(Chithrupa Ramesh) School of Electrical Engineering, Royal Institute of Technology,
Stockholm, Sweden.
E-mail address: firstname.lastname@ee.kth.se
(Karl Johansson) School of Electrical Engineering, Royal Institute of Technology, Stockholm, Sweden.
E-mail address: firstname.lastname@ee.kth.se

