SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

1

On Distributed Linear Estimation With Observation
Model Uncertainties

arXiv:1709.02040v1 [] 7 Sep 2017

Alireza Sani, Student Member, IEEE, Azadeh Vosoughi, Senior Member, IEEE,

Abstract—We consider distributed estimation of a Gaussian
source in a heterogenous bandwidth constrained sensor network,
where the source is corrupted by independent multiplicative and
additive observation noises, with incomplete statistical knowledge
of the multiplicative noise. For multi-bit quantizers, we derive
the closed-form mean-square-error (MSE) expression for the
linear minimum MSE (LMMSE) estimator at the FC. For both
error-free and erroneous communication channels, we propose
several rate allocation methods named as longest root to leaf
path, greedy and integer relaxation to (i) minimize the MSE
given a network bandwidth constraint, and (ii) minimize the
required network bandwidth given a target MSE. We also derive
the Bayesian Cramér-Rao lower bound (CRLB) and compare
the MSE performance of our proposed methods against the
CRLB. Our results corroborate that, for low power multiplicative
observation noises and adequate network bandwidth, the gaps
between the MSE of our proposed methods and the CRLB are
negligible, while the performance of other methods like individual
rate allocation and uniform is not satisfactory.

I. I NTRODUCTION
Large-scale sensor networks, consisting of battery operated
devices with limited sensing, computation, and communication
capabilities, can impact various applications, including environmental monitoring, surveillance, target tracking, and can
be utilized to perform challenging tasks such as distributed
estimation (DES). For DES, sensors send their locally processed observations to a Fusion Center (FC), that is tasked with
estimating an unknown parameter, via fusing the collective
information received from the sensors, such that the estimation
error at the FC is minimized, subject to practical network
resource constraint(s), such as transmit bit or transmit power
constraint. In this work, we address two important problems
pertaining bandwidth-constrained DES in a sensor network:
(i) given a network bandwidth constraint, we investigate
the quantization rate allocation schemes that minimize Mean
Square Error (MSE) at the FC, (ii) given a target MSE at the
FC, we explore the quantization rate allocation schemes that
minimize the required network bandwidth.
DES has been of great research interest in signal processing
society [2]–[20]. Several researchers have studied quantization
design, assuming that sensors’ quantized observations are sent
over bandwidth-constrained (otherwise error-free) communication channels [2], [3], [6]–[8]. Note that placing a cap on the
total bandwidth can further improve energy efficiency [21]–
[25], because data communication is a major contributer to
the network energy consumption. In particular, [2] designed
Part of this research was presented at the IEEE 82nd Vehicular Technology
Conference, 2015 [1]. This research is supported by the NSF under grants
CCF-1341966 and CCF-1319770.

quantizers that maximize Bayesian Fisher information (FI). [3]
designed identical one-bit quantizers that minimize CramérRao lower bound (CRLB). For a homogeneous network with
Maximum-Likelihood Estimator (MLE) at the FC, [7] investigated one-bit quantizers. For estimating a random parameter
with uniform variable rate quantizers, [8] investigated the
tradeoff between fine quantization of observations of only
a few sensors and coarse quantization of observations of
as many sensors as possible, as well as its effect on FI,
subject to a network bandwidth constraint. For a heterogeneous
network with best linear unbiased estimator (BLUE) at the
FC, [6] proposed a suboptimal bit allocation scheme that
minimizes MSE, subject to a network bandwidth constraint.
For a homogeneous network with MLE at the FC, [5] designed
one-bit quantizers, where each sensor optimizes its quantizer
threshold. For DES of an unknown deterministic parameter
vector, [4] proposed a joint compression-quantization scheme
that first reduces a sensor’s observation to a scalar and then
applies a one-bit quantizer, such that CRLB is minimized. For
a homogeneous network with one bit quantizers at the sensors,
[10] designed a universal linear estimator at the FC, assuming
that the deterministic unknown and the additive observation
noise are in symmetric bounded intervals around zero.
An inherently related problem to DES is the quadratic
Gaussian CEO problem in information theory [26]–[29], where
a team of agents observe a Gaussian source, corrupted by
independent additive Gaussian noises. Agents communicate
their coded messages over error-free channels to the CEO
(FC), that is tasked with estimating the Gaussian source with
minimal MSE distortion. For this problem [26] conjectured a
rate-distortion region, where the decay rate of MSE is inversely
proportional to total source coding rate of agents when the
number of agents goes to infinity. Later, [28] found the ratedistortion region explicitly for arbitrary number of agents. It
is worthwhile to emphasize the differences between DES and
the quadratic Gaussian CEO problem. Addressing the CEO
problem (from the information theoretic perspective) is built
upon fundamental assumptions that are often challenging to be
satisfied in practical settings and hence the produced results
(including the rate allocation schemes in [27], [29]) cannot
be directly applied to sensor networks. In particular, sensors
sample their observations continuously and use infinite-length
complex source codes (jointly typical sequences) to compress
their samples into bits, and the FC utilizes the best decoder
(estimator), in terms of minimizing MSE. These assumptions
disregard several key characteristics of sensor networks, including hardware complexity, computational capability, power,
and delay constraints. In contrast, DES keeps the delay and

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

computational complexity low, as each sensor has one sample
(of an unknown or a vector of unknowns) to quantize.
The bulk of literature on DES assumes that the observation (sensing) model is completely specified [2]–[17], while
some assume that the statistical knowledge of the additive
observation noise is incomplete (noise model uncertainty) [7],
[10]. Centralized estimation with multiplicative and additive
observation noises has been studied before in [30]–[32].
Despite its great importance, few researchers have studied
DES with both multiplicative and additive observation noises.
DES with Gaussian multiplicative and additive observation
noises, one-bit quantizers at the sensors and MLE at the FC
has been investigated very recently in [33], [34], respectively,
for vector and scalar unknown deterministic parameters. We
also note that most DES literature is focused on one-bit
quantization [2]–[5], [7], [9], [10], [15], [33], [34], assuming
that the dynamic range of the unknown is equal to or less than
that of the additive observation noise [10]. Interestingly, [7]
argued that there can be a significant gap between the CRLB
performance based on one-bit quantization and the clairvoyant
benchmark (unquantized observations are available at the FC),
when the dynamic range of the unknown is large with respect
to the additive observation noise variance. Also, [33] reported
that in presence of multiplicative noise, low power additive
observation noise can negatively impact the performance of
MLE based on one-bit quantization. The works in [7], [33]
motivate us to consider DES based on multi-bit quantization.
Our Contributions: We consider DES of a Gaussian source,
corrupted by independent multiplicative and additive observation noises, in a heterogeneous bandwidth-constrained
network. Similar to [6], [7], we choose the total number
of quantization bits as the measure of network bandwidth.
Different from [33], [34], we assume that the distribution of
the multiplicative observation noise is unknown and only its
mean and variance are known (observation model uncertainties). To overcome the limitations caused by the observation
model uncertainties, the FC employs linear minimum MSE
(LMMSE) estimator to fuse the quantization bits received from
sensors over orthogonal channels. We consider both error-free
and erroneousness communication channels [21], [35], using
binary symmetric channel (BSC) model. We focus on the
effects of multiplicative noise, observation model uncertainties
and quantization errors on the accuracy of estimating the
Gaussian source. We derive a closed-form expression for the
MSE of LMMSE estimator and consider two system-level
constrained optimization problems with respect to the sensors’
quantization rates: in (P1) we minimize the MSE given
a network bandwidth constraint, and in (P2) we minimize
the required network bandwidth given a target MSE. To
address these two problems, we propose several rate allocation
schemes. In addition, we derive the Bayesian CRLB and
compare the MSE performance of the proposed schemes
against the CRLB.
II. S YSTEM M ODEL AND P ROBLEM S TATEMENT
We consider a network with K spatially distributed heterogeneous sensors and a FC, where the FC is tasked with
estimating a realization of a Gaussian source θ ∼ N (0, σθ2 ),

2

via fusing the collective received data from all sensors. Each
sensor makes a noisy observation of θ, where both multiplicative and additive observation noises are involved. Let
xk denote the scalar noisy observation of θ at sensor k. We
assume the following observation model:
xk = hk θ + nk ,

for

k = 1, ..., K,

(1)

where hk and nk are multiplicative and additive observation
noises, respectively. Also, hk , nk , θ are uncorrelated. We
assume nk ∼ N (0, σn2 k ), E{hk } = 1 ∀k, and1 var(hk ) = σh2 k .
Sensor k employs a uniform quantizer with Mk quantization
levels and quantization step size ∆k . The quantizer maps
xk into a quantization level mk ∈ {mk,1 , ..., mk,Mk }, where
k )∆k
for i = 1, ..., Mk . We assume xk lies
mk,i = (2i−1−M
2
in the interval [−τk , τk ] almost surely, for some reasonably
k
large value of τk , and we let ∆k = M2τ
. These imply
k −1
that the uniform quantization mapping rule can be described
as the following: if xk ∈ [mk,i − ∆2k , mk,i + ∆2k ], then
mk = mk,i , if xk ≥ τk , then mk = τk , and if xk ≤ −τk ,
then mk = −τk . Following quantization, sensor k maps the
index i of mk,i into a bit sequence of length rk = log2 Mk
and sends rk bits to the FC. Sensors transmit over orthogonal bandwidth-constrained error-free communication channels.
Error-free communication channel model has been used before
in several classical works on DES, examples are [1]–[8]. In
Section VIII we extend our analytical results to the case
where these channels are modeled as independent BSC with
different error probabilities.
PK To capture the network bandwidth
constraint we assume
k=1 rk ≤ Btot . In the absence of
knowledge of joint distribution of mk ’s and θ, we resort
to the linear minimum MSE (LMMSE) estimator [36] to
form the estimate θ̂ = Gm at the FC, where G is the
1 × K linear estimation operator and m = [m1 , ..., mK ]T
is the vector of transmitted quantization levels for all sensors.
The LMMSE estimator has a low computational complexity
and only requires the knowledge of moments E{θmT } and
E{mmT } to form θ̂. Let D = E{(θ − θ̂)2 } denote the MSE
corresponding to the LMMSE estimator, where D depends on
rk ∀k. We consider two system-level constrained optimization
problems with respect to optimization variables rk ∀k. In
the first problem, we minimize D subject to the network
bandwidth constraint. In the second problem, we minimize
the total number of transmitted bits subject to the constraint
on D. In other words, we are interested to solve the following

1 For the general case E{h } = µ we can scale x and obtain x0 =
k
k
k
k
h0k θ + n0k , where x0k = xk /µk , h0k = hk /µk , n0k = nk /µk , E{h0k } = 1,
2 /µ2 , n0 ∼ N (0, σ 2 /µ2 ). Thus without loss of generality,
var(h0k ) = σh
nk
k
k
k
k
we assume E{hk } = 1, ∀k.

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

3

two constrained optimization problems:
(P1)

minimize D({rk }K
k=1 )
s.t.

K
X

k=1

(P2)

(2)

rk ∀k

rk ∀k

s.t.

E{mmT } = σθ2 11T + diag(

rk ≤ Btot , rk ∈ Z+ , ∀k,

minimize

K
X

(3)

rk

k=1
D({rk }K
k=1 )

≤ D0 , rk ∈ Z+ , ∀k,

θ̂ = Gm where G = E{θm }(E{mm })

(8)

k=1

k

Proposition 1 summarizes the expressions for θ̂, D in (4),(5).

III. C HARACTERIZING MSE FOR LMMSE ESTIMATOR
We wish to characterize D in terms of the optimization
variables {rk }K
k=1 . From [36] we have:
T

1
1
, ...,
),
α1
αK

where αk−1 = σk2 +σ2k . Applying matrix inversion Lemma [36]
to (8) we find:

α2k
αk − −2 P
, if k = l
σθ + K
T −1
k=1 αk
[(E{mm }) ]k,l =
α
α
k
l
− −2 PK
,
if k 6= l
σ +
α
θ

where D0 is the pre-determined upper bound on D.

T

where σk2 = σθ2 σh2 k + σn2 k . Consequently matrix E{mmT }
can be written as the following:

−1

D = σθ2 − E{θmT }(E{mmT })−1 E{θmT }T ).

,

Proposition 1. The LMMSE estimator θ̂ and its corresponding
MSE D, based on the quantized observations {mk }K
k=1 are:
θ̂ =

(4)

ck mk where ck =

k=1

(5)

To find E{θmT } and E{mmT } in (4), (5) we need to delve
into statistics of the quantization errors.
• Characterizing E{θmT } and E{mmT }: For sensor k,
let the difference between observation xk and its quantized
version mk , i.e., k = xk −mk , be the corresponding quantization error. In general, k ’s are mutually correlated and also
are correlated with xk ’s. However, in [37] it is shown that,
when highly correlated Gaussian random variables are coarsely
quantized with uniform quantizers of step sizes ∆k ’s, quantization errors can be approximated as mutually independent
random variables, that are uniformly distributed in the interval
[− ∆2k , ∆2k ], and are also independent of quantizer inputs. Here,
since θ and nk ’s in (1) are Gaussian, conditioned on hk ’s
observations xk ’s are correlated Gaussian that are quantized
with uniform quantizers of quantization step sizes ∆k ’s. Thus
k ’s are approximated as mutually independent zero mean
∆2
uniform random variables with variance σ2k = 12k , that are also
independent of xk ’s (and hence independent of θ and nk ’s).
Using the aforementioned assumptions and approximations for
the quantization errors, kth element of E{θmT } becomes:

K
X

D=

σθ−2

+

1
PK

σθ−2

k=1

+

αk

.

αk
PK

k=1

αk

,
(9)

Examining (9), we note that αk represents the contribution
of sensor k in reducing the overall MSE at the FC. Also,
αk can be viewed as an indicator for the quality of received
message from sensor k: the larger αk is, the more reliable is
the received message. It is easy to verify that αk is increasing
in rk and decreasing in σk2 .
Remark 1: When all observations xk ’s are available at
the FC with full precision (so-called centralized
estimation),
PK
the LMMSE estimator would be θ̆ =
k=1 bk xk , where
σ −2
Pk
−2 , with its
σθ−2 + K
k=1 σk
1
P
−2 . This clairvoyant
σθ−2 + K
k=1 σk

bk =

corresponding MSE Dc =

estimator can be used as our
performance benchmark, since D > Dc .
Proposition 2. In a network with homogeneous sensors, i.e.,
σk2 = σ 2 , ∀k, and all sensors quantize their observations with
identical quantizers of step size ∆, the MSE gap between two
linear estimators θ̂ and θ̆ is:
D − Dc =

12(K +

∆2
K∆2
. (10)
−2 2 ≤
2
12K
+ σ ))(K + σθ σ )

σθ−2 (σ 2

Based on (10), if ∆ → 0, then D → Dc . Additionally, if
K → ∞, then D → Dc even for large ∆. These conclusions
= Ehk {hk E{θ2 } + E{nk }E{θ} − E{k }E{θ}} = σθ2 . (6)
still hold true in a network with heterogeneous sensors, where
T
2 T
T
Hence we have E{θm } = σθ 1 , where 1 = [1, ..., 1] . sensor k quantizes with step size of ∆k . If (max∆k ) → 0, then
αk → σk−2 and according to (9), D → Dc . On the other hand,
Similarly, for (k, l)th element of E{mmT } we have:
according to (9) and noting that αk > 0 for active sensors,
E{mk ml } = Ehk ,hl {E{mk ml |hk , hl }} =
(7) MSE always decreases as the number of active sensors in(a)
creases. Thus as K → ∞ , we have D → Dc → 0.
Ehk ,hl {E{(hk θ + nk − k )(hl θ + nl − l )|hk , hl }} =
E{θmk } = Ehk {E{θmk |hk }} = Ehk {E{θ(xk − k )|hk }}

Ehk ,hl {hk hl E{θ2 } + E{nk nl } + E{k l }}=

Ehk ,hl {hk hl }E{θ2 } + E{nk nl } + E{k l },

where for (a) we have used the assumptions that (i) nk ’s and
θ are uncorrelated, (ii) k ’s and θ are uncorrelated (iii) nk ’s
and k ’s are uncorrelated. Having (7), and noting that hk ’s are
uncorrelated with unit means, we reach:
(
σθ2 + σk2 + σ2k , if k = l
E{mk ml } =
σθ2 ,
if k 6= l

IV. S OLVING C ONSTRAINED P ROBLEM (P1)
Since the optimization variables rk ’s are integer and D is
a non-linear function of rk ’s, (P1) is a non-linear integer
programing problem and is NP-hard.PEven if the inequality
K
constrain holds with equality, i.e.,
k=1 rk = Btot , solv
tot −1
ing (P1) requires a brute-force evaluation over K+B
K−1
choices. For K = 50 and Btot = 60 bits, the number of
evaluations would be in the order of 1031 . The following

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

4

lemmas help us find strategies that reduce the computational
complexity required for solving (P1).
Lemma
1. Minimizing D in (9), is equivalent to maximizing
PK
α
k=1 k with the same constraints as in (2).
Proof. Since σθ−2 > 0, it is axiomatic.

Lemma 2. Suppose {rk∗ }K
k=1 is the optimal solution to (P1).
PK
Then k=1 rk∗ = Btot .
∆2

Proof. Note αk is a function of rk through σ2k = 12k =
τk2
∂D
3(2rk −1)2 . It is easy to verify that ∂rk ≤ 0 and D is a
decreasing function of rk ’s. Thus the optimal
PK solution satisfies
the network bandwidth constraint, i.e., k=1 rk∗ = Btot .

Lemma 3. Without loss of generality, suppose sensors are
2
sorted2 such that σ12 ≤ σ22 ≤ ... ≤ σK
. Then the optimal
∗
∗
solution satisfies ri ≥ rj for i < j.
Proof. Suppose {rk }K
k=1 is the optimal solution, such that
ri < rj for i < j. Also, suppose {rk0 }K
k=1 is a solution
of (P1), which is not necessarily optimal, such that rk0 =
rk for k 6= i, j and ri0 = rj , rj0 = ri . Consider the following:
K
X

k=1

+

+

αk (rk0 ) −

z

σi2 +

z

σj2 +

K
X

αk (rk ) =

k6=i,j

k=1
=δ2

1
τ2
3(2 −1)2
rj

1
τ2
3(2rj −1)2

z
K
X

}|
−

1
σi2 +

=δ3

}|
−

}|

αk (rk ) −

K
X

{

αk (rk )

6=i,j

{

τ2
3(2ri −1)2

1
σj2 +

=δ1

{

τ2
3(2ri −1)2

>0.

PK
One can verify δ1 = 0, δ2 > 0, δ3 > 0, thus k=1 αk (rk0 ) >
PK
k=1 αk (rk ). According to (9), the MSE associated with
{rk0 }K
k=1 should be less than that of the optimal solution
{rk }K
k=1 , which is a contradiction. In this proof, we assumed
τk = τ, ∀k, although the proof is still valid for unequal
τk ’s, provided that τi ≤ τj , which is satisfied if we choose
τk2 ∝ var(xk ) = σθ2 + σk2 .
In the next subsections, we propose four methods for solving
(P1): A) longest root to leaf path method, which is optimal
with less computational complexity than that of brute force, B)
greedy method, C) integer relaxation method, D) individual
rate allocation method. The suboptimal B), C), D) methods
have moderate to low computational complexity.
A. Longest Root to Leaf Path Method
We view (P1) as the problem of finding the longest “root
to leaf” path in a weighted directed binary tree, where there is
a constraint on the number of P
edges from “root to leaf” [38].
K
In fact our objective function k=1 αk can be viewed as the
length of the path to bePmaximized, where the constraint on
K
the number of edges is k=1 rk ≤ Btot . Fig. 1 demonstrates
2 We

the problem for K = 3 and Btot = 5 bits. The nodes are
tagged with indices of sorted sensors and visiting node k is
translated to “allocating one bit to sensor k”. The edge3 weight
wk (r) is the weight of the edge entering node k and r is the
number of prior visits of node k, i.e., wk (r) = αk (r + 1) −
αk (r) = αk (number of bits allocated to sensor k so far+1)−
αk (number of bits allocated to sensor k so far). For instance,
the green path in Fig. 1 is associated with the rate allocation
[r1 , r2 , r3P
] = [3, 2, 0], and the corresponding objective function
K
value is k=1 αk = w1 (1)+w1 (2)+w2 (0)+w2 (1) = α1 (3)+
α2 (2). To solve (P1), one needs to construct the associated
binary tree with structures conforming to lemmas 2 and 3,
then uses a search algorithm, such as depth first search (DFS)
[38] to discover all possible “root to leaf” paths, and choose
the path that results in the maximum objective function value.
For K = 3 and Btot = 5 bits, Fig. 1 shows there exist
5 “root to leaf” paths, all conforming to lemmas 2 and 3,
corresponding to 5 distinct rate allocation among 3 sensors
[r1 , r2 , r3 ] ∈ {[5, 0, 0], [4, 1, 0], [3, 2, 0], [3, 1, 1], [2, 2, 1]}. We
recognize these as different partitions of the integer number
5, with 3 or fewer addends [39], i.e., the number of possible
“root to leaf” paths in a binary tree constructed as explained,
conforming to lemmas 2 and 3, and characterizing (P1), is
equal to the number of solutions to the following equation:

assume sorted sensors throughout this work for all scenarios.

(11)

r1 + r2 + ... + rK = Btot
s.t. r1 ≥ r2 ≥ ... ≥ rK

rk ∈ Z+ .

Although the number of ways one can partition an integer
number does not have a closed form formula, the literature
[39] provides some useful asymptotic formulas or recurrence
relations. Suppose qk (n) is the number of solutions to (11),
then we have the recurrence relation qk (n) = qk−1 (n)−qk (n−
k), with q0 (n) = 0 , qk (1) = 1 [39]. For K = 50 and Btot = 60
bits, qK (Btot ) is in the order of 106 , which is much smaller
than that of brute force 1031 . The computational complexity
of this method is still high for very large networks, e.g., K ≥
100, and hence its application is most beneficial for finding
the optimal solution of small to moderate size networks.
B. Greedy Method
Recall from Lemma 1 that the maximum
PK reduction in D
corresponds to the maximum increase in k=1 αk (rk ). Hence,
our proposed greedy method in each iteration allocates one
bit
PKto the sensor that guarantees the maximum increase in
k=1 αk (rk ), i.e., in each iteration the algorithm loads one
bit on sensor k ∗ where k ∗ = argmax Ik (rk ) = argmax(αk (rk +
k

k

1)−αk (rk )). The iteration ends when all Btot bits are allocated
to the sensors. Following algorithm illustrates the details:
For K = 3 and Btot = 5 bits Fig. 1 shows the accepted
decisions by the greedy method at each iteration/decision
epoch with green arrows and the rejected decisions with red
arrows. Note that the initial point is always rk = 1, rk = 0 for
k = 2, ..., K, since the first bit is always allocated to sensor 1
(for sorted sensors sensor 1 has the largest αk or smallest σk2 ).
The second bit can be allocated to either sensor 1 or sensor 2,
3 For

definition of weight wk (0), we consider αk (0) = 0.

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

2 K
Data: Btot , {τk }K
k=1 , {σk }k=1
Result: rate allocation {rk∗ }K
k=1
initialization;
r1 = 1, rk = 0 for k = 2, ..., K, S = {1, 2}
for i = 1 : Btot do
k ∗ = argmax(αk (rk + 1) − αk (rk ))
k∈S

rk∗ = rk∗ + 1
S = {k|rk < rk−1 } ∪ {1}
end
Algorithm: greedy method for rate allocation in (P1)

5

Hearing all Ij , j 6= k, sensor k increases rk by one if it has
the largest Ik among all sensors. Doing the mentioned process
for Btot times would complete the rate allocation.
C. Integer Relaxation Method
Convex relaxation for solving combinatorial optimization
problems such as (P1) is a rather old technique, that has been
widely used in research and applied to a variety of applications
[40], [41]. Relaxing the integer constraint on rk ’s and letting
them be positive numbers and using lemmas 1 and 2, we
consider the following relaxed problem:
maximize

(P0 1)

rk ∀k

s.t.

K
X

k=1

K
X

(12)

αk (rk )

k=1

rk = Btot , rk ∈ R+ , ∀k.

The Lagrangian for (P0 1) is:
L({rk , µk }K
k=1 , λ) =
Fig. 1: Longest root to leaf path and Greedy method
i.e., k ∗ = argmax(αk (rk + 1) − αk (rk )), this is equivalent to
k∈{1,2}

k∗ =1

making the decision w1 (1) ≷ w2 (0) (look at the weights on
k∗ =2

the edges in Fig. 1). The sequence of green arrows in Fig.1,
1 → 1 → 2 → 1 → 3 is associated with the rate allocation
[r1 , r2 , r3 ] = [3, 1, 1].
In the following, we look at the computational complexity
of the greedy method in two cases: case (a) Btot ≤ K, in this
case the first bit has to be allocated to sensor 1, the second
bit can be allocated to either sensor 1 or sensor 2. In general
the ith bit, for 1 ≤ i ≤ Btot , can be allocated to one of
at most i sensors (sensor 1,..., sensor i). In other words, in
the ith decision epoch, greedy method should find the best
sensor among eligible candidates in set S = {k|k ≤ i, rk <
rk−1 } ∪ {1}, where S has a maximum of i elements. Thus
allocating Btot bits among K sensors needs calculation of
Ik (rk ) = αk (rk + 1) − αk (rk ) for at most 2 + 3 + ... + Btot =
B 2 +B
( tot 2 tot )−1 times. Case (b) Btot > K, adopting a reasoning
similar to case (a), in this case allocation of the first K bits
2
needs at most ( K 2+K )−1 calculations. Each of the remaining
Btot − K bits can be allocated to one of at most K sensors,
leading into (Btot − K) × K number of calculations at most.
Hence, the overall number of evaluations at most would be
2
( K 2+K ) − 1 + (Btot − K) × K ≈ K(Btot − K/2). For K = 50
and Btot = 60 bits, where Btot > K, the number of evaluations
would be in order of 103 .
Remark 2: In the absence of a powerful FC, the proposed greedy algorithm can be implemented in a distributed
way, assuming sensors can broadcast and hear the broadcast
messages by other sensors. Sensor k calculates the value
Ik (rk ) = αk (rk + 1) − αk (rk ), and broadcasts the value.

K
X

k=1

αk (rk ) − µk rk + λ(

K
X

k=1

rk − Btot ).

In the following we apply the first order KKT necessary
optimality conditions for (P0 1) which generate a closed-form
solution for rk ’s. Afterwards, we prove that the obtained solution satisfies the second order sufficient optimality conditions.
• Necessary Optimality Conditions
After solving the KKT conditions corresponding to (12), we
find:

+


q
2
(13)
rk† = 0.5 log2 τk2 (λ† − σk2 − λ† − 2λ† σk2 )−1 − log2 3

where [x]+ = max(0, x) and λ† in (13) is the solution to
following equation:
†

g(λ, {σk2 , τk2 }K
k=1 ) =

†

K
Y

k=1

τk−2 (λ − σk2 −

q
λ2 − 2λσk2 ) = T, (14)

†
2
in which K = max{k|λ† > 2σK
† , rk > 0}, and T =
4
. Consider a new equation which is obtained by
3
replacing K † in (14) with M . The new equation, which we
refer to as (140 ), does not necessarily have a real solution for
2
λ, such that λ > 2σM
for any value of M ∈ {2, 3, ..., K}. In
order to find the requirements for (140 ) to yield a real solution
for λ, we present the following Lemma and ensuing discussion. For simplicity, we drop the parameters {σk2 , τk2 }M
k=1 in
g(λ, {σk2 , τk2 }M
k=1 ) and indicate it as g(λ, M ).
−Btot −K †

†

Lemma 4. The function g(λ, M ) is a decreasing function of
2
λ, for λ > 2σM
.
QM
Proof. Considerpg(λ, M ) =
k=1 gk (λ), where gk (λ) =
τk−2 (λ − σk2 − λ2 − 2λσk2 ). We can verify that gk (λ)’s are
λ−σ 2
k (λ)
strictly decreasing in λ, because dgdλ
= 1 − √ 2 k 2 < 0.
λ −2λσk

Since all gk (λ)’s are strictly decreasing and positive, i.e.,
gk (λ) > 0, ∀k, we conclude that g(λ, M ) is a strictly
decreasing function in λ.

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

Having Lemma 4, we consider two scenarios that occur
2 ,
when solving g(λ, M ) = T : case (i) when T ≤ g(λ)|λ=σM
in this case according to Lemma 4 we have a unique real
2 , in this case
solution for λ; case (ii) when T > g(λ)|λ=σM
there is no real solution for λ. Hence we need to increase the
2
value of g(λ)|λ=σM
to reach T . The only way to accomplish
this is decreasing the number of active sensors that contribute
2
2 and deactivating sensors with largest σ
to g(λ)|λ=σM
k values,
until we find a real solution for λ or only one active sensor
remains. In other words, solving (140 ) in case (ii) translates
into obtaining the set of active sensors A = {1, 2, ..., K † } and
allocating Btot among these active sensors.
Remark 3: The solution in (13) can be implemented in a
distributed fashion. FC solves (14) and broadcasts λ† . Each
sensor calculates its own rk† using λ† via (13). If a sensor
finds its rate to be zero or a non-real value, it means that the
sensor must be inactive. The integer relaxation method has a
very low computational complexity, since it requires finding
the root of the monotonic function in (14) once and, and then
calculating the rates via (13) for a maximum of K times.
We can find an approximate closed form solution for (14)
under the special condition when (λ − σk2 )2 is large compared
to σk4 . Rewriting the function gk (λ) = τk−2 [λ − σk2 − ((λ −
σk2 )2 − σk4 )1/2 ] and keeping only the first two terms in the
binomial expansion of the term ((λ−σk2 )2 −σk4 )1/2 , we obtain
4
4
σk
σk
gk (λ) ≈ 2τ 2 (λ−σ
2 ) ≈ 2τ 2 λ . Substituting the approximation
k
k
Q k
Q −1 4 −2 −1
in (14), we reach
σk τk λ , based
k gk (λ) ≈
k2
on which the Lagrange multiplier can be approximated as
Btot
Q
λ† ≈ 1.5 η 2 4 K † , where η = k σk2 τk−1 . Substituting the
approximation for λ† in (13) gives the following:
+

τk
Btot
+
log
(η
)
.
(15)
rk† ≈
2
K†
σk2
Examining (15), we note that first term inside the bracket
is common among active sensors and can be perceived as
average rate, whereas the second term (which depends on τk ,
σk2 ) differs among active sensors, such that an active sensor
with a larger ratio στk2 is allocated a larger rk† . Consistent
k
with the assumption in the proof of Lemma 3, suppose
τk = κvar(xk ) = κ(σθ2 + σk2 ). Interestingly, the second term
σ2
σ2
in (15) takes the form log2 (κη(1 + σθ2 )), where the ratio σθ2 ,
k
k
can be viewed as the observation SNR in (1). We consider two
scenarios: (i) high observation SNR: the quantization rates are
large (fine quantization) and less sensors become active for a
given Btot . (ii) low observation SNR: the quantization rates
are smaller (coarse quantization) and more sensors become
active for the same Btot value, compared with that of scenario
(i). Substituting (15) in (9) and after some simplifications we
establish the bound D ≤ Dc (1 +

−

Btot

2
K†
σK
†4
3η 2

).

• Sufficient Optimality Conditions
The objective and equality constraint functions in (12)
are twice differentiable. Hence, the second order sufficient
optimality conditions for the solution in (13) and (14) to be

6

strict minimum for (P0 1) are [42, p.301, proposition 3.2.1]:


λ=λ†
y > 0, ∀y 6= 0
−y T ∇2r L({rk , µk }K
,
λ)|
†
k=1
rk =rk ∀k
X †
with [∇(
rk − Btot )]T y = 0, (16)
k∈A

where
is the Hessian matrix of the
Lagrangian in (13), and ∇( k∈A rk† − Btot ) is the gradient of
the equality constraint in (12), both evaluated at the solution
in (13) and (14). It is easy to verify that the Hessian matrix
is diagonal with entries:
∇2r L({rk , µk }K
k=1 , λ)
P

=β

[∇2r L({rk , µk }K
k=1 , λ)]k,k

z }| {
2
2
2
2 σk − σk
= σk (ln4)
, ∀k. (17)
2
2
(σk + σk )3

Noting that the denominator in (17) and σ2k (ln4)2 are positive
numbers we probe into β evaluated at the solution in (13):
q
2
β † = (σ2k − σk2 ) = λ† − 2σk2 − λ† − 2λ† σk2 < 0. (18)

The inequality in (18) is true, because β (which is a function
2
of λ† ) is decreasing in λ† andqnoting that λ† > 2σK
† in (18),

we have Sup(λ† − 2σk2 −
k,λ†
h
λ=λ†
2
∇r L({rk , µk }K
k=1 , λ)|r =r †
k

k

2

λ† − 2λ† σk2 ) = 0. Therefore
i
< 0 ∀k, confirming that
∀k
k,k

the sufficient optimality conditions in (16) are satisfied.
• Migration to Integer Solution
We describe an approach for migrating from the continuous
solution in (13) to an integer solution satisfying the integer
constraint in (2). We round the rates to nearest integers [43],
[44]. In case the rounding violates the bandwidth constraint,
we reduce the smallest rate by one, because this sensor is more
likely to be the weakest player in the network (in the sense
that it has the least contribution to D) until the bandwidth
constraint is satisfied. Although rounding the rates to nearest
integers may sounds trivial, our simulation results corroborate
that the performance loss is negligible, while at the same
time it keeps the rate allocation scheme simple and easily
implementable.
D. Individual Rate Allocation Method
Examining (10) closely we realize that allocating Btot
among sensors in order to minimize D presents a tradeoff between the number of active sensors and quantization
accuracy. If Btot is distributed among only few sensors, we
can have fine quantization, i.e., small K and small ∆. On
the other hand, if Btot is distributed among many sensors, we
can only have coarse quantization, i.e., large K and large ∆.
Consider a network with homogeneous sensors σk2 = σ 2 , ∀k.
Given Btot , there exists an optimal number of active sensors
K opt , associated with an optimal quantizationP
rate ropt , where
K
K opt ropt = Btot . Thus the maximization of k=1 αk = Kα,
where we substitute K = Btot /r, reduces to the following
one dimensional simple search for ropt in the set Sh =
{1, ..., Btot }:
ropt = argmin{r(σ 2 + τ 2 3−1 (2r − 1)−2 )},
r∈Sh

(19)

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

 tot 
and consequently K opt = B
r opt . Modifying the solution in
(19) for heterogeneous networks, we reach the following:
rksopt = argmin{r(σk2 + τk2 3−1 (2r − 1)−2 )},

(20)

r∈Sk

in which S1 = {1, ..., Btot }, Sk = {1, ..., Btot −
P
k−1 sopt
} for k = 2, ..., K, K sopt = max {k| Sk 6= {0}},
i=1 ri
sopt
rk
and K sopt are the rates and number of active sensors,
respectively. Note that the solution in (20) is integer and
unique (since the objective function in (20) is convex for
r > 0). The drawback of the proposed rate allocation method
is that for large Btot , all Btot bits may not be allocated to
PK sopt sopt
sensors, i.e.,
< Btot , causing the solution in
k=1 rk
(20) to deviate from the optimal solution according to Lemma
2. This method is similar to the one in [6], with the difference
that starting from sensor 1, we update and reduce the search
domain, i.e., Sk for the next sensor. This accelerates the rate
allocation process. Additionally, search domain reduction in
some scenarios would help to use all Btot bits by activating
more sensors with coarse quantizers though. The proposed
method exhibits a moderate computational complexity, since
it only requires solving (20) for a maximum of K times and
it is almost fully distributed [6].
V. S OLVING C ONSTRAINED P ROBLEM (P2)
Different from (P1), satisfying the MSE constraint
D({rk }K
k=1 ) ≤ D0 in (P2) enforces the number of active
sensors to exceed a minimum number Kmin . Lemma 5 provides Kmin .
Lemma 5. To satisfy the the constraint D({rk }K
k=1 ) ≤ D0
we needPat least Kmin active sensors, where Kmin =
K
min{K| k=1 σk−2 > D00 = D0 −1 − σθ−2 }.

Proof. Considering (9) and the definition
that
PK of αk ’s, 0we find−1
D({rk }K
D0 = D0 −
k=1 ) ≤ D0 is equivalent to
k=1 αk ≥ P
K
σθ−2 . Thus (P2) is equivalent to minimizing k=1 rk such
PK
that k=1 αk ≥ D00 and rk ∈ Z+ , ∀k. Since αk > 0, ∀k,
PK
PK
1
we can increase k=1 αk = k=1 ( σ2 +σ
2 ), via increasing
k
k
the number of active sensors, until the MSE constraint is
satisfied. This implies that the minimum number of active
sensors can be found by letting σ2k = 0, ∀k, i.e., Kmin =
PK
min{K| k=1 σk−2 > D00 }.
In the following we propose three methods for solving
(P2): A) greedy method, B) integer relaxation method, C)
individual rate allocation method. We obtain these methods
via applying some modifications to the proposed methods in
section IV.
A. Greedy Method
According to Lemma 5, we need at least Kmin active
sensors. Therefore, we initiate the algorithm with rk = 1 for
k ∈ {1, ..., Kmin } and let rk = 0 otherwise, and go through
the greedy method until the MSE constraint is satisfied.

7

2 K
Data: D0 , σθ2 , {τk }K
k=1 , {σk }k=1
∗ K
Result: rate allocation {rk }k=1
initialization;
rk = 1 for k ∈ {1, ..., Kmin } o.w. rk = 0,
S = {1, Kmin + 1}, D00 = D0 −1 − σθ−2 ,
PKmin
αk (rk )
d = k=1
while d < D00 do
k ∗ = argmax(αk (rk + 1) − αk (rk ))
k∈S

rk∗ = rk∗ + 1
d = d − αk∗ (rk∗ − 1) + αk∗ (rk∗ )
S = {k|rk < rk−1 } ∪ {1}
end
Algorithm: greedy method for rate allocation in (P2)

B. Integer Relaxation Method

Let (P0 2) be the corresponding relaxed problem of (P2).
Solving the first order KKT necessary optimality conditions
for (P0 2) yields a similar solution to (13) as the following:

+


q
2
‡
−1
2 ‡
2
2
‡
‡
− log2 3
(21)
rk = 0.5 log2 τk (λ − σk − λ − 2λ σk )

PK
One can show that the objective function, i.e.,
k=1 rk is
a strictly increasing function of αk ’s. Hence, the optimal
0
solution
satisfy the MSE constraint as equality,
PKto (P 2) must
i.e., k=1 αk = D00 . Using the MSE equality constraint we
find that λ‡ in (21) is the solution to the following equation:

‡
f (λ, {σk2 }K
k=1 )

‡

=

K
X

k=1

λ−

p

1
= D00 ,
λ2 − 2λσk2

(22)

‡
2
where K ‡ = max{k|k ≥ Kmin , λ‡ > 2σK
‡ , rk > 0}.
Similar to what we did for the solution in (13), one can verify
that the solution in (21) satisfies the second order sufficient
optimality conditions in [42, p.301, proposition 3.2.1]. Note
that (22) does not necessarily have a real solution for λ. We
first let K ‡ = K, i.e., the largest possible value for K ‡
in the feasible set F = {Kmin , ..., K} and solve (22). If
there is no real solution for λ we decrease the number of
active sensors by one, i.e., K ‡ = K − 1, and solve (22).
We continue reducing the number of active sensors one by
one until we reach a real solution for λ‡ or K ‡ = Kmin
(the smallest possible value for K ‡ in the feasible set). Even
when K ‡ = Kmin it is still possible that solving (22) does
min
not yield a real solution for λ‡ . Since f (λ, {σk2 }K
k=1 ) is
an increasing function of λ, this scenario would occur when
min
2
D00 < f (λ, {σk2 }K
. In this scenario we let
k=1 )|λ=2σK
min
‡
2
2
λ = 2σKmin . Substituting λ‡ = 2σK
in (21) and then
min
PKmin
‡
rk ’s in k=1 αk , we obtain:
!
K
K
min
min
X
X
1
p
2
αk =
|λ=2σK
> D00 ,
2 − 2λσ 2
min
λ
−
λ
k
k=1
k=1

implying that the MSE constraint is met. Using similar ap-

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

proximation that led us to (15), we can approximate (21) as:
K

rk‡ ≈ [log2 (

8

In the following, we find E{Fa }, E{Fb }. We have:
=1

X
τk
)+0.5log2 (K ‡ 3−1 (
σk−2 −D00 )−1 )]+ . (23)
2
σk
k=1

Equation (23) shows as target MSE approaches its feasible minimum, i.e., as D0 → σ−2 +P1K σ−2 and D00 →
k=1 k
θ
PK
‡
−2
σ
,
the
rates
r
’s
become
very
large,
i.e., rk‡ → ∞.
k=1 k
k
C. Individual Rate Allocation Method
Following a similar reasoning to the one provided in Section
IV-D for a homogeneous network and recalling the discussion
on satisfying the MSE constraint as equality in Section V-B,
we conclude that, given D00 , there exists an optimal number of
active sensors K opt , associated with an optimal quantization
rate ropt , where K opt α(ropt ) = D00 and our problem is to
minimize K opt ropt subject to this MSE equality constraint.
This optimization problem for a heterogeneous network, reduces to almost the same as in (20), with a difference that the
search domain includes any positive integer number, i.e.,
rksopt = argmin{r(σk2 + τk2 3−1 (2r − 1)−2 )},

(24)

r∈Z+

K sopt = min{k|Kmin ≤ k ≤ K,

k
X
i=1

αi (ri ) ≥ D00 }.

(25)

Note that there is no need to solve (24) for all sensors, since
the rate allocation continues only till we find K sopt in (25).

VI. C RAM ÉR R AO LOWER B OUND
We derive the CRLB for any Baysian estimator of θ
based on quantized observations {mk }K
k=1 . Assuming that the
} = 0 [36]
regularity condition is satisfied, i.e., E{ ∂ lnp(m,θ)
∂θ
we write the Fisher information:

E{Fa } =
E{Fb } =

K Z
X

k=1
K Z
X
k=1

p(θ)
p(θ)

z }| {
Mk
X
∂2(
sk,i (θ))
i=1

∂2θ

Mk
X
i=1

dθ = 0,

1
(ṡk,i (θ))2 dθ,
sk,i (θ)

where sk,i (θ) = p(mk = mk,i |θ) = p{mk,i − ∆2k ≤ hk θ +
∂s (θ)
nk ≤ mk,i + ∆2k |θ} and ṡk,i (θ) = k,i
. To complete the
∂θ
derivations of F we need to characterize sk,i (θ) and ṡk,i (θ).
Combining all above and recalling θ ∼ N (0, σθ2 ), we obtain:
K Mk Z
(ṡk,i (θ))2
θ
1
1 XX
(27)
F =
φ( )dθ + 2 ,
σθ
s
(θ)
σ
σ
k,i
θ
θ
i=1
k=1

where φ(.) is the standard normal probability density function
(pdf). Equation (27) is true for arbitrarily distributed hk ’s with
E{hk } = 1 ∀k, and var(hk ) = σh2 k . When hk ’s are Gaussian
we have:
ζk,i+1 − θ
ζk,i − θ
sG
) − Φ( q
),
k,i (θ) = Φ( q
θ2 σh2 k + σn2 k
θ2 σh2 k + σn2 k
where ζk,i = mk,i − ∆2k , ζk,i+1 = mk,i + ∆2k are the quantizer
boundaries, and Φ(.) is the cumulative distribution function
(CDF) of a standard normal random variable. Deriving ṡG
k,i (θ)
is straightforward and reduces to subtraction of two scaled
standard normal pdf.

VII. E XTENSION TO E RRONEOUS C HANNELS
To obtain our results so far we have focused on errorfree communication channel model, i.e., the quantization bits
from the sensors are available at the FC, to feed the LMMSE
estimator. The results can be extended to independent BSCs
with different error probabilities pk . Suppose sensor k uses
binary natural coding (BNC) to code its quantized message
mk , that is sent through a BSC with error probability pk , and
m̂k is the corresponding recovered quantization level at the
FC, where in general m̂k 6= mk , due to channel errors.

∂ 2 lnp(m, θ)
∂ 2 lnp(m|θ)
∂ 2 lnp(θ)
} = −E{
}−E{
}. (26) A. LMMSE Estimator and its corresponding MSE
2
2
∂ θ
∂ θ
∂2θ
The LMMSE estimator and its MSE would have the same
Assuming that
PKmk ’s conditioned on θ are independent, i.e., forms as in (4) and (5), with the difference that vector m
lnp(m|θ) = k=1 lnp(mk |θ), the first and second derivatives is replaced with vector m̂. We characterize E{θm̂ } and
k
of the log-likelihood function become:
E{m̂ m̂ } as the following:
F=−E{

k

∂lnp(m|θ)
=
∂θ

K
X

k=1

∂ 2 lnp(m|θ)
=
∂2θ
K
X

1
∂p(mk |θ)
,
p(mk |θ)
∂θ

z
K
X

k=1

=Fa

}|
{
2
1
∂ p(mk |θ)
p(mk |θ)
∂2θ

1
∂p(mk |θ) 2
−
(
) .
p2 (mk |θ)
∂θ
k=1
|
{z
}
=Fb

l

E{θm̂k } = E{E{θm̂k |θ, mk }} = E{θE{m̂k |mk }}.

(28)

With BNC of bit sequences and BSC model we have
E{m̂k |mk } = (1 − 2pk )mk [45]. Thus (28) reduces to
E{θm̂k } = (1−2pk )E{θmk }, where E{θmk } is characterized
in (6). For E{m̂k m̂l }, k 6= l and k = l we have:
(a)

E{m̂k m̂l } = E{E{m̂k m̂l |θ, mk , ml }} =

E{E{m̂k |mk }E{m̂l |ml }} = (1 − 2pk )(1 − 2pl )E{mk ml },
(b)

E{m̂2k } = E{E{m̂2k |mk }} = gk E{m2k } + Rk ,

(29)

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

Gaussian, uniform, and Laplacian distributions. We observe
that the analytical MSE is a good approximation of simulated
MSE for almost all scenarios, and the approximation accuracy
improves as K increases and/or kh decreases. Also, except for
small K and very high kh , the distribution of hk has negligible
effect on the approximation accuracy.
1

analytical
simulated (h ∼ Gaussian)
simulated (h ∼ uniform)
simulated (h ∼ Laplace)

0.9

0.35

kh = 4

0.3

MSE

0.6
0.5

0.2

0.3

0.15

j=1

eji
k p(mk = mk,j |θ),

eij
k

where
is the probability of receiving level mk,i , while level
mk,j is transmitted from sensor k. Note that eij
k can be found
n(j,i,rk )
in terms of pk , i.e., eji
=
(p
)
(1
−
pk )rk −n(j,i,rk ) ,
k
k
where n(j, i, rk ) is the Hamming
between BNC
Prk distance
rk −l
representations
of
m
=
b
2
and mk,i =
k,j
k,j,l
l=1
Prk
rk −l
b
2
.
To
sum
up,
F
becomes:
k,i,l
l=1
P
2
Mk ji
Mk Z
K X
ek ṡk,j (θ)
X
j=1
θ
P
 φ( )dθ + σθ−2 .
F = σθ−1
Mk ji
σ
θ
e s (θ)
i=1
k=1

j=1 k

k,j

VIII. N UMERICAL AND S IMULATION R ESULTS
In this section, we corroborate our analytical results with
numerical simulations. These results validate the accuracy of
our analysis and illustrate the effectiveness and superiority of
the proposed rate allocation schemes. We consider networks
of sizes K = 5, 10, 50 and conduct simulations for over 105
observation channels with randomly generated {σn2 k , σh2 k }K
k=1
and depict the average performance for all rate allocation
methods (greedy, integer relaxation (relaxed), individual rate
allocation (IRA), order aware (OA) uniform, and uniform).
We generate σn2 k such that E{σn2 k } = 1 or E{σn2 k } = kn .
To investigate the effect of multiplicative observation noise
variance on the network dynamics and performance, we let
E{σh2 k } = kh = 0.1, 1, 2, 4 to indicate low, moderate, high, and
very high multiplicative noise variance.
Figs. 2a and 2b compare the analytical MSE in (9) and
simulated MSE for K = 10, 50, when greedy method is employed4 . The simulations are conducted for hk ’s drawn from

4 Integer relaxation and IRA exhibit similar results, and the plots are omitted,
for the sake of saving space.

kh = 4

0.1

kh = 1

0.05

0.1

Mk
X

0.25

0.4

kh = 1

p(m̂k = mk,i |θ) =

analytical
simulated (h ∼ Gaussian)
simulated (h ∼ uniform)
simulated (h ∼ Laplace)

0.4

0.7

0.2

B. CRLB and Fisher Information Expressions
To find F based on m̂k ’s, we need to find the counterpart
of (26), where m is replaced with m̂. For independent BSC
model, m̂k ’s conditioned
on θ would be independent, leading
PK
to lnp(m̂|θ) = k=1 lnp(m̂k |θ). Following similar steps as
in Section VI, we find new Fa to be zero. New Fb can be
found by replacing p(mk = mk,i |θ) with p(m̂k = mk,i |θ) in
the derivations. All that remains is to characterize:

0.5
0.45

0.8

MSE

where (a) in (29) is obtained using the facts that (i) given
mk , ml then m̂k , m̂l are independent, (ii) given θ, then
mk , ml are uncorrelated (since nk , nl , hk , hl are all uncorrelated). And gk = (1 − pk )rk −1 (1 + pk (rk − 5)) and Rk =
(4/3)(1 − pk )rk −1 pk τk2 (2rk + 1)(2rk − 1)−1 . To obtain (b)
in (29) we assume at most one bit in a sequence of rk bits
can be flipped due to the channel errors (roughly speaking
pk  rk−1 ). This is a reasonable assumption noting that for
a poor channel with pk ≈ 0.1 and typical quantization rates
of rk ≤ 6, flipping more than one bit in an rk -bit sequence
is unlikely [45]. Note that E{mk ml } and E{m2k } in (29) are
characterized in (7). Having (28), (29), the LMMSE estimator
and its corresponding MSE are characterized for BSC model.

9

0

0
5

10

Btot

15

(a) K = 10

20

25

5

10

15

20

25

30

35

40

45

Btot

(b) K = 50

Fig. 2: Analytical and simulated MSE of greedy vs. Btot
Figs. 3 compares the analytical MSE for different methods,
K = 5 and kh = 0.1, 1, 2. We observe the MSE performance
gap between uniform (including OA uniform), and greedy and
integer relaxation are remarkable. As Btot increases, the performance of greedy, integer relaxation, uniform, OA uniform
approaches that of the clairvoyant centralized estimation. However, there is a persistent gap between the performance of IRA
and the clairvoyant case, even for large Btot . The performance
of greedy and integer relaxation are almost the same for all
scenarios. Similar observations are valid for K = 50 and the
plots are omitted due to lack of space. For large K and small
kh the performance of the individual rate allocation competes
with greedy and integer relaxation methods, however, for small
K or high kh it loses the competition. On the other hand,
when Btot is relatively small compared to K, greedy, integer
relaxation, and IRA have the same performance. As expected,
we observe larger kh (larger K) leads to a larger (smaller)
MSE for all methods.
Fig. 4 depicts the MSE performance of different methods
and the associated CRLB for K = 5, where hk ’s are drawn
from Gaussian distribution for CRLB. The σn2 k ’s and σh2 k ’s are
independently generated with Chi-Square distribution σh2 k ∼
χ2 (kh ), σn2 k ∼ χ2 (1). For kh = 1 (moderate multiplicative
noise), there is a noticeable gap between the MSE and the
associated CRLB for all methods, whereas for kh = 0.1 (low
noise) and large Btot , this gap tends to be very small. This is
in agreement with the result that the MSE of MMSE estimator
for a Gaussian linear observation model achieves the CRLB
[36]. In fact, for kh = 0 the observation model in (1) becomes
the linear Gaussian model xk = θ + nk and when Btot → ∞,
LMMSE estimator in (4) becomes MMSE estimator, which
achieves the CRLB. Similar observations are valid for K = 50
and the plots are omitted due to lack of space.
Figs. 5a and 5b depict the number of active sensors versus
Btot for all methods and K = 10, 50. For kh = 2 (high noise)
more sensors become active to reduce the noise effect, by averaging over observations coming from more sensors, leading to
smaller quantization rates (coarser quantization). On the other

50

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

0.6

0.5
0.4
0.3

0.8

greedy
relaxed
IRA
OA uniform
uniform
clairvoyant

0.7

analytical MSE

0.6

analytical MSE

0.8

greedy
relaxed
IRA
OA uniform
uniform
clairvoyant

0.7

0.6

0.5
0.4
0.3

0.2

0.2

0.1

0.1

5

10

Btot

15

20

0.5
0.4
0.3
0.2
0.1

0

0

greedy
relaxed
IRA
OA uniform
uniform
clairvoyant

0.7

analytical MSE

0.8

10

0
5

25

10

(a) kh = 0.1

Btot

15

20

25

5

10

(b) kh = 1

Btot

15

20

25

(c) kh = 2

Fig. 3: MSE of different rate allocation methods vs. Btot for K = 5
0.7

MSE
CRLB
clairvoyant

0.6

0.5

MSE and CRLB

0.5

0.4
kh = 0.1
0.3
kh = 1

0.2

0.4

kh = 0.1

7

0.3
kh = 1

6

0.1

0.1

0

0
5

10

15

20

5

25

10

15

25

Btot

Btot

(a)

20

greedy

(b) relaxed

0.7

0.7

MSE
CRLB
clairvoyant

0.6

5

4

kh = 2
3

MSE and CRLB

MSE and CRLB

0.3

kh = 0.1
kh = 1

0.1

20

25

5

10

Btot

15

20

25

Btot

(c) IRA

(d) uniform

Fig. 4: MSE and CRLB vs. Btot for K = 5

10

50

9

45
40

8

35

# active sensors

active sensors

7
6
5
4
3

greedy
relaxed
IRA
uniform

kh = 2

2
1
0

5

10

15

20

25

Btot

(a) K = 10

30

30
25
20

kh = 2

15

kh = 0.1

35

40

45

5

kh = 2

4

20

25

30

35

40

45

1
0

50

100

Btot

150

200

(b) K = 50

Fig. 6: Average quantization rate of active sensors vs. Btot

0
15

6

0.3

0.2

10

15

(a) K = 10

kh = 0.1

0.1

5

10

Btot

kh = 1
0.4

0.2

0

kh = 0.1

7

2
5

0.5

0.4

8

2

1
0.5

greedy
relaxed
IRA
uniform

9

3

MSE
CRLB
clairvoyant

0.6

10

greedy
relaxed
IRA
uniform

kh = 0.1

0.2

average rate per sensor

MSE and CRLB

sensors with finer quantization). These observations illustrate
the trade-off mentioned in explanations following (15) and in
subsection IV-D.

MSE
CRLB
clairvoyant

0.6

average rate per sensor

0.7

10

greedy
relaxed
IRA
uniform

kh = 0.1

5
0
0

50

100

Btot

150

200

(b) K = 50

Fig. 5: Number of active sensors vs. Btot
hand, for hk = 0.1 (low noise) less sensors become active,
leading to larger quantization rates (finer quantization). These
observations illustrate the trade-off mentioned in explanations
following (15) and in subsection IV-D. Note that greedy and
integer relaxation methods activate fewer sensors, compared
with those of IRA and uniform methods, and still provide
better MSE performance (see also Fig. 3). Figs. 6a and 6b
illustrate the average quantization rates of active sensors versus
Btot for all methods and K = 10, 50. For kh = 2 (high noise)
the average quantization rates are smaller (more active sensors
with coarser quantization). On the other hand, for hk = 0.1
(low noise) the average quantization rates are larger (less active

250

Fig. 7 illustrates
PK the required bandwidth, i.e., sum of quantization rates k=1 rk versus a target MSE (to be satisfied),
for all methods and K = 50. Note that greedy and integer
relaxation methods require much less bandwidth to achieve
the target MSE, compared with those of IRA, uniform, and
OA uniform. Similar observations are valid for K = 5 and the
plot is omitted due to lack of space. These figures (excluding
IRA) show that, more bandwidth is required to (i) satisfy a
smaller target MSE, (ii) satisfy a fixed target MSE for larger
kh , (iii) satisfy a fixed target MSE for smaller K. In some
sub-figures the required bandwidth for some target MSE values
are left blank, since the target MSE is not achievable for that
particular network setting. Note that IRA method is different
from greedy and integer relaxation methods, since it is blind
to target MSE and K value (see (24),(25)), i.e., the assigned
quantization rates are independent of the target MSE and K
value and the number of active sensors is kept at minimum,
such that the target MSE is satisfied. For illustrative purposes,
consider a large and easy-to-be-satisfied MSE target, such that
it lies in the interval [0.7, 1]. Such a target MSE most likely
can be satisfied with one active sensor (see Figs. 8a, 8b), and
one bit (see Fig. 9) in greedy and integer relaxation methods.
However, since IRA is blind to the target MSE, it assigns a
quantization rate to the only active sensor, according to the
observation channel quality σ12 , that is likely to be larger than
one bit (in fact, the smaller kh is, the larger rk is).
Figs. 8a and 8b depict the number of active sensors when the
target MSE is met for all methods and K = 10, 50. Note that
greedy and integer relaxation methods activate fewer sensors
to satisfy the target MSE, compared with those of IRA and

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

10 3

greedy
relaxed
IRA
OA uniform
uniform

10 2

11

greedy
relaxed
IRA
OA uniform
uniform

greedy
relaxed
IRA
OA uniform
uniform

10 2

10 1

bandwidth

bandwidth

bandwidth

10 2

10 1

10 1

10 0

10 0
0

0.1

0.2

0.3

0.4

0.5

0.6

O (target MSE)

(a)

0.7

0.8

0.9

1

0

0.1

0.2

0.3

0.4

(b)

kh = 0.1

0.5

0.6

O (target MSE)

0.7

0.8

10 0

0.9

0

0.1

0.2

0.3

0.4

(c)

kh = 1

0.5

0.6

O (target MSE)

0.7

0.8

0.9

1

kh = 2

Fig. 7: Required bandwidth of different rate allocation methods vs. target MSE for K = 50
10

8

40

10 0

CRLB
analytical MSE
simulated MSE

kh = 2

35

kh = 2

kh = 0.1

5
4

30
25
20

kh = 0.1
15

3

10

2

5
0

1
0.1

0.2

0.3

0.4

0.5

0.6

O (target MSE)

0.7

0.8

0.9

1

0

0.1

0.2

(a) K = 10

0.3

0.4

0.5

0.6

O (target MSE)

0.7

0.8

0.9

1

(b) K = 50

MSE and CRLB

7
6

greedy
relaxed
IRA
uniform

45

# active sensors

# active sensors

50

greedy
relaxed
IRA
uniform

9

10 -1

p = 10−2

p = 10−4

Fig. 8: Number of active sensors vs. target MSE
10 -2

uniform and OA uniform. For kh = 2 (high noise), all methods
require more active sensors to satisfy the target MSE (similar
observations to those of Fig.5). Figs. 9a and 9b illustrate
7

kh = 2

4

3

kh = 0.1

6

kh = 2

5
4
3

2

2
1

1
0.1

0.2

0.3

0.4

0.5

0.6

O (target MSE)

(a) K = 10

0.7

0.8

0.9

1

60

80

100

120

140

160

180

Fig. 10: MSE and CRLB of greedy vs. Btot for K = 50

7

average rate per sensor

kh = 0.1
5

40

Btot

greedy
relaxed
IRA
uniform

8

average rate per sensor

20

9

greedy
relaxed
IRA
uniform

6

p = 10−1

0

0.1

0.2

0.3

0.4

0.5

0.6

O (target MSE)

0.7

0.8

0.9

(b) K = 50

Fig. 9: Quantization rate of active sensors vs. target MSE
the average quantization rates of active sensors versus target
MSE, for all methods and K = 10, 50. Similar conclusions to
those for Fig. 7 can be made here. Note that IRA loses the
competition to uniform methods for large target MSE values.
These figures show that, larger average quantization rate is
required to (i) satisfy a smaller target MSE, (ii) satisfy a fixed
target MSE for larger kh , (iii) satisfy a fixed target MSE for
smaller K (compare the average rate of all algorithms except
IRA in Fig. 9). In the figures the average quantization rates for
some target MSE values are left blank, since the target MSE is
not attainable for that particular network setting. Combining
the observations from Figs. 7, 8, 9, we conclude that IRA
method is not suitable to address (P2). To show the effect
of erroneous communication channels, Fig. 10 depicts the
analytical and simulated MSE and compare them with CRLB
for p = 10−1 , 10−2 , 10−4 , K = 50, and kh = 1, when greedy
method is employed. As expected, the analytical MSE is very

1

accurate unless for large error probability p = 0.1 (this is
expected since to derive (29), we assume pk  rk−1 , which is
not true for p = 0.1).
• Comparison with Rate-Distortion (R−D) Bound in [27]:
As we mentioned in Section I, DES and the quadratic Gaussian
CEO problem are different and hence the R−D bounds in [27]–
[29] are less relevant to the problem in hand. Interestingly, our
simulations show that in some scenarios, even the CRLB based
on multi-bit quantization can reach the R − D bound in [27].
Fig. 15 compares the CRLB based on one-bit and multi–bit
quantization with the R − D bound for different K and kn .
We use the R−D bound in [27] given below, which is for a
heterogeneous network with limited number of agents:
+

K̃
2 Y
σ
K̃
 ,
R(D) = 0.5 log2 ( θ
( 2
(30)
1
1 ))
D
σ
(
nk D̃(K̃) − D )
k=1
)−1 and K̃ is the largest
where D̃(K̃) = (σθ−2 +σn−2
+...+σn−2
1
K̃
value that satisfies

K̃
2
σn

K̃

1
−( D̃(1K̃) − D
) ≥ 0. Note that the gap

between the R−D bound and the CRLB based on multi-bit
quantization is not persistent: as Btot increases for a fixed K,
the gap fades away and the latter approaches the former.
• One-bit MLE and MAP vs. Multi-bit LMMSE: As we
mentioned in Section I, there is a significant gap between
the CRLB performance based on one-bit quantization and the
clairvoyant benchmark (unquantized observations are available
at the FC), when the dynamic range of θ is large with respect

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

to σn2 k [7]. To illustrate this, Fig. 11 plots classical CRLB
(ṡ (θ))2 −1
= ( sk,i
)
versus θ in a homogeneous network with
k,i (θ)
σn2 = 1, assuming θ ∈ [−4, 4]. corresponding to one bit [34]
and two bit quantization. We observe the classical CRLB
corresponding to two bit quantization is significantly better
than that of one bit quantization, when θ is larger than σn2 .
One expects similar observation holds when we compare MSE
of MLE corresponding to one bit and two bit quantization.
In Appendix X we provide an analytical reason for the poor
performance of CLRB and MLE based on one bit quantization.
When pdf of θ is known a priori, Maximum A Posteriori
estimator (MAP) can be used instead of MLE. Figs. 13 and
14 plot MSE of one-bit MLE, one-bit MAP, and the proposed
multi-bit LMMSE estimators, as well as the associated (one-bit
and multi-bit) CRLBs and R −D for a heterogeneous network
of size K = 50 versus kn and Btot , respectively. As expected,
in all cases the proposed multi-bit LMMSE outperforms onebit MLE and one-bit MAP. Fig. 13 also shows that for small σn2
one-bit MLE and one-bit MAP perform poorly. Also, as σθ2
becomes larger the performance gap between one-bit MLE,
one-bit MAP and the proposed multi-bit LMMSE increases
significantly, as expected from our analysis in Section X.

IX. C ONCLUSIONS
We considered DES of a Gaussian source in a heterogenous
bandwidth constrained WSN, where the source is corrupted
by independent multiplicative and additive observation noises,
with incomplete statistical knowledge of the multiplicative
noise. For uniform multi-bit quantizers, we derived the closedform MSE expression for the LMMSE estimator at the FC, and
verified the accuracy of our derivations via simulations. For
both error-free and erroneous communication channels (using
BSC model) we proposed several rate allocation methods to (i)
minimize the MSE given a network bandwidth constraint, and
(ii) minimize the required network bandwidth given a target
MSE. We also derived the Bayesian CRLB and compared the
MSE performance of our proposed methods against the CRLB.
Our results corroborate that, for low power multiplicative
observation noises and adequate network bandwidth, the gaps
between the MSE of our proposed methods and the CRLB
are negligible, while the performance of other methods like
individual rate allocation in [6], and uniform is not satisfactory. Through analysis and simulations, we showed that onebit MLE and one-bit MAP in the literature have poor performance, when the realizations of unknown is large (compared
with the observation noise variances), whereas our proposed
multi-bit LMMSE significantly outperforms estimators based
on one-bit quantization.

X. A PPENDIX
For a homogeneous network, the likelihood
PK function correθ
),
sponding to one bit quantization is L(θ) = k=1 lnΦ(yk v(θ)

12

p
where v(θ) = θ2 σh2 + σn2 and yk = sign(hk θ + nk ). Therefore, the one-bit MLE is θ̂M L =
argmax L(θ). We have:
θ∈[−θmax ,θmax ]

=la (θ)

}|

z
{"

=lb (θ)

}|
#{
N−
∂L
σn2
θ
N+
−
L̇(θ) =
= 2 φ(
)
θ
−θ
∂θ
v (θ) v(θ) Φ( v(θ)
) Φ( v(θ)
)
z

(31)

where N + and N − are number of +1’s and −1’s in all yk ’s,
respectively, and N + +N − = K. Since la (θ) > 0, the solution
of lb (θ) = 0 is θ̂M L . However, in some cases this equation has
no solution in [−θmax , θmax ] and L(θ) is strictly monotonic.
One instance is when both multiplicative and additive noises
are very small, such that all yk ’s are equal to +1 almost
surely, i.e., N + = K, N − = 0, and hence θ̂M L = θmax . This
poor performance of MLE for low power noises is unintuitive.
Below, we provide an analytical explanation for this. Let Ps
denote the probability of L(θ) being strictly monotonic (i.e.,
the probability of lb (θ) > 0 ∨ lb (θ) < 0). We find:


−θ
θ
) + ΦK−q (
)
(32)
Ps =
ΦK−q (
v(θ)
v(θ)

q 
X
θ
−θ
×
Φi (
)Φq−i (
)
v(θ)
v(θ)
i=0
j
k
−θmax
where q = Φ( v(θ
) . Interestingly, Ps is an even function
max )
of σ|θ|n and can get very close to one (plot of Ps versus σ|θ|n
is deleted due to lack of space). In fact, when σ|θ|n is large
enough MLE becomes independent of yk ’s and realization of
θ, i.e., θ̂M L = ±θmax , leading to poor estimation performance.
The larger is the dynamic range of θ (compared with σn2 ),
the larger is the estimation error of one-bit MLE. When pdf
of θ is known a priori, MAP can be used instead of MLE,
where θ̂M AP = argmax[L(θ) + ln(σθ−1 φ( σθθ ))], i.e., θ̂M AP is
the solution of L̇(θ) = σθ2 . As we discussed before, in some
θ

cases L̇(θ) becomes almost independent of yk ’s and realization
of θ, leading to poor estimation performance. Figs. 12.a and
12.b depict Mont Carlo averages of θ̂M L and θ̂M AP versus
realizations of θ for σn2 = 1, σh2 = 0.1, 1, K = 5, 50. These
figures show that as the realization of θ becomes large enough,
the estimates become more inaccurate, i.e., θ̂M L approaches
±θmax and θ̂M AP gets independent of θ realization, and is
determined5 by σθ2 , σh2 , K, causing severe estimation errors.
R EFERENCES
[1] A. Sani and A. Vosoughi, “On quantizer design for distributed estimation
in bandwidth constrained networks,” in 2015 IEEE 82nd Vehicular
Technology Conference (VTC2015-Fall), Sept 2015, pp. 1–2.
[2] A. Vempaty, H. He, B. Chen, and P. Varshney, “On quantizer design for
distributed bayesian estimation in sensor networks,” Signal Processing,
IEEE Transactions on, vol. 62, no. 20, pp. 5359–5369, Oct 2014.
[3] S. Kar, H. Chen, and P. Varshney, “Optimal identical binary quantizer
design for distributed estimation,” Signal Processing, IEEE Transactions
on, vol. 60, no. 7, pp. 3896–3901, July 2012.
[4] J. Fang and H. Li, “Hyperplane-based vector quantization for distributed
estimation in wireless sensor networks,” Information Theory, IEEE
Transactions on, vol. 55, no. 12, pp. 5682–5699, Dec 2009.
5 In contrast to θ̂
M L , our simulations suggest, provided that θmax > 3σθ ,
θ̂M AP does not depend on θmax in these cases.

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

14

one bit quantizer(σh2 = 1)

2 bit quantizer (σh2 = 0)

10 1

one bit quantizer (σh2 = 0)
two bit quantizer (σh2 = 0)
one bit quantizer (σh2 = 1)
two bit quantizer (σh2 = 1)

10 6

CRLB (Non-Bayesian)

2 bit quantizer (σh2 = 1)

10
8
6
4

10 4

10

10 1

one bit - CRLB
multi bit - CRLB
multibit LMMSE
one bit MAP
one bit MLE
R-D bound

10 0

CRLB and MSE

one bit quantizer (σh2 = 0)

12

CRLB

θ=2

10 8

16

2

10 -1

10

one bit - CRLB
multi bit - CRLB
multibit- LMMSE
one bit - MAP
one bit - MLE
R-D bound

10 0

CRLB and MSE

σn2 = 1

18

13

-2

10 -1

10 -2

10 0

10 -3
10 -3

2

10 -2

0
-4

-3

-2

-1

0

θ (unknown)

1

2

3

4

2 =1
(a) CRLB vs θ with σn

0.2

0

0.5

1

1.5

2

2.5

σn2

3

3.5

4

4.5

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0

kn

5

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

kn

2 = 0, σ 2 = 1
(a) σh
θ

2 with θ = 2
(b) CRLB vs σn

0.2

(b)

10 1

2 = 0, σ 2 = 8
σh
θ

10 1

Fig. 11: CRLB correspoding to one bit and two bit quantizers
one bit - CRLB
multi bit - CRLB
multi bit - LMMSE
one bit - MLE
one bit - MAP

10 0

5

θ̂M L

K=5

CRLB and MSE

CRLB and MSE

10 0

10 -1

one bit - CRLB
multi bit - CRLB
multi bit - LMMSE
one bit - MLE
one bit - MAP

10 -2

K = 50

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

10 -1

10 -2
0.2

2

0.4

0.6

0.8

2 6= 0, σ 2 = 1
(c) σh
θ

σh2 = 0.1
σh2 = 0
-5
-5

-4

-3

-2

-1

0

θ

1

2

3

4

5

1

1.2

1.4

1.6

1.8

2

kn

kn

2 6= 0, σ 2 = 8
(d) σh
θ

Fig. 13: Performance comparison of one-bit and multi-bit
estimation vs. kn , with K = 50, Btot = 50, θmax = 2σθ

(a) θ̂M L vs. realization of θ with θmax = 5
10 1

5

K = 50
σh2
σh2

-5
-5

-4

-3

-2

-1

0

θ

1

2

3

10 -1

one bit - CRLB
multi bit - CRLB
multi bit - LMMSE
one bit - MLE
one bit - MAP
R-D bound

10 -1

= 0.1
=0
10 -2

10 -2

4

5

5

15

20

25

30

35

40

45

5

50

10

15

20

(a)

Fig. 12: θ̂M L and θ̂M AP vs. realizations of θ

25

30

40

45

50

2 = 0, σ 2 = 8
(b) σh
θ

2 = 0, σ 2 = 1
σh
θ
10 1

10 1

one bit - CRLB
multi bit - CRLB
multi bit - LMMSE
one bit - MLE
one bit MAP
CRLB and MSE

35

Btot

Btot

(b) θ̂M AP vs. realization of θ with θmax = 5

[5] ——, “Distributed adaptive quantization for wireless sensor networks:
From delta modulation to maximum likelihood,” IEEE Transactions on
Signal Processing, vol. 56, no. 10, pp. 5246–5257, Oct 2008.
[6] J. Li and G. AlRegib, “Rate-constrained distributed estimation in wireless sensor networks,” Signal Processing, IEEE Transactions on, vol. 55,
no. 5, pp. 1634–1643, May 2007.
[7] A. Ribeiro and G. Giannakis, “Bandwidth-constrained distributed estimation for wireless sensor networks-part i: Gaussian case,” Signal
Processing, IEEE Transactions on, vol. 54, no. 3, pp. 1131–1143, March
2006.
[8] S. Marano, V. Matta, and P. Willett, “Quantizer precision for distributed
estimation in a large sensor network,” Signal Processing, IEEE Transactions on, vol. 54, no. 10, pp. 4073–4078, Oct 2006.
[9] O. Dabeer and A. Karnik, “Signal parameter estimation using 1bit dithered quantization,” IEEE Transactions on Information Theory,
vol. 52, no. 12, pp. 5389–5405, Dec 2006.
[10] Z.-Q. Luo, “Universal decentralized estimation in a bandwidth constrained sensor network,” IEEE Transactions on Information Theory,
vol. 51, no. 6, pp. 2210–2219, June 2005.
[11] A. Sani and A. Vosoughi, “On distributed vector estimation for power
and bandwidth constrained wireless sensor networks,” IEEE Transactions on Signal Processing, vol. PP, no. 99, pp. 1–1, 2016.
[12] M. Ahmed, T. Al-Naffouri, M.-S. Alouini, and G. Turkiyyah, “The effect
of correlated observations on the performance of distributed estimation,”
Signal Processing, IEEE Transactions on, vol. 61, no. 24, pp. 6264–
6275, Dec 2013.
[13] S. Talarico, N. Schmid, M. Alkhweldi, and M. Valenti, “Distributed
estimation of a parametric field: Algorithms and performance analysis,”
Signal Processing, IEEE Transactions on, vol. 62, no. 5, pp. 1041–1053,

10

CRLB and MSE

K=5

10 0

CRLB and MSE

CRLB and MSE

θ̂M AP

10 0

0

10 1

one bit - CRLB
multi bit - CRLB
multi bit - LMMSE
one bit - MLE
one bit - MAP
R-D bound

10 0

10 -1

one bit - CRLB
multi bit - CRLB
multi bit - LMMSE
one bit - MLE
one bit - MAP

10 0

10 -1

5

10

15

20

25

30

35

Btot

2 6= 0, σ 2 = 1
(c) σh
θ

40

45

50

5

10

15

20

25

30

35

40

45

50

Btot

2 6= 0, σ 2 = 8
(d) σh
θ

Fig. 14: Performance comparison of one bit and multi bit
estimation vs. Btot , K = 50, θmax = 2σθ
March 2014.
[14] I. Nevat, G. Peters, and I. Collings, “Random field reconstruction with
quantization in wireless sensor networks,” Signal Processing, IEEE
Transactions on, vol. 61, no. 23, pp. 6020–6033, Dec 2013.
[15] T. C. Aysal and K. E. Barner, “Constrained decentralized estimation
over noisy channels for sensor networks,” IEEE Transactions on Signal
Processing, vol. 56, no. 4, pp. 1398–1410, April 2008.
[16] J.-J. Xiao, S. Cui, Z.-Q. Luo, and A. Goldsmith, “Power scheduling of
universal decentralized estimation in sensor networks,” Signal Processing, IEEE Transactions on, vol. 54, no. 2, pp. 413–422, Feb 2006.
[17] A. Behbahani, A. Eltawil, and H. Jafarkhani, “Decentralized estimation
under correlated noise,” Signal Processing, IEEE Transactions on,
vol. 62, no. 21, pp. 5603–5614, Nov 2014.
[18] A. Sani and A. Vosoughi, “Resource allocation optimization for dis-

SUBMITTED TO JOURNAL OF TRANSACTIONS ON SIGNAL PROCESSING,

one bit - CRLB
multi bit - CRLB
R-D bound

kn = 1

bounds

10 -1

10 -2

10 -3

kn = 0.1

5

10

15

20

25

30

35

40

45

50

Btot

(a) K = 5
one bit - CRLB
multi bit - CRLB
R-D bound

bounds

kn = 1
10 -2

kn = 0.1

5

10

15

20

25

30

35

40

45

50

(b) K = 50

Fig. 15: R−D and CRLB for K = 5, 50 and σθ2 = 1

[19]
[20]
[21]
[22]

[23]

[24]
[25]
[26]
[27]

[28]

[29]

[30]
[31]

tributed vector estimation with digital transmission,” in Signals, Systems
and Computers, 2014 48th Asilomar Conference on, Nov 2014, pp.
1463–1467.
——, “Bandwidth and power constrained distributed vector estimation
in wireless sensor networks,” in Military Communications Conference,
MILCOM 2015 - 2015 IEEE, Oct 2015, pp. 1164–1169.
——, “Noise enhanced distributed bayesian estimation,” in 2017 IEEE
International Conference on Acoustics, Speech and Signal Processing
(ICASSP), March 2017, pp. 4217–4221.
J. Proakis, Digital Communications, ser. Electrical engineering series.
McGraw-Hill, 2001. [Online]. Available:
https://books.google.com/books?id=sbr8QwAACAAJ
K. Kotobi and S. G. Bilén, “Spectrum sharing via hybrid cognitive
players evaluated by an m/d/1 queuing model,” EURASIP Journal on
Wireless Communications and Networking, vol. 2017, no. 1, p. 85, May
2017. [Online]. Available: https://doi.org/10.1186/s13638-017-0871-x
S. Hatamnia, S. Vahidian, S. Assa, B. Champagne, and M. AhmadianAttari, “Network-coded two-way relaying in spectrum-sharing systems
with quality-of-service requirements,” IEEE Transactions on Vehicular
Technology, vol. 66, no. 2, pp. 1299–1312, Feb 2017.
M. K. Kiskani and B. H. Khalaj, “Novel power control algorithms
for underlay cognitive radio networks,” in 2011 21st International
Conference on Systems Engineering, Aug 2011, pp. 206–211.
B. S. M. Mousaie, M. Soltanian, “Comsens: Exploiting pilot diversity
for pervasive integration of communication and sensing in mimo-tddframeworks,” in MILCOM, 2017.
H. Viswanathan and T. Berger, “The quadratic gaussian ceo problem,”
IEEE Transactions on Information Theory, vol. 43, no. 5, pp. 1549–
1559, Sep 1997.
J. Chen, X. Zhang, T. Berger, and S. B. Wicker, “An upper bound on
the sum-rate distortion function and its corresponding rate allocation
schemes for the ceo problem,” IEEE Journal on Selected Areas in
Communications, vol. 22, no. 6, pp. 977–987, Aug 2004.
Y. Oohama, “Rate-distortion theory for gaussian multiterminal source
coding systems with several side informations at the decoder,” IEEE
Transactions on Information Theory, vol. 51, no. 7, pp. 2577–2593,
July 2005.
H. Behroozi and M. R. Soleymani, “Distortion sum-rate performance
of successive coding strategy in quadratic gaussian ceo problem,” IEEE
Transactions on Wireless Communications, vol. 6, no. 12, pp. 4361–
4365, December 2007.
Y. C. Eldar, A. Ben-Tal, and A. Nemirovski, “Robust mean-squared error
estimation in the presence of model uncertainties,” IEEE Transactions
on Signal Processing, vol. 53, no. 1, pp. 168–181, Jan 2005.
Y. C. Eldar, “Minimax estimation of deterministic parameters in linear
models with a random model matrix,” IEEE Transactions on Signal
Processing, vol. 54, no. 2, pp. 601–612, Feb 2006.

14

[32] A. Wiesel, Y. C. Eldar, and A. Yeredor, “Linear regression with gaussian
model uncertainty: Algorithms and bounds,” IEEE Transactions on
Signal Processing, vol. 56, no. 6, pp. 2194–2205, June 2008.
[33] J. Zhu, X. Wang, X. Lin, and Y. Gu, “Maximum likelihood estimation from sign measurements with sensing matrix perturbation,” IEEE
Transactions on Signal Processing, vol. 62, no. 15, pp. 3741–3753, Aug
2014.
[34] J. Zhu, X. Lin, R. S. Blum, and Y. Gu, “Parameter estimation from
quantized observations in multiplicative noise environments,” IEEE
Transactions on Signal Processing, vol. 63, no. 15, pp. 4037–4050, Aug
2015.
[35] V. Naghshin, M. C. Reed, and Y. Liu, “On the performance analysis
of finite wireless network,” in 2015 IEEE International Conference on
Communications (ICC), June 2015, pp. 6530–6535.
[36] S. Kay, Fundamentals of statistical signal processing: estimation theory.
Prentice Hall,Upper Saddle River,NJ, 1993.
[37] B. Widrow and I. Kollar, Quantization Noise: Roundoff Error in Digital
Computation, Signal Processing, Control and Communications. Cambridge Univ. Press, 2008.
[38] R. Garnier and J. Taylor, Discrete Mathematics: Proofs, Structures and
Applications, Third Edition, 3rd ed. Bristol, PA, USA: Taylor & Francis,
Inc., 2009.
[39] G. E. Andrews, The Theory of Partitions. Cambridge University
Press, 1984, cambridge Books Online. [Online]. Available:
http://dx.doi.org/10.1017/CBO9780511608650
[40] S. Joshi and S. Boyd, “Sensor selection via convex optimization,” Signal
Processing, IEEE Transactions on, vol. 57, no. 2, pp. 451–462, Feb
2009.
[41] A. Sani, M. M. Feghhi, and A. Abbasfar, “Discrete bit loading and power
allocation for ofdma downlink with minimum rate guarantee for users,”
AEU - International Journal of Electronics and Communications,
vol. 68, no. 7, pp. 602 – 610, 2014. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S1434841114000077
[42] D. P. Bertsekas, Nonlinear Programming.
Belmont, MA: Athena
Scientific, 1999.
[43] A. Sani and A. Abbasfar, “Qos-aware discrete bit loading for ofdma
networks,” in 2013 International Conference on Computing, Networking
and Communications (ICNC), Jan 2013, pp. 994–998.
[44] ——, “Discrete bit-loading for ofdma downlink with qos consideration,”
in 2012 15th International Telecommunications Network Strategy and
Planning Symposium (NETWORKS), Oct 2012, pp. 1–6.
[45] H. Leung, C. Seneviratne, and M. Xu, “A novel statistical model for
distributed estimation in wireless sensor networks,” IEEE Transactions
on Signal Processing, vol. 63, no. 12, pp. 3154–3164, June 2015.

