Barankin Vector Locally Best Unbiased Estimates∗

arXiv:1706.10062v1 [stat.ML] 30 Jun 2017

Bruno Cernuschi-Frı́as†

Abstract
The Barankin bound is generalized to the vector case in the mean
square error sense. Necessary and sufficient conditions are obtained to
achieve the lower bound. To obtain the result, a simple finite dimensional
real vector valued generalization of the Riesz representation theorem for
Hilbert spaces is given. The bound has the form of a linear matrix inequality where the covariances of any unbiased estimator, if these exist,
are lower bounded by matrices depending only on the parametrized probability distributions.

Keywords: Parameter estimation, unbiased estimation, optimal estimator,
Barankin bound, performance bounds, linear matrix inequalities, minimal covariance matrix, Cramer-Rao bound.

1

Introduction

The problem considered, following Barankin, [2], and results in Banach, [1], is
the optimal unbiased estimation of a deterministic vector of parameters ν of
a family of probability measures Pν , or more generally a known real vector
function of these parameters g(νν ), using a realization of a vector random variX drawn from Pν T . The first issue is to find a function ψ such that
able
R
X ) dPν = g(νν ), for all ν in some admissible set. This problem is a vector
ψ (X
integral equation and may or may not have a solution, [6, 20]. Furthermore, even
if it has solution, it may not have a solution with finite covariance matrix for ν T .
Barankin, under very simple hypothesis, [2], gives an if and only if condition
for the existence of a minimal s-th variance unbiased estimator for the scalar
case, which is tighter than the classical Cramer-Rao or Bhattacharyya bounds
if they exist. In recent years the Barankin bound has attracted attention, since
there are problems for which the Cramer-Rao or Bhattacharyya bounds give
no satisfactory solution, see e.g. [22], and references there. Following [2], the
problem studied here is under what conditions there exists a finite covariance
∗ This work was partially supported by the Universidad de Buenos Aires, UBA, grant
UBACYT No. 20020130100357BA, and the Consejo Nacional de Investigaciones Cientı́ficas y
Técnicas, CONICET, Argentina.
† B. Cer nuschi-Frı́as is with the Universidad de Buenos Aires, Facultad de Ingenierı́a, and
the Instituto Argentino de Matemáticas, IAM, CONICET, Casilla 8, Sucursal 12(B), 1412
Buenos Aires, Argentina (e-mail: bcf@acm.org).

1

B. Cernuschi-Frı́as

2

vector unbiased estimator of the true vector parameter ν T , and in that case if
a minimal covariance vector unbiased estimator exists.
In Section 2 an overview is presented of the relevant results of measure theory and the Lebesgue integral related to the Barankin formulation. In Section
3 the vector Barankin bound generalization is presented as a linear matrix inequality (LMI). In Section 4 the Barankin functional analysis formalization is
generalized to handle the vector case. In Section 5 a finite dimensional real vector valued generalization of the Riesz representation theorem for Hilbert spaces
is presented. In Section 6 necessary and sufficient conditions are given for the
existence of an optimal vector estimator attaining the bound given by the LMI
obtained in Section 3. In Section 7 other alternative LMI formulations for the
existence of an optimal vector estimator are given.

2
2.1

Formalization of the vector estimation problem
Measure theoretic setup

Let (Ω, F ) be a measurable space, where Ω is a well defined abstract set, and
F is a sigma-algebra of subsets of Ω, [14]. Let Θ be an abstract arbitrary set
of sub-indexes with no conditions on its structure as in [2], p. 477. Let B
be a collection of probability measures Pθ for
 the measurable space (Ω, F ),
indexed by the sub-indexes θ ∈ Θ, i.e. B = Pθ : θ ∈ Θ , as in [2] p. 477.
Hence for each θ ∈ Θ, the triple (Ω, F , Pθ ) is a probability space. Let X
be a vector random variable, i.e. a measurable function from the measurable
space (Ω, F ) to the measurable space (RdS , BdS ), where RdS is the vector
dS -dimensional real space, and BdS is the Borel sigma-algebra for RdS , that
is the minimal sigma-algebra generated, e. g., by the open sets of RdS . Then
X is a real vector random variable iff ∀B ∈ BdS we have X −1 (B) ∈ F , if
and only if each component of the vector is a real random variable, [17] p. 19.
Define for each θ ∈ Θ the measure Pθ , for the measurable space (RdS , BdS ),
induced by the random variable X , [10] p. 34, i.e. for each B ∈ BdS define
X −1 (B)). Hence for each θ ∈ Θ the random variable X induces
Pθ (B) = Pθ (X
the probability space (RdS , BdS , Pθ ).
Let ψ be a real measurable vector function from (RdS , BdS ) to (RdP , BdP ),
that is ψ : RdS → RdP , and for each B ∈ BdP we have ψ −1 (B) ∈ BdS .
For the measurable vector function ψ from (RdS , BdS ) to (RdP , BdP ), define
ψ ]i , which is a measurable function
the i-th component of the vector ψ as [ψ
from (RdS , BdS ) to (R, B) for each 1 ≤ i ≤ dP iff ψ is measurable. Define
L1 (RdS , BdS , Pθ ) as the collection of all
R the measurable vector functions ψ from
ψ ]i dPθ < +∞, for 1 ≤ i ≤ dP , equiva(RdS , BdS ) to (RdP , BdP ), such that [ψ
ψ ]i ∈ L1 (RdS , BdS , Pθ ), for all 1 ≤ i ≤ dP , so that L1 (RdS , BdS , Pθ ) =
lently, [ψ
d P
L1 (RdS , BdS , Pθ ) .
X ) is a random variable from (Ω, F ) to (RdP , BdP ), since for
Hence ψ (X

B. Cernuschi-Frı́as

3


−1

X ) (B) = X −1 ψ −1 (B) , but B ∈ BdP so that
B ∈ BdP we have ψ (X

ψ −1 (B) ∈ BdS and then X −1 ψ −1 (B) ∈ F .
Define the integral of a vector of functions as a vector whose elements are
the integrals of each function. Then, [10] p. 45:
Z
Z
X ) dPθ = ψ dPθ
ψ (X
Note that the integral on the left is with respect to the probability space
(Ω,F ,Pθ ), while the integral on the right is with respect to the probability
X ) and ψ as an estimaspace (RdS , BdS , Pθ ). We will refer indistinctly to ψ (X
tor, with the understanding that they refer to different probability spaces linked
by the previous equality of integrals.
dS
ψ] =
R For ψ ∈ LR1 (R , BdS , Pθ ) define the expectation of ψ as Eθ [ψ
X ) dPθ = ψ dPθ .
ψ (X
We assume that the random variable X is drawn from some specific probability measure (p. m.) PθT , with θT ∈ Θ, i. e. we will use the realization
of this random variable to obtain the estimator for g(θT ). The random vector
X ) is an unbiased estimator for g(θ), ∀θ ∈ Θ , gR: Θ → RdP , if the
ψ
R (X
R integral
X ) dPθ is well defined, ∀θ ∈ Θ, and we have ψ (X
X ) dPθ = ψ dPθ =
ψ (X
g(θ), ∀θ ∈ Θ.
Then the first issue posed in the introduction may be formally stated as:
Problem 2.1 (Basic Problem). Given a function g : θ → RdP , defined for each
θ ∈ Θ, and a family of p.m.’s indexed by θ ∈ Θ, find an unbiased
estimator,
R
dS
X
,
B
i.e.
find
a
function
ψ
∈
L
(R
,
P
),
∀θ
∈
Θ,
such
that
ψ
(X
) dPθ =
d
1
θ
S
R
ψ dPθ = g(θ), for all θ ∈ Θ.
Define the integral of a matrix Ψ of dimensions N × M , N, M ∈ N, whose
elements belong to L1 (RdS , BdS , Pθ ), as a matrix whose
elements are
R
R the inX )] = Ψ(X
X ) dPθ = Ψ dPθ .
tegrals of the elements of Ψ, so that Eθ [Ψ(X
For a measurable square integrable function f : (RdS , BdS , PθT ) → (R, B), i.e.
f ∈ L2 (RdS , BdS , PθT ),
Define L2 (RdS , BdS , PθT ) as the collectionR of all the measurable functions u
2
from (RdS , BdS ) to (RdP , BdP ), such that [u]i dPθT < +∞, for 1 ≤ i ≤ dP ,
dS
equivalently, [u]i ∈ L2 (R , BdS , PθT ), for all 1 ≤ i ≤ dP . If the non-centered
second order moments of the components of the estimator ψ exist for θT , i.e.
ψ ]i ∈ L2 (RdS , BdS , PθT ), for 1 ≤ i ≤ dP , so that ψ ∈ L2 (RdS , BdS , PθT ) ≡
[ψ
dP
L2 (RdS , BdS , PθT ) , then, the first order moments
of the components of the
R
ψ ]i [ψ
ψ ]j dPθT , are well defined
estimator exist for θT . Also, the correlations [ψ
and are finite for all i 6= j, 1 ≤ i, j ≤ dP , and using the Cauchy-Schwarz
R
ψ ]i [ψ
ψ ]j dPθT ≤ k[ψ
ψ ]i kL2 [ψ
ψ ]j
inequality, we obtain
[ψ
. Additionally asL2

X ) is unbiased ∀θ ∈ Θ, hthen the covariance matrix of ψ (X
X )iexists for
sume ψ (X
R
ψ) =
ψ (X
X ) − g(θT )) (ψ
ψ (X
X ) − g(θT ))T dPθT =
θT , and we have CovθT (ψ
(ψ
h
i


ψ − g(θT )) (ψ
ψ − g(θT ))T = EθT ψ ψ T − g(θT ) g(θT )T .
EθT (ψ

B. Cernuschi-Frı́as

4

In the same direction of [2], with s = r = 2, instead of the general Problem
2.1, we pose the problem in terms of estimators with finite covariance matrix at
θT :
Problem 2.2 (Finite Covariance Problem). Given a function g : θ → RdP ,
defined for each θ ∈ Θ, and a family of p.m.’s indexed by θ ∈ Θ, find a function
ψ ∈ L2 (RdS , BdS , PθT ), with ψ ∈ L1 (RdS , BdS , Pθ ), ∀θ ∈ Θ, such that
R
ψ dPθ = g(θ), for all θ ∈ Θ. If there are several solutions find, if possible, a
solution with minimal covariance matrix at θT .

2.2

Centered definitions

Define ϕ = ψ − g(θT R), and h(θ) = g(θ) −R g(θT ) so that h(θT ) = 0. If ψ is
unbiased,
R then, since ψ dPθ = g(θ) and g(θT ) dPθ = g(θT ), forR all θ ∈ Θ,
ψ − g(θT )) dPθ = g(θ) − g(θT ), for all θ ∈ Θ, so that ϕ dPθ =
then, (ψ
T
h(θ) ∀θ ∈ Θ, and CovθT (ψ
Rψ ) = EθT ϕ ϕ . Also, if ψ is unbiased, then, since
h(θT ) = 0, then EθT[ ϕ ] = ϕ dPθT = 0.

2.3

Barankin formulation: basic hypothesis

Following Barankin we will introduce some simple additional hypothesis resumed in Barankin’s Postulate in [2] p. 481.
Hypothesis 2.1. The set Θ is an arbitrary index set with no conditions on its
structure, [2] p. 477, and B is a collection
of probability measures Pθ for the

measurable space (Ω, F ), i.e. B = Pθ : θ ∈ Θ as in [2], p. 477. The random
variable X : (Ω, F ) → (RdS , BdS ) is drawn from the probability measure (p.
m.) PθT , with θT ∈ Θ. Assume that for each θ ∈ Θ the p.m. Pθ is absolutely
continuous with respect to PθT , i.e Pθ << PθT , with θT ∈ Θ.
Lemma 2.1. If Hypothesis 2.1 is true then for each θ ∈ Θ the p.m. Pθ is
absolutely continuous with respect to PθT , i.e. Pθ << PθT .
Proof. Assume B ∈ BdP is such that PθT (B) = 0, then since PθT (B) =
X −1 (B)), we obtain PθT (X
X −1 (B)) = 0. But Pθ << PθT , hence
PθT (X
−1
X (B)) = 0. Since Pθ (B) = Pθ (X
X −1 (B)), then Pθ (B) = 0.
Pθ (X
Observation 2.1. In the case in which every index θ ∈ Θ is a possible candidate
for θT , then, Hypothesis 2.1 should require that for each θ1 ∈ Θ the p.m. Pθ1
should be absolutely continuous with respect to each other p.m. Pθ2 with
θ2 ∈ Θ, and then Pθ1 << Pθ2 for all θ1 , θ2 ∈ Θ.
As a consequence of the previous hypothesis and lemma, the Radon-Nykodim
derivatives dPθ /dPθT and dPθ /dPθT exist for all θ ∈ Θ, [15] p. 315.
Definition 2.1. Define π(θ) = dPθ /dPθT , with π(θ) ≡ πθ (x, θT ), x ∈ RdS , so
X , θT ). Define B0 = {π(θ) : θ ∈ Θ}, see [2], p. 481.
that dPθ /dPθT = πθ (X
R

We have π(θ) ≥ 0Rw.p. 1, for all θ ∈ Θ, [15]
R p. 315, π(θT ) = 1 w.p. 1, and
X , θT )dPθT = dPθ /dPθT dPθT = dPθ = 1, for all θ ∈ Θ.
πθ (X

B. Cernuschi-Frı́as

5

Hypothesis 2.2.
1. Assume that for each θ there is one and only one π(θ) ∈
B0 , i.e. the correspondence π : Θ → B0 is one-to-one.
2. There are at least two values θ1 , θ2 ∈ Θ, such that g(θ1 ) 6= g(θ2 ).
Observation 2.2. Item 1 avoids the identifiability problem, [12], pp. 58 and
191. Item 2 implies that we do not consider estimators which are
R constant with
dP
X ) dPθ1 =
probability
1:
if
it
was
ψ
=
α
w.p.
1
for
some
α
∈
R
,
then
ψ (X
0
0
R
R
X
α 0 dPθ1 = α 0 , similarly
ψ
(X
)
dP
=
α
,
but
since
we
assume
that ψ is
θ
0
2
R
R
X ) dPθ1 = g(θ1 ) and ψ (X
X ) dPθ2 = g(θ2 ), and
unbiased, it should be ψ (X
then it should be g(θ1 ) = α 0 = g(θ2 ), which is a contradiction. Additionally,
Hypothesis 2.2 implies that there exists at least a θ0 ∈ Θ such that g(θ0 ) 6= 0.
Nonetheless, see e.g. [2] p. 482 and [7] p. 2440, for some comments regarding
constant estimators.

2.4

Barankin postulate

The following hypothesis is Barankin’s Postulate in [2], p. 481, for s = r = 2.
Hypothesis 2.3 (Barankin, [2], Postulate p. 481). Assume that
π(θ) ∈ L2 (RdS , BdS , PθT )

∀θ ∈ Θ

equivalently B0 ⊆ L2 (RdS , BdS , PθT ).

R
R
R
Observation 2.3. Since π(θ) dPθT = dPθ /dPθT dPθT = dPθ = 1, for all
θ ∈ Θ, then kπ(θ)kL2 6= 0, for all θ ∈ Θ, equivalently kukL2 6=
R 0, for all u ∈ B0 .
If not, kπ(θ)kL2 = 0 implies π(θ) = 0 w.p. 1, and then π(θ) d PθT = 0,
contradiction. Additionally note, taking in account Hypothesis 2.2, that B0
has at least two elements.
Suppose ψ ∈ L2 (RdS R, BdS , PθT ), since π(θ) ∈ L2 (RdS , BdS , PθT ) for all
θ ∈ Θ, then the integrals ψ π(θ) dPθT are well defined for all θ ∈ Θ, and we
have all the equivalent forms:
Z
Z
Z
dPθ
dPθT = ψ dPθ
ψ π(θ) dPθT = ψ
dPθT
Z
Z
dPθ
X ) dPθ = ψ (X
X)
dPθT = EθT[ ψ π(θ)]
= ψ (X
dPθT
If ψ ∈ L2 (RdS , BdS , PθT ) is unbiased, then for ϕ = ψ − g(θT ) we have
Z
EθT[ ϕ π(θ)] = ϕ π(θ) dPθT = h(θ)
∀θ∈Θ

(2.1)

The introduction of the functions π reduces the consideration of the multiple probability spaces L2 (RdS , BdS , Pθ ), ∀θ ∈ Θ, to a single probability space
L2 (RdS , BdS , PθT ).

B. Cernuschi-Frı́as

2.5

6

Probability density function form

Call λ the Lebesgue measure for the measurable space (RdS , BdS ), i.e. the
measure that assigns to parallelepipeds in RdS the value given by the product
of the lengths of the edges of the parallelepiped in each direction. Alternatively
call dλ = dx, with x ∈ RdS . If in turn we have PθT << λ, i.e. the p.m. PθT
is absolutely continuous with respect to the Lebesgue measure, then, Pθ <<
PθT << λ, so that Pθ << λ, and then the Radon-Nykodim derivatives dPθ /dλ
exist, for all θ ∈ Θ. These derivatives are the probability density functions (pdf)
pθ ≡ pθ (x) ≡ dPθ /dλ with x ∈ RdS . Since, [15] p. 328,
pθ =

dPθ dPθT
dPθ
=
= π(θ) pθT
dλ
dPθT dλ

λ−ae

then, if ψ is unbiased
Z
g(θ) =

2.6

Z
ψ dPθ =

Z
ψ pθ dλ =

ψ π(θ) pθT dλ

The Main Problem

With all the previous considerations we may formalize the generalization to the
vector case of the Barankin formulation as:
Problem 2.3 (Main Problem). Given a function g : θ → RdP , defined for each
θ ∈ Θ, and a family of p.m.’s indexed by θ ∈ Θ, that satisfy theR Hypothesis 2.1,
2.2, and 2.3, find a function ψ ∈ L2 (RdS , BdS , PθT ), such that ψ π(θ) dPθT =
g(θ), ∀θ ∈ Θ. If there are several solutions find, if possible, a solution with
minimal covariance matrix at θT .
The solution to this problem is given below in Theorem 7.3.

3

Matrix bound

For a vector a in a finite vector space denote [a]i the i-th component of the
vector. For a matrix A, define [A]i as the i-th column of the matrix and [A]i,j
h
i
as i-th, j-th element of the matrix. We have [A]i,j = [A]j . Denote AT the
i

transpose of the matrix A, Det(A) the determinant of A, and Tr [A] the trace
of A. A square symmetric real matrix A ∈ RN ×N is a symmetric non-negative
definite (s.n.n.d.) matrix iff, xT Ax ≥ 0, for all x ∈ RN . A real s.n.n.d. matrix
A ∈ RN ×N is a symmetric positive definite (s.p.d.) matrix if Det(A) 6= 0, iff
xT Ax > 0, for all x 6= 0. Two s.n.n.d. matrices A ∈ RN ×N and B ∈ RN ×N are
comparable in the Löwner partial order, [24] p. 166, if either A − B is s.n.n.d.
and then A ≥ B, or if B − A is s.n.n.d. and then B ≥ A, else, they are not
comparable. For A, B and C s.n.n.d of dimensions N × N , then if A ≥ B and
B ≥ C then A ≥ C, and if A ≥ B and B ≥ A, then A = B, see e.g. [7]
Lemma 3 p. 2444. If A ∈ RN ×N is s.p.d., and S ∈ RN ×M is arbitrary, such

B. Cernuschi-Frı́as

7

that S T AS = 0, then S = 0, see e.g. [7] Lemma 2 p. 2444. For A ∈ RN ×N
1/2
denote the Frobenius norm as kAkF = Tr(AAT )
.
The following lemma is a variant of the information inequality [25] p. 172,
[13] Lemma 1 p. 1288, [19] pp. 326–328.
Lemma 3.1. Let (X, X, µ) be an arbitrary measure space. Let L2 (X, X, µ) be
the collection of all the measurable square integrable real valued functions from
d
dρ
X to R. Let dγ , dρ , RdA ∈ N, γ ∈ (L2 (X, X, µ)) γ , ρ ∈ (L
A∈
R 2 (X,T X, µ)) , and
dA ×dρ
T
dγ ×dρ
R
. Call F = γ ρ dµ, F ∈ R
, and B = ρ ρ dµ, B ∈ Rdρ ×dρ .

−1
R
If Det A B AT 6= 0, then γ γ T dµ ≥ F AT A B AT
A F T , with
equality if and only if there exists a matrix Λ0 ∈ Rdγ ×dA such that γ = Λ0 A ρ
−1
µ-almost-everywhere (µ-ae), and in that case, it is Λ0 = F AT A B AT
.
R
T
Proof. For each Λ ∈ Rdγ ×dA , let M(Λ) = (γγ − Λ A ρ ) (γγ − Λ A ρ ) dµ, M(Λ) ∈
dγ ×dγ
dγ ×dA
. Then M(Λ) is s.n.n.d. for all Λ ∈ R
. We have M(Λ) =
RR
γ γ T dµ − F AT ΛT − Λ A F T + Λ A B AT ΛT . By assumption
Det A B AT 6= 0, so that the matrix A B AT is invertible. Define Λ0 =
−1
−1
R
F AT A B AT
, then, M(Λ0 ) = γ γ T dµ − F AT A B AT
A FT ≥ 0

R
−1
so that γ γ T dµ ≥ F AT A B AT
A F T , and there is equality iff
M(Λ0 ) = 0. From the definition of M(Λ), if there exists Λ? ∈ RRdγ ×dA such
that γ R= Λ? A ρ µ-ae, then M(Λ? ) = 0. In that case it will be γ ρ T dµ =
Λ? A
ρ ρ T dµ, so that F = Λ? A B, and then F AT = Λ? A B AT .
−1
Since by hypothesis A B AT is invertible, then Λ? = F AT A B AT
=
Λ0 so that M (Λ0 ) = M (Λ? ) = 0, and then we obtain the equality. Con−1
R
versely, if γ γ T dµ = F AT A B AT
A F T , take Λ0 ∈ Rdγ ×dA , as

−1
Λ0 = F AT A B AT
, so that by the definition of M(Λ) it results M(Λ0 ) =
−1
R
R
T
(γγ − Λ0 A ρ ) (γγ − Λ0 A ρ ) dµ = Zγ γ T dµ − F AT A B AT
A
F T = 0.
T
Hence Tr (M(Λ0 ))
=
Tr
(γγ − Λ0 A ρ ) (γγ − Λ0 A ρ ) dµ
=
Z
Pdγ
2
[γγ − Λ0 A ρ ]i dµ = 0, and then γ = Λ0 A ρ µ-ae.
i=1
The following definition specifies all the elements required in the proposed
linear matrix inequality (LMI) generalized Barankin bound.
Definition 3.1. Given arbitrary dM ∈ N and dA ∈ N, an arbitrary real matrix
A of dimensions dA × dM , A ∈ RdA ×dM , and arbitrary indexes θi ∈ Θ, for
1 ≤ i ≤ dM , define τ T = (θ1 , θ2 , . . . , θdM ), τ ∈ ΘdM , and define the quad-tuple
q, as q = (dM , dA , A, τ ). Define h(θ) = g(θ) − g(θT ).
Define β T (ττ ) = (π(θ1 ), π(θ2 ), . . . 
, π(θdM )), i.e. β (ττ ) ∈ B0 dM , define the dP ×
dM real matrix G(ττ ) as G(ττ ) = g(θ1 ) − g(θT ), g(θ2 ) − g(θT ), . . . , g(θdM ) −



g(θT ) = h(θ1 ), h(θ2 ), . . . , h(θdM ) , and define the dM × dM real matrix


B(ττ ) as B(ττ ) = E β (ττ ) β T (ττ ) .

B. Cernuschi-Frı́as

8


Define CA as the collection of all the quad-tuples q with Det A B(ττ ) AT ) =
6 0.,
i.e.

CA = q : ∀ dM ∈ N, ∀ dA ∈ N, ∀ A ∈ RdA ×dM ,

∀ τ ∈ ΘdM , with Det A B(ττ ) AT ) 6= 0
.
Definition 3.2. Call Ug the family of all the finite covariance at θT unbiased
estimators of g(θ), for all θ ∈ Θ, for Problem 2.3. Define WA as the collection
of matrices of the form:
−1
W (q) = G(ττ ) AT A B(ττ ) AT
A GT (ττ )
∀q ∈ CA

i.e. ∀dM ∈ N, ∀dA ∈ N, ∀A ∈ RdA ×dM , ∀ττ ∈ ΘdM , with Det A B(ττ ) AT ) 6= 0,
with G(ττ ) and B(ττ ) as in Definition 3.1. Hence WA = {W (q) : q ∈ CA }. The
matrices W (q) will be called the Barankin covariance lower bound matrices for
Problem 2.3.
Let S(B0 ) be the linear span of B0 , i.e. S(B0 ) = {u ∈ L2 (RdS , BdS , PθT ) : u =
PdM
i=1 ai πi w.p. 1, ∀dM ∈ N, ∀ ai ∈ R for 1 ≤ i ≤ dM , ∀ πi ∈ B0 for 1 ≤ i ≤
dM }.
The following theorem gives the first half of the Barankin vector bound.
Theorem 3.1. If for Problem 2.3 there exists a finite covariance at θT unbiased
X ) ∈ Ug for g(θ), ∀θ ∈ Θ, then, see Definition 3.1,
estimator ψ (X
−1
ψ ) ≥ G(ττ )AT AB(ττ )AT
CovθT (ψ
AGT (ττ ) ∀q ∈ CA
(3.1)
i.e. (3.1) is true for the set of conditions CA : ∀ dM ∈ N, ∀dA ∈ N, ∀ A ∈
RdA ×dM , ∀ τ ∈ ΘdM , with Det A B(ττ ) AT 6= 0. There is equality in (3.1) for
some ψ ∗ ∈ Ug and some q∗ = (d∗M , d∗A , A∗ , τ ∗ ), q∗ ∈ CA , if and only if there
exists a matrix Λ∗ ∈ RdP ×dA such that ϕ ∗ = ψ ∗ − g(θT ) = Λ∗ A∗ β (ττ ∗ ) w.p. 1,
ϕ∗ ]i is a linear combination of elements in B0
if and only if each component [ϕ
dP
w.p. 1 for 1 ≤ i ≤ dP , i.e. ϕ ∗ = ψ ∗ − g(θT ) ∈ S(B0 ) , see Definition 3.2.
Proof. The proof will follow from Lemma 3.1. Let ψ ∈ Ug be an arbitrary
finite covariance at θT unbiased estimator for Problem 2.3. Take an arbitrary
dM ∈ N, and an arbitrary τ ∈ ΘdM , see Definition 3.1. Since ψ is unbiased, see
(2.1),


 
EθT ϕ β T (ττ ) = EθT[ ϕ π(Θ1 )] , . . . , EθT[ ϕ π(ΘdM )]
Z
Z

=
ϕ π(Θ1 ) dPθT , . . . , ϕ π(ΘdM ) dPθT
Z

Z
Z
=
ϕ dPθ1 , ϕ dPθ2 , . . . , ϕ dPθdM


= h(θ1 ), h(θ2 ), . . . , h(θdM ) = G(ττ )

B. Cernuschi-Frı́as

9





ψ − g(θT )) β T (ττ ) , see Definition 3.1, and
then, G(ττ ) = EθT ϕ β T (ττ ) = EθT (ψ
this
is true for any unbiased estimator ψ ∈ Ug . Additionally, we have,
R
A β (ττ ) β T (ττ ) AT dPθT = A B(ττ ) AT , see Definition 3.1. Take an arbitrary
dA ∈ N and a matrix A ∈ RdA ×dM such that Det(A B(ττ )AT ) 6= 0 otherwise arbitrary. Then the result follows from Lemma 3.1 with γ = ϕ , ρ = β (ττ ), F = G(ττ ),
and B = B(ττ ). The first if and only if equality condition follows directly from
Lemma 3.1. As for the second equality condition, if there is equality in (3.1)
for some ψ ∗ ∈ Ug and some q∗ = (d∗M , d∗A , A∗ , τ ∗ ) ∈ CA , then from Lemma 3.1,
∗
there exists Λ∗ ∈ RdP ×dA such that ϕ ∗ = ψ ∗ − g(θT ) = Λ∗ A∗ β (ττ ∗ ) w.p. 1.
d∗
ϕ∗ ]i is a linear combination w.p. 1 of
Since β (ττ ∗ ) ∈ B0M , then each component [ϕ
dP
∗
elements in B0 , for 1 ≤ i ≤ dP , i.e. ϕ = ψ ∗ − g(θT ) ∈ S(B0 ) . Conversely,
suppose that ψ ∗ ∈ Ug , with ϕ ∗ = ψ ∗ −g(θT ), and that each component [ ϕ ∗ ]i is a
d P
linear combination w.p. 1 of elements in B0 , i.e. ϕ ∗ = ψ ∗ − g(θT ) ∈ S(B0 ) .
Since each [ ϕ ∗ ]i ∈ S(B0 ) w.p. 1, then, there exist Mi ∈ N, ai ∈ RMi ,
and τ i ∈ ΘMi , such that [ ϕ ∗ ]i = aTi β (ττ i ) w.p. 1 for 1 ≤ i ≤ dP . Define

PdP
Mα = i=1
Mi , and τ Tα = τ T1 , τ T2 , . . . , τ TdP , τ α ∈ ΘMα . Call β α = β (ττ α ),
β α ∈ B0 Mα . Define the real matrix
Aα ∈ RdP ×Mα , as the block-diagonal ma
T
T
T
trix Aα = Diag a1 , a2 , . . . , adP , where each block aTi is of dimension 1 × Mi ,
for 1 ≤ i ≤ Mi , so that ϕ ∗ = Aα β α . Starting with the second component of
β α , see Observation 2.3, delete the i-th component if it is a linear combination
w.p. 1 of the previous components. There will remain Mγ ∈ N elements, with
1 ≤ Mγ ≤ Mα , see Observation 2.3. Call τ γ ∈ ΘMγ the non-deleted indexes of
M
the previous elimination procedure. Call β γ = β (ττ γ ), β γ ∈ B0 γ , so that the
components of β γ are linearly independent w.p. 1. Then, there exists a real matrix Aγ ∈ RMα ×Mγ such that β α = Aγ β γ w.p. 1, and then ϕ ∗ = Aα Aγ β γ w.p.
1. Define the quad-tuple qγ = (Mγ , Mγ , Iγ , τ γ ), where Iγ is the
 identity matrix
of dimensions Mγ × Mγ . Call Bγ = B(ττ γ ) = EθT β γ β Tγ , Bγ ∈ RMγ ×Mγ ,
so that Det(Bγ ) 6= 0. If not, there would exist α ∈ RMγ , with α 6= 0,
such that α T Bγ α = 0, but α T Bγ α = α T EθT β γ β Tγ α = EθT α T β γ β Tγ α =
h
2 i
EθT α T β γ
= α T β γ L , and then it would be α T β γ L = 0, which is a con2
2
tradiction since the components of β γ are linearly independent w.p. 1. Hence,

Det(Iγ Bγ IγT ) = Det(Bγ ) 6= 0, so that qγ ∈ CA . Since G(ττ γ ) = EθT ϕ ∗ β Tγ =

h
i

EθT Aα Aγ β γ β Tγ = Aα Aγ EθT β γ β Tγ = Aα Aγ Bγ , then W (qγ ) =
G(ττ γ ) Bγ−1 GT (ττ γ ) = Aα Aγ Bγ Bγ−1 Bγ ATγ ATα = Aα Aγ Bγ ATγ ATα .
h
i


T
ψ ∗ ) = EθT ϕ∗ (ϕ
ϕ∗ )T
But CovθT (ψ
= EθT Aα Aγ β γ (Aα Aγ β γ )
=
h
i
T
T
T
T
T
∗
ψ ) = W (qγ ).
Aα Aγ EθT β γ β γ Aγ Aα = Aα Aγ Bγ Aγ Aα , and then CovθT (ψ
The converse of this theorem, is given in Theorem 7.3, see Section 7.3.
Observation 3.1. The previous proof shows that ifψ ∈ Ug is a finite covariance

unbiased estimator for Problem 2.3, then G(ττ ) = h(θ1 ), h(θ2 ), · · · , h(θdM ) =

B. Cernuschi-Frı́as

10



EθTϕ β T (ττ ),
so
that
the
value
of
EθT ϕ β T (ττ ) is independent of the estimator ψ ∈ Ug as a consequence of the
unbiasedness of ψ , see (2.1).
Observation 3.2. Theorem 3.1 shows that any other finite covariance at θT unbiased estimator will satisfy (3.1). Then, the covariance matrix of any unbiased
estimator in Ug is comparable, in the Löwner partial order, with any of the
matrices in WA . Hence:
ψ) ≥ W
CovθT (ψ

∀ ψ ∈ Ug and ∀ W ∈ WA

dP
with equality if and only if ϕ = ψ −g(θT ) ∈ S(B0 ) . The covariance matrices
of estimators in Ug need not be comparable between them, as well as, Barankin
bound matrices in WA need not be comparable between them.

4
4.1

Functional analysis setup
Definition of the operator LB0 : B0 → RdP

From Hypothesis 2.3 we have B0 ⊆ L2 (RdS , BdS , PθT ). The subset B0 is not
a linear subspace, since any π ∈ B0 , is a Radon-Nykodim derivative of a p.m.
with respect to the p.m. PθT , then π ≥ 0 w.p. 1, [15], p. 315, with kπkL2 6= 0,
see Observation 2.3, so that −π cannot belong to B0 .
Let u0 be an arbitrary element in B0 . To this particular element u0 ∈ B0
corresponds a unique θ0 ∈ Θ, such that u0 ≡ π(θ0 ), see Hypothesis 2.2, so that
θ0 = π −1 (u0 ), and, to this index θ0 corresponds a unique well defined value
dP
element
h(θ0 ) = g(θ
 0 ) − dg(θT ) ∈ R . Hence, to u0 ∈ B0 corresponds a unique −1

−1
P
h π (u0 ) ∈ R which we define as LB0 u0 , so that LB0 (u0 ) = h π (u0 ) .
Hence,
LB0 (π(θ)) = h(θ)
∀θ ∈ Θ
(4.1)

−1
equivalently LB0 (u) = h π (u) , for all u ∈ B0 . Note that LB0 (π(θT )) =
h(θT ) = 0. Then, we may establish a direct relation from B0 to RdP , as an
operator LB0 from B0 to RdP , i.e. LB0 : B0 → RdP . This operator is not
(without additional conditions) necessarily linear nor bounded. The operator
LB0 is completely defined by the collection of Radon-Nykodim derivatives in
B0 , i.e. the elements π(θ) ∈ B0 , for all θ ∈ Θ, and the vectors g(θ) ∈ RdP ,
for all θ ∈ Θ, and does not depend on the existence or not of any unbiased
estimator, and if it exists, on whether it has finite covariance at θT or not.

4.2

Barankin formulation

The key observation made by Barankin, [2], for dP = 1, where he considers B0 ⊆
Lr (RdS , BdS , PθT ), r ≥ 1, is that if we are able to find an integral representation
of the operator LB0 , then the problem is solved.
In Barankin, [2], the answer is given by the Riesz Representation Theorem
which finds an element in the conjugate space φ0 ∈ Ls (RdS , BdS , PθT ), with

B. Cernuschi-Frı́as

11

R
1/s + 1/r = 1, such that LB0 (u) = φ0 u dPθT , ∀u ∈ B0 , with minimum
s-norm, i.e. minimum s-th variance. In our case, we generalize to vector estimates, i.e. dP > 1, but we will only consider the case s = r = 2 which is
the traditional variance and covariance matrices case, which is the most important in applications. To solve the problem the idea is to generalize the Riesz
representation theorem to the vector case. The Riesz representation theorem
requires that the represented functional be defined from a linear space to the
reals. Since B0 is not a linear subspace, Barankin, see [1], pp. 479-480, extends
the operator LB0 to a linear operator over the whole space, using indirectly
the Hahn-Banach theorem, invoking a condition first used by Riesz and generalized by Helly as exposed in [1] footnote in p. 56, see also [18]. In the next
sub-section we generalize the Helly-Riesz-Banach condition to handle the vector
case. In Section 5 we generalize the Riesz representation theorem to the vector
case without requiring the Hahn-Banach theorem, and in Section 6 we apply
these results to solve Problem 2.3.

4.3

Vector generalized Barankin hypothesis: Helly, Riesz,
Banach,Barankin (HRBB)

The following is the generalization of the hypothesis in [2], pp. 480 and 483–484,
see also [1], Theorems 4 and 5 pp. 55–57. This condition will be called here the
HRBB condition for Helly, Riesz, Banach, Barankin. For u ∈ L2 (RdS , BdS , PθT ),
1/2
R 2
u dPθT
define the semi-norm kukL2 =
, and call kxkRdP the standard Euclidean norm for x ∈ RdP .
Definition 4.1. (HRBB condition) The functions h(θ) = g(θ) − g(θT ), h(θ) ∈
RdP , and π(θ) ∈ B0 , ∀θ ∈ Θ, satisfying the Hypothesis 2.1, 2.2, and 2.3, for
Problem 2.3, satisfy the HRBB condition iff: ∃KH ∈ R+ , i.e. KH ≥ 0, such
that:
dM
dM
X
X
(4.2)
≤ KH
ai π(θi )
ai h(θi )
i=1

R dP

i=1

L2

for all dM ∈ N, for all ai ∈ R, i = 1, 2, · · · , dM , for all θi ∈ Θ, i = 1, 2, · · · , dM .

5

Generalized Riesz representation theorem

Here a generalization is given of the Riesz Representation Theorem for Hilbert
spaces real functionals, see e.g. [3] p. 112, to operators from an arbitrary Hilbert
space H , separable or not, to the real finite dimensional vector space RdP , with
dP ≥ 1. The proof given here does not require the Hahn-Banach extension
theorem, and then, the non-denumerable Axiom of Choice is not required, or
some less stringent variant, [18]. The bound proposed in Helly’s theorem, [1]
pp. 55–56, is generalized, and will be called the operator OP-HRBB (Helly,
Riesz, Banach, Barankin) condition.

B. Cernuschi-Frı́as

5.1

12

The Theorem.

Let H denote an arbitrary Hilbert space with semi-inner product hu, viH ,
1/2
∀u, v ∈ H , and semi-norm kukH = hu, uiH . If kukH = 0, then we say that
u = 0 in semi-norm, (i.s.n.). Equivalently u = 0 i.s.n. iff kukH = 0. Define
u = v i.s.n., iff ku − vkH = 0.
Theorem 5.1. Let H be an arbitrary Hilbert space. Let B0 be a non-empty
arbitrary subset of H , B0 6= ∅, B0 ⊆ H . Let LB0 be an operator from B0 to
RdP , LB0 : B0 → RdP , such that there exists at least one u0 ∈ B0 for which
LB0 (u0 ) 6= 0. Assume that the operator LB0 satisfies the following condition,
that will be called the operator HRBB condition (OP-HRBB): ∃KH ∈ R+ , i.e.
KH ≥ 0, such that:
k

dM
X

ai LB0 (ui ) kRdP ≤ KH k

i=1

dM
X

ai ui kH

(5.1)

i=1

for all dM ∈ N, for all ai ∈ R, i = 1, 2, · · · , dM , for all ui ∈ B0 , i =
1, 2, · · · , dM .
Call C(B0 ) ⊆ H the minimal closed linear space containing B0 , i.e. the
closed linear span of B0 , [11] p. 11. Then:
1. The operator LB0 may be extended to a bounded linear operator LC from
C(B0 ) to RdP , LC : C(B0 ) → RdP , with LC (u) = LB0 (u), for all u ∈ B0 .
2. The operator LC has the following representation: There exists dL ∈ N,
1 ≤ dL ≤ dP , and there exist orthonormal u
bi ’s, u
bi ∈ C(B0 ), for 1 ≤ i ≤
dL , such that:
LC (u) =

dL
X

hu, u
bi iH LC (b
ui )

∀u ∈ C(B0 )

(5.2)

i=1

Observation 5.1. The standard Riesz representation theorem, corresponds to
B0 = S(B0 ) = C(B0 ) = H , and dP = 1. In that case the operator LB0 is
taken as a bounded linear operator, so that the OP-HRBB condition is satisfied,
and then the conclusion is given by (5.2) with dL = 1.

5.2
5.2.1

Proof of the generalized Riesz representation theorem
Extension of the operator LB0 to the span of B0 , LS : S(B0 ) →
RdP

This extension follows the exposition of Banach in [1] pp. 55–56. Assume
the OP-HRBB condition is true. Call S(B0 ) the linear span i.s.n. of B0 , i.e.
PdM
S(B0 ) = {u ∈ H : u = i=1
ai πi i.s.n., ∀dM ∈ N, ∀ ai ∈ R, for 1 ≤ i ≤
dM , ∀ πi ∈ B0 , for 1 ≤ i ≤ dM }. The span S(B0 ) is called [B0 ] in [2] p.
495. Clearly, S(B0 ) is a linear space. With the help of the OP-HRBB condition

B. Cernuschi-Frı́as

13

extend the operator LB0 : B0 → RdP to an operator LS : S(B0 ) → RdP by
the following procedure: for each u ∈ S(B0 ) there exist, dependent on each
u, dM ∈ N, ai ’s ∈ R, 1 ≤ i ≤ dM , πi ’s ∈ B0 , 1 ≤ i P
≤ dM , such that u =
P
dM
dM
a
π
i.s.n.
Define
L
(u)
for
u
∈
S(B
)
as
L
(u)
=
S
0
S
i=1 ai LB0 (πi ). This
i=1 i i
procedure gives a well defined value for LS (u), since for any other decomposition
Pd0M 0 0
Pd0M 0
of u = j=1
aj πj i.s.n., resulting in L0S (u) = j=1
aj LB0 (πj0 ), because of the
OP-HRBB condition we will have:
k LS (u) −

L0S (u)

kRdP ≤ KH k

dM
X
i=1

0

ai πi −

dM
X

a0j πj0 kH = 0

j=1

Pd0M 0
PdM
0
so that
j=1 aj LB0 (πj ). The important result here is
i=1 ai LB0 (πi ) =
that now S(B0 ), unlike B0 , is a linear space, and that LS : S(B0 ) → RdP is
a bounded linear operator with bound KH , i.e. k LS (u) kRdP ≤ KH k u kH ,
∀u ∈ S(B0 ), and LS (u) = LB0 (u), ∀u ∈ B0 .
Observation 5.2. Barankin, [2] pp. 480 and 483–484, following [1] Theorems
2 and 4, p. 55, invokes the Hahn-Banach theorem, see e.g. [11] p. 78 or [1]
Theorem 1 p. 27, to extend the operator LS to the whole space. The HahnBanach theorem requires the Axiom of Choice or some slightly less stringent
condition, see e.g. [18]. In [1] arbitrary Banach spaces are considered. The
fact that here we work with Hilbert spaces, permits us to avoid the use of the
Hahn-Banach theorem, and then, the non-denumerable Axiom of Choice is not
required.
5.2.2

Extension of the operator LS to the closure of the span of B0 ,
LC : C(B0 ) → RdP

Define the closure of the span of B0 as C(B0 ) = Closure(S(B0 )), i.e. C(B0 ) =
{u ∈ H : ∃ (un )n∈N with un ∈ S(B0 ) ∀n ∈ N, such that k un − u kH → 0}. It
is readily checked that C(B0 ) is a closed linear subspace of H . The set C(B0 )
is called {B0 } in [2], p. 494. Extend the operator LS : S(B0 ) → RdP to an operator LC : C(B0 ) → RdP by continuity: Let u ∈ C(B0 ), then there exists a sequence (un )n∈N of elements un ∈ S(B0 ) such that k un − u kH → 0. Hence this
sequence is a Cauchy fundamental sequence, i.e. for each  > 0 there exists N ()
such that ∀ n, m ≥ N () we have k un − um kH < . But since LS is a bounded
linear operator, then k LS (un ) − LS (um ) kRdP ≤ KH k un − um kH < KH .
Then, (LS (un ))n∈N is a Cauchy fundamental sequence in the complete finite dimensional vector space RdP , [3] p. 23, hence there exists a limit in RdP . Call that
limit LC (u), so that k LS (un ) − LC (u) kRdP → 0, and then LS (un ) − LC (u) → 0
component by component (c.b.c.), i.e. [LS (un )]i − [LC (u)]i → 0, for 1 ≤
i ≤ dP . The value LC (u) is well defined: assume that for some other sequence (u0j )j∈N of elements u0j ∈ S(B0 ) with k u0j − u kH → 0, we ob
tain using the previous procedure a limit L0C (u) for the sequence LS (u0j ) j∈N ,
i.e. k LS (u0j ) − L0C (u) kRdP → 0. We have: k LS (u0j ) − LS (un ) kRdP =
k LS (u0j − un ) kRdP ≤ KH k u0j − un kH = KH k u0j − u − (un − u) kH

B. Cernuschi-Frı́as

14

≤ KH (k u0j − u kH + k un − u kH ). Then, k LC (u) − L0C (u) kRdP = k LC (u) −
LS (un ) − (L0C (u) − LS (u0j ))
−(LS (u0j ) − LS (un )) kRdP
≤

0
0
0
k LC (u) − LS (un ) kRdP + k LC (u) − LS (uj ) kRdP + KH k uj − u kH +

k un − u kH , so that taking the limits n → ∞, and j → ∞, we obtain
LC (u) = L0C (u), so that the value LC (u) ∈ RdP is independent of the chosen sequence. Hence LC (u) is a well defined operator from the closed linear subspace C(B0 ) ⊆ H to RdP . It is immediate to show that this operator is linear and that LC (u) = LS (u), ∀u ∈ S(B0 ), and then LC (u) =
LS (u) = LB0 (u), ∀u ∈ B0 . Finally, let’s show that the operator LC is bounded
with bound KH . Let u ∈ C(B0 ), and (un )n∈N a sequence of elements un ∈
S(B0 ) such that k u − un kH → 0, and then kLC (u) − LS (un )kRdP → 0.
≤ ku − un kH , then kun kH → kukH . Hence,
Since kukH − kun kH
kLC (u)kRdP = kLC (u) − LS (un ) + LS (un )kRdP ≤ kLC (u) − LS (un )kRdP +
kLS (un )kRdP ≤ kLC (u) − LS (un )kRdP + KH kun kH . Taking the limit
n → ∞, we obtain k LC (u) kRdP ≤ KH k u kH . Hence LC (u) is a bounded
linear operator from C(B0 ) to RdP , such that LC (u) = LS (u), ∀u ∈ S(B0 ), and
LC (u) = LS (u) = LB0 (u), ∀u ∈ B0
5.2.3

Null space NL and topological complement NL⊥ of the operator
LC : C(B0 ) → RdP

Define the kernel or null space of the operator LC as NL = {u ∈ C(B0 ) :
LC (u) = 0}. It is readily seen that NL is a closed linear subspace of C(B0 ),
NL ⊆ C(B0 ) ⊆ H . The orthogonal complement of NL with respect to C(B0 )
is NL⊥ = {u ∈ C(B0 ) : hu, wiH = 0 ∀w ∈ NL }. Note that the orthogonal
complement of NL⊥ with respect to C(B0 ) is NL . It is readily shown that NL⊥
is a closed linear subspace of C(B0 ), NL⊥ ⊆ C(B0 ) ⊆ H . Next, let’s show that
C(B0 ) = NL⊥ ⊕ NL , i.e. for each u ∈ C(B0 ) there exist unique elements i.s.n.
v ∈ NL⊥ and w ∈ NL , such that u = v + w i.s.n. We have:
Fact 1, (Minimum Distance to a Convex Set, [11] p. 8) Let u ∈ C(B0 ),
since NL is a closed convex subset of the complete Hilbert vector space H ,
there exists a w(u) ∈ NL , such that k u − w(u) k H ≤ k u − z k H , ∀z ∈ NL ,
and that element is unique i.s.n., i.e. if there exists another w0 (u) ∈ NL such
that k u − w0 (u) k H ≤ k u − z k H , ∀z ∈ NL , then k w(u) − w0 (u) k H = 0.
Fact 2, (Principle of Orthogonality, [11] p. 9) Define v(u) = u − w(u),
then v(u) is orthogonal to each of the elements in NL , so that v(u) ∈ NL⊥ .
Additionally, if Fact 2 is true then Fact 1 is true. The element w(u) is defined
as the orthogonal projection of u on the closed subspace NL denoted as w(u) =
Proj(u | NL ), similarly v(u) = Proj(u | NL⊥ ).
Hence u ∈ C(B0 ) may be decomposed as u = v(u) + w(u) i.s.n. with
v(u) ∈ NL⊥ and w(u) ∈ NL . This decomposition is unique i.s.n.: if we also
may write u = v 0 (u) + w0 (u) i.s.n., with v 0 (u) ∈ NL⊥ and w0 (u) ∈ NL , then
v(u)−v 0 (u) = w0 (u)−w(u) i.s.n. with v(u)−v 0 (u) ∈ NL⊥ and w0 (u)−w(u) ∈ NL
by linearity. Then, k v(u) − v 0 (u) k2H = hv(u) − v 0 (u), v(u) − v 0 (u)iH =
hv(u) − v 0 (u), w0 (u) − w(u)iH = 0. Similarly k w(u) − w0 (u) k2H = 0. Hence

B. Cernuschi-Frı́as

15

NL and NL⊥ are topological complements, [11] p. 93, i.e. C(B0 ) = NL⊥ ⊕ NL .
5.2.4

Images of B0 , S(B0 ), C(B0 ) and NL⊥

The previous properties are valid if we replace the space RdP with an arbitrary
Banach space. The following properties depend strongly on the finite dimensional character of RdP . The main property is that NL⊥ is a finite dimensional
sub-space of H as shown below.
Call I[B0 ] the image of the operator LB0 : B0 → RdP , then, I[B0 ] ⊆
dP
R . Since RdP has dimension dP then any dP + 1 vectors in RdP are linearly
dependent, and there are dP linearly independent vectors that constitute a basis
for RdP , see e.g. [5] pp. 178-179. Since I[B0 ] ⊆ RdP then any dP + 1 vectors
in I[B0 ] are linearly dependent. Since by hypothesis there exists at least one
u0 ∈ B0 such that LB0 (u0 ) 6= 0, then there exists dL ∈ N with 1 ≤ dL ≤ dP ,
such that any dL + 1 vectors in I[B0 ] are linearly dependent, and there are
dL linearly independent vectors LB0 (b
π1 ), LB0 (b
π2 ), · · · , LB0 (b
πdL ) that belong
to I[B0 ] with π
bi ∈ B0 , for 1 ≤ i ≤ dL . Note that I[B0 ] is not necessarily a
linear subspace.
The elements π
bi ∈ B0 for 1 ≤ i ≤ dL , are linearly independent i.s.n.,
i.e. whenever
there
are real coefficients ai ∈ R for 1 ≤ i ≤ dL , for which we
PdL
have k i=1
ai π
bi kH = 0, then ai = 0 for 1 ≤ i ≤ dL . If not, there would
PdL
exist ai ’s, ai ∈ R for 1 ≤ i ≤ dL , not all null, such that k i=1
ai π
bi kH =
PdL
0, but then, because of the OP-HRBB condition k
a
L
(b
π
i
B
i ) kRdP ≤
0
i=1
PdL
PdL
KH k i=1
ai π
bi kH , see (5.1), it would be k
a
L
(b
π
)
k
d
B0
i
R P = 0, iff
i=1 i
PdL
πi ) = 0, but the LB0 (b
πi )’s are l.i., so that it should be ai = 0,
i=1 ai LB0 (b
1 ≤ i ≤ dL , which is a contradiction.
Next, decompose each π
bi as in the previous item 5.2.3, i.e. for 1 ≤ i ≤ dL ,
π
bi = vbi + w
bi
i.s.n., where vbi = Proj(b
πi | NL⊥ ) and
⊥
bi ∈ NL ⊆ C(B0 ). Note
w
bi = Proj(b
πi | NL ), so that vbi ∈ NL ⊆ C(B0 ) and w
that, even though π
bi ∈ B0 , and then π
bi ∈ S(B0 ), in general it may happen
that vbi ∈
/ S(B0 ) and w
bi ∈
/ S(B0 ). Since, see item 5.2.2, LB0 (b
πi ) = LC (b
πi ) =
LC (b
vi + w
bi ) = LC (b
vi ) + LC (w
bi ) = LC (b
vi ), then the vectors LC (b
vi )’s are linearly independent. Hence, the elements vbi ’s are linearly independent i.s.n.: if
not,
would exist ai ’s, ai ∈ R for 1 ≤ i ≤ dL , not all null, such that
Pdthere
L
k i=1
ai vbi kH = 0. Then, since the extension LC is a bounded linear operaPdL
PdL
tor, see item 5.2.2, then k i=1
ai LC (b
vi ) kRdP ≤ KH k i=1
ai vbi kH , so that it
PdL
PdL
would be k i=1 ai LC (b
vi ) kRdP = 0, iff i=1 ai LC (b
vi ) = 0. But the LC (b
vi )’s
are l.i., so that it should be ai = 0, 1 ≤ i ≤ dL , which is a contradiction.
Since the vbi ’s are linearly independent i.s.n., and they all belong to NL⊥ ,
use the Gram-Schmidt procedure, see e.g. [5], p. 204, to obtain dL orthonormal
elements u
bi ∈ NL⊥ ⊆ C(B0 ) ⊆ H , that span the same space than the vbi ’s, so
that kb
ui kH = 1, hb
ui , u
bj iH = 0 for i 6= j, and hb
ui , wiH = 0, for 1 ≤ i ≤ dL and
∀ w ∈ NL . Hence, each u
bi is a linear combination i.s.n. of the vbi ’s, and since
this transformation is invertible, then each vbi is a linear transformation i.s.n. of
the u
bi ’s. The vectors LC (b
ui ) are linearly independent: if not, there would exist

B. Cernuschi-Frı́as

16

PdL
ai LC (b
ui ) = 0, but
ai ’s, ai ∈ R for 1 ≤ i ≤ dL , not all null, such that i=1
PdL
PdL
bi ∈ NLE. Then, for 1 ≤ k ≤ dL , we
bi )E= 0, so that Di=1 ai u
then, DLC ( i=1 ai u
PdL
PdL
PdL
ai hb
uk , u
bi iH =
bi
= i=1
bi
= 0. But u
bk , i=1 ai u
have u
bk , i=1 ai u
H

H

ak kb
uk k2H = ak , so that ak = 0 for 1 ≤ k ≤ dL , which is a contradiction.
Call IL the span of the linearly independent vectors LC (b
πi ), for 1 ≤ i ≤ dL ,
so that I[B0 )] ⊆ IL . Since LC (b
πi ) = LC (b
vi ), then IL is the span of the linearly
independent vectors LC (b
vi ), for 1 ≤ i ≤ dL . Since each vbi is a linear combination i.s.n. of the linearly independent i.s.n. elements u
bi , then, since LC is a
linear operator, each vector LC (b
vi ) is a linear combination of the linearly independent vectors LC (b
ui ) and vice-versa, and then IL is the span of the linearly
independent vectors LC (b
ui ), for 1 ≤ i ≤ dL .
Call I[S(B0 )] the image of the operator LS : S(B0 ) → RdP . Recall that
if u ∈ S(B0 ) then LS (u) = LC (u). Clearly, IL ⊆ I[S(B0 )]. If u ∈ S(B0 )
then u is a linear combination i.s.n. of a finite number of elements in B0 ,
and then the vector LC (u) ∈ I[S(B0 )], is the same linear combination of the
corresponding finite number of vectors in I[B0 ]. But since each vector in I[B0 ]
is a linear combination of the vectors LC (b
ui ), for 1 ≤ i ≤ dL , then LC (u) is a
linear combination of the independent vectors LC (b
ui ), for 1 ≤ i ≤ dL , hence
LC (u) ∈ IL , so that I[S(B0 )] = IL . If u ∈ S(B0 ), then since I[S(B0 )] = IL ,
PdL
there exist αi (u) ∈ R, 1 ≤ i ≤ dL , such that LC (u) =
ui ).
i=1 αi (u) LC (b
PdL
Define w(u) = u − i=1 αi (u) u
bi , then LC (w(u)) = 0, so that w(u) ∈ NL and
PdL
PdL
bi ∈ NL⊥ . Hence u ∈ S(B0 ) may be written as u = i=1
αi (u) u
bi +
i=1 αi (u) u
w(u) i.s.n.
Observation 5.3. Note that, whenever u ∈ C(B0 ) may be written as u =
PdL
bi + w(u), where w(u) ∈ NL and the u
bi ’s, 1 ≤ i ≤ dL , are ori=1 αi (u) u
thonormal elements in NL⊥ , then we have
2
kukH

=

dL
X

2

2

|αi (u)| + kw(u)kH

(5.3)

i=1

Call I[C(B0 )] the image of the operator LC : C(B0 ) → RdP . Since
I[S(B0 )] ⊆ I[C(B0 )] then IL ⊆ I[C(B0 )]. Fix an arbitrary u ∈ C(B0 ), then
there exists a sequence (un )n∈N of elements un ∈ S(B0 ) such that kun −ukH →
0, and kLC (un ) − LC (u)kRdP → 0. Since kun − ukH → 0, then (un )n∈N
is a Cauchy fundamental sequence in H . Since un ∈ S(B0 ), then, there
exist sequences (αi (un ))n∈N , with αi (un ) ∈ R for 1 ≤ i ≤ dL , ∀n ∈ N,
and a sequence (w(un ))n∈N , with w(un ) ∈ NL , ∀n ∈ N, such that un =
PdL
bi + w(un ). Since (un )n∈N is a Cauchy fundamental sequence
i=1 αi (un ) u
in H , with un ∈ S(B0 ) ⊆ C(B0 ), then, (5.3) shows that the sequences
(αi (un ))n∈N for 1 ≤ i ≤ dL are Cauchy fundamental sequences of real numbers,
and the sequence (w(un ))n∈N is a Cauchy fundamental sequence of elements
in NL ⊆ C(B0 ) ⊆ H . Since the reals are complete, there exist real numbers
ai ∈ R for which αi (un ) → ai for 1 ≤ i ≤ dL , and, since H is complete and NL
is closed, there exists an element η ∈ NL such that kw(un ) − ηkH → 0. Define

B. Cernuschi-Frı́as

17

PdL
ai u
bi + η, then u0 ∈ C(B0 ). Then (5.3) shows that kun − u0 kH → 0.
u0 = i=1
0
Since ku − u kH ≤ ku − un kH + kun − u0 kH , taking the limit, we obtain
u = u0 i.s.n. Hence for each u ∈ C(B0 ) we have found real numbers αi (u) ∈ R,
for 1 ≤ i ≤ dL , and an element w(u) ∈ NL such that
u=

dL
X

αi (u) u
bi + w(u)

i.s.n.

∀u ∈ C(B0 )

(5.4)

i=1

PdL
αi (u) LC (b
ui ), so that LC (u) ∈ IL , and then I[C(B0 )] =
Then, LC (u) = i=1
IL . Additionally, since for arbitrary u ∈ C(B0 ), from (5.4), we have
PdL
PdL
αi (u) u
bi
αi (u) u
bi i.s.n., then, u ∈ NL⊥ iff u = i=1
Proj(u | NL⊥ ) = i=1
i.s.n., and then NL⊥ is a finite dimension subspace, NL⊥ ⊆ C(B0 ) ⊆ H , with
dimension dL , even though H might be a non-separable space.
Hence, we have I[NL ] = {0}, and I[B0 ] ⊆ IL = I[S(B0 )] = I[C(B0 )] =
I[NL⊥ ].
Generalized Riesz representation of the operator LC : C(B0 ) →
RdP
PdL
From (5.4), it is LC (u) = i=1
αi (u) LC (b
ui ), for all u ∈ C(B0 ). Since the u
bi ’s
are
orthonormal
and
perpendicular
to
w(u),
then,
hu,
u
b
i
=
k
H
DP
E
dL
bi + w(u), u
bk
= αk (u). Hence
i=1 αi (u) u

5.2.5

H

LC (u) =

dL
X

hu, u
bi iH LC (b
ui )

∀u ∈ C(B0 )

i=1

see (5.2), which is the vector generalized Riesz representation for the extension
LC of an operator LB0 : B0 → RdP , B0 ⊆ H , satisfying the OP-HRBB
condition.

6

Optimal estimator under the HRBB condition

The space L2 (RdS , BdS , PθT ) is a Hilbert
space, [15] p. 194, with semi-inner
R
product hu1 , u2 iH = hu1 , u2 iL2 = u1 u2 dPθT , ∀ u1 , u2 ∈ L2 (RdS , BdS , PθT ),
1/2
R 2
semi-norm kukH = kukL2 =
u dPθT
, and equality in semi-norm (i.s.n.)
given by equality with probability 1 (w.p. 1).
Lemma 6.1. If the HRBB condition holds for Problem 2.3, see Definition 4.1,
then there exists a finite covariance unbiased estimator ψbc ∈ Ug .
Proof. Since L2 (RdS , BdS , PθT ) is a Hilbert space, then we take the elements
of H as the functions in L2 (RdS , BdS , PθT ). Define the operator LB0 (u) =
h π −1 (u) , for all u ∈ B0 , see Section 4.1. Since the HRBB condition holds for

B. Cernuschi-Frı́as

18

Problem 2.3, see (4.2), then the OP-HRBB condition holds, see (5.1), and then
we may apply Theorem 5.1. From (5.2) we obtain
LC (u) =

dL
X

hu, u
bi iH LC (b
ui ) =

i=1

Z
=

u

dL
X

Z
LC (b
ui )

uu
bi dPθT

(6.1)

i=1

"d
L
X

#
u
bi LC (b
ui ) dPθT

∀u ∈ C(B0 )

i=1

where the u
bi ’s are orthonormal, with u
bi ∈ NL⊥ ⊆ C(B0 ), for 1 ≤ i ≤ dL . Note
the importance of working with finite dimensions dP and dL , with 1 ≤ dL ≤ dP ,
since this permits exchanging sums and integrals invoking elementary properties
of Lebesgue integrals. Define
ϕ
bc =

dL
X

u
bi LC (b
ui )

(6.2)

i=1

R
so that LC (u) = ϕ
bc u dPθT , ∀ u ∈ C(B0 ).
Since each LC (b
ui ) is some constant real vector, i.e. LC (b
ui ) ∈ RdP , for
dS
1 ≤ i ≤ dL , and each u
bi ∈ L2 (R , BdS , PθT ), then each component of the
vector ϕ
bc is square integrable, i.e. [b
ϕbc ]i ∈ L2 (RdS , BdS , PθT ), equivalently ϕ
bc ∈
dS
L2 (R , BdS , PθT ). Then ϕ
bc is a measurable function from L2 (RdS , BdS , PθT )
PdL
X) =
X ) LC (b
X) :
bc (X
to RdP , so that ϕ
bi (X
ui ) is a random vector, ϕ
bc (X
i=1 u
Ω → RdP , that does not depend on the sub-indexes θ ∈ Θ. Additionally since
ϕ
bc ∈ L2 (RdS , BdS , PθT ), then ψbc = ϕ
bc +g(θT ) has finite covariance as previously
discussed in Section 2.1. Since LC (u) = LB0 (u) if u ∈ B0 , see Section 5.2.2,
and, for each u ∈ B0 there exists θ ∈ Θ such that u = π(θ), see Hypothesis
2.2, and LB0 (π(θ)) = h(θ), ∀θ ∈R Θ, see (4.1), then,
R using (6.1) and (6.2),
bc π(θ) dPθT = ϕ
bc (dPθ /dPθT ) hdPθTi =
h(θ) = LB0 (π(θ)) = LC (π(θ)) = ϕ
R
R
X ) dPθ = Eθ [b
ϕ
bc dPθ = ϕ
bc (X
ϕbc ], ∀θ ∈ Θ, see (2.1). Then, Eθ ψbc =
R
R
X ) dPθ = g(θ), ∀θ ∈ Θ. Hence, ψbc (X
X ) is unbiased for all
ψbc dPθ = ψbc (X
θ ∈ Θ, and then ψbc ∈ Ug .
Definition 6.1. Define the HRBB estimator as ψbc = ϕ
bc + g(θT ), where ϕ
bc is
given by (6.2) as discussed in Lemma 6.1, so that ψbc ∈ Ug .
Definition 6.2 (Barankin-efficient estimator). A finite covariance unbiased esψ) ≥
timator ψb ∈ Ug for Problem 2.3, will be called Barankin-efficient, if CovθT (ψ
b
b
ψ), for all ψ ∈ Ug . Equivalently, ψ is a minimum-covariance unbiased
CovθT (ψ
estimator for Problem 2.3.
Definition 6.3. Let W be a collection of real s.n.n.d. matrices of dimensions
N × N . A s.n.n.d. matrix A ∈ RN ×N is an upper (lower) bound for W if
A ≥ W (A ≤ W ), ∀W ∈ W . Define, if it exists, the matrix-supreme (msup) of
the matrices in W , as a real s.n.n.d. matrix A of dimensions N × N , such that

B. Cernuschi-Frı́as

19

A ≥ W , ∀W ∈ W , and such that for each  ∈ R+ ,  > 0, there exists W () ∈ W
such that kA − W ()kF < . The notation will be A = msup W . If A ∈ W ,
then A will be called the matrix-maximum of W .

W ∈W

Define L as the real matrix, L ∈ RdP ×dL , with columns [L]i = LC (b
ui ), for
b T = (b
b , see (6.2). Since
1 ≤ i ≤ dL , and define u
u1 , u
b2 , · · · , u
bdL ), so that ϕ
bc = L u
bu
b T = IdL , where IdL
the u
bi ’s are orthonormal in L2 (RdS , BdS , PθT ), then EθT u
b
is the identity matrix
h of dimensions
i dL × dL . From (6.2), we have: CovθT (ψ c ) =


T
T
T
T
T
b (L u
b ) = L EθT u
bu
b L = L IdL L = L LT so
EθT ϕ
bc ϕ
bc = EθT L u
that
dL
X
T
b
ψ
CovθT ( c ) = L L =
LC (b
ui ) LC (b
ui )T
(6.3)
i=1

Theorem 6.1. If the HRBB condition holds for Problem 2.3, see Definition
4.1, then the HRBB estimator ψbc ,see Definition 6.1, is an unbiased Barankinefficient estimator, and CovθT ψbc = msup WA .
W ∈WA

Proof. The HRBB estimator ψbc is unbiased and has finite covariance as a consequence of Lemma 6.1. To show that it is Barankin-efficient let’s consider the
following two cases.
d
b ∈ S(B0 ) L . Since ϕ
b,
1) All the u
bi ’s belong to S(B0 ). Then, u
bc = L u
d P
then ϕ
bc ∈ S(B0 ) , and then, see Theorem 3.1 and Observation 3.2, we
have equality in (3.1). More precisely, since each u
bi ∈ S(B0 ), then, there
exist Mi ∈ N, ai ∈ RMi , and τ i ∈ ΘMi , such that u
bi = aTi β (ττ i ) w.p. 1,

P
c
dL
c =
bT = τ T1 · · · τ TdL , τb ∈ ΘM .
for 1 ≤ i ≤ dL . Define M
i=1 Mi , and τ

T
c
Call βb = β (b
τ ), βb ∈ B0 M , so that βb = β T (b
τ ) = β T (ττ 1 ) · · · β T (ττ dL ) , and
h
Ti
c×M
c
c
b = Eθ βb βb , B
b ∈ RM
b ∈ RdL ×M
B
. Define the real matrix A
, as the
T

b = Diag aT , aT , . . . , aT , where each block aT is of
block-diagonal matrix A
1
2
i
dL
b βb w.p. 1. Since
b = A
dimension 1 × Mi , for 1 ≤ i ≤ Mi , so that u


dS
bu
b T = IdL .
the u
bi ’s are orthonormal
in
L
(R
,
B
,
P
),
we
have
E
u
2
d
θ
θ
S
T
T


T 
h


Ti
b βb A
b βb
b Eθ βb βb A
b B
b A
bT , then
bT = A
bu
b T = EθT A
Since EθT u
= A
T


bB
bA
bT = Id , so that Det A
bB
bA
bT = 1. Since ψbc ∈ Ug is unbiased, then,
A
L
h
i




bT =
τ ) = EθT ϕ
bc β T (b
τ ) = EθT L u
τ ) = L EθT u
b β T (b
bβ
see Observation 3.1, G(b
h
h


Ti
Ti
b βb βb
b Eθ βb βb
b B.
b Take q
c, dL , A,
b τb , see
b = M
L E θT A
= LA
= L A
T


bB
bA
bT = 1, and then W (b
b ∈ CA since Det A
q) ∈
Definition 3.1, so that q

−1
bT A
bB
bA
bT
b GT (b
bT A
b GT (b
WA . Hence W (b
q) = G(b
τ) A
A
τ ) = G(b
τ) A
τ) =
bB
bA
bT )(A
bB
bT A
bT )LT = L LT = Covθ (ψbc ), see (6.3). Then Covθ (ψbc ) ∈
L (A
T

T

B. Cernuschi-Frı́as

20

WA , and then, see (3.1) and Observation 3.2, W (b
q) = CovθT (ψbc ) is a matrixmaximum for the matrices W ∈ WA and a matrix-minimum for the covariances
of any unbiased estimator ψ ∈ Ug , so that ψbc is a minimal covariance unbiased
estimator, i.e. the unbiased HRBB estimator ψbc is Barankin-efficient.
2) At least for one i∗ , 1 ≤ i∗ ≤ dL , we have that u
bi∗ belongs to C(B0 )
and u
bi∗ ∈
/ S(B0 ). Since each u
bi belongs to C(B0 ), then there exist sequences
(b
si (m))m∈N , for 1 ≤ i ≤ dL , with sbi (m) ∈ S(B0 ), 1 ≤ i ≤ dL , ∀m ∈ N, such that
lim kb
ui − sbi (m)kL2 = 0, for 1 ≤ i ≤ dL . As before, for each sbi (m) ∈ S(B0 ),
m→∞

there exist Mi (m) ∈ N, ai (m) ∈ RMi (m) , and τ i (m) ∈ ΘMi (m) , such that
bi − aTi (m) β (ττ i (m)) L2 = 0. Desbi (m) = aTi (m) β (ττ i (m)) w.p. 1, and lim u
m→∞

P
T
dL
c(m) =
define τb (m) =
τ T1 (m) · · · τ TdL (m) ,
fine M
i=1 Mi (m),

T
c
τb(m) ∈ ΘM (m) , and βb = β T (b
τ (m)) = β T (ττ 1 (m)) · · · β T (ττ d (m)) , βb ∈
m

L

m

c(m)
M
B0
.

c
b
Define the real matrix A(m)
∈ RdL ×M (m) , as the block-diagonal matrix

b
A(m)
= Diag aT1 (m), . . . , aTdL (m) , where each block aTi (m) is of dimension 1×

dL
Mi (m), for 1 ≤ i ≤ dL . Define b
sT (m) = (b
s1 (m) · · · sbdL (m)), b
,
s(m) ∈ S(B0 )


T
b
b
so that b
s(m) = A(m)
β m w.p. 1. Define S(m) = EθT b
s(m) b
s (m) , S(m) ∈
RdL ×dL .
Then,
S(m)
=
EθT b
s(m) b
sT (m)
=
h
h
T i
T i
c
c
b
bT (m). Call B(m)
b
b
A(m)
EθT βb mβb m A
= EθT βb mβb m , B(m)
∈ RM (m)×M (m) , so
b
b
bT (m). Since ψbc is unbiased, see Observation 3.1, we
that S(m) = A(m)
B(m)
A




b β T (b
have G (b
τ (m))
=
EθT ϕ
b β T (b
τ (m))
=
EθT L u
τ (m))
=
h
 T ci
h
 Ti
b
L EθT b
s(m) + [b
u−b
s(m)] βb m = L EθT A(m)
βb m + [b
u−b
s(m)] βb m =
h
T i
b
b
L A(m)
B(m)
+ L EθT (b
u−b
s(m)) βb m .


c(m), dL , A(m),
b
b(m) = M
Define q
τb(m) , then, see Appendix Lemma A.1,


b
b
bT (m) 6= 0, so that, for m ≥ M0 ,
for m ≥ M0 , we have Det A(m)
B(m)
A
b(m) ∈ CA , and then W (b
q
q(m)) ∈ WA . Then, after some algebra, for m ≥ M0
we obtain:

bT (m)
W (b
q(m)) = G(b
τ (m)) A

−1
b
b
bT (m)
b
A(m)
B(m)
A
A(m)
GT (b
τ (m))


= L S(m) LT + L EθT (b
u−b
s(m)) b
sT (m) LT
h
i
T
+ L EθT b
s(m) (b
u−b
s(m)) LT


−1
+ L EθT (b
u−b
s(m)) b
sT (m) S(m)
h
i
T
EθT b
s(m) (b
u−b
s(m)) LT
so that, see Appendix Lemma A.1, W (b
q(m)) → L LT = CovθT (ψbc ), see (6.3),
component by component and then in Frobenius norm.

B. Cernuschi-Frı́as

21

Unlike the previous case, if for some i∗ , 1 ≤ i∗ ≤ dL , we have that u
bi∗
belongs to C(B0 ) and u
bi∗ ∈
/ S(B0 ), then CovθT (ψbc ) = L LT ∈
/ WA . If not,
CovθT (ψbc ) ∈ WA , and then we have equality in (3.1), so that, see Theorem 3.1,
[b
ϕbc ]i ∈ S(B0 ), for each 1≤ i ≤ dL . But ϕ
bc = Lb
u, so that LT ϕ
bc = LT Lb
u, with
L = LC (b
u1 ) · · · LC (b
udL ) . Since the dL columns of L are linearly independent
α = 0,
then Det(LT L) 6= 0, if not there exists α ∈ RdP , α 6= 0, such that LT Lα
α = kLα
αk2RdP = 0, and then Lα
α = 0, contradiction. Hence
so that α T LT Lα
−1 T
b = LT L
u
L ϕ
bc , and then [b
u]i ∈ S(B0 ), for each 1 ≤ i ≤ dL , contradiction.
ψ ∈ Ug , see Theorem 3.1, it is CovθT (ψ
ψ ) ≥ W (b
ψ )−
Since ∀ψ
q(m)), then CovθT (ψ
W (b
q(m)) ≥ 0, so that, taking the limit, see Appendix Lemma A.2, we have
ψ ) − CovθT (ψbc ) ≥ 0, and then CovθT (ψ
ψ ) ≥ CovθT (ψbc ), ∀ ψ ∈ Ug . Then,
CovθT (ψ
ψ ) ≥ CovθT (ψbc ), ∀ ψ ∈ Ug ,
even though CovθT (ψbc ) ∈
/ WA , we have CovθT (ψ
b
and, see Theorem 3.1, CovθT (ψ c ) ≥ W , ∀ W ∈ WA . Furthermore, as previously shown, there exists a sequence Wm ∈ WA , Wm ≡ W (b
q(m)), such that
b
b
CovθT (ψ c ) − Wm
→ 0. Hence, though CovθT (ψ c ) is not a matrix-maximum
F

for WA , it is a matrix-supreme for WA , and CovθT (ψbc ) is a matrix-minimum for
all the covariances of the estimators in Ug , so that ψbc is Barankin-efficient.
Observation 6.1. A key point in Theorem 6.1 is that if WA is bounded above,
then, the optimal covariance CovθT (ψbc ) may be obtained as the matrix-supreme,
see Definition 6.3, of the matrices W ∈ WA , see (3.1).
CovθT (ψbc ) = msup G(ττ ) AT

−1
A B(ττ ) AT )
A GT (ττ )

q∈CA

and the matrix-supreme will be a matrix-maximum if and only if ψbc − g(θT ) =

dP
ϕ
bc ∈ S(B0 )
.

7
7.1

LMI equivalent formulation
Equivalence of the LMI bound and the HRBB condition

The statement that the Barankin covariance lower bounds WA are bounded
above, is a disguised form of the HRBB condition, as a matter of fact the
converse is also true, see Lemma 7.1 and Theorem 7.1 below.
Lemma 7.1. If the Barankin covariance lower bounds WA are bounded above,
i.e. the collection WA is bounded, see Definition 3.2, then the HRBB condition
holds, see Definition 4.1.
Proof. Call BW the bound for WA , i.e.
BW ≥ W (q) = G(ττ ) AT

−1
A B(ττ ) AT )
A GT (ττ )

(7.1)

B. Cernuschi-Frı́as

22

for all q ∈ CA . Since this is true for matrices A of all sizes dA ∈ N for a given
dM ∈ N, A ∈ RdA ×dM , in particular is true when dA = 1, i.e. when A has
one single row. Call aT = (a1 , a2 , · · · , adM ) the single row, so that A = aT ,
with a ∈ RdM . Then, A B(ττ ) AT = aT B(ττ ) a = aT EθT β (ττ ) β T (ττ )  a =
h
2
P


2 i
dM
EθT aT β (ττ ) β T (ττ ) a
= EθT aT β (ττ )
a
π(θ
)
=
= EθT
i
i
i=1
PdM

i=1

2

ai π(θi )

.

Observe that aT B(ττ ) a is a non-negative scalar, i.e.

L2
+

aT B(ττ ) a ∈ R , and since we assumed that q ∈ CA then aT B(ττ ) a 6= 0,
2
PdM
> 0. On the other hand,
as a matter of fact aT B(ττ ) a =
i=1 ai π(θi )
L2
P
dM
ai h(θi ). Then, (7.1) takes the form
G(ττ ) AT = G(ττ ) a = i=1
 P
T
dM
ai h(θi )
a
h(θ
)
i
i
i=1

P

dM
i=1

BW ≥

PdM

i=1

2

ai π(θi )

L2

Hence,
Tr
Tr [BW ] ≥


P

dM
i=1

ai h(θi )
PdM

i=1

1/2

Call KH = (Tr [BW ])

.

Since, Tr

 P

dM
i=1

T 
ai h(θi )

2

ai π(θi )


P

dM
i=1

L2

 P
T 
dM
ai h(θi )
a
h(θ
)
=
i
i=1 i

2
PdM
PdM
.
ai h(θi ) d /
ai h(θi ) d then, KH ≥
i=1 ai π(θi )
i=1
L2
R P
R P
PdM
PdM
, ∀dM ∈ N, ∀ai ∈ R,
≤ KH
Hence
i=1 ai π(θi )
i=1 ai h(θi )
L2
R dP
PdM
6= 0.
1 ≤ i ≤ dM , ∀θi ∈ Θ, 1 ≤ i ≤ dM , such that
i=1 ai π(θi )
L2
PdM
PdM
= 0, then
If
i=1 ai π(θi )
i=1 ai π(θi ) = 0 w.p. 1. Take an arbi-

PdM

i=1

L2

trary u∗ ∈ B0 , then, see Observation 2.3, ku∗ kL2 6= 0. Call θ∗ = π −1 (u∗ ).
PdM
∗
∗
Then
i=1 ai π(θi ) + (1/n) u = (1/n) u w.p. 1, for all n ∈ N, so that
PdM
∗
= k(1/n) u∗ kL2 6= 0, for all n ∈ N. Hence
i=1 ai π(θi ) + (1/n) u
L2

PdM
∗
the previously obtained inequality applies,
≤
i=1 ai h(θi ) + (1/n)h θ
RdP
PdM
∗
KH
=
KH
(1/n) ku∗ kL2 ,
so that
i=1 ai π(θi ) + (1/n)u
L2

PdM
∗
→ 0, as n → +∞.
Since
i=1 ai h(θi ) + (1/n) h θ
dP


PdM
PdRM
∗
=
− (1/n) h θ∗
≤
i=1 ai h(θi )
i=1 ai h(θi ) + (1/n)h θ
RdP
R dP


PdM
∗
+ (1/n) h θ∗ RdP then, taking the limit
i=1 ai h(θi ) + (1/n) h θ
R dP
PdM
n → +∞, it results
= 0.
i=1 ai h(θi )
d
R

P

B. Cernuschi-Frı́as
PdM

23
PdM

, ∀dM ∈ N, ∀ai ∈ R,
ai π(θi )
L2
PdM
≥ 0, as required
1 ≤ i ≤ dM , ∀θi ∈ Θ, 1 ≤ i ≤ dM , such that
i=1 ai π(θi )
Hence

i=1

ai h(θi )

RdP

≤ KH

i=1

L2

by Definition 4.1, so that the HRBB condition holds.
Theorem 7.1. The HRBB condition holds, see Definition 4.1, if and only if
the collection WA is bounded, see Definition 3.2.
Proof. If HRBB holds, see Theorem 6.1, then there exists ψbc ∈ Ug such that

ψbc is Barankin-efficient, and then CovθT ψbc ≥ W , ∀ W ∈ WA , so that WA is
bounded. The converse follows as a consequence of Lemma 7.1.
X ) for
Lemma 7.2. If there exists a finite covariance unbiased estimator ψ (X
g(θ), ∀θ ∈ Θ for Problem 2.3, then WA is bounded above, see Definition 3.2.
Proof. Since a finite covariance unbiased estimator ψ exists, then (3.1) shows
that WA is bounded.

7.2

Other equivalent LMI bounds

One of the key ideas in Barankin’s paper is the use of the free coefficients ai ’s,
see [2] p. 480, that here take the form of the matrices A’s. As discussed in [2],
and here below, the matrices A are not required for the determination of the
optimal matrix bound, but, they are most useful when one needs to compare the
Barankin bound with other bounds, such as Cramer-Rao, Bhattacharyya, etc.
For the scalar case see [2] Corollaries 5–1 p. 487 and 6–1 p. 488. For the vector
Cramer-Rao bound, compare the results here with e.g. [21] and references there.
Definition 7.1. Define the pair d = (dM , τ ), where dM ∈ N, and τ ∈ ΘdM .
Define CB as the collection of all the pairs d with Det B(ττ ) 6= 0, with B(ττ ) as
in Definition 3.1, so that


CB = d : ∀ dM ∈ N, ∀ τ ∈ ΘdM , with Det B(ττ ) 6= 0
Define WB as the collection of matrices
−1 T
V (d) = G(ττ ) B(ττ )
G (ττ )

∀d ∈ CB

with G(ττ ) as in Definition 3.1. Equivalently WB = V (d) : d ∈ CB .
PdM
Define the function g(θ) as B0 -compatible if whenever i=1
ai π(θi ) = 0
PdM
w.p. 1, we have i=1 ai h(θi ) = 0, with dM ∈ N, ai ∈ R, θi ∈ Θ, for 1 ≤ i ≤ dM .
Note that if g(θ) is not B0 -compatible then no unbiased estimator exists for
g(θ), ∀θ ∈ Θ, for Problems 2.1, 2.2 or 2.3. If g(θ) is B0 -compatible, then for
τ ∈ ΘdM and a ∈ RdM , if aT β (ττ ) = 0, then G(ττ ) a = 0, and for A ∈ RdA ×dM , if
A β (ττ ) = 0, then G(ττ )AT = 0. Hence, if τ 1 ∈ ΘdA and we have A β (ττ ) = β (ττ 1 ),
then G(ττ )AT = G(ττ 1 ).

B. Cernuschi-Frı́as

24

Theorem 7.2. The collection WA , see Definition 3.2, is bounded above, if and
only if the collection WB is bounded above and g(θ) is B0 -compatible.
Proof. Assume WA is bounded. Then, there exists a s.n.n.d. matrix B1 ∈
RdP ×dP , such that B1 ≥ W (q), ∀q
d0 ∈ CB , with
 ∈ CA . Take0 an arbitrary
0
0
0
0
0
0
d = dM , τ , so that Det B(ττ ) 6= 0. Define q = dM , dM , Id0M , τ 0 , where

Id0M is the identity matrix of dimensions d0M × d0M . Since Det Id0M B(ττ 0 )IdT0 =
M

Det B(ττ 0 ) 6= 0, then q0 ∈ CA , and we have W (q0 ) = V (d0 ), so that B1 ≥ V (d0 ),
∀d0 ∈ CB , and then WB is bounded. Since WA is bounded, then the HRBB condition holds, see Lemma 7.1, and then (4.2) shows that g(θ) is B0 -compatible.
Conversely, assume WB is bounded. Then, there exists a s.n.n.d. matrix
B2 ∈ RdP ×dP , such that B2 ≥ V (d), ∀d ∈ CB . Take an arbitrary q0 ∈ CA , with

0
0
T 
q0 = d0M , d0A , A0 , τ 0 , with A0 ∈ RdA ×dM , so that Det A0 B(ττ 0 ) (A0 ) 6= 0. As
in the proof of the last part of Theorem 3.1, obtain d?τ ∈ N, 1 ≤ d?τ ≤ d0M ,
?
0
?
A? ∈ RdM ×dτ , and τ ? ∈ Θdτ , by elimination of the components of the vector
β (ττ 0 ) which are linear combinations w.p. 1 of previous components,
so that



β (ττ 0 ) = A? β (ττ ? ), with Det B(ττ ? ) = Det EθT β (ττ ? ) β T (ττ ? ) 6= 0.




T
Then, B(ττ 0 ) = EθT β (ττ 0 ) β T (ττ 0 ) = A? EθT β (ττ ? ) β T (ττ ? ) (A? ) =
T
T
A? B(ττ ? ) (A? ) . Since g(θ) is B0 -compatible, then G(ττ 0 ) = G(ττ ? ) (A? ) .
Hence,
W (q0 ) = G(ττ 0 ) (A0 )

T
T

T −1 0
A0 B(ττ 0 ) (A0 ) )
A G(ττ 0 )
T

= G(ττ ? ) (A? ) (A0 )

T

A0 A? B(ττ ? ) (A? ) (A0 )

T −1

A0 A? GT (ττ ? )


Define d? = d?τ , τ ? so that V (d? ) = G(ττ ? )B −1 (ττ ? )GT (ττ ? ). Then, the Appendix Lemma A.4 shows that V (d? ) ≥ W (q0 ), so that B2 ≥ V (d? ) ≥ W (q0 ).
Hence B2 ≥ W (q0 ), for all q0 ∈ CA , so that WA is bounded.
For an arbitrary symmetric matrix W call λM (W ) ∈ R its greatest eigenvalue. The operator norm kAkop of a matrix A ∈ RN ×M is its greatest singular
value, [4] p. 12, i.e. the non-negative square root of the greatest eigenvalue
1/2
of the matrix AT A, so that kAkop = λM (AT A)
. For s.n.n.d. matrices
singular values and eigenvalues coincide, [23] p. 19, so that if W ∈ W , then
kW kop = λM (W ). Define a k-identity matrix as a matrix of the form K IM
where K ∈ R and IM is the identity matrix of dimensions M × M . Then, we
have
Lemma 7.3. If X is a s.n.n.d. matrix, X ∈ RM ×M , then, for K ∈ R, we have
K IM ≥ X if and only if K ≥ λM (X).
Proof. Since X is symmetric, then it is diagonalizable, so that there exist an
orthogonal matrix Q ∈ RM ×M , and a diagonal matrix Λ ∈ RM ×M , such
that X = Q Λ QT . Then, since λM (X) IM ≥ Λ, we have λM (X) IM =
Q λM (X) IM QT ≥ Q Λ QT = X. Hence, if K ≥ λM (X), then K IM ≥
λM (X) IM ≥ X. Conversely if K IM ≥ X, then K IM ≥ Q Λ QT , so that

B. Cernuschi-Frı́as

25

QT K IM Q ≥ Λ, and since QT K IM Q = K IM , then K IM ≥ Λ. Hence
K ≥ λM (X).
Lemma 7.4. A non-empty collection W of s.n.n.d. matrices W ∈ RM ×M is
upper bounded if and only if there exists KW ∈ R, such that KW IM ≥ W ,
∀W ∈ W .
Proof. If there exists KW ∈ R, such that KW IM ≥ W , ∀W ∈ W , then by
definition KW IM is a matrix bound for W , and then W is bounded. Conversely,
assume W is bounded. Then there exists BW ∈ RM ×M s.n.n.d., such that BW ≥
W , ∀W ∈ W . Since BW is symmetric, then there exists the real maximum
eigenvalue λM (BW ) ∈ R. Take KW ∈ R such that KW ≥ λM (BW ). Then, from
Lemma 7.3, KW IM ≥ BW ≥ W , ∀W ∈ W .

7.3

Main Theorem

Collecting all the previous results, we have
Theorem 7.3 (Main Theorem). The following statements for Problem 2.3 are
equivalent, meaning that if any one of them is true, then they are all true:
1. A finite covariance vector unbiased estimator exists, i.e. Ug is not empty.
2. A Barankin-efficient vector unbiased estimator exists.
3. The HRBB condition holds.
4. The collection WA is bounded.
5. There exists K ∈ R+ such that K IdP ≥ W , ∀W ∈ WA .
6. The collection WB is bounded, and g(θ) is B0 -compatible.
7. There exists K ∈ R+ such that K IdP ≥ W , ∀W ∈ WB , and g(θ) is
B0 -compatible.
Proof. 3) ⇒ 2) ⇒ 1) follows from Theorem 6.1, 3) ⇔ 4) from Theorem 7.1, 1)
⇒ 4) from Lemma 7.2, 4) ⇔ 5) and 6) ⇔ 7) from Lemma 7.4, finally 4) ⇔ 6)
follows from Theorem 7.2.
Corollary 7.3.1. As a corollary, the collection Ug is empty iff WA is not
bounded, i.e. for each k ∈ N there exists Wk? ∈ WA such that kWk? kF ≥
kWk? kop = λM (Wk? ) > k, so that limk→+∞ kWk? kF = +∞. Note that Ug is
empty either because there are no unbiased estimators for g(θ), ∀θ ∈ Θ, or if
there exist, they don’t have finite finite covariance matrix at θT , see Definition
3.2.

B. Cernuschi-Frı́as

A

26

APPENDIX

Lemma A.1. Let H be an arbitrary Hilbert space. Let the elements ui ∈ H ,
for 1 ≤ i ≤ dL < +∞, be orthonormal so that kui kL2 = 1, for 1 ≤ i ≤ dL , and
hui , uj iH = 0, for i 6= j, 1 ≤ i, j ≤ dL , and then hui , uj iH = δi,j . Assume that
for each ui , for 1 ≤ i ≤ dL , there exist sequences si (m) m∈N , with si (m) ∈ H ,
∀m ∈ N, for 1 ≤ i ≤ dL , such that lim kui − si (m)kH = 0, for 1 ≤ i ≤ dL .
m→∞


Define S(m) ∈ RdL ×dL , as a matrix with i-th, j-th element S(m) i,j =
hsi (m), sj (m)iH , for 1 ≤ i, j ≤ dL , ∀m ∈ N. Then:
1. kS(m) − IdL kF → 0, as m → ∞, where IdL is the identity matrix of
dimensions dL × dL .
2. Det(S(m)) → 1, and S −1 (m) − IdL

F

→ 0, as m → ∞.

3. lim hui − si (m), sj (m)iH = 0, for all 1 ≤ i, j ≤ dL .
m→∞

Proof. a) From the Cauchy-Schwarz inequality we obtain hui − si (m), uj iH ≤
kui − si (m)kH , so that lim hui − si (m), uj iH = 0, for all 1 ≤ i, j ≤ dL .
m→∞

b) Also hui − si (m), uj − sj (m)iH ≤ kui − si (m)kH kuj − sj (m)kH , and
then lim hui − si (m), uj − sj (m)iH = 0, for all 1 ≤ i, j ≤ dL .
m→∞

c) We have hsi (m), sj (m)iH = hsi (m) − ui + ui , sj (m) − uj + uj iH =
hsi (m) − ui , sj (m) − uj iH + hui , sj (m) − uj iH + hsi (m) − ui , uj iH +
hui , uj iH . Taking the limit, and using a) and b), lim hsi (m), sj (m)iH = δi,j ,
m→∞

for all 1 ≤ i, j ≤ dL . Then S(m) → IdL component by component, and then
in Frobenius norm. Since the determinant of a matrix is an algebraic sum of a
finite number of products of a finite number of elements of the matrix, see [5] p.
319, then Det (S(m)) → Det(IdL ) = 1, so that ∃M0 ∈ N, such that ∀m ≥ M0 ,
it will be Det (S(m)) ≥ 1/2, and then Det (S(m)) 6= 0. Similarly, since the
elements of the inverse of a matrix are the quotients of algebraic sums of a finite
number of products of a finite number of elements of the matrix divided the
determinant, see [5] p. 325, then S −1 (m) has a limit A0 component by component, so that S(m) S −1 (m) → IdL A0 , but since S(m) S −1 (m) = IdL , ∀m ∈ N,
then A0 = IdL , and then S −1 (m) → IdL component by component as m → ∞,
and then in Frobenius norm, or any other matrix norm, so that we have shown
items 1) and 2).
d) We have hui − si (m), sj (m)iH = hui − si (m), sj (m) − uj + uj iH =
hui − si (m), sj (m) − uj iH + hui − si (m), uj iH . Applying a) and b) we obtain lim hui − si (m), sj (m)iH = 0, for all 1 ≤ i, j ≤ dL , so that we have
m→∞

shown item 3).

Lemma A.2. Let An n∈N be a sequence of s.n.n.d. matrices An ∈ RN ×N ,
An ≥ 0, ∀n ∈ N , such that there exists A ∈ RN ×N for which An → A c.b.c and
then in Frobenius norm. Then the matrix A is s.n.n.d.

B. Cernuschi-Frı́as

27

α.
Proof. Take α ∈ RN arbitrary, since N is finite, then limn→+∞ α T Anα = α T Aα
α ≥ 0,
But α T Anα ≥ 0, ∀n ∈ N, so that limn→+∞ α T Anα ≥ 0, and then α T Aα
α ∈ RN .
∀α
The following lemma is a LMI weighted form of the Cauchy-Schwarz inequality for matrices, [8] p. 1093. For convenience, a proof is given here.
Lemma A.3. Let M, N, P ∈ N. Let H ∈ RM ×M be an arbitrary real s.p.d.
matrix, and let X ∈ RN ×M and Y ∈ RP ×M , be otherwise arbitrary real matrices
such that Det(Y HY T ) 6= 0. Then
−1
X H XT ≥ X H Y T Y H Y T
Y H XT
with equality if and only if there exists Λ ∈ RN ×P , such that X = Λ Y , if and
only if
−1
X =X H YT Y H YT
Y
T

Proof. Let Λ ∈ RN ×P . Define T (Λ) = (X − Λ Y ) H (X − Λ Y ) , so that T (Λ)
is s.n.n.d., ∀Λ ∈ RN ×P . Define


−1 
−1 T
R(Λ) = Λ − X H Y T Y H Y T
Y H YT Λ−X H YT Y H YT
−1
and D = X H X T − X H Y T Y H Y T
Y H X T . Note that R(Λ) ≥
N ×P
0, ∀Λ ∈ R
. Then T (Λ) = R(Λ) + D ≥ 0, ∀Λ ∈ RN ×P . For Λ0 =
−1
X H YT Y H YT
, we have R(Λ0 ) = 0, so that T (Λ0 ) = D ≥ 0, and then
the LMI is obtained. If there is equality then D = 0, and then T (Λ) = R(Λ),
∀Λ ∈ RN ×P . In particular for Λ0 we have R(Λ0 ) = 0, and then T (Λ0 ) = 0.
−1
But, since H is s.p.d. then X = Λ0 Y = X H Y T Y H Y T
Y . As for
the converse, if there exists Λ1 such that X = Λ1 Y , then T (Λ1 ) = 0, since
R(Λ1 ) ≥ 0 by definition, and D ≥ 0 as previously shown, then, R(Λ1 ) = 0
and D = 0, because T (Λ1 ) = R(Λ1 ) + D. From D = 0 we obtain the equality
in the LMI inequality, and from R(Λ1 ) = 0, we obtain that Λ1 = Λ0 , because

−1
Det Y H Y T 6= 0 and then Y H Y T is s.p.d. If X = X H Y T Y H Y T
Y,
multiply both sides on the right by H X T , and then the equality for the LMI
is obtained.
The following lemma, cf. [9] Lemma 2.4.1, may be interpreted as a LMI
generalization of the Rayleigh quotient, [12] p. 117.
Lemma A.4. Let M, N, P ∈ N. Let B ∈ RM ×M be an arbitrary real s.p.d.
matrix, and let G ∈ RN ×M and A ∈ RP ×M , be otherwise arbitrary real matrices
such that Det(ABAT ) 6= 0. Then
−1
G B −1 GT ≥ G AT A B AT
A GT
with equality if and only if there exists Λ0 ∈ RN ×P , such that G = Λ0 A B, if
and only if
−1
G = G AT A B AT
AB

B. Cernuschi-Frı́as

28

Proof. Since B is s.p.d. then it has a unique s.p.d. square root B 1/2 , [16] p.
405, and we have Det(B) 6= 0 and Det(B 1/2 ) 6= 0. The result follows from the
previous Lemma A.3 taking, X = G B −1/2 , Y = A B 1/2 , and H as the identity
matrix of dimensions M × M .

References
[1] Stefan Banach, Théorie des opérations linéaires, Druk M. Garasinski, Warsaw, 1932.
[2] E. W. Barankin, Locally best unbiased estimates, The Ann. of Mathematical
Statistics 20 (1949), no. 4, 477–501.
[3] Richard Beals, Advanced mathematical analysis; periodic functions and distributions, complex analysis, laplace transform and applications, SpringerVerlag, 1973.
[4] Rajendra Bhatia, Positive definite matrices, Princeton University Press,
Princeton, N.J, 2007.
[5] Garrett Birkhoff and Saunders Mac Lane, A survey of modern algebra,
fourth ed., Macmillan, 1977.
[6] A. V. Bitsadze, Integral equations of first kind, World Scientific, Singapore
River Edge, NJ, 1995.
[7] B. Cernuschi-Frı́as, F. Gama, and D. Casaglia, Deepest minimum criterion
for biased affine estimation, IEEE Transactions on Signal Processing 62
(2014), no. 9, 2437–2449.
[8] John S. Chipman, On least squares with insufficient observations, Journal
of the American Statistical Association 59 (1964), no. 308, 1078–1111.
[9] John S. Chipman, Advanced econometric theory, Routledge, New York,
2011.
[10] Kai Lai Chung, A course in probability theory, second ed., Academic Press,
1974.
[11] John B. Conway, A course in functional analysis, Springer-Verlag, 1985.
[12] R. O. Duda and P. E. Hart, Pattern classification and scene analysis, first
ed., Wiley, New York, 1973.
[13] J. D. Gorman and A. O. Hero, Lower bounds for parametric estimation with
constraints, IEEE Transactions on Information Theory 36 (1990), no. 6,
1285–1301.
[14] Paul Richard. Halmos, Measure theory, Springer, 1974.

B. Cernuschi-Frı́as

29

[15] Edwin Hewitt and Karl Stromberg, Real and abstract analysis: a modern
treatment of the theory of functions of a real variable, Springer, 1975.
[16] R. A. Horn and C. R. Johnson, Matrix analysis, first ed., Cambridge University Press, Cambridge, UK, 1985.
[17] R. G. Laha and V. K. Rohatgi, Probability theory, Wiley, 1979.
[18] Lawrence Narici and Edward Beckenstein, The Hahn-Banach theorem: the
life and times, Topology and its Applications 77 (1997), no. 2, 193 – 211.
[19] C. R. Rao, Linear statistical inference and its applications, second ed.,
Wiley, New York, 1973.
[20] Saburou Saitoh and Y. Sawano, Theory of reproducing kernels and applications, Springer, Singapore, 2016.
[21] P. Stoica and Boon Chong Ng, On the Cramer-Rao bound under parametric
constraints, IEEE Signal Processing Letters 5 (1998), no. 7, 177–179.
[22] H. L. Van Trees and K. L. Bell, Bayesian bounds for parameter estimation
and nonlinear filtering/tracking, first ed., Wiley, IEEE Press, Piscataway,
NJ, 2007.
[23] Xingzhi Zhan, Matrix inequalities, Springer, Berlin New York, 2002.
[24] Fuzhen Zhang, Matrix theory: Basic results and techniques, first ed.,
Springer-Verlag New York, New York, 1999.
[25] Fuzhen Zhang, The Schur complement and its applications, Springer, New
York, 2005.

