Secure Estimation under Causative Attacks
Saurabh Sihag Ali Tajer


Implementing the inverse type-II seesaw mechanism into the 3-3-1 model
    Carlos Antônio de Sousa Pires ^a, Felipe Ferreira de Freitas^b, Jing Shu^b,c,d,e, Li Huang^b,c,f and Pablo Wagner Vasconcelos Olegário^a
    December 30, 2023
============================================================================================================================================




This paper considers the problem of secure parameter estimation when the estimation algorithm is prone to causative attacks.  Causative attacks, in principle, target decision-making algorithms (e.g., inference and learning algorithms) to alter their decisions by making them oblivious to specific attacks. Such attacks influence inference algorithms by tampering with the mechanism through which the algorithm is provided with the statistical model of the population about which an inferential decision is made. Causative attacks are viable, for instance, by contaminating the historical or training data, or by compromising an expert who provides the model. In the presence of causative attacks, the inference algorithms operate under a distorted statistical model for the population from which they collect data samples.  This paper introduces specific notions of secure estimation and provides a framework under which secure estimation under causative attacks can be formulated. A central premise underlying the secure estimation framework is that forming secure estimates introduces a new dimension to the estimation objective, which pertains to detecting attacks and isolating the true model. Since detection and isolation decisions themselves are imperfect, their inclusion induces an inherent coupling between the desired secure estimation objective and the auxiliary detection and isolation decisions that need to be formed in conjunction with the estimates. This paper establishes the fundamental interplay among the decisions involved, and characterizes the general decision rules in closed-forms for any desired estimation cost function. Furthermore, to circumvent the computational complexity associated with growing parameter dimension or attack complexity, a scalable estimation algorithm and its attendant optimality guarantees are provided. Finally, the theory developed is applied to secure parameter estimation in a sensor network operating in an adversarial environment.






§ INTRODUCTION





 §.§ Motivation



Statistical inference offers mechanisms for deducing the statistical properties of a population based on the data sampled from the population. Inference problems, broadly, focus on discerning the statistical model of the population or forming estimates about an unknown parameter that specifies the statistical model of the population. Anomaly detection,  which has diverse applications in intrusion detection, fraud detection, fault detection, system health monitoring, and event detection, constitutes a major class of inference problems in which the objective is raising alarms when the data pattern (e.g., statistical model) deviates significantly from the expected patterns. Effective detection of anomalies in the data strongly hinges on the known rules for distinguishing normal and abnormal data segments. These rules, for instance, can be specified by an expert or by leveraging the historical data, depending on the context of the application. These rules, subsequently, can be used for designing inference algorithms for detecting the anomalies with specified guarantees on relevant figures of merit. 



 
 
While anomaly detection, which in essence copes with the vulnerability of the sampled data to being contaminated or compromised, has been studied extensively (c.f. <cit.>), the vulnerability of the inference algorithms to being compromised is far less-investigated. The nature of security vulnerabilities that inference algorithms are exposed to is fundamentally distinct from that of data. Specifically, in the case of compromised sample data, the information of the decision algorithm about the model remains intact, while the data fed to the algorithm is anomalous, in which case the counter-measures consist of winnowing out the compromised samples and forming inferential rules that are robust against them. In contrast, attacks on the algorithms can be exerted by providing the algorithm with an incorrect statistical model for the data. This is viable by, for instance, contaminating the historical data or by confusing the expert that produces a model, which are critical for furnishing the true model for the statistical model of the data. Therefore, when the sampled data is compromised, an inference algorithm produces decisions based on an un-compromised known model for the data, while the data that it receives and processes are compromised. On the other hand, when the historical data or the expert are compromised, an inference algorithm functions based on an incorrect model for the data, in which case even un-compromised sampled data produces unreliable decisions. 


The aforementioned security vulnerabilities for the inference algorithms can be capitalized on by adversaries in order to force an inference algorithm to deviate from its optimal structure and produce decisions in ways that serve an adversary's purposes. Such attacks on decision algorithms are often referred to as causative attacks, through which an adversary aims to (i) make the inference algorithms oblivious to specific attacks, or (ii) degrade the performance of the inference algorithm in the presence of such an attack <cit.>. 

While the notion of secure decision-making in adjacent domains (e.g., machine learning and data mining) has been heavily investigated in recent years, the fundamental limits of secure statistical inference are not well-investigated, and all the limited existing studies remain rather ad-hoc. In this paper, we provide a framework for secure parameter estimation under the potential presence of causative attacks. We establish the fundamental tradeoffs involved in decision-making under causative attacks and characterize the optimal decision rules for securely estimating the parameters and concurrently detecting the presence of the attackers. Furthermore, we provide scalable algorithms for addressing settings in which the size or complexity of the attacks grow, and provide optimality guarantees on the performance of these algorithms. A summary of the content and the contributions are discussed in Section <ref>, and the relevant literature on secure statistical inference are reviewed in Section <ref>.





 §.§ Overview and Contributions



To lay the context for discussing the problem investigated, consider the canonical parameter estimation problem in which we have a collection of probability distributions {P_X: X∈} defined over a common measurable space. The objective is to estimate X, which lies in a known set ⊆ℝ^p, from data samples ≜ [Y_1,…, Y_n], where the sample Y_r is distributed according to P_X and  lies in a known set ⊆ℝ^m. We denote the probability density functions (pdfs) that the statistician assumes about the underlying distributions of X and Y_r by π and f (· X), respectively, i.e., 

    Y_r ∼  f (·  X) ,       X ∼ π .

For convenience in the analysis, we will assume that the pdfs do not have any non-zero probability masses over lower-dimensional manifolds. The objective of the statistician is formalizing a reliable estimator 

    X̂():^n↦ .


Causative Attacks:  In an adversarial environment, a malicious attacker might launch a causative attack to influence (degrade) the quality of X̂(). The purpose of such an attack is to compromise the process that underlies acquiring the statistical models. We emphasize that such an attack is different from those that aim to compromise the data, e.g., false data injection attacks, which aim to distort the data samples . Consequently, the effect of a causative attack is misleading the statistician about the true model f (·  X) that it assumes about the data. Such attacks are possible by compromising the historical (or training) data that is used for defining a model for the data. Depending on the specificity and the extent of a causative attack, e.g., the fraction of the historical or training data that is compromised, the true model f (· X) can deviate to alternative forms, the space of which we denote by . The attack can affect the statistical distribution of any number of the m coordinates of . There are two major aspects to selecting  as viable model space. 


  * An attack is effective if the compromised model is sufficiently distinct from the model assumed by the statistician. Hence, even though in general  can be any representation of possible kernels f (· X) mapping  to ℝ^m, only a subset of such  mappings suffices to describe the set of effective attacks.

  * There exists a tradeoff between the complexity of the model space and its expressiveness. Specifically, if it is overly expressive, it can represent the possible compromised models with a more refined accuracy at the expense of more complex inferential rules. 

 The specifics of the attack model will be discussed in Section <ref>. 




(q,β)-Security: The potential presence of an adversary introduces a new dimension to the estimation problem in (<ref>). Specifically, on the one hand, the stochastic model of the data can be altered by an attack and detecting whether the data model is compromised, itself being an inference task,  is never perfect. On the other hand,  designing an optimal estimation rule strongly hinges on successfully isolating the true model. Hence, there exists an inherent coupling between the original estimation  problem of interest and the introduced auxiliary problem (i.e., detecting the presence of an attacker and isolating the true model). Based on this observation, in an adversarial setting, there exists uncertainty about the true model, based on which the quality of the estimator is expected to degrade with respect to an attack-free setting. We are interested in establishing the fundamental interplay between the quality of discerning the true model and the degradation level in the estimation quality. To establish this interplay, we say that an estimator is (q,β)-secure if its estimation cost is weaker than that of the attack-free setting by a factor q∈[1,+∞), while missing at most β∈ (0,1] fraction of the attacks[Estimation costs and the associated estimation degradation factor q will be defined  in Section <ref>.].

In this paper, we pursue three intertwined objectives. First, we characterize the fundamental tradeoffs between q and β and delineate a tradeoff curve. Secondly, we characterize the inference rules in closed-forms and provide a secure estimation algorithm that achieves the optimal tradeoffs for any desired point on the tradeoff curve. Finally, to circumvent the computational complexity as the the dimension of the data, p, grows, or the complexity of the attacks scales up (e.g., the number of coordinates compromised),  we provide a scalable algorithm that has low computational complexity with guaranteed optimality in the asymptote of large data dimension p.






 §.§ Related Studies


The problem of secure inference is studied primarily in the context of sensor networks. The study in <cit.>, in particular, considers a two-sensor network in which one sensor is known to be secured, and one sensor is vulnerable to attacks. The objective is forming an estimate based on the mean-squared error criterion, for which a heuristic detection-driven estimator is designed. According to this design, first, a decision is formed about whether the unsecured sensor is attacked. If it is deemed to be attacked, then the estimator will rely only on the secured sensor, and otherwise, it uses the data from both sensors. Unlike in <cit.>, we consider a model with arbitrary size, assume that all data coordinates are vulnerable to the attack, and characterize the optimal decision structure, which turns out to be different from being a detection-driven design studied in <cit.>. Through a case study, we will also show the rather significant improvement in the estimation quality when using the optimal rules, compared to the rules specified in <cit.>.  

The adversarial setting defined in this paper is also similar to the widely-studied Byzantine attack models in sensor networks, in which the data generated by the compromised sensors are modified arbitrarily by the adversaries in order to degrade or the inference quality. An overview of the impact of Byzantine attacks on inference quality in sensor networks and relevant mitigation strategies are discussed in <cit.>. Detection-driven estimation strategies (i.e., when attack detection precedes the estimation routine) when the effect of the Byzantine attacks are characterized by randomly flipping of the information bits generated by the sensors are discussed in <cit.>. Furthermore, attack-resilient target localization strategies are investigated in <cit.> and <cit.>, where it is assumed that the attacker adopts a fixed strategy for maximum disruption to the inference. In these studies, nevertheless, an attacker may deviate from the worst-case strategy of incurring the maximum damage in order to launch a less powerful but sustained attack, which may not be detected perfectly. Finally, strategies for isolating the compromised nodes in sensor networks are investigated in <cit.>. The emphasis of these studies is primarily focused on detecting attacks, or isolating the attacked sensors, which is different from the scope of our paper, which is focused on parameter estimation.



The problem of secure estimation in linear dynamical systems in the context of cyber-physical systems has been studied extensively in the recent years (c.f. <cit.>). The studies that are more relevant to the scope of this paper include <cit.>, <cit.>, and <cit.>, which focus on the robust estimation of the states in dynamic systems. Specifically, a coding-theoretic interplay between the number of sensors compromised and the guarantees on perfect state recovery is characterized in <cit.>, a Kalman filter-based approach for identifying the most reliable set of sensors to make an inference from is investigated in <cit.>, and designing estimators that are robust against dynamical model uncertainty is studied in <cit.>. The degradation in estimation performance in a dynamical system consisting of a single sensor network is studied from the adversary's perspective in <cit.>, where the bounds on the degradation in estimation performance with degrees of stealthiness of the attacker are characterized.

All the aforementioned studies that involve secure estimation, irrespective of their focus or objective, conform in their design principle, which decouples the estimation decisions from all other decisions involved (e.g., attack detection or attacked sensor isolation), and produces either detection-driven estimators or estimation-driven detection routines. In the detection-driven estimation routines, an initial decision regarding the presence of an adversary (e.g., based on Neyman-Pearson theory) is followed by an optimal estimator based on the detection decision (e.g., Bayesian estimation). Such approaches implicitly assume that the detection decision has been perfect. Similarly, in an estimation-driven approach, the unknown parameter is first estimated, and then a detection decision is made (e.g., the generalized likelihood ratio test). Such approaches achieve optimality conditions only asymptotically, i.e., under the assumption of having an infinite number of measurements.  The premise that decoupling such intertwined estimation and detection problems into independent estimation and detection routines is sub-optimal is well-investigated (e.g., in <cit.>). 

 
















§ DATA MODEL AND DEFINITIONS






 §.§ Attack Model


Our focus is on the canonical estimation problem in (<ref>). The objective is to form an optimal estimate X̂() (under the general cost functions specified later) in the potential presence of a causative attack. Under the attack-free setting, the data is assumed to be generated according to the known distribution

    Y_r ∼  f (· X) ,       X ∼ π ,    for  r ∈{1,… n} .

In an adversarial setting, an adversary, depending on its strength and preference, can launch an attack that can compromise the underlying process that the statistician uses for acquiring f(· X). An attack will be carried out for the ultimate purpose of degrading the estimation quality of X. We assume that the adversary can corrupt the data model of up to K ∈{1,…,m} coordinates of . Hence, for a given K, there exist T = ∑_i=1^Km  i number of attack scenarios under which the compromised data models are distinct. Define ≜{ S_1,…, S_T } as the set of all possible combinations of attack scenarios, where S_i⊆{1,…,m} describes the set of coordinates the models of which are compromised under  scenario i∈{1,…,T}.

Under the attack scenario i∈{1,…,T}, the joint distribution of Y_r deviates from f and changes to a model in the space _i. As discussed earlier, there exists a tradeoff between the expressiveness of this space and the complexity of the ensuing inferential rules. Specifically, a larger space _i can distinguish different attack strategies with a more accurate resolution at the expense of high complexity in the analysis and the resulting decision rules. Also, the model can be effective if it encompasses sufficiently distinct models. Throughout the analysis of the paper, we assume that _i≜{f_i(· X)}, i.e., _i consists of one alternative distribution. This is primarily for the convenience in notations, and all the results presented can be generalized to any arbitrary space with countable elements. Based on this model, when the data models in the coordinates contained in S_i are compromised, the joint distribution changes from  f(· X) to f_i(· X).


Different attack scenarios might occur with different likelihoods, e.g., compromising one coordinate is easier than compromising two, and it might turn out to be more likely. To distinguish such likelihoods we adopt a Bayesian framework in which we define  ϵ_0 as the prior probability of having an attack-free scenario and define ϵ_i as the prior probability of the event that the attacker compromises the model under the coordinates specified by S_i. A block diagram of the attack model and the inferential goals to be characterized, which are discussed in the remainder of this section, is depicted in Fig. <ref>. Finally, we define the marginal pdf of the data at coordinate l∈{1,…,m} under the attack-free setting and when the coordinate is compromised by g_l^0 and g_l^1, respectively.










 §.§ Compound Decisions

The estimation objective constantly faces the uncertainty about whether an adversary exists. Furthermore, when one is deemed to exist, there is additional uncertainty pertaining to the number and the locations of the coordinates in which the data is being compromised. Hence, forming the estimate X̂() is inherently entwined with discerning the true model of the data. Decoupling the decisions for isolating the model and estimating the parameter under the isolated model does not generally render optimal performance. In fact, there exist extensive studies on formalizing and analyzing such compound decisions, which generally aim to decouple the inferential decisions. In <cit.>, it is shown that generalized likelihood ratio test (GLRT) is not always optimal and necessary and sufficient conditions for its asymptotic optimality are provided. Moreover, note that GLRT utilizes maximum likelihood estimates of unknown parameters in its decision rule and thus, it is primarily focused on the detection performance. In <cit.>, the problem of signal detection and estimation in noise is investigated under the Bayes criterion.  In <cit.> and <cit.>, non-asymptotic frameworks  for optimal joint detection and estimation are developed. Specifically, in <cit.>, a binary hypothesis testing problem is investigated in which one hypothesis is composite and consists of an unknown parameter to be estimated. In <cit.>, the theory in <cit.> is extended to a composite binary hypothesis testing problem in which both hypotheses correspond to composite models. 






 §.§ Decision Cost Functions





  §.§.§ Attack Detection Costs


The possibility of having multiple alternatives to the attack-free model renders the model detection problem as the following (T+1)- composite hypothesis testing problem.

    [                                      _̋0 :                    ∼ f(| X),  with X ∼π(X);                                      _̋i : ∼ f_i(| X),  with X ∼π(X) ,      ∈{1,…, T} ] .
 
where _̋0 is the hypothesis corresponding to the attack-free setting, and _̋i is the hypothesis corresponding to an attack launched at the coordinates in S_i ∈. For the convenience in notations, throughout the rest of the paper we denote the attack-free data model by f_0(· X), i.e., f_0(· X)=f(· X). To formalize relevant costs for the detection decisions, we define ∈{_̋0,…,_̋T } as the decision on the hypothesis testing problem in (<ref>), and ∈{_̋0,…,_̋T } as the true hypothesis. We adopt a general randomized test δ()≜ [ δ_0(),…,δ_T() ] for discerning the true hypothesis, where δ_i() ∈ [0,1] denotes the probability of deciding in favor of _̋i. Clearly,

    ∑_i=0^T δ_i() = 1  .

Hence, the likelihood of deciding in favor of _̋j while the true model is _̋i is given by 

    (=_̋j|=_̋i) = ∫_δ_j()f_i()   .

We define _ md as the aggregate probability of incorrectly identifying the true model under the presence of compromised coordinates, i.e.,

    _ md ()   ≜(≠|≠_̋0)
       = 1/(≠_̋0)∑_i=1^T(≠_̋i | = _̋i) (=_̋i) 
        = ∑_i=1^Tϵ_i /1 - ϵ_0·(≠_̋i | = _̋i) .





Furthermore, we define _ fa as the aggregate probability of erroneously  declaring that a set of coordinates are compromised, while operating in an attack-free scenario. We have

    _ fa ()    ≜(≠_̋0 | = _̋0) =∑_i=1^T (=_̋i|=_̋0) .





  §.§.§ Secure Estimation Costs



In this subsection, we define two estimation cost functions for capturing the  fidelity of the estimate X̂() we aim to form for X. For this purpose, we adopt a generic and non-negative cost function (X,U()) to quantify the discrepancy between the ground truth X and a generic estimator U(). Based on this, corresponding to each model _̋i and given data  we first define the average posterior cost function as

    _p,i(U() |) ≜_i[ (X,U()) |]  ,    ∀ i∈{0,…,T} ,

where the conditional expectation is with respect to X when the true model is _̋i. Besides this, due to having distinct data models under different attack models, we consider having possibly distinct estimators under different models. Driven by this, we consider the design for an estimate for X under each model. We denote the estimate of X under model _̋i by X̂_i(), and accordingly, we define

    () ≜ [X̂_0(),…, X̂_T()] .

Considering such distinct estimators, the estimation cost (X,X̂_i()) is relevant only if the decision is _̋i. Hence, for any generic estimator U_i() of X under model _̋i, we define the decision-specific average cost function as

    J_i(δ_i,U_i()) ≜_i[(X,U_i())| = _̋i]  ,    ∀ i ∈{0,…,T} ,

where the conditional expectation is with respect to X and . Accordingly, we define an aggregate average estimation cost according to

    J(δ,) ≜max_i ∈{0,…,T} J_i(δ_i,U_i()) ,

where we have defined ≜ [U_0(),…,U_T()]. Finally, corresponding to the attack-free scenario, in which the only possible data model is the assumed model f, corresponding to any generic estimator V() we define the average estimation according to

    J_0(V) = [(X,V())]  ,

where the expectation is with respect to X and  under model f. It is noteworthy that J_0 defined in (<ref>) is fundamentally different from J(,) defined in (<ref>), since the former is the estimation cost when there is no alternative to f (i.e., the attack-free scenario), while the latter is the estimation cost in an adversarial setting in which we have decided that the attacker has not compromised the data, which being a detection decision is never perfect and can be inaccurate with a non-zero probability. The role of J_0(V) in our analysis is furnishing a baseline for the estimation quality in order to assess the impact of the potential presence of an adversary on the estimation quality. 
















§ SECURE PARAMETER ESTIMATION


In this section, we formalize the secure estimation problem. The core premise underlying the notion of secure estimation presented is that there exists an inherent interplay between the quality of estimating X and the quality of isolating the true model governing the data. Specifically, perfect detection of an adversary's attack model is impossible. At the same time, the estimation quality strongly relies on  the successful isolation of the true data model. Lack of a perfect decision about the data model is expected to degrade the estimation quality compared to the attack-free scenario. To quantify such an interplay as well as the degradation in estimation quality with respect to the attack-free scenario, we provide the following definition. 

For a given estimator V in the attack-free scenario, and a secure estimation procedure specified by the detection and estimation rules (,) in the adversarial scenario, we define the estimation degradation factor (EDF) as

    q(,,V) ≜(,)/_0(V) .


Based on this definition, next we define the performance region, which encompasses all the pairs of decision qualities q(,,V) and _ md() over the space of all possible decision rules (,,V). 

We define the performance region as the region of all simultaneously achievable estimation quality q(,,V) and detection performance _ md(). 

By leveraging the characteristics of the performance region, next we define the notion of (q,β)-security, which is instrumental in defining the secure estimation problem of interest. For this purpose, note that EDF normalizes the estimation cost in the adversarial setting by that of the attack-free scenario. The two estimation cost functions involved in q(,,V) can be computed independently, and as a result, determining their attendant decision rules can be carried out independently.  For this purpose, we define V^* as the optimal decision rule under the attack-free setting, and J^*_0 as the corresponding estimation cost, i.e.,

    V^* ≜min_V J_0(V) ,         J^*_0 ≜min_V J_0(V) .


An estimation procedure specified by (,,V^*) for the adversarial scenario is said to be (q,β)-secure if the decision rules (,) yield the minimal EDF  among all the decision rules corresponding to which the average rate of missing the attacks does not exceed β∈ (0,1], i.e.,

    q ≜min_,q(,,V^*)  ,         _ md() ≤β .


The performance region, and its boundary that specifies the interplay between q and β are illustrated in Fig. <ref>. Based on these definitions, we aim to characterize:







  * The region of all simultaneously achievable values of q(,,V^*) and _ md(), which is illustrated by the dashed region in Fig. <ref>.

  * The (q,β)-secure decision rules (,,V^*) that solve (<ref>), and specify the boundary of the performance region, which is illustrated by a solid line as the boundary of the performance region in Fig. <ref>.

By noting that q(,,V^*)=J(,)/J^*_0, where J^*_0 is a constant, the performance region and the (q,β)-secure decision rules are found as the solutions to

    (β)    ≜min_,    J(,)
    s.t.   _ md() ≤β .

Solving (β) ensures that the likelihood of missing an attack is confined below β. However, it is insensitive to the rate of the false alarms that the decision rules can generate. In case the statistician wishes to also control the rate of false alarms, that is the rate of erroneously declaring an attack while there is no attack, we can further extend the notion of (q,β)-security as follows. 

An estimation procedure is (q,α,β)-secure if it is (q,β)-secure and the likelihood of false alarms does not exceed α∈(0,1]. 

The optimal decision rules that yield (q,α,β)-secure decisions can be found as the solution to

    (α,β)    = min_,    J(,)
    s.t.   _ md() ≤β
       _ fa () ≤α .


It can be easily verified that (β)  = (1,β).

 
The probabilities _ md() and _ fa() cannot be made arbitrarily small simultaneously. Specifically, from the Neyman-Pearson theory <cit.>, it can be readily verified that for any given α, there exists a value β^∗(α), which specifies the smallest feasible value for β.

We characterize the optimal solution to problems (α,β) and (β) in closed-forms in Section <ref>. Close scrutiny of the optimal decision rules indicates that the complexity of the rules grows exponentially with the dimension of X, i.e., p, and the number of coordinates that an adversary can compromise, i.e. K. To address the computational complexity, we address the scalability issue in Section <ref>. Specifically,  we provide alternative low-complexity decision rules and show that despite their simple structure, they satisfy asymptotic optimality guarantees. 




§ SECURE PARAMETER ESTIMATION: OPTIMAL DECISION RULES



In this section, we characterize an optimal solution to the more general problem (α,β), i.e., the estimators {X̂_i(): i ∈{ 0,…,T}} and the detectors {δ_i() : i ∈{ 0,…,T}}. We will also specify how the decision rules simplify for characterizing the solution to the problem (β)  = (1,β). In order to proceed, we start by providing the expansion of the decision error probability terms _ md() and _ fa() in terms of the data models and decision rules. By noting  (<ref>) and leveraging (<ref>), we have 

    _ md ()   = ∑_i=1^Tϵ_i/1 - ϵ_0∑_j≠ ij=0^T∫_δ_j()f_i()   .

Similarly, by noting (<ref>)



and based on (<ref>), we have 

    _ fa()    =∑_i=1^T∫_δ_i()f_0()  .

By using the expansions in (<ref>) and (<ref>), the problem of interest in (<ref>) can be equivalently cast as

    (α,β)    = min_(δ, U)   (, )
    s.t.   ∑_i=1^Tϵ_i/1 - ϵ_0∑_j≠ ij=0^T∫_δ_j()f_i() ≤β
       ∑_i=1^T∫_δ_i()f_0() ≤α .

The roles of the estimators { U_i(): i ∈{ 0,…,T}} appear only in the utility function (,). This allows for decoupling the optimization problem (α,β) into two sub-problems, as formalized in Theorem <ref>.

The optimal secure estimators of X under different models, i.e., =[X̂_0,…, X̂_T] is the solution to

    =  min_(,)  .

Furthermore, the solution of (α,β), and subsequently the design of the attack detectors, can be found  by equivalently solving 

    (α,β)    = min_δ   (δ,)
     s.t.   ∑_i=1^Tϵ_i/1 - ϵ_0∑_j≠ ij=0^T∫_δ_j()f_i() ≤β
       ∑_i=1^T∫_δ_i()f_0() ≤α .


This theorem establishes  a property for the optimal estimator  in (<ref>). By leveraging this property, and also taking into account the decoupled structure of the problem (α,β) in (<ref>), in the following theorem we provide optimal designs for the secure estimators. Interestingly, it is shown that the optimal estimator under each model can be specified by optimizing a relevant cost function defined exclusively for that model.
 
For the optimal secure estimators  we have


  * The minimizer of the estimation cost J_i(δ_i, U_i()), i.e., the estimation cost function under model _̋i, is given by

    U^*_i() ≜inf_U_i()_ p,i(U_i() |) .


  * The optimal estimator =[X̂_0,…, X̂_T], specified in (<ref>), is given by

    X̂_i() = U^*_i()  .


  * The cost function (, ) is given by 

    (, ) = max_i {∫_δ_i()_ p,i^∗()f_i() d/∫_δ_i()f_i()d}  ,

where we have defined

    ^*_ p,i() ≜inf_U_i()_ p,i(U_i() |)  .




See Appendix <ref>.

For illustration purposes, in the next corollary, we provide the closed-forms of these decision rules when the distributions {f_i(· X): i∈{0,…, T}}  are all Gaussian. 

When the data models are Gaussian such that

    f_i(· X) ∼ N(θ_i, X)  ,      θ_i∈ℝ ,

where the mean values are distinct, and 

    X  ∼ X^-1(ζ,ϕ) ,

where X^-1(ζ,ϕ) denotes the inverse chi-squared distribution with parameters ζ and ϕ, s.t., ζ+n >4,
for the optimal secure estimators , we have:


  * The minimizer of the estimation cost J(δ_i, U_i()), i.e., the estimation cost function under model _̋i, is given by

    U_i^∗()  = ζϕ + ∑_r=1^n Y_r- θ_i^2_2 /ζ + n -2 .


  * The optimal estimator =[X̂_0,…, X̂_T], specified in (<ref>), is given by

    X̂_i() = U^*_i()  .


  * The cost function (, ) is given by 

    (, ) = max_i {∫_δ_i()_ p,i^∗()f_i() d/∫_δ_i()f_i()d}  ,

where we have

    ^*_ p,i()  = 2(ζϕ + ∑_r=1^nY_r-θ_1^2)^2/(ζ_i + n - 2)^2(ζ+n-4) .




Next, given the optimal estimators , we characterize the optimal detection rules in the next theorem. The main observation is that even though we started by considering general randomized decision rules, these rules in their optimal forms reduce to deterministic ones. Furthermore, the decisions rules depend on the estimation costs that are computed based on the optimal estimation costs. These estimation costs make the decisions coupled. In order to proceed, we first show that problem (α,β) in (<ref>) can be solved by leveraging the result of the following theorem, which specifies an auxiliary convex problem in a variational form.
 
For any arbitrary u∈ℝ_+, we have (α,β)≤ u if and only if (α,β,u)≤ 0, where we have defined

    (α,β,u)    = min_δ   γ
    
     s.t.   ∫_δ_i() f_i() [^*_ p,i() - u] ≤γ,   ∀i∈{ 0,…,T}
       ∑_i=1^T ϵ_i/1-ϵ_0∑_j≠ ij=0^T  ∫_δ_j()f_i()  ≤β+γ
       ∑_i=1^T∫_δ_i()f_0() ≤α+γ .

Furthermore, (α,β,u) is convex, and (α,β,u)=0 has a unique solution in u, which we denote by u^*. 


See Appendix <ref>.

The point  u^* has a pivotal role in specifying the optimal detection decision rules. We define the constants {ℓ_i: i∈{ 0,…,T+2}} as the dual variables in the Lagrange function associated with the convex problem (α,β,u^*). Given u^* and {ℓ_i: i∈{ 0,…,T+2}}, the optimal detection rules can be characterized in closed-forms, as specified in the following theorem.

The optimal decision rules for isolating the compromised coordinates are given by

    δ_i() =     1,     if    i = i^∗
        0,    if    i ≠ i^∗ ,

where we have defined

    i^∗≜_i ∈{ 0,…,T }  A_i   .

Constants {A_0,… ,A_T} are specified by the data models, u^*, and its associated Langrangian multipliers {ℓ_i: i∈{ 0,…,T+2}}. Specifically, we have

    A_0    ≜ℓ_0 f_0() [^*_ p,0() - u^*] + ℓ_T+1∑_i=1^Tϵ_i/1-ϵ_0 f_i() ,

and for i∈{ 1,…,T } we have

    A_i    ≜ℓ_i f_i() [^*_ p,i() - u^*] + ℓ_T+1∑_j=1, j≠ i^Tϵ_j/1-ϵ_0 f_j() +  ℓ_T+2f_0() .



See Appendix <ref>.

In the next corollary, we provide the closed-forms of these decision rules when the distributions {f_i(· X): i∈{0,…, T}}  are all Gaussian. 

When the data models {f_i(· X): i∈{0,…, T}} have the following Gaussian distributions

    f_i(· X) ∼ N(θ_i, X)  ,      θ_i∈ℝ ,

where the mean values are distinct,  and

    X  ∼ X^-1(ζ,ϕ) ,

the optimal decision rules for isolating the compromised coordinates are given by

    δ_i() =     1,     if    i = i^∗
        0,    if    i ≠ i^∗ ,

where we have defined

    i^∗≜_i ∈{ 0,…,T }  A_i   .

Constants {A_0,… ,A_T} are specified by the data models, u^*, and its associated Langrangian multipliers {ℓ_i: i∈{ 0,…,T+2}}. Specifically, we have

    A_0    ≜ℓ_0 f_0() (^*_ p,0() - u^*) + ℓ_T+1∑_i=1^Tϵ_i/1-ϵ_0 f_i() ,

and for i∈{ 1,…,T } we have

    A_i    ≜ℓ_i f_i() (^*_ p,i() - u^*) + ℓ_T+1∑_j≠ ij=1^Tϵ_j/1-ϵ_0 f_j() +  ℓ_T+2f_0() .

where ^*_ p,i() is evaluated using (<ref>) and we have

    f_i() = (ζϕ )^ζ/2/π^n/2Γ(ζ/2)·Γ(ζ + n)/2/( ζϕ + ∑_r=1^n Y_r - θ_i^2 )^ζ + n/2 .


By setting T= 1, n = 1,  θ_0=0, θ_1 = 2, and selecting ζ = 4, ϕ = 1, Fig. <ref> depicts the performance region and the associated (q,β)-security curve, which shows the tradeoff between the quality of the detection and the degradation in the estimation quality. It is noteworthy that this tradeoff is inherently due to the formulation of the secure estimation problem. Essentially, problem (α,β) as specified in (<ref>), is designed to trade the quality of detection in favor of improving the estimation cost.




Based on all the decision rules specified in the section, and the detailed steps of specifying the parameters involved in characterizing the decision rules, we provide Algorithm 1, which summarizes all the steps for solving (α,β) for any feasible pair of α and β.












 






§ SCALABLE SECURE PARAMETER ESTIMATION 

As the data dimension m grows, the number of possible data models T grows exponentially. This, subsequently, leads to an exponential growth in the complexity of forming the decision rules, e.g., the number of Lagrangian multipliers needed for characterizing the detection rules scales linearly in T. This can render Algorithm 1 computationally prohibitive. In order to circumvent the computational complexity, in this section, we develop a scalable approach to secure estimation, which exhibits optimality properties too. The core idea is to break down the problem into m coordinate-level problems, treat them individually, and then aggregate the individual decisions. Specifically, the decisions involve a binary decision on whether the data is compromised. If the data is deemed to be compromised, then each coordinate is tested individually, and an estimate of X is formed based on the data of that coordinate. Individual coordinate-level estimates are then tested for reliability, and combined to form an aggregate estimate for X. Since we need to perform only m single-coordinate binary detection decisions followed by forming a coordinate-level estimation routine, the computational complexity scales only linearly in the number of coordinates m, as opposed to exponentially for forming the optimal decision rules.



 §.§ Binary Attack Detection



In the first stage, we perform a binary test to detect whether data is compromised at all. This is carried out by solving a  binary composite hypothesis testing problem given by

    [                    _0 : ∼ f (| X),  with X ∼(X);                    _1 :  ∼f̂(| X),  with X ∼(X);                         ] ,
 
where _0 is the hypothesis corresponding to the attack-free setting, and _1 signifies to the presence of an attack. Probability distribution f̂ is a mixed distribution given by

    f̂() ≜1/1-ϵ_0∑_i=1^Tϵ_i f_i() .

Similarly to the optimal approach of Section <ref>, to design the decision rules we define the randomized test δ̂≜ [ δ̂_0(),δ̂_1()  ], in which δ̂_i() ∈[0,1] is the probability of deciding in favor of ̋̂_i, and we have δ_0() + δ_1() = 1.  Furthermore, define _ d∈{_0, _1 } and _ d∈{_0, _1} as the decision and the true hypotheses. Hence, the false alarm rate is given by 

    (_ d = _1 |_ d = _0) = ∫δ̂_1() f()   ,

and the miss-detection rate is given by

    (_ d = _0 |_ d = _1) = ∫δ̂_0() f̂()   .

Since our aim is to maximize the true detection rate (or minimize (<ref>)) in the first step, we design the decision rule δ̂ using the Neyman Pearson approach. Following the Neyman-Pearson approach, as noted in Remark <ref>, we have the following decision rules for the binary detection decision  <cit.>.

The optimal attack detection rule that minimized the rate of missing the attacks subject to a constraint on the false alarms is given by 

    δ̂_0()     = 1_{f̂()/f() < γ} + (1-ϱ)·1_{f̂()/f() = γ} , 
    δ̂_1()     = 1_{f̂()/f() > γ} + ϱ·1_{f̂()/f() = γ}  .

where the threshold γ and the probability term ϱ are chosen such that the false alarm constraint is satisfied with equality. 













 §.§ Isolating Compromised Coordinates



The outcome of the binary detection rule in Section <ref> is ruling whether there exists an attack in the data. If an attack is deemed to exists, in the second step, we carry out an isolation decision, the role of which is identifying the compromised coordinates. In this subsection, we start by analyzing the optimal isolation rules when an attack is deemed to exist in Section <ref>, and characterize the performance of the optimal decision rules. This analysis will serve as a baseline for comparing the performance of any alternative low-complexity isolation rule. Next, we provide an isolation rule in Section <ref> that has low-complexity and can achieve the same performance asymptotically as that of the optimal decision rule.



  §.§.§ Optimal Isolation Rule



If the attack detection rules specified by (<ref>)-(<ref>) determine that the data in one or more coordinates are compromised, in the next step we aim to identify the compromised coordinates. Isolating the  set of compromised coordinates is equivalent to solving the following T-hypothesis testing problem.

    [                                         _̋i : ∼ f_i(| X),  with X ∼π(X) ,      i∈{1,…, T} . ]
 
Define _ is∈{_̋1,…,_̋T} and _ is∈{_̋1,…,_̋T} as the decision formed and the true hypothesis, respectively.  We define the randomized test δ̂_1 () ≜ [ δ̂_11(), …, δ̂_1T()  ], in which δ̂_1i() ∈[0,1] is the probability of deciding in favor of ̋̂_i given that we have already decided that part of the data is compromised, i.e., ̋̂_1 is the true hypothesis in (<ref>). Accordingly, we define _ is(δ̂_1) as the probability of making an erroneous  decision on the problem in (<ref>), given that the decision in the detection step was _1. Therefore, _ is(δ̂_1) is given by 

    _ is (δ̂_1)    ≜(_ is≠_ is|_ d = ̋̂_1)
       = ∑_i = 1^T ∑_j = 1, j≠ i^T (_ is = _̋j |_ is = _̋i, _ d = ̋̂_1)·(_ is = _̋i_ d = ̋̂_1)  .

We also denote the error exponent of _ is (δ̂_1) as the number of the samples n grows according to

    ψ(δ̂_1) ≜ - lim_n→∞log_ is (δ̂_1)/n .

In the next theorem, we characterize the optimal decision rule (δ̂_1) and the associated error exponent. For this purpose, we denote the Chernoff information between two probability measures with probability density functions g and h by

    C(g,h)≜ -logmin_α∈ (0,1)∫ g^α (x) h^1-α(x)  x̣ .


The decision rule δ̂_1 that minimizes _ is (δ̂_1) is given by

    δ̂_1i() =     1,     if    i = i^∗
        0,    if    i ≠ i^∗ ,

where

    i^∗ = _j ∈{1,…,T} f_j()(_ is= _̋j |_ d = ̋̂_1) .
 






See  Appendix <ref>.

Note that the optimal decision rules will have the same computational complexity as the optimal rules characterized in Theorem <ref>. 


 
 

  §.§.§ Asymptotically Optimal Isolation Rule

 
 
The optimal decision rule in Theorem <ref> has similar drawbacks in terms of computational complexity in high dimensions as the optimal decision rules discussed in Section <ref>. In this subsection, we provide an alternative isolation rule for discerning the compromised coordinates at a lower computational complexity, and show that this rule is optimal in the asymptote of large number of samples, i.e.,  n→∞.  We denote the alternative low-complexity decision rule by  δ̅_1 and start by providing lower and upper bounds on _ is(δ̅_1), and show that these bounds exhibit the same error exponents. To specify a lower bound we first leverage the fact that _ is(δ̂_1) ≤_ is (δ̅_1), which is true due to the optimality of _ is(δ̂_1), and define _ is(X,(δ̅_1)) as the value of _ is (δ̅_1) when the unknown parameter X is 
fully known (e.g., provided by a genie).  Clearly, when X is fully known, the error probability decreases, as part of the model uncertainty due to the unknown parameter is removed, and the decision problem reduces to a purely detection problem. To specify the upper bound on _ is (δ̅_1), we  define  _ is (X_c,δ̅_1) as the value of _ is (δ̅_1) when X is
assumed to take an arbitrary value X_c∈ℝ^p.  Note that replacing X by arbitrarily chosen X_c induces sub-optimality compared to the optimal case in which optimal estimation is performed. The following lemma specifies  the bounds on _ is (δ̅_1).


There exists some X_c  ∈ℝ^p, such that

    _ is^l≜_ is(X,(δ̂_1))  ≤_ is(δ̂_1) ≤_ is(δ̅_1)  ≤_ is^u≜_ is(X_c,(δ̅_1))  .


This lemma  is instrumental in establishing the error exponent of the alternative low-complexity decision rule that we will provide. To proceed, corresponding to each coordinate l∈{1,…,m} we define the likelihood ratio term 

    _l ()≜∏_r=1^n g_l^1(Y_r(l))/g_l^0(Y_r(l))  ,

where g_l^0 and g_l^1 denote the marginal pdfs of the data at coordinate l∈{1,…,m} under the attack-free setting and when the coordinate is compromised, respectively. In the next theorem, we show that a decision rule based on calculating these likelihood ratio terms suffices to reach a decision that achieves the same error exponent as the optimal decision rule specified in (<ref>).




The isolation rule

    δ̅_1i() =     1,     if    i = i^∗
        0,    if    i ≠ i^∗  ,

in which we have defined

    i^∗ = _i ∈{1,… T}∏_v∈ S_i_v() ,

has the following error exponent

    ψ(δ̅_1) = min_i≠ j ∈{1,…,T} C(f_i,f_j)  .

which is equal to ψ(δ̂_1), i.e., the error exponent of the optimal rule.


See Appendix <ref>.

Therefore, the error probabilities corresponding to the optimal decision rule in Theorem <ref> and the low-complexity alternative rule Theorem <ref> decay at the same rate with the increasing number of samples n, rendering the decision rule in Theorem <ref> asymptotically optimal. Clearly, the decision rule based on the marginal likelihood ratios significantly reduces the computational complexity for isolating the attacked coordinates. This reduction in computational complexity is viable at the expense of accuracy of the isolation when we have a small number of samples. In the next subsection, we discuss how the attack detection and coordinate isolation decisions characterized so far are leveraged for forming an estimate of X. 




 §.§ Coordinate-based Secure Estimation


The decision rules in Section <ref> and Section <ref> have already produced decisions about whether each individual coordinate is compromised. In the third stage of the decisions, we form one estimate of X based on all the n samples available at each coordinate. This leads to forming m distinct estimates for X, one corresponding to each coordinate. Clearly, not all the estimates can be deemed reliable, especially when some of the coordinates are compromised. For this purpose, after forming the m estimates we perform a reliability test, the purpose of which is discarding the unreliable estimates and retaining and aggregating the reliable ones. We provide relevant estimation cost functions in Section <ref>, and characterize the reliability test in Section <ref>.



  §.§.§ Cost Functions




Based on the sequence of the data points at each coordinate, we form an estimate of X corresponding to each coordinate, resulting in m distinct estimates for X. Clearly, different coordinates produce estimates of X with potentially different qualities. Motivated by the fact that the decisions formed in the previous steps are not perfect and a coordinate might yield an unreliable estimate for X, we consider performing a reliability test on the decision produced at each coordinate. For this purpose, at each coordinate l and based on the data available at coordinate l, which we denote by _l, we perform a binary test to decide whether the estimate based on the data from coordinate l is reliable or unreliable, denoted by ^̋ r_l and ^̋ u_l, respectively. 

We define ^ r_l ∈{^̋ r_l, ^̋ u_l} as the decision formed about the reliability of the estimate, and let the randomized test δ̅_l(_l)= [δ̅_l^ r(_l), δ̅^ u_l(_l) ] be  the decision rule to decide upon the reliability of the estimate from coordinate l, where δ̅_l^ r(_l) is the probability of deciding in favor of ^̋ r_l and  δ̅_l^ u(_l) is the probability of deciding in favor of ^̋ u_l. Furthermore, we define δ_l () ≜  [δ^0_l(_l), δ^1_l(_l)], where δ^1_l(_l) denotes the  probability that coordinate l is being compromised based on data  and the decision rules in Section <ref> and Section <ref>, and subsequently, δ^0_l(_l)=1-δ^1_l(_l). Hence, the likelihood of forming a reliable estimate at coordinate l given that we have decided that coordinate l is not compromised is given by

    _0(^ r_l = ^̋ r_l)    = ∫δ^0_l(_l) δ̅_l^ r(_l) g_l^0(_l) _l  .

Similarly,  the likelihood of forming a reliable estimate at coordinate l given that we have decided that coordinate l is compromised is given by

    _1(^ r_l = ^̋ r_l)     = ∫δ^1_l(_l) δ̅_l^ r(_l) g_l^1(_l) _l  .

Furthermore, consider U^0_l and U^1_l as the estimates of X at coordinate l when we have decided that the coordinate is attack-free and  compromised, respectively. Hence, the cost associated with U_l^j when the decision of the reliability test is ^̋ r_l is defined as 

    _l^j(δ_l^j,δ̅_l^ r,U_l^j)    ≜_j[(X,U_l^j) |_ l^ r = _̋l^ r] 
       = ∫∫δ^j_l(_l) δ̅^ r_l(_l)(U_l^j,X) g_l^j(_l | X) (X) X̣_l/∫∫δ^j_l(_l) δ̅^ r_l(_l) g_l^j(_l | X) (X) X̣_l 
       = ∫δ_l^j(_l) δ̅_l^ r(_l)^j_l(U_l^j|_l) g_l^j(_l) _l/∫δ^l_j(_l) δ̅^ r_l(_l) g_l^j(_l ) _l  ,

where ^j_l(U_l^j|_l) is the posterior estimation cost at coordinate l.  We define the optimal average estimation cost as

    ^j_l(_l) ≜min_U_l^j_l^j(U_l^j|_l) ,

and the optimal coordinate-level estimators as

    X̂_l^j(_l) ≜min_U_l^j_l^j(U_l^j|_l) .





  §.§.§ Reliability Decision Rules



For the probabilities of forming a reliable estimate at coordinate l defined in (<ref>) and (<ref>) we have

    _j(_l^ r   = _̋l^ r)    = ∫δ_l^j(_l) δ̅_l^ r(_l) g_l^j(_l) _l ,
       ≤∫δ_l^j(_l)  g_l^j(_l) _l
        =  ρ^j_l  ,

where (1-ρ_l^0) and (1-ρ_l^0) are the Type-I and Type-II probabilities of mis-classifying the status of coordinate l. Therefore, the probability of forming a reliable estimate, when coordinate l is deemed to be compromised or attack-free is upper bounded by ρ^1_l and ρ_l^0, respectively. This implies that only a fraction of the decisions on the true model of data at coordinate l will provide reliable estimates.  We wish to have decision rules that control the fraction of the reliable estimates to be beyond a pre-specified  level. Obviously, these desired levels should also be within the feasible range. For this purpose, we select ν_l^j∈[0,ρ_l^j],  and impose a lower bound on the fraction of the reliable estimates that the reliability test retains according to 

    _j(_l^ r= _̋l^ r)≥ν_l^j  .
















Based on the cost functions and decision constraints, the decision rules δ̅_l and the estimators U_l^j are determined by solving m problems in parallel, where the problem to be solved corresponding to coordinate l is given by

    S(ν^j_l) ≜min_δ̅_l, U_l^j   _l^j(δ_l^j,δ̅_l^ r,U_l^j)
    s.t.    _j(_l^ r = _̋l^ r) ≥ν_l^j
       .

Similarly to the discussion in Section <ref>, since the effect of the estimators U_l^j appears only in the cost function _l^j(δ_l^j,δ̅_l^ r,U_l^j), the optimization problem in (<ref>) can be decoupled into two subproblems as formalized in the following theorem.

For given ν_l^0 and ν_l^1, the optimal decision rules for the problems S(ν_l^0) and S(ν_l^1) are given by

    ^0_l(_l)    ≷^_̋l^ u__̋l^ rγ_l^0   ,
    ^1_l(_l)    ≷^_̋l^ u__̋l^ rγ_l^1   ,

where γ_l^1 and γ_l^0 are selected such that the constraints in (<ref>) are satisfied with equality. The optimal estimate that solve S(ν_l^0) and S(ν_l^1) are the estimators X̂_l^0(_l) and X̂_l^1(_l), respectively, defined in (<ref>).




See Appendix <ref>.


For given data , performing the reliability test in Theorem <ref> retains the estimates deemed reliable. These estimates, subsequently, can be aggregated to form a single estimate for X. The estimation approach, which consists of global binary attack detection, followed by coordinate-level isolation of compromised coordinates and local estimates, and completed by aggregating the reliable coordinate-level estimates, renders a scalable approach to the secure estimation of interest. This approach is significantly less complex as compared to the optimal estimation rules characterized in Theorem <ref>. Finally, we remark that aggregating the local estimates is a well-investigated problem, to which there exists a broad range of solutions with varying degrees of optimality under various settings. Representative approaches to  such aggregation include the studies in <cit.>. A summary of the steps involved in the scalable approach is provided in Algorithm 2.




 





§ CASE STUDIES: SECURE ESTIMATION IN SENSOR NETWORKS

We use the example of a sensor network consisting of two sensors and a fusion center (FC) to evaluate the estimation frameworks presented in this paper. Each sensor is collecting a stream of data. Sensor i∈{1,2} collects n measurements, denoted by _i=[Y^i_1,…,Y^i_n], where each sample Y^i_j∈ℝ in an attack-free environment is related to X according to 

    Y_j^i = h^iX + N_j^i ,

where h^i models the channel connecting sensor i to the FC and  N_j^i accounts for the additive channel noise. Different noise terms are assumed to be independent and identically distributed (i.i.d.) generated according to a known distribution. We will consider two adversarial scenarios that impact the data model in (<ref>) and evaluate the optimal performance, as well as the application of the asymptotically optimal scalable algorithm when the number of the samples n tends to infinity.



 §.§ Case 1: One Sensor Vulnerable to Causative Attacks

We start by considering an adversarial setting in which the data model of the measurements from only one sensor (sensor 1) are vulnerable to a causative attack, while the other sensor (sensor 2) remains attack-free. Under such a setting, we have only one attack scenario, i.e., T=1 and S_1={1}. Accordingly, we have ϵ_0+ϵ_1=1. Under the attack-free scenario, we assume that the noise terms N_j^i are i.i.d. and distributed according to (0,σ^2_n), i.e.,

    Y_j^i | X ∼(h^i X, σ_n^2) .

When data from sensor 1 is compromised, the actual conditional distribution of  Y_j^1|X is distinct from the above distribution assumed by the statistician. The inference objective under such a setting, in principle, becomes similar to the adversarial setting of <cit.>, which focuses on data injection. Hence, in order to be able to compare the performance of the optimal framework with that of <cit.>, we assume that the conditional distribution of  Y_j^1|X when sensor 1 is under a causative attack is (h^i X,σ^2_n) ∗ Unif[a,b], where a,b∈ℝ are fixed constants and ∗ denotes convolution. Therefore, the composite hypothesis test for estimating X and discerning the model in (<ref>) simplifies to the following binary test with the prior probabilities ϵ_0 and ϵ_1, in which we have defined =[_1,_2]. 

    [                        _̋0 : ∼ f_0(| X),  with X ∼(0,σ^2);                        _̋1 : ∼ f_1(| X),  with X ∼(0,σ^2);                              ] ,

Figure <ref> depicts the variations of the estimation quality, captured by q, versus the tolerable miss-detection rate β, where it is observed that the estimation quality improves monotonically as β increases, and it reaches its maximum quality when β=1. 
This observation is in line with what is expected analytically from the formulation of the secure parameter estimation problems in (<ref>) and (<ref>). 


A similar setting is studied in <cit.>, where the attack is induced additively into the data of sensor 1 and can be any real number. This setting can be studied in the context of causative attacks where the attacker's mode of compromising the data is adding a disturbance which has uniform distribution. Figure <ref> also compares the estimation quality of the methodology developed in this paper, with that obtained by applying the methodology of <cit.>, which characterizes a single point in the (q,β) plane. Specifically, in <cit.>, an estimator is designed to obtain the most robust estimate by exploring the dependence of the estimation quality on the false alarm probability, using which an optimal false alarm probability α^∗ is obtained, which in turn, fixes the miss-detection error probability, and does not provide the flexibility to change the miss-detection rate β. 





The results presented in Fig. <ref> correspond to σ = 3, σ_n = 1, h^1 = 1, h^2 = 4, a = -40, b = 40. The upper bound on _ fa is set to α^∗ = 0.1, where α^∗ is obtained using the methodology in <cit.>. 







 §.§ Case 2: Both Sensors Vulnerable to Causative Attacks

We again consider the same model for X, and in this setting, we assume that data from both the sensors are vulnerable to being compromised. We assume that the attacker can compromise the data of at most one sensor at any instant. Under such a setting, we have T=2, S_1={1}, and S_2={2}. Therefore, under the adversarial setting, the sensor measurements follow the following composite hypothesis model

    [                    _̋0 : ∼ f_0(| X),  with X ∼(X);                    _̋1 : ∼ f_1(| X),  with X ∼(X);                    _̋2 : ∼ f_2(| X),  with X ∼(X);                          ] ,

where _̋0 corresponds to the attack-free setting, and hypothesis _̋i corresponds to the data of sensor i being compromised. Motivating by the fact that the sensor with the higher gain h^i is expected to generate a better estimate, we explore a scenario in which the sensor with the higher gain is more likely to be attacked. Hence, we select the parameters h^1 = 1, and h^2 = 2, and set the probabilities (ϵ_0,ϵ_1,ϵ_2)=(0.2,0.2,0.6). We set the distribution of X to Unif[-2,2]. We assume that Y_j^i, for i∈{1,2}, given X, is distributed according to N(h^i X,1) in the attack-free setting. When sensor i is compromised, we assume that Y_j^i, for i∈{1,2}, given X, follows the distribution N(h^iX,5). 



Figure <ref> depicts the performance region defined in Fig. <ref>, which specifies the variations of q versus β for three different values of α. The region spanned by the plots between q and β for different values of α is the feasible region of operation for the secure estimation methodology developed in this paper. This provides the FC with the flexibility to adjust the emphasis on each of the estimation or detection decisions. As expected, the estimation quality improves monotonically as α and β increase.



 §.§ Case 3: Evaluation of Scalable Secure Parameter Estimation Framework

Before comparing the estimation performance from the scalable approach developed in Section <ref> and the optimal approach, we illustrate the asymptotic optimality of the decision rule proposed in Theorem <ref>. Consider a 2 sensor network, where the measurements of at least one sensor are compromised at any instant. The measurements of the sensors follow a similar model as described in  (<ref>). 
Assuming that the parameter X follows the distribution (0,σ^2), Y_j^i given X is distributed according to (h^i X,σ_1^2) under the attack-free scenario and according to (h^iX,σ_2^2) when sensor i is compromised, we illustrate the variations of -log(_ is (δ̅_1)) versus the number of observations at each sensor in  Fig. <ref>. For the results in Fig. <ref>, we set h^1 =1, h^2 = 4, σ^2 = 4, σ_1^2 = 2 and σ_2^2 = 6 . The error probabilities corresponding to both decision rules decay exponentially at the same rate with the increase in number of observations at each sensor. 





In order to compare the performance of the scalable secure estimation framework with that of the optimal framework, we choose ν_l^0 and ν_l^1, for l ∈{1,2}, according to the steps described in Algorithm 2.


We aggregate the reliable local estimates to form an optimal linear estimate at the FC using the fusion strategy for sensor networks with star topology in <cit.>. 


























The decision on the reliability of the estimate is formed using the decision rules in Theorem <ref>, following which the local linear estimates from the sensors deemed to provide reliable estimates are aggregated at the FC. To compare with the estimation degradation factor q for an optimal framework, we evaluate the estimation degradation factor for the scalable secure estimation framework and denote it by q̂. The estimation performances for the scalable framework and the optimal decision rules are compared in the following table:
																
For the results presented in Table <ref>, we have set h^1 = h^2 =1. We assume that the parameter X is distributed according to (0,3), N_j^i is distributed according to (0,1) under the attack-free scenario, and according to (0,1)∗ Unif[-10,10] when data from sensor i is compromised. As expected, the optimal decision rules result in superior estimation quality as compared to that obtained from the scalable framework, i.e., q̂ > q. 



§ CONCLUSION

We have formalized and analyzed the problem of secure parameter estimation problem under the potential presence of causative attacks on the estimation algorithm. Under causative attacks, the information of the estimation algorithm about the  statistical model of the sampled data is compromised. This leads the estimation algorithm exhibit degraded performance compared to the attack-free setting. We have provided closed-form optimal decision rules that ensure the best estimation quality (minimum estimation cost) while controlling the error in detecting the attacks and isolating the true model of the data. We have shown that the design of optimal estimators is intertwined with the detection rules for deciding upon the true model of the data. Based on this, we have designed the optimal decision rules, which combine both estimation performance and detection power. Based on this vision, the decision-maker can place any desired emphasis on the estimation and detection routines involved. to study the trade-off between the two. We have also provided case studies by applying the theory developed to sensors networks, where sensors face security vulnerabilities. Finally, to circumvent the computational complexity associated with growing the data dimension or attack complexity, we have provided a low-complexity secure estimation algorithm that is optimal in the asymptote of high data dimension. 





§ PROOF OF THEOREM <REF>



From (<ref>) we have

    _i(δ_i,U_i)    = [ (X,U_i())| = _̋i] 
       = ∫_∫_δ_i() (X,U_i())f_i(|X)(X)X̣/∫_δ_i()f_i() .

Using the definition of _ p,i(U_i()|) from (<ref>), a lower bound on _i(δ_i,U_i()) is given by

    _i(δ_i,U_i)    =∫_δ_i()_ p,i(U_i()|) f_i() /∫_δ_i()f_i()
       ≥∫_δ_i()inf_ U_i()_ p,i(U_i()|) f_i() /∫_δ_i()f_i() ,

which implies that

    _i(δ_i,U_i)   ≥∫_δ_i()_ p,i^∗()f_i() /∫_δ_i()f_i() .

Based on the definition of X̂_i() provided in (<ref>), this lower bound is clearly achieved when the estimator U_i() is chosen to be

    X̂_i() = inf_U_i()_ p,i(U_i() |) ,

which proves that the estimator characterized in (<ref>) is an optimal estimator that minimizes the cost _i(δ_i,U_i). The corresponding minimum average estimation cost is

    _i(δ_i,X̂_i) = ∫_δ_i()_ p,i^∗()f_i() /∫_δ_i()f_i()  .

Next, we prove that 

    max_i min_{_i(δ_i,U_i)}≡min_max_i {_i(δ_i,U_i)}  .
 
Recall from (<ref>), the overall estimation cost (δ,) is defined as

    (δ,) = max_i {_i(δ_i,U_i) }  .

Define (Ω,δ,) as a convex function of _i(δ_i,U_i), i ∈{0,…,T }, given by

    (Ω,δ,) ≜∑_i=0^TΩ_i _i(δ_i,U_i)  ,

where Ω = [Ω_0,…,Ω_T], and Ω_i satisfy 

    ∑_i=0^TΩ_i = 1  ,  and Ω_i ∈ [0,1] .

We can represent (δ,) as a function of (Ω,δ,) in the following form

    (δ,)    = max_Ω(Ω,δ,) .

Let Ω^∗ = {Ω_j^∗:j=0,…,T } be defined as

    Ω^∗   ≜max_Ω(Ω,δ,) ,

where Ω_j^∗ = 1 if 

    j = max_i {_i(δ_i,U_i) }  .

From (<ref>) and (<ref>), we observe that

    max_Ωmin_(Ω,δ,)    = max_Ω(Ω,δ,)
       ≥min_max_Ω(Ω,δ,) .

Also, at the same time, we have 

    max_Ω(Ω,δ,)    ≥max_Ωmin_(Ω,δ,)  ,

which implies that

    min_max_Ω(Ω,δ,)    ≥max_Ωmin_(Ω,δ,)  .

From (<ref>) and (<ref>), it is easily concluded that

    max_Ωmin_(Ω,δ,) = min_max_Ω(Ω,δ,)  ,

which completes the proof for (<ref>). Using the results in (<ref>) and (<ref>), the cost function (δ, ) is given by

    (δ, )    = min_max_i{_i(δ_i,U_i) }
       = max_i min_{_i(δ_i,U_i) }
       = max_i {_i(δ_i,X̂_i) }
       = max_i {∫_δ_i()_ p,i^∗()f_i() /∫_δ_i()f_i()}  .




§ PROOF OF THEOREM <REF>


The function _i(δ_i,U_i) is a quasi-convex function. Since the weighted maximum function preserves the quasi-convexity, it is concluded that _i(δ_i,X̂_i) is a quasi-convex function from its definition in (<ref>). Therefore, we can find the solution by solving an equivalent feasibility problem given below <cit.>. Specifically, for u ∈ℝ_+, it is easily observed that 

    (δ, ) ≤ u    ≡∫_δ_i() f_i() (^∗_ p,i() - u)≤ 0, for i∈{0,…,T} .

Hence, the feasibility problem equivalent to (<ref>) is given  by

    (α,β)    = min_δ    u
    
    s.t.   ∫_δ_i() f_i() (^∗_ p,i() - u)≤ 0,   ∀ i∈{ 0,…,T}
       ∑_j=1^T∑_i=0,i≠ j^Tϵ_j/1-ϵ_0∫_δ_i()f_j() ≤β
       ∑_i=1^T∫_δ_i()f_0()≤α .

The above problem is feasible if (α,β) ≤ u, where (α,β) represents the lowest possible value of u for which the problem is feasible and all the constraints are satisfied. Given an interval [u_0,u_1] containing (α,β), the optimal detection rule δ and the optimal estimation cost (α,β) can be determined by a bi-section search between u_0 and u_1 iteratively, solving the feasibility problem in each iteration. 
To solve the feasibility problem, we define an auxiliary convex optimization problem 

    (α,β,u)    = min_δ   η
    
    s.t.   ∫_δ_i() f_i() (_ p,i^∗() - u)≤η,   ∀ i∈{ 0,…,T}
       ∑_j=1^T∑_i=0,i≠ j^Tϵ_j/1-ϵ_0∫_δ_i()f_j() ≤β + η
       ∑_i=1^T∫_δ_i()f_0()≤α + η .

Algorithm 1 summarises the steps for determining (α,β).



§ PROOF OF THEOREM <REF>



To solve the problem in (<ref>), a Lagrangian function is constructed according to

    (δ,η,ℓ)    ≜(1-∑_i=0^T+2ℓ_i)η
          + ∑_i=0^Tℓ_i ∫_δ_i()f_i()(^∗_ p,i() - u)
          + ℓ_T+1∑_j=1^T∑_i=0,i≠ j^Tϵ_j/1-ϵ_0∫_δ_i()f_j() - ℓ_T+1β
          + ℓ_T+2∑_i=1^T∫_δ_i()f_0()- ℓ_T+2α ,

where ℓ≜[ ℓ_0,…,ℓ_T+2] are the non-negative Lagrangian multipliers selected to satisfy the constraints in (<ref>), such that 

    ∑_i=0^T+2ℓ_i = 1 .

The Lagrangian dual function is given by

    d(ℓ)    ≜min_δ,η(δ,η,ℓ)
       = min_δ(∑_i=0^T∫_δ_i() A_i ) - ℓ_T+1β - ℓ_T+2α ,

where 

    A_0    ≜ℓ_0 f_0() [^∗_ p,0() - u] + ℓ_T+1∑_i=1^Tϵ_i/1-ϵ_0f_i() ,

and for i ∈{ 1,…,T },

    A_i    ≜ℓ_i f_i() [^∗_ p,i() - u] + ℓ_T+1∑_j=1,j≠ i^Tϵ_j/1-ϵ_0 f_j() +  ℓ_T+2f_0() .

Therefore, the optimum detection rules that minimize d(ℓ) are given by:

    δ_i() =     1,   if    i = i^∗
        0,   if    i ≠ i^∗,

where

    i^∗ = _i∈{ 0,…,T } A_i   .

Hence, the proof is concluded.




§ PROOF OF THEOREM <REF> 

We can write _ is(δ̂_1) as 

    _ is(δ̂_1) =   ℙ(_ is≠_ is|_ d = ̋̂_1)
    
    =   ℙ(_ is≠_ is|_ d = ̋̂_1, _ d = _0) ℙ( _ d = _0 |_ d = _1) 
          + ℙ(_ is≠_ is|_ d = ̋̂_1, _ d = _1) ℙ( _ d = _1 |_ d = _1) .

Note that ℙ(_ is≠_ is|_ d = ̋̂_1, _ d = _0) = 1 for any decision rule δ̂_1, and the terms (_ d = ̋̂_1 |_ d = ̋̂_1)  and (_ d = ̋̂_0 |_ d = ̋̂_1) are independent of the decision rule δ̂_1.  Therefore, minimizing _ is(δ̂_1) is equivalent to minimizing ℙ(_ is≠_ is|_ d = ̋̂_1, _ d = _1). Furthermore,

    ℙ(_ is   ≠_ is|_ d = ̋̂_1, _ d= _1)    
        = ℙ(_ is≠_ is , _ d = ̋̂_1|_ d = _1) (_ d = ̋̂_1)/ℙ( _ d = ̋̂_1, _ d = _1)
       = (_ d = ̋̂_1)/ℙ( _ d = ̋̂_1, _ d = _1)
           ×∑_i=1^T( ∑_j=1, j≠ i^T (_ is = _̋i, _ d = ̋̂_1|_ is = _̋j, _ d = ̋̂_1) (_ is = _̋j |_ d = ̋̂_1)) 
       =  (_ d = ̋̂_1)/ℙ( _ d = ̋̂_1, _ d = _1)
           ×∑_i=1^T((1 -   (_ is = _̋i, _ d = ̋̂_1|_ is = _̋i, _ d = ̋̂_1)) (_ is = _̋i |_ d = ̋̂_1))
       = (_ d = ̋̂_1)/ℙ( _ d = ̋̂_1, _ d = _1)×∑_i=1^T((1 -   ∫_R_i ∩R̂_1 f_i()) (_ is = _̋i |_ d = ̋̂_1)) ,

where R_i ⊆^n is the region corresponding to the decision _ is = _̋i and R̂_1⊆^n is the region corresponding to the decision _ d = ̋̂_1.  Since we form the decision _ is only when _ d = _1, we conclude that R_i ⊆R̂_1 and hence, R_i ∩R̂_1 = R_i. Therefore,

    ℙ(_ is≠_ is|_ d = ̋̂_1, _ d = _1)    =  (_ d = ̋̂_1)/ℙ( _ d = ̋̂_1, _ d = _1)
         ×∑_i=1^T((1 -   ∫_R_i f_i()) (_ is = _̋i |_ d = ̋̂_1)) ,

and ℙ(_ is≠_ is|_ d = ̋̂_1, _ d) is minimized when the regions R_i are chosen using the decision rule

    δ̂_1i() =     1,     if    i = i^∗
        0,    if    i ≠ i^∗ ,

where

    i^∗ = _j ∈{1,…,T} f_j()(_ is= _̋j |_ d = ̋̂_1) .
 





§ PROOF OF THEOREM 5


 (<ref>) can also be written as

    (| = _̋1,  = _̋1)     = (, = _̋1, =_̋1)/( = _̋1, = _̋1)
       = (,  = _̋1 | = _̋1)( = _̋1)/( = _̋1, = _̋1)
       =∑_i = 1^t( ∑_j = 1, j≠ i^t(_ is= ^i,  = _̋1 |_ is= _j) (_ is= _j| = _̋1) ) (=_̋1)/( = _̋1,  = _̋1)
       = ∑_i=1^t((1 - (_ is= _i,= _̋1|_ is= _i)(_ is= _i | = _̋1)) (=_̋1)/( = _̋1,  = _̋1)
       = (1 - ∑_i=1^t(=̣_i,=_̋1|_ is= _i)(_ is= _i | = _̋1))(=_̋1)/( = _̋1,  = _̋1)
       = (1 - (∑_i=1^t∫_∈ R_i ∩ R_1 f_i() ) (_ is= _i | = _̋1))(=_̋1)/( = _̋1,  = _̋1)

where R_1 = {f_m()/f_0() > γ} and R_i is the region corresponding to decision _i when = _̋1. Therefore, R_i ⊂ R_1, which implies that R_i ∩ R_1 = R_i. Therefore,

    (| = _̋1,  = _̋1)    = (1 - ∑_i=1^t∫_∈ R_i f_i()  (_ is= _i | = _̋1))(=_̋1)/( = _̋1,  = _̋1) ,

and the decision rule that minimizes (| = _̋1,  = _̋1) and consequently _ is, is given by

    ^̣∗ = _i, where  i = _j f_j()(_ is= _j | = _̋1) .

The error exponent analysis is provided in the proof for Theorem 6.




§ PROOF OF THEOREM 6


The error probability _ is has the component (| = _̋1,  = _̋1) that is dependent on the decision rule for (<ref>). 

    (| = _̋1,  = _̋1)     = (, = _̋1|=_̋1) (=_̋1)/( = _̋1, = _̋1)

and

    (, = _̋1|=_̋1) = ∑_j = 1, j≠ i^t(_ is= _i |_ is= _j) (_ is= _j | = _̋1) ,

for some decision rule $̣. Let(_is= _j |= _̋1) = ϵ_j/1-ϵ_0 = ϵ̂_j. Since the decision rule given in Theorem <ref> is optimal, the following equation is valid 

    ∑_i = 1^t( ∑_j = 1, j≠ i^t(^̣∗ = _i |_ is= _j) ϵ̂_j) ≤∑_i = 1^t( ∑_j = 1, j≠ i^t(_ is= _i |_ is= _j) ϵ̂_j)

for any decision rule$̣. Define $̣ as

    _ is= _i, where  i = argmax_j ∏_v∈ S_j_v .

We will first find an upper bound on the error probability(,= _̋1|=_̋1), corresponding to decision rule$̣. Note that there exists a value X̂ in the parameter space, such that 

    ∑_i = 1^t( ∑_j = 1, j≠ i^t(_ is= _i |_ is= _j) ϵ̂_j) ≤∑_i = 1^t( ∑_j = 1, j≠ i^t(_ is= _i |_ is= _j, X = X̂) ϵ̂_j) .

The average error probability in (<ref>) can be related to pairwise average error probability by

    ∑_i = 1^t( ∑_j = 1, j≠ i^t(_ is= _i |_ is= _j, X = X̂) ϵ̂_j) = 
    ∑_j=1, i<j^t(ϵ̂_j + ϵ̂_i) ^_i _j ,

where 

    ^_i _j = ϵ̂_j/ϵ̂_i + ϵ̂_j(_ is= _i |_ is= _j, X = X̂) + ϵ̂_i/ϵ̂_j + ϵ̂_i(_ is= _j |_ is= _i,  X = X̂)  .

Therefore,

    ∑_i = 1^t( ∑_j = 1, j≠ i^t(_ is= _i |_ is= _j, X = X̂) ϵ̂_j)    ≤t(t-1)/2max_i ≠ j^_i _j ,
       ≤t(t-1)/2max_i ≠ j{(_ is= _j |_ is= _i,  X = X̂), (_ is= _i |_ is= _j,  X = X̂) } .

Let t' = t(t-1)/2. Note that

    (_ is= _j |_ is= _i,  X = X̂)    ≤∫_R_ij f_i(|X̂)  ,

where R_ij = {∏_v1∈ S_j_v1≥∏_v2∈ S_i_v2}. Denote B_i as the set of sensors deemed to be compromised in _̋i but not in _̋j, given by S_i - S_i ∩ S_j and correspondingly, B_j as the set S_j - S_i ∩ S_j, with B_i = r_i and B_j = r_j. Also, since the sensor measurements become independent conditioned on X,

    f_i(|X̂) = ∏_m_1 ∈ S_i (∏_c=1^kg_1(Y_m_1^c - h_m_1X̂)) ∏_b ∉S_i (∏_d=1^kg_0(Y_b^d - h_bX̂)) .

Then, the region R_ij is equivalent to 

    {(∏_m_1 ∈ B_j(η_m_1∏_c = 1^kg_1(Y_m_1^c - h_m_1X̂))∏_b ∈ B_i((1-η_b)∏_d = 1^kg_0(Y_b^d - h_bX̂))) .

 
    . ≥(∏_m_1 ∈ B_i(η_m_1∏_c = 1^kg_1(Y_m_1^c - h_m_1X̂))∏_b ∈ B_j((1-η_b)∏_d = 1^kg_0(Y_b^d - h_bX̂)) ) }

Therefore,

    (_ is= _j |_ is= _i,  X = X̂)    ≤∫_R_ij(∏_m_1 ∈ B_i (∏_c = 1^kg_1(Y_m_1^c - h_m_1X̂))∏_b ∈ B_j(∏_d = 1^kg_0(Y_b^d - h_bX̂)) ) ^B_i ∪ B_j
       = 1/∏_m_1 ∈ B_iη_m_1∏_b∈ B_j(1-η_b)∫_R_ijmin{_i()c,τ_jd}^B_i ∪ B_j

where 

    c = ∏_m_1 ∈ B_j (∏_c = 1^kg_1(Y_m_1^c - h_m_1X̂))∏_b ∈ B_i(∏_d = 1^kg_0(Y_b^d - h_bX̂) ,


    d = (∏_m_1 ∈ B_i ( ∏_c = 1^kg_1(Y_m_1^w1 - h_m_1X̂))∏_b ∈ B_j(∏_d = 1^kg_0(Y_b^d - h_bX̂)) )  ,


    τ_i = ∏_m_1 ∈ B_jη_m_1∏_b ∈ B_i(1-η_b) ,


    τ_j = ∏_m_1 ∈ B_jη_m_1∏_b ∈ B_j(1-η_b) ,

and ^B_i ∪ B_j is the set of measurements of sensors in B_i ∪ B_j.
Using the inequality min(a,b)  ≤ a^λ b^1-λ∀ a,b >0 and λ∈ [0,1],

    (_ is= _j |_ is= _i,  X = X̂)    ≤τ_ij^1-λ∫  c^λ d^1-λ^B_i ∪ B_j ,

where τ_ij = (∏_m_1 ∈ B_jη_m_1/(1-η_m_1)) (∏_b∈ B_i(1-η_b)/η_b). Also,

    ∫  c^λ d^1-λ^B_i ∪ B_j


    = ∏_m_1 ∈ B_j(∏_c =1^k∫ g_1(Y_m_1^c - h_m_1X̂)^λ g_0(Y_m_1^c - h_m_1X̂)^1-λ dY_m_1^c)∏_b ∈ B_i(∏_d =1^k∫ g_0(Y_b^d - h_bX̂)^λ g_1(Y_b^d - X̂)^1-λ dY_b^d)

Similarly, 

    (_ is= _i |_ is= _j,  X = X̂)    ≤τ_ij^-λ∫  c^λ d^1-λ^B_i ∪ B_j .

From (<ref>), (<ref>), (<ref>) and (<ref>),

    ∑_i = 1^t( ∑_j = 1, j≠ i^t(_ is= _i |_ is= _j) ϵ̂_j)    ≤  t'max_i,jmax{τ_ij^1-λ,τ_ij^-λ}∫  c^λ d^1-λ^B_i ∪ B_j .

The above inequality holds for all λ∈ [0,1], and note that max{τ_ij,τ_ij^-1}≥ 1. Therefore, 

    ∑_i = 1^t( ∑_j = 1, j≠ i^t(_ is= _i |_ is= _j) ϵ̂_j)    ≤  t'max_i,jmax{τ_ij,τ_ij^-1}min_λ∫  c^λ d^1-λ^B_i ∪ B_j ,
       ≤ t'τmin_λmax_i,j∫  c^λ d^1-λ^B_i ∪ B_j ,

where τ = max_i ≠ j τ_ij is a finite constant. 
Using the assumption that the pdf of the random variables corresponding to channel noise have infinite support, and by change of variables,

    ∫  c^λ d^1-λ^B_i ∪ B_j   =  ∏_m_1 ∈ B_j(∏_c =1^k∫ g_1(Y_m_1^'c)^λ g_0(Y_m_1^'c )^1-λ dY_m_1^'c)∏_b ∈ B_i(∏_d =1^k∫ g_0(Y_b^'d)^λ g_1(Y_b^'d)^1-λ dY_b^'d) 
       = (∫ g_1(Y)^λ g_0(Y )^1-λ dY)^r_j k (∫ g_0(Y)^λ g_1(Y )^1-λ dY)^r_i k

Denote ϕ_1 as the error exponent corresponding to error probability (| = _̋1,  = _̋1) under decision rule $̣.ϕ_1is given by

    ϕ_1    = lim_k →∞-log ((| = _̋1,  = _̋1)) /k ,
       = lim_k →∞log(( = _̋1 | = _̋1))/k - lim_k →∞log (( ,  = _̋1| = _̋1)) /k .

Note that(= _̋1 |= _̋1))approaches 1 ask →∞, and using (<ref>) and (<ref>), we have

    ϕ_1    ≥ - lim_k →∞t' τ/k - lim_k →∞log(min_λmax_i,j∫  c^λ d^1-λ^B_i ∪ B_j)/k ,
       ≥lim_k →∞min_i,jmax_λ-log(∫  c^λ d^1-λ^B_i ∪ B_j )/k .

Using (<ref>),

    ϕ_1    ≥lim_k →∞min_i,jmax_λ  rk (C(g_1,g_0,λ) +C(g_0,g_1,λ) /k ,

whereC(g_u,g_v,λ) = -log(∫g_u^λ(Y) g_v^1-λ(Y) dY). Note thatmax_λ C(g_u,g_v,λ)is the Chernoff information between the pdfsg_uandg_v, which we denote byC^∗(g_u,g_v)andC^∗(g_u,g_v) ≥0. Also, minimizing over the pair of hypotheses_iand_jimplies that

    r_i    = r_j = 1,  when K=1 ,
    
    r_i    = 1, r_j = 0  when  K > 1  and   C^∗(g_0,g_1) ≥  C^∗(g_1,g_0) ,
    
    r_i    = 0, r_j = 1  when  K > 1  and   C^∗(g_1,g_0) >  C^∗(g_0,g_1) .

Therefore,

    ϕ_1    ≥    C^∗ (g_1,g_0) + C^∗ (g_0,g_1),  if K=1 ,
       min{ C^∗ (g_1,g_0), C^∗ (g_0,g_1) }, otherwise ,

We now find a lower bound on the error probability(|= _̋1, = _̋1). DenoteX_tas the true value of parameterX. Therefore, under the assumption of having perfect knowledge of parameterX,

    ∑_i = 1^t( ∑_j = 1, j≠ i^t(^̣∗ = _i |_ is= _j) ϵ̂_j)    ≥∑_i = 1^t( ∑_j = 1, j≠ i^t(^̣∗ = _i |_ is= _j, X = X_t) ϵ̂_j) ,
       = ∑_i = 1^t(^̣∗≠_i |_ is= _i, X = X_t) ϵ̂_i ,
       ≥max_i,j{(^̣∗≠_i |_ is= _i, X = X_t) ϵ̂_i, (^̣∗≠_j |_ is= _j, X = X_t) ϵ̂_j} ,

Denoteϕ_2as the error exponent of(|= _̋1=_̋1), corresponding to decision rule^̣∗. Therefore,

    ϕ_2    = lim_k →∞ -log(∑_i = 1^t( ∑_j = 1, j≠ i^t(^̣∗ = _i |_ is= _j) ϵ̂_j) )/k ,
       ≤lim_k →∞  -log( max_i,j{(^̣∗≠_i |_ is= _i, X = X_t) ϵ̂_i, (^̣∗≠_j |_ is= _j, X = X_t) ϵ̂_j} )/k ,
       = lim_k →∞  -log( max_i,j{(^̣∗≠_i |_ is= _i, X = X_t) , (^̣∗≠_j |_ is= _j, X = X_t)} )/k .

Given parameterX, the sensor observations are conditionally independent of each other over time and space. Under this condition, we adopt the approach used in <cit.> for further analysis. Defineb_i,j^λ(Y^w|X_t)as

    b_i,j^λ(Y^w| X_t)    ≜f̃_i ^λ (Y^w| X_t) f̃_j ^1-λ(Y^w | X_t)/∫f̃_i ^λ (Y^w| X_t) f̃_j ^1-λ(Y^w | X_t)dY^w  ,

and let

    λ^∗   = _λ∈ [0,1] -log(∫ f_i^λ(| X_t) f_j^1-λ(| X_t) ) ,
       = _λ∈ [0,1] -∑_w=1^klog(∫f̃_i^λ(Y^w| X_t) f̃_j^1-λ(Y^w| X_t) dY^w) ,
       = _λ∈ [0,1] -k log(∫f̃_i^λ(Y^w| X_t) f̃_j^1-λ(Y^w| X_t) dY^w) .

It is easy to verify that

    max_λ∈ [0,1] -log(∫f̃_i^λ(Y^w| X_t) f̃_j^1-λ(Y^w| X_t) dY^w)
    
    = D_ KL(b_i,j^λ^∗(·| X_t) f̃_i(·| X_t) ) = D_ KL(b_i,j^λ^∗(·| X_t)  f̃_j(·| X_t))  ,

whereDdenotes the KL-divergence.
Define a probability measureas

    (| X_t) ≜∏_w=1^k b_i,j^λ^∗(Y^w | X_t) .

Define a random processM_las 

    M_l ≜∑_w=1^l( log( b_i,j^λ^∗(Y^w | X_t)/f̃^j(Y^w| X_t) - [ log( b_i,j^λ^∗(Y^w | X_t)/f̃^j(Y^w| X_t) |F_w-1] ) ,

whereF_w-1is the sigma field generated by(Y^1,…,Y^w-1), and the expectation is with respect to the probability measure. It can be easily verified that the processM_lis a stable martingale. According to the martingale stability theorem <cit.>, 

    lim_k →∞ M_k/k 0 .

This can be equivalently written in the following form: For anyν>0,

    lim_k →∞(  M_k/k > ν) = 0 ,

Since, for a givenX_t,Y^wis conditionally independent of past observations, 

    [ log( b_i,j^λ^∗(Y^w | X_t)/f̃^j(Y^w| X_t) |F_w-1]  = D_ KL(b_i,j^λ^∗(·| X_t)||f̃_j(·| X_t)) = D_ KL(b_i,j^λ^∗(·| X_t)||f̃_i(·| X_t)) .

Using (<ref>) and by restructuring (<ref>), we obtain

    lim_k →∞(∏_w=1^k f̃_j(Y^w| X_t) > exp(- k D_ KL(b_i,j^λ^∗(·| X_t)||f̃_j(·| X_t)) - kν) ×∏_w=1^k b_i,j^λ^∗(Y^w | X_t)) = 1 ,

Note that either(d^∗ ≠_i |X_t) ≥1/2or(d^∗ = _i |X_t) ≥1/2.
Assume that(d^∗ = _i |X_t) ≥1/2holds. Then, 

    lim_k →∞(∏_w=1^k f̃_j(Y^w | X_t) > exp(- k D_ KL(b_i,j^λ^∗(·| X_t)||f̃_j(·| X_t)) - kν) ×∏_w=1^k b_i,j^λ^∗(Y^w | X_t),d^∗ = _i |  X_t) ≥1/2 - κ ,

for anyκ>0. (<ref>) is equivalent to 

    lim_k →∞∫ _R∏_w=1^k b_i,j^λ^∗(Y^w | X_t) ≥1/2 - κ ,

whereRis the region{d^∗ = _i }∩{∏_w=1^k b_i,j^λ^∗(Y^w |X_t) < exp( k D_KL(b_i,j^λ^∗(·|X_t)||f̃_j(·|X_t)) + kν)×∏_w=1^k f̃_j(Y^w |X_t)}.
Clearly,

    ∫ _R∏_w=1^k b_i,j^λ^∗(Y^w | X_t)    ≤exp( k D_ KL(b_i,j^λ^∗(·| X_t)||f̃_j(·| X_t)) + kν) ∫ _R∏_w=1^k f̃_j(Y^w | X_t) ,
       ≤exp( k D_ KL(b_i,j^λ^∗(·| X_t)||f̃_j(·| X_t)) + kν)∫ _R_2 f_j(| X_t) ,
       = exp( k D_ KL(b_i,j^λ^∗(·| X_t)||f̃_j(·| X_t)) + kν) (^̣∗≠_j |_ is= _j, X_t)

where regionR_2is{d^∗ ≠_j }. From (<ref>) and (<ref>), it can be concluded that 

    (^̣∗≠_j |_ is= _j, X_t) ≥( 1/2 - κ) exp(- k D_ KL(b_i,j^λ^∗(·| X_t)||f̃_j(·| X_t)) - kν) ,

for anyκ, ν>0. Similarly, if the case(d^∗ ≠_i |X_t) ≥1/2holds,

    (^̣∗≠_i |_ is= _i, X_t) ≥( 1/2 - κ) exp(- k D_ KL(b_i,j^λ^∗(·| X_t)||f̃_i(·| X_t)) - kν) .

Using (<ref>) and (<ref>) in (<ref>), 

    ϕ_2    ≤lim_k →∞ -max_i,j{log ((^̣∗≠_j |_ is= _j, X_t)) ,log((^̣∗≠_i |_ is= _i, X_t))}/k 
       = lim_k →∞1/k(-( 1/2 - κ) + kν) + lim_k →∞min_i,j (k D_ KL(b_i,j^λ^∗(·| X_t)||f̃_j(·| X_t)),k D_ KL(b_i,j^λ^∗(·| X_t)||f̃_i(·| X_t)) /k .

Sinceκandνcan be made arbitrarily close to 0, and using (<ref>),

    ϕ_2    ≤min_i,jmax_λ∈ [0,1] -log(∫f̃_i^λ(Y^w| X_t) f̃_j^1-λ(Y^w| X_t) dY^w) ,
       = min_i,j (r_j C^∗ (g_1,g_0) + r_i C^∗ (g_0,g_1)) ,

which implies that

    ϕ_2 ≤    C^∗ (g_1,g_0) + C^∗ (g_0,g_1),  if K=1 ,
       min{ C^∗ (g_1,g_0), C^∗ (g_0,g_1) }, otherwise .

From (<ref>),ϕ_2 ≥ϕ_1, and therefore, from (<ref>) and (<ref>), it can be concluded that

    ϕ_1 = ϕ_2 =      C^∗ (g_1,g_0) + C^∗ (g_0,g_1),  if K=1 ,
       min{ C^∗ (g_1,g_0), C^∗ (g_0,g_1) }, otherwise .



§ PROOF OF THEOREM <REF>


For the decision rule defined in Theorem <ref>, we denote the decision formed as_is ∈{_̋1,…, _̋T}to distinguish it from the decision formed by the optimal decision rule in Lemma <ref>. Also, from Lemma <ref>,

    -log_ is^u/n≤ -log_ is(δ̅_1)/n≤ -_ is(δ̂_1)/n≤-log_ is^ l/n .
 
Next, we find an upper bound on_is^ u.  WhenXis guessed to beX_c,  the probability that the decision formed by the rule in Theorem <ref> is_̋jwhen the true model is_̋iis given by(_is = _̋j |_is = _̋i, X_c). Assuming that the probabilitiesϵ_i , fori ∈{1,…,T}, are independent of the choice ofX_c, we have 

    _ is^u   = ∑_i = 1^T∑_i≠ j
    j = 1^Tϵ_j (_ is = _̋i |_  is = _̋j,  X_c)
       = ∑_j = 2
     i<j^T (ϵ_j + ϵ_i)^ij ,

where we have defined

    ^ij   ≜ϵ_j/ϵ_i + ϵ_j·(_ is = _̋i |_ is = _̋j,  X_c) + ϵ_i/ϵ_i + ϵ_j·(_ is = _̋j |_ is = _̋i, X_c) .

Therefore,

    _ is^u   ≤T(T-1)/2max_i≠ j; i,j,∈{1,… T}^ij ,
       ≤T(T-1)/2max_i≠ j; i,j∈{1,… T}{(_ is = _̋i |_ is = _̋j, X_c), (_ is = _̋j |_ is = _̋i, X_c) } .

LetT' ≜T(T-1)/2. Note that

    (_ is = _̋j |_ is = _̋i, X = X_c) ≤∫_R_ij f_i(| X_c) ,

whereR_ij = {: ∏_a ∈S_j _a ≥∏_b ∈S_i _b }.
DefineB_ias the set of coordinates deemed to be compromised in_̋ibut not in_̋j, i.e.,B_i ≜S_i - S_i∩S_j, withB_i = r_i. Similarly, defineB_j ≜S_j - S_i ∩S_j, withB_j = r_j. Also, since we haveg_l^j, forl ∈{1,…,n}, j∈{0,1}, to be conditionally independent distributions givenX_c, we have 

    f_i(| X_c) = ∏_a ∈ S_i(∏_c=1^n g_a^1(Y_c(a)| X_c)) ∏_b ∈S̅_i(∏_d = 1^n g_b^0(Y_d(b)| X_c)) ,

whereS̅_i ≜{1,…,m} \S_i.  Then, the regionR_ijis equivalent to

    {: ( ∏_a ∈ B_j∏_c = 1^n g_a^1(Y_c(a)| X_c)) ( ∏_b ∈ B_i∏_d = 1^n g_b^0(Y_b(d) | X_c)) ≥.
       .( ∏_a ∈ B_i∏_c = 1^n g_a^1(Y_c(a)| X_c))( ∏_b ∈ B_j∏_d = 1^n g_b^0(Y_d(b)| X_c)) } .

Therefore, 

    (_ is = _̋j |_ is = _̋i,X_c)    ≤∫_R_ij( ∏_a ∈ B_i∏_c = 1^n g_a^1(Y_c(a)| X_c))( ∏_b ∈ B_j∏_d = 1^n g_b^0(Y_d(b)| X_c)) _B_i∪ B_j ,
       = ∫_R_ijmin{t_1,t_2}_B_i ∪ B_j ,

where_B_i ∪B_jis the data for the coordinates in the setB_i ∪B_jand

    t_1   ≜( ∏_a ∈ B_i∏_c = 1^n g_a^1(Y_c(a)| X_c))( ∏_b ∈ B_j∏_d = 1^n g_b^0(Y_d(b)| X_c)) ,
    
    t_2   ≜( ∏_a ∈ B_j∏_c = 1^n g_a^1(Y_c(a)| X_c)) ( ∏_b ∈ B_i∏_d = 1^n g_b^0(Y_b(d) | X_c)) ,

Using the inequality 

    min(a,b) ≤ a^λb^1-λ,    ∀ a,b>0,   and   λ∈ [0,1] ,

and 

    ∫_R_ijf_i(| X_c)≤∫ f_i(| X_c) ,

we can modify the inequality in (<ref>) as

    (_ is = _̋j |_ is = _̋i, X_c)  ≤∫ t_1^λt_2^1-λ^B_i ∪ B_j ,    ∀ λ∈ [0,1] .

Following the similar line of analysis as in (<ref>)-(<ref>), it can be verified that

    (_ is = _̋i |_ is = _̋j,X_c)≤∫ t_1^λt_2^1-λ_B_i ∪ B_j .

The inequalities in (<ref>) and (<ref>) hold for allλ∈[0,1]. Hence, from (<ref>), (<ref>) and (<ref>), 

    _ is^ u   ≤ T' min_λ∈ [0,1]∫ t_1^λ t_2^1-λ_B_i ∪ B_j .
 Under the assumption that the pdfs  of the random variables Y_a(b) | X, belong to the location-scale family of distributions with infinite support and only the location parameter a function of X, we use the change of variablesỸ_a(b) = Y_a(b) |X_cto obtain

    ∫ t_1^λ t_2^1-λ_B_i ∪ B_j   = ( ∏_a ∈ B_j∏_c = 1^n ∫ (g_a^1(Ỹ_a(c))) ^λ (g_a^0(Ỹ_a(c)))^1-λdỸ_a(c))
         ×( ∏_b ∈ B_i∏_d = 1^k ∫ (g_b^1(Ỹ_b(d)))^λ (g_b^0(Ỹ_b(d)))^1-λdỸ_b(d)) .

Note that

    - log_ is^u/n =  - 1/nlog(max_i,jmin_λ∫ t_1^λ t_2^1-λ_B_i ∪ B_j) .

By taking the limitn→∞and using the monotonicity of thelogfunction, we have

    -lim_n →∞log_ is^u/n    = lim_n→∞1/nmin_i,jmax_λ -log(∫ t_1^λ t_2^1-λ_B_i ∪ B_j)
       = lim_n →∞1/nmin_i,jmax_λ( ∑_a ∈ B_j-nlog(∫ (g_a^1(Y))^λ (g_a^0(Y))^1-λdY). 
       .   +∑_b ∈ B_i -nlog(∫ (g_b^0(Y))^λ(g_b^1(Y))^1-λdY)) .

Therefore,

    -lim_n →∞log_ is^u/n     =min_i,jmax_λ( ∑_a ∈ B_j-log(∫ g_a^1(Y)^λ g_a^0(Y)^1-λdY) . 
            + .∑_b ∈ B_i -log(∫ g_b^0(Y)^λg_b^1(Y)^1-λdY)) .














We now find a lower bound on_is^l. Under the perfect knowledge ofX, we have

    _ is^ l   = ∑_i=1^T ∑_j=1
    j≠ i^T (_ is = _̋j |_ is = _̋i, X) ϵ_i ,
       = ∑_i=1^T (_ is≠_̋i|_ is = _̋i,X)ϵ_i ,
       ≥max_i,j{(_ is = _̋j |_ is = _̋i, X) ϵ_i, (_ is = _̋i |_ is = _̋j, X) ϵ_j } .

Note that

    lim_n →∞ - log(_ is^l)/n   ≤lim_n→∞ -1/nlog(max_i,j{(_ is = _̋j |_ is = _̋i, X) ϵ_i, (_ is = _̋i |_ is = _̋j, X) ϵ_j}) ,
       = lim_n →∞ - 1/nlog(max_i,j{(_ is = _̋j |_ is = _̋i, X) , (_ is = _̋i |_ is = _̋j, X) }) .

For further analysis, we adopt the approach used in <cit.>. Under model_̋i, the data pointsY_ware distributed according to the pdff_i, forw ∈{1,…n}andi ∈{1,…T}. Defineb_i,j^λ(Y_w) as the pdf 

    b_i,j^λ(Y_w) ≜f_i^λ (Y_w | X) f_j^1-λ(Y_w | X)/∫f_i^λ (Y_w | X) f_j^1-λ(Y_w | X) dY_w ,

and let 

    λ^∗   = _λ∈ [0,1] -log( ∫ f_i^λ(| X) f_j^1-λ (| X))
       = _λ∈ [0,1] -∑_w = 1^n log( ∫f_i^λ(Y_w | X) f_j^1-λ (Y_w | X) dY_w)
       = _λ∈ [0,1] -n log( ∫f_i^λ(Y_w | X) f_j^1-λ (Y_w | X) dY_w) .

It can be readily verified that

    max_λ∈ [0,1] -log( ∫f_i^λ(Y_w | X) f_j^1-λ (Y_w | X) dY_w) 
    = D_ KL(b_i,j^λ^∗f_i(.| X)) =  D_ KL(b_i,j^λ^∗f_j(.| X)) ,

whereD_KL(fg)denotes the Kullback-Liebler divergence between distributionsfandg. Define the probability measureas

    (| X) ≜∏_w = 1^n b_i,j^λ^∗(Y_w) ,

and define a random processM_ras 

    M_r ≜∑_w = 1^r ( log( b_i,j^λ^∗(Y_w)/f_j(Y_w | X)) - [ log(b_i,j^λ^∗(Y_w)/f_j(Y_w | X)) | F_w-1] ) ,

whereF_w-1is theσ-field generated by{Y_1,…,Y_w-1}, and the expectation is with respect to the probability measure. It can be verified that the processM_ris a stable martingale. According to the martingale stability theorem <cit.>, we have

    lim_r→∞M_r/r 0 .

This can be equivalently written in the following form:

    lim_r →∞( M_r/r > ν) = 0 ,   ∀ν > 0 .

Since for a givenXthe random vectorsY_ware i.i.d., we have 

    [ log(b_i,j^λ^∗(Y_w)/f_j(Y_w | X))  |  F_w-1]    = D_ KL(b_i,j^λ^∗f_i(.| X)) =  D_ KL(b_i,j^λ^∗f_j(.| X)) .

By using (<ref>) and restructuring (<ref>), we obtain

    lim_n→∞( ∏_w=1^n f_j(Y_w| X) > exp(-n D_ KL(b_i,j^λ^∗f_j(.| X)) - nν) ×∏_w=1^n b_i,j^λ^∗(Y_w)) = 1 .

Under the probability measure, we denote the probability of deciding_̋ias the true model whenXis given as(_is = _̋i |X)and note that either(_is ≠_̋i|X) ≥1/2or(_is = _̋i|X) ≥1/2. Then, using the assumption that(_is = _̋i|X) ≥1/2holds, from (<ref>) we get 

    lim_n→∞( ∏_w=1^n f_j(Y_w| X) > exp(-n D_ KL(b_i,j^λ^∗f_j(.| X)) - nν) ×∏_w=1^n b_i,j^λ^∗(Y_w), _ is = _̋i| X) ≥1/2 - κ ,

for anyκ>0. Using (<ref>), we conclude that (<ref>) is equivalent to

    lim_n→∞∫_R ∏_w=1^n b_i,j^λ^∗(Y_w) dY_w ≥1/2 - κ ,

whereRis the region 

    {: {_ is = _̋i| X} and {∏_w=1^n b_i,j^λ^∗(Y_w| X) <exp(n D_ KL(b_i,j^λ^∗f_j(.| X)) + nν) ×∏_w=1^n f_j(Y_w| X) }} .

In the regionR, we have

    ∫_R ∏_w=1^n b_i,j^λ^∗(Y_w| X) dY_w    ≤exp(n D_ KL(b_i,j^λ^∗f_j(.| X)) + nν) ∫_R ∏_w=1^n f_j(Y_w| X) ,
       ≤exp(n D_ KL(b_i,j^λ^∗f_j(.| X)) + nν) ∫_R_2∏_w=1^n f_j(Y_w| X) ,
       = exp(k D_ KL(b_i,j^λ^∗f_j(.| X)) + nν) ·(_ is≠_̋j|_ is = _̋j, X) ,

where regionR_2is{ _is ≠_̋j|X}and also,R ⊆R_2. From (<ref>) and (<ref>), it is concluded that

    (_ is≠_̋j|_ is = _̋j,X) ≥(1/2 - κ)exp(-n D_ KL(b_i,j^λ^∗f_j(.| X)) - nν) ,
  
for anyν,κ>0. Similarly, if the case(_is ≠_̋i|X) ≥1/2holds,

    (_ is≠_̋i|_ is = _̋i,X) ≥(1/2 - κ)exp(-n D_ KL(b_i,j^λ^∗f_i(.| X)) - nν) .

Using  (<ref>), (<ref>) and (<ref>), we have

    lim_n →∞ - log(_ is^ l)/n≤   lim_n →∞ -1/nmax_i,j{log((_ is≠_̋j|_ is = _̋j,X)), log((_ is≠_̋i|_ is = _̋i,X))} ,
    
    =   lim_n →∞1/n(-(1/2 - κ) + κν) 
          + lim_n →∞1/nmin_i,j{ n D_ KL(b_i,j^λ^∗f_i(.| X)), n D_ KL(b_i,j^λ^∗f_j(.| X))} .

Using the fact thatκandνcan be made arbitrarily close to0, and using (<ref>), we get

    lim_n →∞ - log(_ is^ l)/n≤   min_i,jmax_λ∈ [0,1] -log( ∫f_i^λ (Y_w | X) f_j^1-λ (Y_w | X) Ỵ_w)
     =   min_i,jmax_λ{∑_a ∈ B_j -log( ∫ (g_a^1(Y))^λ (g_a^0(Y))^1-λỴ)  .  
                  + . ∑_b ∈ B_i -log( ∫ (g_b^0(Y)^λ (g_b^1(Y))^1-λỴ)} 
       = min_i≠ jC(f_i,f_j) ,

where (<ref>) follows from the assumption that the pdfsg_a^jbelong to the location-scale family of distributions and have infinite support. From (<ref>), (<ref>), it is clear that 

    lim_n →∞ - log_ is^ l/k = lim_n →∞ - log_ is^u/n .
 
Then, using (<ref>), we conclude that the error exponents of_is(δ̂_1)and_is(δ̅_1)are the same and are given by (<ref>).




§ PROOF OF THEOREM <REF>

We aim to design the estimatorU_l^jand the decision ruleδ̅_lthat minimize the utility function_l^j(δ_l^j,δ̅_l^r,U_l^j) subject to the constraint 

    _j(_l^ r = _̋l^ r)    = ∫δ_l^j(_l) δ̅_l^ r(_l) g_l^j(_l) _l ≥ 1 - ν_l^j .

Since the effect of estimator only appears in, the optimization problem can be decoupled similarly to the approach followed earlier. The optimal estimator can be readily verified to be 

    X̂^j_l(_l) = __l^j ,

whereX̂_l^j(_l) is defined in (<ref>). In order to design the decision rules, we start by noting that for a decision ruleδ̅_l, such that, 

    ∫δ_l^j(_l) δ̅_l^ r(_l) g_l^j(_l) _l>1 - ν_l^j ,

we can design another decision ruleΔ_l  ≜[Δ_l^r(_l), Δ_l^u(_l)], such that,∫δ_l^j(_l) Δ^r_l(_l) g_l^j(_l) _l = 1 - ν_l^jwith the same estimation performance. To show this, we set 

    Δ^ r_l(_l) = (1 - ν_l^j) δ̅^ r_l(_l)/∫δ_l^j(_l) δ̅^ r_l(_l) g_l^j(_l) _l  ,

which satisfies (<ref>) with equality. Using (<ref>), note thatΔ^r_l(_l) < δ̅^r_l(_l), which implies thatΔ_lis a valid decision rule. We can easily verify that

    _l^j(δ_l^j,Δ^ r_l,X̂_l^j) = _l^j(δ_l^j,δ̅^ r_l,X̂_l^j) ,
 
which implies that the estimation performance is the same for both decision rules, and therefore, we can restrict our design for the optimum decision rule to the class of rules that satisfy (<ref>) with equality. Under the equality condition,∫δ_j^i(_i) ,δ̅^r_l(_l) g_l^j(_l) _l = 1 - ν_l^j, in which case minimizing_l^j(δ_l^j,δ̅^r_l,X̂_l^j)is equivalent to minimizing∫δ_l^j(_l) δ̅^r_l(_l)_l^j(_l) g_l^j(_l) _l. Letγ_l^j ≥0be the solution to 

    _j(γ_l^j ≥_l^j(_l)) = ∫ _R̂δ^j_l(_l) g_l^j(_l) _l = 1 - ν_l^j ,

where regionR̂is{_l: γ_l^j ≥_l^j(_l) }. Then,

    ∫δ^j_l(_l) δ̅^ r_l(_l)_l^j(_l) g_l^j(_l) _l - γ_l^j(1-ν_l^j) 
       = ∫δ^j_l(Y_l) δ̅^ r_l(_l)_l^j(_l) g_l^j(_l) _l - γ_l^j∫δ^j_l(_l) δ̅^ r_l(_l)g_l^j(_l) _l
       = ∫δ^j_l(_l) δ̅^ r_l(_l)(_l^j(_l)-γ_l^j) g_l^j(_l) _l
       ≥∫_R̂δ^l_j(_l) (_l^j(_l)-γ_l^j) g_l^j(_l) _l 
        = ∫_R̂δ^l_j(_l) _l^j(_l) g_l^j(_l) _l -γ_l^j _j(γ_l^j ≥_l^j(_l)) 
       = ∫_R̂δ^j_l(_l) _l^j(_l) g_l^j(_l) _l -γ_l^j (1-ν_l^j) .

Clearly,∫δ^j_l(_l) δ̅^r_l(_l)_l^j(_l) g_l^j(_l) _l  ≥∫_R̂ δ^j_l(_l) _l^j(_l) g_l^j(_l) _l and therefore, the decision ruleδ̅^r_l(_l)given by

    δ̅_l^ r(_l) = 1_{γ_l^j ≥_l^j(_l) } ,

is optimal as it ensures optimal estimation performance and satisfies the constraint in (<ref>).
IEEEtran