In this paper, a cross-layer framework to jointly optimize spectrum sensing and scheduling in resource constrained agile wireless networks is presented. A network of secondary users (SUs) accesses portions of the spectrum left unused by a network of licensed primary users (PUs). A central controller (CC) schedules the traffic of the SUs, based on distributed compressed measurements collected by the SUs. Sensing and scheduling are jointly controlled to maximize the SU throughput, with constraints on PU throughput degradation and SU cost. The sparsity in the spectrum dynamics is exploited: leveraging a prior spectrum occupancy estimate, the CC needs to estimate only a residual uncertainty vector via sparse recovery techniques. The high complexity entailed by the POMDP formulation is reduced by a low-dimensional belief representation via minimization of the Kullback-Leibler divergence. It is proved that the optimization of sensing and scheduling can be decoupled. A partially myopic scheduling strategy is proposed for which structural properties can be proved showing that the myopic scheme allocates SU traffic to likely idle spectral bands. Simulation results show that this framework balances optimally the resources between spectrum sensing and data transmission. This framework defines sensing-scheduling schemes most informative for network control, yielding energy efficient resource utilization.
@cite_18: This two-part paper presents a feedback-based cross-layer framework for distributed sensing and estimation of a dynamic process by a wireless sensor network (WSN). Sensor nodes wirelessly communicate measurements to the fusion center (FC). Cross-layer factors such as packet collisions and the sensing-transmission costs are considered. Each SN adapts its sensing-transmission action based on its own local observation quality and the estimation quality feedback from the FC under cost constraints for each SN. In this second part, low-complexity myopic sensing-transmission policies (MPs) are designed to optimize a trade-off between performance and the cost incurred by each SN. The MP is computed in closed form for a coordinated scheme, whereas an iterative algorithm is presented for a decentralized one, which converges to a local optimum. The MP dictates that, when the estimation quality is poor, only the best SNs activate, otherwise all SNs remain idle to preserve energy. For both schemes, the threshold on the estimation quality below which the SNs remain idle is derived in closed form, and is shown to be independent of the number of channels. It is also proved that a single channel suffices for severely energy constrained WSNs. The proposed MPs are shown to yield near-optimal performance with respect to the optimal policy of Part I, also in this issue, at a fraction of the complexity, thus being more suitable for practical WSN deployments.
@cite_22: This paper is concerned with model reduction for complex Markov chain models. The Kullback-Leibler divergence rate is employed as a metric to measure the difference between the Markov model and its approximation. For a certain relaxation of the bi-partition model reduction problem, the solution is shown to be characterized by an associated eigenvalue problem. The form of the eigenvalue problem is closely related to the Markov spectral theory for model reduction. This result is the basis of a heuristic proposed for the m-ary partition problem, resulting in a practical recursive algorithm. The results are illustrated with examples.
@cite_7: Sensor systems typically operate under resource constraints that prevent the simultaneous use of all resources all of the time. Sensor management becomes relevant when the sensing system has the capability of actively managing these resources; i.e., changing its operating configuration during deployment in reaction to previous measurements. Examples of systems in which sensor management is currently used or is likely to be used in the near future include autonomous robots, surveillance and reconnaissance networks, and waveform-agile radars. This paper provides an overview of the theory, algorithms, and applications of sensor management as it has developed over the past decades and as it stands today.
@cite_8: We consider the optimal sensor scheduling problem formulated as a partially observed Markov decision process (POMDP). Due to operational constraints, at each time instant, the scheduler can dynamically select one out of a finite number of sensors and record a noisy measurement of an underlying Markov chain. The aim is to compute the optimal measurement scheduling policy, so as to minimize a cost function comprising of estimation errors and measurement costs. The formulation results in a nonstandard POMDP that is nonlinear in the information state. We give sufficient conditions on the cost function, dynamics of the Markov chain and observation probabilities so that the optimal scheduling policy has a threshold structure with respect to a monotone likelihood ratio (MLR) ordering. As a result, the computational complexity of implementing the optimal scheduling policy is inexpensive. We then present stochastic approximation algorithms for estimating the best linear MLR order threshold policy.
@cite_21: In this paper, a framework for dynamic high-dimensional hypothesis testing in wireless sensor networks is presented. The sensor nodes (SNs) collect and transmit to a fusion center (FC), in a distributed fashion, compressed measurements of a time-correlated hypothesis vector. The FC, based on the measurements collected, tracks the hypothesis vector, and feeds back minimal information about the uncertainty in the current estimate, which enables adaptation of the SNs' data collection and transmission strategy. The policy of the SNs is optimized with the overall objective of minimizing the detection error probability, under sensing and transmission cost constraints incurred by each SN. A Bernoulli approximation on the detection error is employed, which enables a significant reduction in the optimization complexity and the design of scalable estimators based on sparse approximation recovery algorithms. Simulation results demonstrate that, for a target 5 detection error, the adaptive scheme attains 90 and 50 cost savings with respect to a memoryless scheme which does not exploit the time-correlation and a non-adaptive one, respectively.
@cite_6: In this paper, we study the problem of tracking an object moving randomly through a network of wireless sensors. Our objective is to devise strategies for scheduling the sensors to optimize the tradeoff between tracking performance and energy consumption. We cast the scheduling problem as a partially observable Markov decision process (POMDP), where the control actions correspond to the set of sensors to activate at each time step. Using a bottom-up approach, we consider different sensing, motion and cost models with increasing levels of difficulty. At the first level, the sensing regions of the different sensors do not overlap and the target is only observed within the sensing range of an active sensor. Then, we consider sensors with overlapping sensing range such that the tracking error, and hence the actions of the different sensors, are tightly coupled. Finally, we consider scenarios wherein the target locations and sensors' observations assume values on continuous spaces. Exact solutions are generally intractable even for the simplest models due to the dimensionality of the information and action spaces. Hence, we devise approximate solution techniques, and in some cases derive lower bounds on the optimal tradeoff curves. The generated scheduling policies, albeit suboptimal, often provide close-to-optimal energy-tracking tradeoffs.
@cite_24: 
@cite_5: Consider a decision maker who is responsible to dynamically collect observations so as to enhance his information about an underlying phenomena of interest in a speedy manner while accounting for the penalty of wrong declaration. Due to the sequential nature of the problem, the decision maker relies on his current information state to adaptively select the most informative'' sensing action among the available ones. In this paper, using results in dynamic programming, lower bounds for the optimal total cost are established. The lower bounds characterize the fundamental limits on the maximum achievable information acquisition rate and the optimal reliability. Moreover, upper bounds are obtained via an analysis of two heuristic policies for dynamic selection of actions. It is shown that the first proposed heuristic achieves asymptotic optimality, where the notion of asymptotic optimality, due to Chernoff, implies that the relative difference between the total cost achieved by the proposed policy and the optimal total cost approaches zero as the penalty of wrong declaration (hence the number of collected samples) increases. The second heuristic is shown to achieve asymptotic optimality only in a limited setting such as the problem of a noisy dynamic search. However, by considering the dependency on the number of hypotheses, under a technical condition, this second heuristic is shown to achieve a nonzero information acquisition rate, establishing a lower bound for the maximum achievable rate and error exponent. In the case of a noisy dynamic search with size-independent noise, the obtained nonzero rate and error exponent are shown to be maximum.
@cite_15: Standard value function approaches to finding policies for Partially Observable Markov Decision Processes (POMDPs) are generally considered to be intractable for large models. The intractability of these algorithms is to a large extent a consequence of computing an exact, optimal policy over the entire belief space. However, in real-world POMDP problems, computing the optimal policy for the full belief space is often unnecessary for good control even for problems with complicated policy classes. The beliefs experienced by the controller often lie near a structured, low-dimensional subspace embedded in the high-dimensional belief space. Finding a good approximation to the optimal value function for only this subspace can be much easier than computing the full value function. We introduce a new method for solving large-scale POMDPs by reducing the dimensionality of the belief space. We use Exponential family Principal Components Analysis (Collins, Dasgupta, & Schapire, 2002) to represent sparse, high-dimensional belief spaces using small sets of learned features of the belief state. We then plan only in terms of the low-dimensional belief features. By planning in this low-dimensional space, we can find policies for POMDP models that are orders of magnitude larger than models that can be handled by conventional techniques. We demonstrate the use of this algorithm on a synthetic problem and on mobile robot navigation tasks.
@cite_12: In this paper, the problem of efficient operation of an energy-constrained, heterogeneous Wireless Body Area Network (WBAN) to optimize an activity detection application is addressed. WBANs constitute a new class of wireless sensor networks that enable diverse applications in healthcare, entertainment, sports, military and emergency situations. A typical WBAN consists of a few, heterogeneous sensors wirelessly coupled to an energy-constrained fusion center which, according to observations of a real-world prototype WBAN, imposes critical restrictions on system lifetime. To address this issue, a novel stochastic control framework is introduced, which considers both sensor heterogeneity and application requirements, for achieving the two-fold goal: energy savings with satisfactory detection performance. An optimal dynamic programming algorithm for the sensor selection problem is also derived. Important properties of the cost functionals are derived and used to design three approximation algorithms, which offer near optimal performance with significant complexity reduction. Simulations on real-world data show energy gains as high as 68 in comparison to an equal allocation scheme with probability of detection error on the order of 10-4.
Active sensor scheduling and adaptation @cite_7 encompass applications such as target tracking @cite_6 @cite_8 , physical activity detection @cite_12 , and sequential hypothesis testing @cite_5 . All these prior works including ours @cite_18 @cite_24 @cite_21 assume that the underlying state is given by nature and is not controlled. In contrast, in this work, states are affected by scheduling decisions, via interference and collisions generated by the SUs to the PUs, and we design joint sensing, estimation and scheduling schemes in wireless networks, which account for the cost of acquisition of state information and its impact on the overall network performance. Complexity reduction of POMDPs via exponential family principal components analysis enables planning on a small dimensional manifold in @cite_15 . Model reduction of complex Markov chain models using the KLD as a metric is investigated in @cite_22 . In contrast, we develop a belief compression method based on Neyman-Pearson formulation of the compressive spectrum sensing problem. Our scheme captures relevant features of the dynamic spectrum access problem, without having to learn key statistics. As in @cite_22 , the KLD measure is also used to project the true belief onto the low-dimensional manifold.