Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Partially observable Markov decision processes (POMDPs) are a widely-used framework to model decision-making with uncertainty about the environment and under stochastic outcome. In conventional POMDP models, the observations that the agent receives originate from fixed known distribution. However, in a variety of real-world scenarios the agent has an active role in its perception by selecting which observations to receive. Due to combinatorial nature of such selection process, it is computationally intractable to integrate the perception decision with the planning decision. To prevent such expansion of the action space, we propose a greedy strategy for observation selection that aims to minimize the uncertainty in state. We develop a novel point-based value iteration algorithm that incorporates the greedy strategy to achieve near-optimal uncertainty reduction for sampled belief points. This in turn enables the solver to efficiently approximate the reachable subspace of belief simplex by essentially separating computations related to perception from planning. Lastly, we implement the proposed solver and demonstrate its performance and computational advantage in a range of robotic scenarios where the robot simultaneously performs active perception and planning. In the era of information explosion it is crucial to develop decision-making platforms that are able to judiciously extract useful information to accomplish a defined task. The importance of mining useful data appears in many applications including artificial intelligence, robotics, networked systems and Internet of things. Generally in these applications, a decision-maker, called an agent, must exploit the available information to compute an optimal strategy toward a given objective. Partially observable Markov decision processes (POMDPs) provide a framework to model sequential decision-making with partial perception of the environment and under stochastic outcomes. The flexibility of POMDPs and their variants in modeling real-world problems has led to extensive research on developing efficient algorithms for finding near-optimal policies. Nevertheless, the majority of previous work on POMDPs either deal with sole perception or sole planning. While independent treatment of perception and planning deteriorates performance, an integrated approach usually becomes computationally intractable. Thereupon, one must establish a trade-off between optimality and tractability when determining how much perception and planning rely on each other. We show that by restricting the perception to the class of subset selection problems and exploiting submodular optimization techniques, it is possible to partially decouple computing perception and planning policies while considering their mutual effect on the overall policy value. In this work, we consider joint perception and planning in POMDPs. More specifically, we consider an agent that decides about two sets of actions; perception actions and planning actions. The perception actions, such as employing a sensor, only affect the belief of the agent regarding the state of the environment. The planning actions, such as choosing navigation direction, are the ones that affect the transition of the environment from one state to another. In subset selection problems, at each time step, due to power, processing capability, and cost constraints, the agent can pick a subset of available information sources along a planning action. The subset selection problem arise in various applications in control systems and signal processing, in wireless sensor networks, as well as machine learning BID7 and have been widely-studied (; BID11 . However, the previous work on sensor selection problems assume that the planning strategy is known, while in this work, we simultaneously learn a selection strategy and a planning strategy. Exact POMDP solvers optimize the value function over all reachable belief points. However, finding exact solution to POMDPs is PSPACE-complete BID18 which deems solving even small POMDPs computationally intractable. This has led to extensive search for nearoptimal algorithms. A common technique is to sample a finite set of belief points that approximate the reachable subspace of belief and apply value iteration over this set, e.g., (; BID3 BID14 ; ; BID19 . BID19 proved that the errors due to belief sampling is bounded where the bound depends on the density of the belief set. A well-established offline POMDP solver is SARSOP BID13 . SARSOP, similar to HSVI , aims to minimize the gap between the lower and upper bounds on the value function by guiding the sampling toward the belief points that are reachable under union of optimal policies. In this paper, we show that the proposed greedy observation selection scheme leads to belief points that are on expectation close to the ones from the optimal (with respect to uncertainty reduction) set of observations, and hence value loss is small. An instance of active perception is dynamic sensor selection. BID12 proposes a reinforcement learning approach that uses Rènyi divergence to compute utility of sensing actions. BID8 formulated a single step sensor selection problem as semi-definite programming, however, it lacks theoretical guarantee. In Kalman filtering setting, developed a greedy selection scheme with near-optimal guarantee to minimize log-determinant of the error covariance matrix of estimated state. Some prior work such as (; ; BID16 model active perception as a POMDP. However, the most relevant work to ours are that of BID0 ;). BID0 proposed ρPOMDP framework where the reward depends on entropy of the belief. introduced POMDP-IR where the reward depends on accurate prediction about the state. established an equivalence property between ρPOMDP and POMDP-IR. Furthermore, they employed the submodularity of value function, under some conditions, to use greedy scheme for sensor selection. The main difference of our work is that we consider active perception as a means to accomplishing the original task while in these work, the active perception is the task itself and hence the POMDP rewards are metrics to capture perception quality. The problem of selecting an optimal set of sensors from a ground set under cardinality constraint is NP-hard . This hardness has motivated design of greedy algorithms since they make polynomial oracle calls to the objective function. Additionally, if the objective function is monotone non-decreasing and submodular, BID17 showed that a greedy selection achieves (1 − 1/e) approximation factor. BID15 and BID7 developed randomized greedy schemes that accelerate the selection process for monotone submodular and weak-submodular objective functions, respectively. BID11; BID10 have introduced different submodular information-theoretic objectives for greedy selection and have studied the theoretical guarantees of their maximization under different constraints. Here, we use the entropy of belief to capture the level of uncertainty in state and aim to select a subset of sensors that leads to maximum expected reduction in entropy. We employ the monotonicity and submodularity of the proposed objective to establish near-optimal approximation factor for entropy minimization. A summary of our contributions are as follows:• Formulating the active perception problem for POMDPs: We introduce a new mathematical definition of POMDPs, called AP 2 -POMDP, that captures active perception as well as planning. The objective is to find deterministic belief-based policies for perception and planning such that the expected discounted cumulative reward is maximized.• Developing a perception-aware point-based value iteration algorithm: To solve AP 2 -POMDP, we develop a novel point-based method that approximates the value function using a finite set of belief points. Each belief point is associated with a perception action and a planning action. We use the near-optimal guarantees for greedy maximization of monotone submodular functions to compute the perception action while the planning action is the of Bellman optimality equation. We further prove that greedy perception action leads to an expected reward that is close to that of optimal perception action. This section starts by giving an overview of the related concepts and then stating the problem. The standard POMDP definition models does not capture the actions related to perception. We present a different definition which we call AP 2 -POMDP as it models active perception actions as well as original planning actions. The active perception actions determine which subset of sensors (observations) the agent should receive. We restrict the set of states, actions, and observations to be discrete and finite. We formally define an AP 2 -POMDP below. DISPLAYFORM0 • S is the finite set of states.• A = A pl × A pr is the finite set of actions with A pl being the set of planning actions and A pr being the set of perception actions. A pr = {δ ∈ {0, 1} n | |δ| 0 ≤ k} constructs an n-dimensional lattice. Each component of an action δ ∈ A pr determines whether the corresponding sensor is selected.• k is the maximum number of sensor to be selected. DISPLAYFORM1 is the probabilistic transition function.• Ω = Ω 1 × Ω 2 ×... × Ω n is the partitioned set of observations, where each Ω i corresponds to the set of measurements observable by sensor i.• O: S × A × Ω → is the probabilistic observation function.• R: S × A pl → R is the reward function, and DISPLAYFORM2 At each time step, the environment is in some state s ∈ S. The agent takes an action β ∈ A pl that causes a transition to a state s ∈ S with probability P r(s |s, β) = T (s, β, s). At the same time step, the agent also picks k sensors by δ ∈ A pr. Then it receives an observation ω ∈ Ω with probability P r(ω|s, β, δ) = O(s, β, δ, ω), and a scalar reward R(s, β). Assumption 1. We assume that the observations from sensors are mutually independent given the current state and the previous action, i.e., ∀I 1, I 2 ⊆ {1, 2, . . ., n}, I 1 ∩ I 2 = ∅: DISPLAYFORM3 Let ζ(δ) = {i|δ(i) = 1} to denote the subset of sensors that are selected by δ. If Assumption 1 holds, then: DISPLAYFORM4 DISPLAYFORM5 The belief of the agent at each time step, denoted by b t is the posterior probability distribution of states given the history of previous actions and observations, i.e., h t = (a 0, ω 1, a 1, . . ., a t−1, ω t).A well-known fact is that due to Markovian property, a sufficient statistics to represent history of actions and observations is belief (Åström, 1965;). Given the initial belief b 0, the following update equation holds between previous belief b and the belief b a,ω b after taking action a = (β, δ) and receiving observation ω: DISPLAYFORM6 The goal is to learn a deterministic policy to maximize E[DISPLAYFORM7 . A deterministic policy is a mapping from belief to actions π : B → A, where B is the set of belief states. Note that B constructs a (|S| − 1)-dimensional probability simplex. The POMDP solvers apply value iteration , a dynamic programming technique, to find the optimal policy. Let V be a value function that maps beliefs to values in R. The following recursive expression holds for V: DISPLAYFORM8 The value iteration converges to the optimal value function V * which satisfies the Bellman's optimality equation BID2. Once the optimal value function is learned, an optimal policy can be derived. An important outcome of FORMULA8 is that at any horizon, the value function is piecewise-linear and convex (PWLC) and hence, can be represented by a finite set of hyperplanes. Each hyperplane is associated with an action. Let α's to denote the corresponding vectors of the hyperplanes and let Γ t to be the set of α vectors at horizon t. Then, DISPLAYFORM9 This fact has motivated approximate point based solvers that try to approximate the value function by updating the hyperplanes over a finite set of belief points. Since the proposed algorithm is founded upon the theoretical from the field of submodular optimization, here, we overview the necessary definitions. Let X to denote a ground set and f a set function that maps an input set to a real number. DISPLAYFORM0 is the marginal value of adding element i to set T 1.Monotonicity states that adding elements to a set increases the function value while submodularity refers to diminishing returns property. Having stated the required , next, we state the problem. Problem 1. Consider a AP 2 -POMDP P = (S, A, k, T, Ω, O, R, γ) and an initial belief b 0. We aim to learn a policy π(b) = (β, δ) such that the expected discounted cumulative reward is maximized, i.e, It is worth noting that the perception actions affect the belief and subsequently the received reward in the objective function. DISPLAYFORM0 3 ACTIVE PERCEPTION WITH GREEDY SCHEME For variety of performance metrics, finding an optimal subset of sensors poses a computationally challenging combinatorial optimization problem that is NP-hard. Augmenting POMDP planning actions with n k active perception actions in a combinatorial expansion of the action space. Thereupon, it is infeasible to directly apply existing POMDP solvers to Problem 1. Instead of concatenating both sets of actions and treating them similarly, we propose a greedy strategy for selecting perception actions that aims to pick the sensors that in minimal uncertainty about the state. The key enabling factor is that the perception actions does not affect the transition, consequently, we can decompose the single-step belief update in into two steps: DISPLAYFORM1 This in turn implies that after a transition is made, the agent should pick a subset of observations that lead to minimal uncertainty in b DISPLAYFORM2 To quantify uncertainty in state, we use Shannon entropy of the belief. For a discrete random variable x, the entropy is defined as H(x) = − i p(x i) log p(x i). An important property of entropy is its strict concavity on the simplex of belief points, denoted by ∆ B (Cover & BID4 . Further, the entropy is zero at the vertices of ∆ B and achieves its maximum, log |S|, at the center of ∆ B that corresponds to uniform distribution, i.e., when the uncertainty about the state is the highest. FIG0 demonstrates the entropy and its level sets for |S| = 3. Since the observation values are unknown before selecting the sensors, we optimize conditional entropy that yields the expected value of entropy over all possible observations. For discrete random variables x and y, conditional entropy is defined as DISPLAYFORM3 . Subsequently, with some algebraic manipulation, it can be shown that the conditional entropy of state given current belief with respect to δ is: DISPLAYFORM4 where ζ(δ) = {i 1, i 2, . . ., i k}. It is worth mentioning that b is the current distribution of s and is explicitly written only for the purpose of better clarity, otherwise, H(s|b, δ) = H(s|δ).To minimize entropy, we define the objective function as the following set function: DISPLAYFORM5 and the optimization problem as: DISPLAYFORM6 We propose a greedy algorithm, outlined in Algorithm 1 to find a near-optimal, yet efficient solution to. The algorithm takes as input the agent's belief and planning action. Then it iteratively adds elements from the ground set (set of all sensors) whose marginal gain with respect to f is maximal and terminates when k observations are selected. Algorithm 1 Greedy policy for perception action DISPLAYFORM7 ζ ← ζ ∪ {j *} 7: end for 8: return δ corresponding to ζ. Next, we derive a theoretical guarantee for the performance of the proposed greedy algorithm. The following lemma states the required properties to prove the theorem. The proof of the lemma follows from monotonicity and submodularity of conditional entropy BID9. See the appendix for the complete proof. Lemma 1. Let Ω = {ω 1, ω 2, . . ., ω n} to represent a set of observations of the state s that conditioned on the state, are mutually independent (Assumption 1 holds). Then, f (ζ), defined in, realizes the following properties: DISPLAYFORM8 2. f is monotone nondecreasing, and 3. f is submodular. The above lemma enables us to establish the approximation factor using the classical analysis in BID17. Theorem 1. Let ζ * to denote the optimal subset of observations with regard to objective function f (ζ), and ζ g to denote the output of the greedy algorithm in Algorithm 1. Then, the following performance guarantee holds: DISPLAYFORM9 Remark 1. Intuitively, one can interpret the minimization of conditional entropy as pushing the agent's belief toward the boundary of the probability simplex ∆ B. Due to convexity of POMDP value function on ∆ B , this in turn implies that the agent is moving toward regions of belief space that have higher value. Although Theorem 1 proves that the entropy of the belief point achieved by the greedy algorithm is close to the entropy of the belief point from the optimal solution, the key question is whether the value of these points are close. We assess this question in the following and show that at each time step, on expectation, the value from greedy scheme is close to the value from optimal observation selection with regard to. To that end, we first show that the distance between the two belief points is upper-bounded. Thereafter, using a similar analysis as that of BID19, we conclude that the difference between value function at these two points is upper-bounded. Theorem 2. Let the agent's current belief to be b and its planning action to be β. Consider the optimization problem in, and let δ * and δ g to denote the optimal perception action and the perception action obtained by the greedy algorithm, respectively. It holds that: Proof. We outline the sketch of the proof and bring the complete proof in the appendix. First, we show that minimizing conditional entropy of posterior belief is equivalent to maximizing KullbackLeibler (KL-) divergence between current belief and the posterior belief, i.e., D KL (b δ,ω b b). Next, we exploit Pythagorean theorem for KL-divergence alongside its convexity to find a relation between DISPLAYFORM10 DISPLAYFORM11 ). Afterwards, using Pinkster's inequality, we prove that the total variation distance between b DISPLAYFORM12 Proof. The proof is omitted for brevity. See the appendix for the proof. In this section, we propose a novel point-based value iteration algorithm to approximate the value function for AP 2 -POMDPs. The algorithm relies on the performance guarantee of the proposed greedy observation selection in previous section. Before describing the new point-based solver, we first overview how point-based solvers operate. Algorithm 2 outlines the general procedure for a point-based solver. It starts with an initial set of belief points B 0 and their corresponding α vectors. Then it performs a Bellman backup for each point to update α vectors. Next, it prunes α vectors to remove dominated ones. Afterwards, it samples a new set of belief points and repeats these steps until convergence or other termination criteria is met. The difference between solvers is in how they apply sampling and pruning. The sampling step usually depends on the reachability tree of belief space, see FIG2. The state-of-the-art point-based methods do not traverse the whole reachability tree, but they try to have enough sample points to provide a good coverage of the reachable space. Note that the combinatorial number of actions due to observation selection highly expand the size of the reachability tree. To avoid dealing with perception actions in the reachability tree, we apply the greedy scheme to make the choice of δ deterministically dependent on β and previous belief. To that end, we modify the BackUp step of point-based value iteration. The proposed BackUp step can be combined with any sampling and pruning method in other solvers, such as the ones developed by Spaan & Vlassis FORMULA6 In point-based solver each witness belief point is associated with an α vector and an action. Nevertheless, for AP 2 -POMDPs, each witness point is associated with two actions, β and δ. We compute δ based on greedy maximization of so that given b and β, δ is uniquely determined. Henceforth, we can rewrite using to obtain: DISPLAYFORM0 whereδ = argmax δ∈A pr f (ζ(δ)) and f is computed atb β b. This way, we can partially decouple the computation of perception action from the computation necessary for learning the planning policy. Inspired by the in the previous section, we propose the BackUp step detailed in Algorithm 3 to compute the new set of α vectors from the previous ones using Bellman backup operation. What distinguishes this algorithm from conventional Bellman backup step is the inclusion of perception actions. Basically, we need to compute the greedy perception action for each belief point and each action (Line 7). This in turn affects computation of Γ b,β,ω t as it represents a different set for each belief point (Lines 9-13). However, notice that this added complexity is significantly lower than concatenating the combinatorial perception actions with the planning actions and using conventional point-based solvers. See the appendix for detailed complexity analysis. To evaluate the proposed algorithm for active perception and planning, we developed a point-based value iteration solver for AP 2 -POMDPs. We initialized the belief set by uniform sampling from ∆ B BID6. To focus on the effect of perception, we did not apply a sampling step, i.e, the belief set is fixed throughout the iterations. However, one can integrate any sampling method such as the ones proposed by; BID13. The α vectors are initialized by 1 1−γ min s,a R(s, a).Ones(|S|) . Furthermore, to speedup the solver, one can employ a randomized backup step, as suggested by. The solver terminates once the difference between value functions in two consecutive iterations falls below a predefined threshold. We also implemented a random perception policy that selects a subset of information sources, uniformly at random, at each backup step. We implemented the solver in Python 2.7 and ran the simulations on a laptop with 2.0 GHz Intel Core i7-4510U CPU and with 8.00 GB RAM. The first scenario models a robot that is moving in a 1-D discrete environment. The robot can only move to adjacent cells and its navigation actions are A pl = {lef t, right, stop}. The robot's transitions are probabilistic due to possible actuation errors. The robot does not have any sensor and it relies on a set of cameras for localization. There is one camera at each cell that outputs a probability for b ∈ B t do 7:δ = Greedy argmax δ∈A pr f (ζ(δ)) 8: DISPLAYFORM0 for α ∈ Γ t−1 do 11: DISPLAYFORM1 DISPLAYFORM2 DISPLAYFORM3 Figure 3: The robot moves in a grid while communicating with the cameras to localize itself. There is a camera at each state on the perimeter. The accuracy of measurements made by each camera depends on the distance of the camera from that state. The robot's objective is to reach the goal state, labeled by star, while avoiding the obstacles.distribution over the position of the robot. The camera's certainty is higher when the robot's position is close to it. To model the effect of robot's position on the accuracy of measurements, we use a binomial distribution with its mean at the cell that camera is on. The binomial distribution represents the state-dependent accuracy. The robot's objective is to reach an specific cell in the map. For that purpose, at each time step, the robot picks a navigation action and selects k camera from the set of n cameras. After the solver terminates, we evaluate the computed policy. To that end, we run 1000 iterations of Monte Carlo simulations. The initial state of the robot is the origin of the map and its initial belief is uniform over the map. Figure 4 -(a) demonstrates the discounted cumulative reward, averaged over 1000 Monte Carlo runs, for random selection of 1 and 2 information sources, and greedy selection of 1 and 2 information sources. It can be seen that the greedy perception policy significantly outperforms the random perception. entropy of greedy perception, compared to random perception, shows less uncertainty of the robot when taking planning actions. See the appendix for further . The second setting is a variant of first scenario where the map is 2-D. Therefore the navigation actions of robot are A pl = {up, right, down, lef t, stop}. The rest of the setting is similar to 1-D case, except the cameras' positions, as they are now placed around the perimeter of the map. Additionally, the robot has to now avoid the obstacles in the map. The reward is 10 at the goal state. -4 at the obstacles, and -1 in other states. We applied the proposed point-based solver with both random perception and greedy perception on the 2-D example. Next, we let the robot to run for a horizon of 25 steps and we terminated the simulations once the robot reached the goal. Figure 5 illustrates the normalized frequency of visiting each state for each perception algorithm. It can be seen that the policy learned by greedy active perception leads to better obstacle avoidance. See the appendix for further . In this paper, we studied joint active perception and planning in POMDP models. To capture the structure of the problem, we introduced AP 2 -POMDPs that have to pick a cardinality-constrained subset of observations, in addition to original planning action. To tackle the computational challenge of adding combinatorial actions, we proposed a greedy scheme for observation selection. The greedy scheme aims to minimize the conditional entropy of belief which is a metric of uncertainty about the state. We provided a theoretical analysis for the greedy algorithm that led to boundedness of value function difference between optimal entropy reduction and its greedy counterpart. Furthermore, founded upon the theoretical guarantee of greedy active perception, we developed a point-based value iteration solver for AP 2 -POMDPs. The idea introduced in the solver to address active perception is general and can be applied on state-of-the-art point-based solvers. Lastly, we implemented and evaluated the proposed solver on a variety of robotic navigation scenarios. In this section, we provide the proofs to the lemmas and theorems stated in the paper. First, in the next lemma, we show that the objective function defined for uncertainty reduction has the required properties for the analysis by BID17, namely being normalized, monotone, and submodular. Lemma 1. Let Ω = {ω 1, ω 2, . . ., ω n} to represent a set of observations of the state s that conditioned on the state, are mutually independent (Assumption 1 holds). Then, f (ζ), defined in, realizes the following properties: DISPLAYFORM0 2. f is monotone nondecreasing, and 3. f is submodular. Proof. Notice thatb β b is explicitly present to determine the current distribution of s and it is not a random variable. Therefore, for simplicity, we omit that in the following proof. It is clear that DISPLAYFORM1 To prove the monotonicity, consider ζ 1 ⊂ [n] and j ∈ [n]\ζ 1. Then, DISPLAYFORM2 where (a) and (c) are due to Bayes' rule for entropy, (b) follows from the conditional independence assumption and joint entropy definition, (d) is due to the conditional independence assumption, and (e) stems from the fact that conditioning does not increase entropy. The monotonicity of the objective function means that if the number of obtained observations are higher, the conditional entropy will be lower, and hence, on expectation, the uncertainty in the state will be lower. Furthermore, from the third line of above proof, we can derive the marginal gain, i.e., the value of adding one sensor, as: DISPLAYFORM3 To prove submodularity, let ζ 1 ⊆ ζ 2 ⊂ [n] and j ∈ [n]\ζ 2. Then, DISPLAYFORM4 where (a) is based on the fact that conditioning does not increase entropy, and (b) from ζ 1 ⊆ ζ 2. The submodularity (diminishing returns property) of objective function indicates that as the number of obtained observations increases, the value of adding a new observation will decrease. In the next theorem, we exploit the properties of the proposed objective function to analyze the performance of the greedy scheme. Theorem 1. Let ζ * to denote the optimal subset of observations with regard to objective function f (ζ), and ζ g to denote the output of the greedy algorithm in Algorithm 1. Then, the following performance guarantee holds: DISPLAYFORM5 Proof. The properties of f stated in Lemma 1 along the theoretical analysis of greedy algorithm by BID17 yields DISPLAYFORM6 Using the definition of f (ζ) and rearranging the terms, we obtain the desired . Before stating the proof to Theorem 2, that bounds the distance of belief points from the greedy and optimal entropy minimization algorithms, we need to present a series of propositions and lemmas. Mutual information between two random variables is a positive and symmetric measure of their dependence and is defined as: DISPLAYFORM0 Mutual information, due to its monotonicity and submodularity, has inspired many subset selection algorithms BID10. In the following proposition, we express the relation between conditional entropy and mutual information. Proposition 1. Minimizing conditional entropy of the state with respect to a set of observations is equivalent to maximizing the mutual information of state and the set of observations. This equivalency is due to the definition of mutual information, i.e., I(s; DISPLAYFORM1 and the fact that H(s) is computed atb β b which amounts to a constant value that does not affect selection procedure. Additionally, notice that is the same as the definition of normalized objective function of greedy algorithm in.Another closely-related information-theoretic concept is Kullback-Leibler (KL-) divergence. The KL-divergence, also known as relative entropy, is a non-negative and non-symmetric measure of difference between two distributions. The KL-divergence from q(x) to p(x) is: DISPLAYFORM2 The following relation between mutual information and KL-divergence exists: DISPLAYFORM3 which allows us to state the next proposition. Proposition 2. The mutual information of state and a set of observations is the expected value of the KL-divergence from prior belief to posterior belief over all realizations of observations, i.e., I(s;, ω ∼ i∈ζ * Ω i to denote prior belief (after taking planning action), posterior belief after greedy perception action, and posterior belief after optimal perception action, respectively. So far, we have established a relation between minimizing the conditional entropy of posterior belief and maximizing the expected KL-divergence from prior belief to posterior belief, i.e., D KL (p g p 0) (See Proposition 1 and Proposition 2). To relate DISPLAYFORM4 DISPLAYFORM5, we state the next lemma. But first, we bring information-geometric definitions necessary for proving the lemma. Definition 4. Let p to be a probability distribution over a finite alphabet. An I-sphere with center p and radius ρ is defined as: DISPLAYFORM6 is called the I-projection of p on Λ. Lemma 2. Instate the definition of p 0, p g, and p *. The following inequality holds on expectation: DISPLAYFORM7 Proof. Consider the set Λ g = {p ∈ ∆ B |H(p) ≥ H(p g) that contain probability distributions whose entropy is lower-bounded by entropy of p g. Since entropy is concave over ∆ B, its hypographs are convex. Consequently Λ g, the projection of a hypograph onto ∆ B, is a convex set. Furthermore, due to monotonicity of conditional entropy, i.e., expected value of entropy over observations, we know that p 0 ∈ Λ g. Besides, Due to optimality of ζ *, it holds that DISPLAYFORM8 which in turn yields p * ∈ ∆ B \Λ g. FIG10 demonstrates these facts for an alphabet of size 3. p g is the I-projection of p * on Λ g. Therefore, by exploiting the analogue of Pythagoras' theorem for KL-divergence (Csiszár, 1975), we conclude: DISPLAYFORM9 A direct of the above lemma, after taking the expectation over i∈[n] ω i, is: DISPLAYFORM10 In the following theorem, we use the stated lemma to bound the expected KL-divergence distance between greedy and optimal selection strategies. Theorem 4. The KL-divergence between p g and p * is upper-bounded, i.e., DISPLAYFORM11 where C 3 is a constant. Proof. Notice that while KL-divergence is not symmetric, the following fact still holds: where (a) follows from the fact that α * is the gradient of the optimal value function, (b) is due toHölder's inequality, and (c) is the of Theorem 2. Taking C 2 = C 1 max{|Rmax|,|Rmin|} 1−γ yields the desired . In this section, we compare the computational complexity of a point-based value iteration method that works with the concatenated action space, with the computational complexity of the proposed point-based method that picks the perception actions based on a greedy approach. First, we compute the computations required for a single backup step in the point-based method with concatenated action space. To that end, consider a fixed set of sampled belief points B. Let Γ to denote the current set of α vectors. Further, for the simplicity of analysis, assume that the number of possible observations from each information source is |Ω i | =Ω, ∀i ∈ [n]. The cardinality of a concatenated action space is |A| = |A pr ||A pl | = n k |A pl |. Therefore, the complexity of a single backup step would be O(On the other hand, applying greedy algorithm to pick a perception action requires O(n × k) calls to an oracle that computes the objective function (or equivalently, the marginal gain). Here the objective function is the conditional entropy whose complexity with a naive approach in the k th iteration is O(Ω k × |S| 2). Therefore, applying Algorithm 3 as the backup step leads to O(|A pl | × |B| × n × k ×Ω k × |S| 2 + |A pl | × |B| ×Ω k × |Γ| × |S| 2 + |B| × |A pl | × |S| ×Ω k) operations. Hence, the proposed approach, as a of exploiting the structure of action space, would lead to significant computational gain, especially for large n. Figure 7 depicts the history of the belief entropy for the 2-D navigation when applying the proposed point-based method with random selection step and the proposed greedy selection step. As expected, the greedy selection leads to smaller entropy and hence, less uncertainty about the state. The corresponding average discounted cumulative reward after running 1000 Monte Carlo simulations is -18.8 for point-based value iteration with random selection step and -14.5 for point-based value iteration with greedy selection step, which demonstrates the superiority of the proposed method. We further analyzed the effect of number of selected cameras on the agent's performance in the 1-D navigation scenario. Figure 8 illustrates the value function for a subset of sampled belief points after the algorithm has been terminated. It can be seen that the diminishing returns property of entropy with respect to number of selected observations is propagated through the value function as well.

Represent the TLDR for finding the corresponding scientific article

We develop a point-based value iteration solver for POMDPs with active perception and planning tasks.

Represent this text to find a much much longer article about it with say hundreds-thousands of words:

The paper designs two algorithms for the stochastic AUC maximization problem with state-of-the-art complexities when using deep neural network as predictive model, which are also verified by empirical studies.


Question: represent the article for finding a single-sentence tldr

Question: Deep neural networks, in particular convolutional neural networks, have become highly effective tools for compressing images and solving inverse problems including denoising, inpainting, and reconstruction from few and noisy measurements. This success can be attributed in part to their ability to represent and generate natural images well. Contrary to classical tools such as wavelets, image-generating deep neural networks have a large number of parameters---typically a multiple of their output dimension---and need to be trained on large datasets. In this paper, we propose an untrained simple image model, called the deep decoder, which is a deep neural network that can generate natural images from very few weight parameters. The deep decoder has a simple architecture with no convolutions and fewer weight parameters than the output dimensionality. This underparameterization enables the deep decoder to compress images into a concise set of network weights, which we show is on par with wavelet-based thresholding. Further, underparameterization provides a barrier to overfitting, allowing the deep decoder to have state-of-the-art performance for denoising. The deep decoder is simple in the sense that each layer has an identical structure that consists of only one upsampling unit, pixel-wise linear combination of channels, ReLU activation, and channelwise normalization. This simplicity makes the network amenable to theoretical analysis, and it sheds light on the aspects of neural networks that enable them to form effective signal representations. Data models are central for signal and image processing and play a key role in compression and inverse problems such as denoising, super-resolution, and compressive sensing. These data models impose structural assumptions on the signal or image, which are traditionally based on expert knowledge. For example, imposing the assumption that an image can be represented with few non-zero wavelet coefficients enables modern (lossy) image compression BID1 and efficient denoising BID6.In recent years, it has been demonstrated that for a wide range of imaging problems, from compression to denoising, deep neural networks trained on large datasets can often outperform methods based on traditional image models BID19 BID0 BID18 BID4 BID22. This success can largely be attributed to the ability of deep networks to represent realistic images when trained on large datasets. Examples include learned representations via autoencoders BID12 and generative adversarial models BID8. Almost exclusively, three common features of the recent success stories of using deep neural network for imaging related tasks are i) that the corresponding networks are over-parameterized (i.e., they have much more parameters than the dimension of the image that they represent or generate), ii) that the networks have a convolutional structure, and perhaps most importantly, iii) that the networks are trained on large datasets. An important exception that breaks with the latter feature is a recent work by Ulyanov et al. BID20, which provides an algorithm, called the deep image prior (DIP), based on deep neural networks, that can solve inverse problems well without any training. Specifically, Ulyanov et al. demonstrated that fitting the weights of an over-parameterized deep convolutional network to a single image, together with strong regularization by early stopping of the optimization, performs competitively on a variety of image restoration problems. This is surprising because it does not involve a training dataset, which means that the notion of what makes an image'natural' is contained in a combination of the network structure and the regularization. However, without regularization the proposed network has sufficient capacity to overfit to noise, preventing meaningful image denoising. These prior works demonstrating the effectiveness of deep neural networks for image generation beg the question whether there may be a deep neural network model of natural images that is underparameterized and whose architecture alone, without algorithmic assistance, forms an efficient model for natural images. In this paper, we propose a simple image model in the form of a deep neural network that can represent natural images well while using very few parameters. This model thus enables image compression, denoising, and solving a variety of inverse problems with close to or state of the art performance. We call the network the deep decoder, due to its resemblance to the decoder part of an autoencoder. The network does not require training, and contrary to previous approaches, the network itself incorporates all assumptions on the data, is under-parameterized, does not involve convolutions, and has a simplicity that makes it amenable to theoretical analysis. The key contributions of this paper are as follows:• The network is under-parameterized. Thus, the network maps a lower-dimensional space to a higher-dimensional space, similar to classical image representations such as sparse wavelet representations. This feature enables image compression by storing the coefficients of the network after its weights are optimized to fit a single image. In Section 2, we demonstrate that the compression is on-par with wavelet thresholding BID1, a strong baseline that underlies JPEG-2000. An additional benefit of underparameterization is that it provides a barrier to overfitting, which enables regularization of inverse problems.• The network itself acts as a natural data model. Not only does the network require no training (just as the DIP BID20); it also does not critically rely on regularization, for example by early stopping (in contrast to the DIP). The property of not involving learning has at least two benefits: The same network and code is usable for a number of applications, and the method is not sensitive to a potential misfit of training and test data.• The network does not use convolutions. Instead, the network does have pixelwise linear combinations of channels, and, just like in a convolutional neural network, the weights are shared among spatial positions. Nonetheless, these are not convolutions because they provide no spatial coupling between pixels, despite how pixelwise linear combinations are sometimes called'1x1 convolutions.' In contrast, the majority of the networks for image compression, restoration, and recovery have convolutional layers with filters of nontrivial spatial extent BID19; BID0; BID18; BID4 BID22. This work shows that relationships characteristic of nearby pixels of natural images can be imposed directly by upsampling layers.• The network only consists of a simple combination of few building blocks, which makes it amenable to analysis and theory. For example, we prove that the deep decoder can only fit a small proportion of noise, which, combined with the empirical observation that it can represent natural images well, explains its denoising performance. The remainder of the paper is organized as follows. In Section 2, we first demonstrate that the deep decoder enables concise image representations. We formally introduce the deep decoder in Section 3. In Section 4, we show the performance of the deep decoder on a number of inverse problems such as denoising. In Section 5 we discuss related work, and finally, in Section 6 we provide theory and explanations on what makes the deep decoder work. Intuitively, a model describes a class of signals well if it is able to represent or approximate a member of the class with few parameters. In this section, we demonstrate that the deep decoder, an untrained, non-convolutional neural network, defined in the next section, enables concise representation of an image-on par with state of the art wavelet thresholding. The deep decoder is a deep image model G: R N → R n, where N is the number of parameters of the model, and n is the output dimension, which is (much) larger than the number of parameters (n N). The parameters of the model, which we denote by C, are the weights of the network, and not the input of the network, which we will keep fixed. To demonstrate that the deep decoder enables concise image representations, we choose the number of parameters of the deep decoder, N, such that it is a small fraction of the output dimension of the deep decoder, i.e., the dimension of the images 1.We draw 100 images from the ImageNet validation set uniformly at random and crop the center to obtain a 512x512 pixel color image. For each image x *, we fit a deep decoder model G(C) by minimizing the loss DISPLAYFORM0 with respect to the network parameters C using the Adam optimizer. We then compute for each image the corresponding peak-signal-to-noise ratio, defined as 10 log 10 (1/MSE), where DISPLAYFORM1 is the image generated by the network, and x * is the original image. We compare the compression performance to wavelet compression BID1 by representing each image with the N -largest wavelet coefficients. Wavelets-which underly JPEG 2000, a standard for image compression-are one of the best methods to approximate images with few coefficients. In Fig. 1 we depict the . It can be seen that for large compression factors (3 · 512 2 /N = 32.3), the representation by the deep decoder is slightly better for most images (i.e., is above the red line), while for smalle compression factors (3 · 512 2 /N = 8), the wavelet representation is slightly better. This experiment shows that deep neural networks can represent natural images well with very few parameters and without any learning. The observation that, for small compression factors, wavelets enable more concise representations than the deep decoder is intuitive because any image can be represented exactly with sufficiently many wavelet coefficients. In contrast, there is no reason to believe a priori that the deep decoder has zero representation error because it is underparameterized. The main point of this experiment is to demonstrate that the deep decoder is a good image model, which enables applications like solving inverse problems, as in Section 4. However, it also suggest that the deep decoder can be used for lossy image compression, by quantizing the coefficients C and saving the quantized coefficients. In the appendix, we show that image representations of the deep decoder are not sensitive to perturbations of its coefficients, thus quantization does not have a detrimental effect on the image quality. Deep networks were used successfully before for the compression of images BID19 BID0 BID18. In contrast to our work, which is capable of compressing images without any learning, the aforementioned works learn an encoder and decoder using convolutional recurrent neural networks BID19 and convolutional autoencoders BID18 based on training data. We consider a decoder architecture that transforms a randomly chosen and fixed tensor B 0 ∈ R n0×k0 consisting of k 0 many n 0 -dimensional channels to an n d × k out dimensional image, where k out = 1 for a grayscale image, and k out = 3 for an RGB image with three color channels. Throughout, n i has two dimensions; for example our default configuration has n 0 = 16 × 16 and n d = 512 × 512. The network transforms the tensor B 0 to an image by pixel-wise linearly combining the channels, upsampling operations, applying rectified linear units (ReLUs), and normalizing DISPLAYFORM0... lin. comb., upsampling, ReLU, CN lin. comb., sigmoid Figure 1: The deep decoder (depicted on the right) enables concise image representations, onpar with state-of-the-art wavelet based compression. The crosses on the left depict the PSNRs for 100 randomly chosen ImageNet-images represented with few wavelet coefficients and with a deep decoder with an equal number of parameters. A cross above the red line means the corresponding image has a smaller representation error when represented with the deep decoder. The deep decoder is particularly simple, as each layer has the same structure, consisting of a pixel-wise linear combination of channels, upsampling, ReLU nonlinearities, and channelwise normalization (CN). the channels. Specifically, the channels in the (i + 1)-th layer are given by DISPLAYFORM1 Here, the coefficient matrices C i ∈ R ki×ki+1 contain the weights of the network. Each column of the tensor B i C i ∈ R ni×ki+1 is formed by taking linear combinations of the channels of the tensor B i in a way that is consistent across all pixels. Then, cn(·) performs a channel normalization operation which is equivalent to normalizing each channel individually, and can be viewed as a special case of the popular batch normalization proposed in BID13. Specifically, let Z i = relu(U i B i C i) be the channels in the i-th layer, and let z ij be the j-th channel in the i-th layer. Then channel normalization performs the following transformation: DISPLAYFORM2, where mean and var compute the empirical mean and variance, and γ ij and β ij are parameters, learned independently for each channel, and is a fixed small constant. Learning the parameter γ and β helps the optimization but is not critical. This is a special case of batch normalization with batch size one proposed in BID13, and significantly improves the fitting of the model, just like how batch norm alleviates problems encountered when training deep neural networks. The operator U i ∈ R ni+1×ni is an upsampling tensor, which we choose throughout so that it performs bi-linear upsampling. For example, if the channels in the input have dimensions n 0 = 16×16, then the upsampling operator U 0 upsamples each channel to dimensions 32 × 32. In the last layer, we do not upsample, which is to say that we choose the corresponding upsampling operator as the identity. Finally, the output of the d-layer network is formed as DISPLAYFORM3 where Fig. 1 for an illustration. Throughout, our default architecture is a d = 6 layer network with k i = k for all i, and we focus on output images of dimensions n d = 512 × 512 and number of channels k out = 3. Recall that the parameters of the network are given by C = {C 0, C 1, . . ., C d}, and the output of the network is only a function of C, since we choose the tensor B 0 at random and fix it. Therefore, we write x = G(C). Note that the number of parameters is given by DISPLAYFORM4 DISPLAYFORM5 where the term 2k i corresponds to the two free parameters associated with the channel normalization. Thus, the number of parameters is N = dk 2 + 2dk + 3k. In the default architectures with d = 6 and k = 64 or k = 128, we have that N = 25,536 (for k = 64) and N =100,224 (k = 128) out of an RGB image space of dimensionality 512 × 512 × 3 = 786,432 parameters. We finally note that naturally variations of the deep decoder are possible; for example in a previous version of this manuscript, we applied upsampling after applying the relu-nonlinearity, but found that applying it before yields slightly better . While the deep decoder does not use convolutions, its structure is closely related to that of a convolutional neural network. Specifically, the network does have pixelwise linear combinations of channels, and just like in a convolutional neural network, the weights are shared among spatial positions. Nonetheless, pixelwise linear combinations are not proper convolutions because they provide no spatial coupling of pixels, despite how they are sometimes called 1 × 1 convolutions. In the deep decoder, the source of spatial coupling is only from upsampling operations. In contrast, a large number of networks for image compression, restoration, and recovery have convolutional layers with filters of nontrivial spatial extent BID19; BID0; BID18; BID4; BID22. Thus, it is natural to ask whether using linear combinations as we do, instead of actual convolutions yields better . Our simulations indicate that, indeed, linear combinations yield more concise representations of natural images than p × p convolutions, albeit not by a huge factor. Recall that the number of parameters of the deep decoder with d layers, k channels at each layer, and 1 × 1 convolutions is N (d, k; 1) = dk 2 + 3k + 2dk. If we consider a deep decoder with convolutional layers with filters of size p × p, then the number of parameters is: DISPLAYFORM0 If we fix the number of channels, k, but increase p to 3, the representation error only decreases since we increase the number of parameters (by a factor of approximately 32). We consider image reconstruction as described in Section 2. For a meaningful comparison, we keep the number of parameters fixed, and compare the representation error of a deep decoder with p = 1 and k = 64 (the default architecture in our paper) to a variant of the deep decoder with p = 3 and k = 22, so that the number of parameters is essentially the same in both configurations. We find that the representation of the deep decoder with p = 1 is better (by about 1dB, depending on the image), and thus for concise image representations, linear combinations (1 × 1 convolutions) appear to be more effective than convolutions of larger spatial extent. In this section, we use the deep decoder as a structure-enforcing model or regularizers for solving standard inverse problems: denoising, super-resolution, and inpainting. In all of those inverse problems, the goal is to recover an image x from a noisy observation y = f (x) + η. Here, f is a known forward operator (possibly equal to identity), and η is structured or unstructured noise. We recover the image x with the deep decoder as follows. Motivated by the finding from the previous section that a natural image x can (approximately) be represented with the deep decoder as G(C), we estimate the unknown image from the noisy observation y by minimizing the loss DISPLAYFORM0 with respect to the model parameters C. LetĈ be the of the optimization procedure. We estimate the image asx = G(Ĉ).We use the Adam optimizer for minimizing the loss, but have obtained comparable with gradient descent. Note that this optimization problem is non-convex and we might not reach a global minimum. Throughout, we consider the least-squares loss (i.e., we take · 2 to be the 2 norm), but the loss function can be adapted to account for structure of the noise. We remark that fitting an image model to observations in order to solve an inverse problem is a standard approach and is not specific to the deep decoder or deep-network-based models in general. Specifically, a number of classical signal recovery approaches fit into this framework; for example solving a compressive sensing problem with 1 -norm minimization amounts to choosing the forward operator as f (x) = Ax and minimizing over x in a 1 -norm ball. We start with the perhaps most basic inverse problem, denoising. The motivation to study denoising is at least threefold: First, denoising is an important problem in practice, second, many inverse problem can be solved as a chain of denoising steps BID15 Figure 2: An application of the deep decoder for denoising the astronaut test image. The deep decoder has performance on-par with state of the art untrained denoising methods, such as the DIP method BID20 and the BM3D algorithm BID5.problem is simple to model mathematically, and thus a common entry point for gaining intuition on a new method. Given a noisy observation y = x + η, where η is additive noise, we estimate an image with the deep decoder by minimizing the least squares loss G(C) − y 2 2, as described above. The in Fig. 2 and Table 1 demonstrate that the deep decoder has denoising performance onpar with state of the art untrained denoising methods, such as the related Deep Image Prior (DIP) method BID20 (discussed in more detail later) and the BM3D algorithm BID5. Since the deep decoder is an untrained method, we only compared to other state-of-the-art untrained methods (as opposed to learned methods such as BID22).Why does the deep decoder denoise well? In a nutshell, from Section 2 we know that the deep decoder can represent natural images well even when highly underparametrized. In addition, as a consequence of being under-parameterized, the deep decoder can only represent a small proportion of the noise, as we show analytically in Section 6, and as demonstrated experimentally in FIG2. Thus, the deep decoder "filters out" a significant proportion of the noise, and retains most of the signal. How to choose the parameters of the deep decoder? The larger k, the larger the number of latent parameters and thus the smaller the representation error, i.e., the error that the deep decoder makes when representing a noise-free image. On the other hand, the smaller k, the fewer parameters, and the smaller the range space of the deep decoder G(C), and thus the more noise the method will remove. The optimal k trades off those two errors; larger noise levels require smaller values of k (or some other form of regularization). If the noise is significantly larger, then the method requires either choosing k smaller, or it requires another means of regularization, for example early stopping of the optimization. For example k = 64 or 128 performs best out of {32, 64, 128}, for a PSNR of around 20dB, while for a PSNR of about 14dB, k = 32 performs best. We next super-resolve images with the deep denoiser. We define a forward model f that performs downsampling with the Lanczos filter by a factor of four. We then downsample a given image by a factor of four, and then reconstruct it with the deep decoder (with k = 128, as before). We compare performance to bi-cubic interpolation and to the deep image prior, and find that the deep decoder outperforms bicubic interpolation, and is on-par with the deep image prior (see Table 1 in the appendix). Finally, we use the deep decoder for inpainting, where we are given an inpainted image y, and a forward model f mapping a clean image to an inpainted image. The forward model f is defined by a mask that describes the inpainted region, and simply maps that part of the image to zero. FIG1 and Table 1 demonstrate that the deep decoder performs well on the inpainting problems; however, the deep image prior performs slightly better on average over the examples considered. For the impainting problem we choose a significantly more expressive prior, specifically k = 320. Image compression, restoration, and recovery algorithms are either trained or untrained. Conceptually, the deep decoder image model is most related to untrained methods, such as sparse representations in overcomplete dictionaries (for example wavelets BID6 and curvelets BID17). A number of highly successful image restoration and recovery schemes are not directly based on generative image models, but rely on structural assumptions about the image, such as exploiting self-similarity in images for denoising BID5 and super-resolution BID7 ).Since the deep decoder is an image-generating deep network, it is also related to methods that rely on trained deep image models. Deep learning based methods are either trained end-to-end for tasks ranging from compression BID19 BID0 BID18 BID4 BID22 to denoising BID4 BID22, or are based on learning a generative image model (by training an autoencoder or GAN BID12 BID8) and then using the ing model to solve inverse problems such as compressed sensing BID3, denoising BID11, phase retrieval, and blind deconvolution BID2, by minimizing an associated loss. In contrast to the deep decoder, where the optimization is over the weights of the network, in all the aforementioned methods, the weights are adjusted only during training and then are fixed upon solving the inverse problem. Most related to our work is the Deep Image Prior (DIP), recently proposed by Ulyanov et al. BID20. The deep image prior is an untrained method that uses a network with an hourglass or encoder-decoder architecture, similar to the U-net and related architectures that work well as autoencoders. The key differences to the deep decoder are threefold: i) the DIP is over-parameterized, whereas the deep decoder is under-parameterized. ii) Since the DIP is highly over-parameterized, it critically relies on regularization through early stopping and adding noise to its input, whereas the deep decoder does not need to be regularized (however, regularization can enhance performance). iii) The DIP is a convolutional neural network, whereas the deep decoder is not. We further illustrate point ii) comparing the DIP and deep decoder by denoising the astronaut image from Fig. 2. In FIG2 we plot the Mean Squared Error (MSE) over the number of iterations of the optimizer for fitting the noisy astronaut image x + η. Note that to fit the model, we minimize the error G(C) − (x + η) 2 2, because we are only given the noisy image, but we plot the MSE between the representation and the actual, true image G(C t) − x 2 2 at iteration t. Here, C t are the parameters of the deep decoder after t iterations of the optimizer. In FIG2 and (c), we plot the loss or MSE associated with fitting the noiseless astronaut image, x (G(C t) − x 2 2 ) and the noise itself, η, (G(C t) − η 2 2 ). Models are fitted independently for the noisy image, the noiseless image, and the noise. The plots in FIG2 show that with sufficiently many iterations, both the DIP and the DD can fit the image well. However, even with a large number of iterations, the deep decoder can not fit the noise well, whereas the DIP can. This is not surprising, given that the DIP is over-parameterized and the deep decoder is under-parameterized. In fact, in Section 6 we formally show that due to the The third panel shows the MSE of the output of DD or DIP for an image consisting purely of noise, as computed relative to that noise. Due to under-parameterization, the deep decoder can only fit a small proportion of the noise, and thus enables image denoising. Early stopping can mildly enhance the performance of DD; to see this note that in panel (a), the minimum is obtained at around 5000 iterations and not at 50,000. The deep image prior can fit noise very well, but fits an image faster than noise, thus early stopping is critical for denoising performance.underparameterization, the deep decoder can only fit a small proportion of the noise, no matter how and how long we optimize. As a consequence, it filters out much of the noise when applied to a natural image. In contrast, the DIP relies on the empirical observation that the DIP fits a structured image faster than it fits noise, and thus critically relies on early stopping. In the previous sections we empirically showed that the deep decoder can represent images well and at the same time cannot fit noise well. In this section, we formally show that the deep decoder can only fit a small proportion of the noise, relative to the degree of underparameterization. In addition, we provide insights into how the components of the deep decoder contribute to representing natural images well, and we provide empirical observations on the sensitivity of the parameters and their distribution. We start by showing that an under-parameterized deep decoder can only fit a proportion of the noise relative to the degree of underparameterization. At the heart of our argument is the intuition that a method mapping from a low-to a high-dimensional space can only fit a proportion of the noise relative to the number of free parameters. For simplicity, we consider a one-layer network, and ignore the batch normalization operation. Then, the networks output is given by DISPLAYFORM0 Here, we take C = (C 0, c 1), where C 0 is a k × k matrix and c 1 is a k-dimensional vector, assuming that the number of output channels is 1. While for the performance of the deep decoder the choice of upsampling matrix is important, it is not relevant for showing that the deep decoder cannot represent noise well. Therefore, the following statement makes no assumptions about the upsampling matrix U 0. Proposition 1. Consider a deep decoder with one layer and arbitrary upsampling and input matrices. That is, let B 0 ∈ R n0×k and U 0 ∈ R n×n0. Let η ∈ R n be zero-mean Gaussian noise with covariance matrix σI, σ > 0. Assume that k 2 log(n 0)/n ≤ 1/32. Then, with probability at least DISPLAYFORM1 The proposition asserts that the deep decoder can only fit a small portion of the noise energy, precisely a proportion determined by its number of parameters relative to the output dimension, n. Our The blue curves show a one-dimensional piecewise smooth signal, and the red crosses show estimates of this signal by a one-dimensional deep decoder with either linear or convex upsampling. We see that linear upsampling acts as an indirect signal prior that promotes piecewise smoothness.simulations and preliminary analytic suggest that this statement extends to multiple layers in that the lower bound becomes 1 − c DISPLAYFORM2, where c is a numerical constant. Note that the lower bound does not directly depend on the noise variance σ since both sides of the inequality scale with σ 2. Upsampling is a vital part of the deep decoder because it is the only way that the notion of locality explicitly enters the signal model. In contrast, most convolutional neural networks have spatial coupling between pixels both by unlearned upsampling, but also by learned convolutional filters of nontrivial spatial extent. The choice of the upsampling method in the deep decoder strongly affects the'character' of the ing signal estimates. We now discuss the impacts of a few choices of upsampling matrices U i, and their impact on the images the model can fit. No upsampling: If there is no upsampling, or, equivalently, if U i = I, then there is no notion of locality in the ing image. All pixels become decoupled, and there is then no notion of which pixels are near to each other. Specifically, a permutation of the input pixels (the rows of B 0) simply induces the identical permutation of the output pixels. Thus, if a deep decoder without upsampling could fit a given image, it would also be able to fit random permutations of the image equally well, which is practically equivalent to fitting random noise. Nearest neighbor upsampling: If the upsampling operations perform nearest neighbor upsampling, then the output of the deep decoder consists of piecewise constant patches. If the upsampling doubles the image dimensions at each layer, this would in patches of 2 d × 2 d pixels that are constant. While this upsampling method does induce a notion of locality, it does so too strongly in the sense that squares of nearby pixels become identical and incapable of fitting local variation within natural images. Linear and convex, non-linear upsampling: The specific choice of upsampling matrix affects the multiscale'character' of the signal estimates. To illustrate this, FIG3 shows the signal estimate from a 1-dimensional deep decoder with upsampling operations given by linear upsampling (x 0, x 1, x 2, . . .) → (x 0, 0.5x 0 + 0.5x 1, x 1, 0.5x 1 + 0.5x 2, x 2, . . .) and convex nonlinear upsampling given by (x 0, x 1, x 2, . . .) → (x 0, 0.75x 0 + 0.25x 1, x 1, 0.75x 1 + 0.25x 2, x 2, . . .). Note that while both models are able to capture the coarse signal structure, the convex upsampling in a multiscale fractal-like structure that impedes signal representation. In contrast, linear upsampling is better able to represent smoothly varying portions of the signal. Linear upsampling in a deep decoder indirectly encodes the prior that natural signals are piecewise smooth and in some sense have approximately linear behavior at multiple scales 6.3 NETWORK INPUT Throughout, the network input is fixed. We choose the network input B 1 by choosing its entries uniformly at random. The particular choice of the input is not very important; it is however desirable that the rows are incoherent. To see this, as an extreme case, if any two rows of B 1 are equal and if the upsampling operation preserves the values of those pixels exactly (for example, as with the linear upsampling from the previous section), then the corresponding pixels of the output image is Figure 6: The left panel shows an image reconstruction after training a deep decoder on the MRI phantom image (PSNR is 51dB). The right panel shows how the deep decoder builds up an image starting from a random input. From top to bottom are the input to the network and the activation maps (i.e., relu(B i C i)) for eight out of the 64 channels in layers one to six. Table 1: Performance comparison of the deep decoder for denoising (DN), superresolution (SR), and inpainting (IP), in peak signal to noise ratio (PSNR). Note that identity corresponds to the PSNR of the noise and corruption in the DN and IP experiments, respectively. also exactly the same, which restricts the range space of the deep decoder unrealistically, since for any pair of pixels, the majority of natural images does not have exactly the same value at this pair of pixels. The deep decoder is tasked with coverting multiple noise channels into a structured signal primarily using pixelwise linear combinations, ReLU activation funcions, and upsampling. Using these tools, the deep decoder builds up an image through a series of successive approximations that gradually morph between random noise and signal. To illustrate that, we plot the activation maps (i.e., relu(B i C i)) of a deep decoder fitted to the phantom MRI test image (see Fig. 6). We choose a deep decoder with d = 5 layers and k = 64 channels. This image reconstruction approach is in contrast to being a semantically meaningful hierarchical representation (i.e., where edges get combined into corners, that get combined into simple sample, and then into more complicated shapes), similar to what is common in discriminative networks. RH is partially supported by NSF award IIS-1816986, an NVIDIA Academic GPU Grant, and would like to thank Ludwig Schmidt for helpful discussions on the deep decoder in general, and in particular for suggestions on the experiments in Section 2.Code to reproduce the is available at https://github.com/reinhardh/ supplement_deep_decoder APPENDIX A PROOF OF PROPOSITION 1Suppose that the network has one layer, i.e., G(C) = relu(U 0 B 0 C 0)c 1. We start by re-writing B 1 = relu(B 0 C 0) in a convenient form. For a given vector x ∈ R n, denote by diag(x > 0) the matrix that contains one on its diagonal if the respective entry of x is positive and zero otherwise. Let c jci denote the i-th column of C j, and denote by W ji ∈ {0, 1} k×k the corresponding diagonal matrix W ji = diag(U j B j c jci > 0). With this notation, we can write DISPLAYFORM0 where [c 1] i denotes the i-th entry of c 1. Thus, G(C) lies in the union of at-most-k 2 -dimensional subspaces of R n, where each subspace is determined by the matrices {W 0j} k j=1. The number of those subspaces is bounded by n k 2. This follows from the fact that for the matrix A:= U 0 B 0, by Lemma 1 below, the number of different matrices W 0j is bounded by n k. Since there are k matrices, the number of different sets of matrices is bounded by n k 2. Lemma 1. For any A ∈ R n×k and k ≥ 5, |{diag(Av > 0)A|v ∈ R k }| ≤ n k.Next, fix the matrixes {W 0j} j. As G(C) lies in an at-most-k 2 -dimensional subspace, let S be a k 2 -dimensional subspace that contains the range of G for these fixed {W 0j} j. It follows that Now, we make use of the following bound on the projection of the noise η onto a subspace. Lemma 2. Let S ⊂ R n be a subspace with dimension. Let η ∼ N (0, I n) and β ≥ 1. Then, P P S c η n, then P X − n ≥ 2 √ nx + 2x ≤ e −x, DISPLAYFORM1 With these, we obtain P [X ≥ 5βn] ≤ e −βn if β ≥ 1,P [X ≤ n/2] ≤ e −n/16.We have noiseless C 1 noisy C 2 noisy C 3 noisy C 4 noisy C 5 noisy C 6 noisy c 7 noisy Figure 7: Sensitivity to parameter perturbations of the weights in each layer, and images generated by perturbing the weights in different layers, and keeping the weights in the other layers constant. A.1 PROOF OF LEMMA 1Our goal is to count the number of sign patterns (Av > 0) ∈ {0, 1}. Note that this number is equal to the maximum number of partitions one can get when cutting a k-dimensional space with n many hyperplanes that all pass through the origin, and are perpendicular to the rows of A. This number if well known (see for example BID21) and is upper bounded by DISPLAYFORM2 Thus, DISPLAYFORM3 where the last inequality holds for k ≥ 5. The deep decoder is not overly sensitive to perturbations of its coefficients. To demonstrate this, fit the standard test image Barbara with a deep decoder with 6 layers and k = 128, as before. We then perturb the weights in a given layer i (i.e., the matrix C i) with Gaussian noise of a certain signal-tonoise ratio relative to C i and leave the other weights and the input untouched. We then measure the peak signal-to-noise ratio in the image domain, and plot the corresponding curve for each layer (see Fig. 7). It can be seen that the representation provided by the deep decoder is relatively stable with respect to perturbations of its coefficients, and that it is more sensitive to perturbations in higher levels. Finally, in FIG7 we depict the distribution of the weights of the network after fitted to the Barbara test image, and note that the weights are approximately Gaussian distributed. The distribution of the weighs is approximately Gaussian.

Represent the short summary for finding the corresponding scientific article (long)

We introduce an underparameterized, nonconvolutional, and simple deep neural network that can, without training, effectively represent natural images and solve image processing tasks like compression and denoising competitively.

We propose a extension of the batch normalization, show a first-of-its-kind convergence analysis for this extension and show in numerical experiments that it has better performance than the original batch normalizatin.