Question: Represent the article for finding a single-sentence TLDR

Question: This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piece-wise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to provide a characterization of the decision boundaries of a simple neural network of the form (Affine, ReLU, Affine). Specifically, we show that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes. The generators of the zonotopes are precise functions of the neural network parameters. We utilize this geometric characterization to shed light and new perspective on three tasks. In doing so, we propose a new tropical perspective for the lottery ticket hypothesis, where we see the effect of different initializations on the tropical geometric representation of the decision boundaries. Also, we leverage this characterization as a new set of tropical regularizers, which deal directly with the decision boundaries of a network. We investigate the use of these regularizers in neural network pruning (removing network parameters that do not contribute to the tropical geometric representation of the decision boundaries) and in generating adversarial input attacks (with input perturbations explicitly perturbing the decision boundaries geometry to change the network prediction of the input).. In addition, and in attempt to understand some of the subtle behaviours DNNs exhibit, e.g. the sensitive reaction of DNNs to small input perturbations, several works directly investigated the decision boundaries induced by a DNN used for classification. The work of showed that the smoothness of these decision boundaries and their curvature can play a vital role in network robustness. Moreover, He et al. (2018a) studied the expressiveness of these decision boundaries at perturbed inputs and showed that these boundaries do not resemble the boundaries around benign inputs. showed that under certain assumptions, the decision boundaries of the last fully connected layer of DNNs will converge to a linear SVM. Also, showed that the decision regions of DNNs with width smaller than the input dimension are unbounded. More recently, and due to the popularity of the piecewise linear ReLU as an activation function, there has been a surge in the number of works that study this class of DNNs in particular. x ⊕ y = max{x, y}, x y = x + y, ∀x, y ∈ T. It can be readily shown that −∞ is the additive identity and 0 is the multiplicative identity. Given the previous definition, a tropical power can be formulated as x a = x x · · · x = a.x, for x ∈ T, a ∈ N, where a.x is standard multiplication. Moreover, the tropical quotient can be defined as: x y = x − y where x − y is the standard subtraction. For ease of notation, we write x a as x a. Now, we are in a position to define tropical polynomials, their solution sets and tropical rationals. f (x) = (c 1 x a1) ⊕ (c 2 x a2) ⊕ · · · ⊕ (c n x an), ∀ a i = a j when i = j. We use the more compact vector notation x a = x In this section, we analyze the decision boundaries of a network in the form (Affine, ReLU, Affine) using tropical geometry. For ease, we use ReLUs as the non-linear activation, but any other piecewise linear function can also be used. The functional form of this network is: f (x) = Bmax (Ax + c 1, 0) + c 2, where max is an element-wise operator. The outputs of the network f are the logit scores. Throughout this section, we assume 2 that A ∈ Z p×n, B ∈ Z 2×p, c 1 ∈ R p and c 2 ∈ R 2. For ease of notation, we only consider networks with two outputs, i.e. B 2×p, where the extension to a multi-class output follows naturally and it is discussed in the appendix. Now, since f is a piecewise linear function, each output can be expressed as a tropical rational as per Theorem 1. If f 1 and f 2 refer to the first and second outputs respectively, we have f 1 (x) = H 1 (x) Q 1 (x) and f 2 (x) = H 2 (x) Q 2 (x), where H 1, H 2, Q 1 and Q 2 are tropical polynomials. In what follows and for ease of presentation, we present our main where the network f has no biases, i.e. c 1 = 0 and c 2 = 0, and we leave the generalization to the appendix. Theorem 2. For a bias-free neural network in the form of f (x): R n → R 2 where A ∈ Z p×n and B ∈ Z 2×p, let R(x) = H 1 (x) Q 2 (x) ⊕ H 2 (x) Q 1 (x) be a tropical polynomial. Then: • Let B = {x ∈ R n : f 1 (x) = f 2 (x)} defines the decision boundaries of f, then B ⊆ T (R(x)). • δ (R(x)) = ConvHull (Z G1, Z G2). Z G1 is a zonotope in R n with line segments The proof for Theorem 2 is left for the appendix. Digesting Theorem 2. Theorem 2 can be broken into two major . The first, which is on the algebra side, i.e. finding the solution set to tropical polynomials, states that the decision boundaries linear pieces separating classes C1 and C2. As per Theorem 2, the dual subdivision of this single hidden neural network is the convex hull between the zonotopes Z G 1 and Z G 2. The normals to the dual subdivison δ(R(x)) are in one-to-one correspondence to the tropical hypersurface T (R(x)), which is a superset to the decision boundaries B. Note that some of the normals to δ(R(x)) (in red) are parallel to the decision boundaries. B is a subset of the tropical hypersurface of the tropical polynomial R(x), i.e. T (R(x)). The second , which is on the geometry side, of Theorem 2 relates the tropical polynomial R(x) to the geometric representation of the solution set to R(x), i.e. T (R(x)), referred to as the dual subdivision, i.e. δ(R(x)). In particular, Theorem 2 states that the dual subdivision for a network f is the convex hull of two zonotopes denoted as Z G1 and Z G2. Note that this dual subdivision is a function of only the network parameters A and B. Theorem 2 bridges the gap between the behaviour of the decision boundaries B, through the super-set T (R(x)), and the polytope δ (R(x)), which is the convex hull of two zonotopes. It is worthwhile to mention that discussed a special case of the first part of Theorem 2 for a neural network with a single output and a score function s(x) to classify the output. To the best of our knowledge, this work is the first to propose a tropical geometric formulation of a super-set containing the decision boundaries of a multi-class classification neural network. In particular, the first of Theorem 2 states that one can alter the network, e.g. by pruning network parameters, while preserving the decision boundaries B, if one preserves the tropical hypersurface of R(x) or T (R(x)). While preserving the tropical hypersurfaces can be equally difficult to preserving the decision boundaries directly, the second of Theorem 2 comes in handy. For a bias free network, π becomes an identity mapping with δ(R(x)) = ∆(R(x)), and thus the dual subdivision δ(R(x)), which is the Newton polytope ∆(R(x)) in this case, becomes a well structured geometric object that can be exploited to preserve decision boundaries. (Proposition 3.1.6) showed that the tropical hypersurface is the skeleton of the dual to δ(R(x)), the normal lines to the edges of the polytope δ(R(x)) are in one-to-one correspondence with the tropical hypersurface T (R(x)). Figure 1 details this intimate relation between the decision boundaries, tropical hypersurface T (R(x)), and normals to δ (R(x)). Before any further discussion, we recap the definition of zonotopes. Equivalently, the zonotope can be expressed with respect to the generator matrix U ∈ R p×n, where Another common definition for zonotopes is the Minkowski sum (refer to appendix A for the definition of the Minkowski sum) of a set of line segments that start from the origin with end points u 1,..., u p ∈ R n. It is also well known that the number of vertices of a zonotope is polynomial in the number of line segments. That is to say, While Theorem 2 presents a strong relation between a polytope (convex hull of two zonotopes) and the decision boundaries, it remains unclear how such a polytope can be efficiently constructed. Although the number of vertices of a zonotope is polynomial in the number of its generating line segments, fast algorithms for enumerating these vertices are still restricted to zonotopes with line segments starting at the origin . Since the line segments generating the zonotopes in Theorem 2 have arbitrary end points, we present the next that transforms these line segments into a generator matrix of line segments starting from the origin, as prescribed in Definition 6. This is essential for the efficient computation of the zonotopes in Theorem 2. Proposition 1. Consider p line segments in R n with two arbitrary end points as follows. The zonotope formed by these line segments is equivalent to the zonotope formed by the line segments {[u training dataset, decision boundaries polytope of original network followed by the decision boundaries polytope during several iterations of pruning with different initializations. The proof is left for the appendix. As per Proposition 1, the generator matrices of zonotopes In what follows, we show several applications for Theorem 2. We begin by leveraging the geometric structure to help in reaffirming the behaviour of the lottery ticket hypothesis. The lottery ticket hypothesis was recently proposed by , in which the authors surmise the existence of sparse trainable sub-networks of dense, randomly-initialized, feedforward networks that-when trained in isolation-perform as well as the original network in a similar number of iterations. To find such sub-networks, propose the following simple algorithm: perform standard network pruning, initialize the pruned network with the same initialization that was used in the original training setting, and train with the same number of epochs. They hypothesize that this should in a smaller network with a similar accuracy to the larger dense network. In other words, a subnetwork can have similar decision boundaries to the original network. While in this section we do not provide a theoretical reason for why this proposed pruning algorithm performs favorably, we utilize the geometric structure that arises from Theorem 2 to reaffirm such behaviour. In particular, we show that the orientation of the decision boundaries polytope δ(R(x)), known to be a superset to the decision boundaries T (R(x)), is preserved after pruning with the proposed initialization algorithm of. On the other hand, pruning routines with a different initialization at each pruning iteration will in a severe variation in the orientation of the decision boundaries polytope. This leads to a large change in the orientation of the decision boundaries, which tends to hinder accuracy. To this end, we train a neural network with 2 inputs (n = 2), 2 outputs, and a single hidden layer with 40 nodes (p = 40). We then prune the network by removing the smallest x% of the weights. The pruned network is then trained using different initializations: (i) the same initialization as the original network , (ii) Xavier , (iii) standard Gaussian and (iv) zero mean Gaussian with variance of 0.1. Figure 2 shows the evolution of the decision boundaries polytope, i.e. δ(R(x)), as we perform more pruning (increasing the x%) with different initializations. It is to be observed that the orientation of the polytopes δ(R(x)) vary much more for all different initialization schemes as compared to the lottery ticket initialization. This gives an indication that lottery ticket initialization indeed preserves the decision boundaries throughout the evolution of pruning. Another approach to investigate the lottery ticket could be by observing the polytopes representing the functional form of the network directly, i.e. δ(H {1,2} (x)) and δ(Q {1,2} (x)), in lieu of the decision boundaries polytopes. However, this does not provide conclusive answers to the lottery ticket, since there can exist multiple functional forms, and correspondingly multiple polytopes δ(H {1,2} (x)) and δ(Q {1,2} (x)), for networks with the same decision boundaries. This is why we explicitly focus our analysis on δ(R(x)), which is directly related to the decision boundaries of the network. Further discussions and experiments are left for the appendix. Network pruning has been identified as an effective approach for reducing the computational cost and memory usage during network inference time. While pruning dates back to the work of and , it has recently gained more attention. This is due to the fact that most neural networks over-parameterize commonly used datasets. In network pruning, the task is to find a smaller subset of the network parameters, such that the ing smaller network has similar decision boundaries (and thus supposedly similar accuracy) to the original over-parameterized network. In this section, we show a new geometric approach towards network pruning. In particu- th node, or equivalently removing the two yellow vertices of zonotope ZG 2 does not affect the decision boundaries polytope which will not lead to any change in accuracy. lar, as indicated by Theorem 2, preserving the polytope δ(R(x)) preserves a superset to the decision boundaries T (R(x)), and thus supposedly the decision boundaries themselves. Motivational Insight. For a single hidden layer neural network, the dual subdivision to the decision boundaries is the polytope that is the convex hull of two zonotopes, where each is formed by taking the Minkowski sum of line segments (Theorem 2). Figure 3 shows an example where pruning a neuron in the neural network has no effect on the dual subdivision polytope and equivalently no effect on the accuracy, since the decision boundaries of both networks remain the same. Problem Formulation. Given the motivational insight, a natural question arises: Given an overparameterized binary neural network f (x) = B max (Ax, 0), can one construct a new neural network, parameterized by some sparser weight matricesÃ andB, such that this smaller network has a dual subdivision δ(R(x)) that preserves the decision boundaries of the original network? In order to address this question, we propose the following general optimization problem The function d defines a distance between two geometric objects. Since the generatorsG 1 and G 2 are functions ofÃ andB (as per Theorem 2), this optimization problem can be challenging to solve. However, for pruning purposes, one can observe from Theorem 2 that if the generatorsG 1 andG 2 had fewer number of line segments (rows), this corresponds to a fewer number of rows in the weight matrixÃ (sparser weights). To this end, we observe that ifG 1 ≈ G 1 andG 2 ≈ G 2, thenδ(R(x)) ≈ δ(R(x)), and thus the decision boundaries tend to be preserved as a consequence. Therefore, we propose the following optimization problem as a surrogate to Problem The matrix mixed norm for C ∈ R n×k is defined as C 2,1 = n i=1 C(i, :) 2, which encourages the matrix C to be row sparse, i.e. complete rows of C are zero. Note thatG 1 = Diag[ReLU(B(1, :))+ReLU(−B(2, :))] Ã,G 2 = Diag[ReLU(B(2, :))+ReLU(−B(1, :))]Ã, and Diag(v) rearranges the elements of vector v in a diagonal matrix. We solve the aforementioned problem with alternating optimization over the variablesÃ andB, where each sub-problem is solved in closed form. Details of the optimization and the extension to multi-class case are left for the appendix. Extension to Deeper Networks. For deeper networks, one can still apply the aforementioned optimization for consecutive blocks. In particular, we prune each consecutive block of the form (Affine,ReLU,Affine) starting from the input and ending at the output of the network. Experiments on Tropical Pruning. Here, we evaluate the performance of the proposed pruning approach as compared to several classical approaches on several architectures and datasets. In particular, we compare our tropical pruning approach against Class Blind (CB), Class Uniform (CU) and Class Distribution (CD);. In Class Blind, all the parameters across all nodes of a layer are sorted by magnitude where x% with smallest magnitudes are pruned. Similar to Class Blind, Class Uniform prunes the parameters with smallest x% magnitudes per node in a layer as opposed to sorting all parameters in all nodes as in Class Blind. Lastly, Class Distribution performs pruning of all parameters for each node in the layer, just as in Class Uniform, but the parameters are pruned based on the standard deviation σ c of the magnitude of the parameters per node. Since fully connected layers in deep neural networks tend to have much higher memory complexity than convolutional layers, we restrict our focus to pruning fully connected layers. We train AlexNet and VGG16 on SVHN, CIFAR10, and CIFAR 100 datasets. We observe that we can prune more than 90% of the classifier parameters for both networks without affecting the accuracy. Moreover, we can boost the pruning ratio using our method without affecting the accuracy by simply retraining the network biases only. Setup. We adapt the architectures of AlexNet and VGG16, since they were originally trained on ImageNet , to account for the discrepancy in the input resolution. The fully connected layers of AlexNet and VGG16 have sizes of and, respectively on SVHN and CIFAR100 with the last layer replaced to 100 for CIFAR100. All networks were trained to baseline test accuracy of (92%,74%,43%) for AlexNet on SVHN, CIFAR10 and CIFAR100, respectively and (92%,92%,70%) for VGG16. To evaluate the performance of pruning, following previous works , we report the area under the curve (AUC) of the pruning-accuracy plot. The higher the AUC is, the better the trade-off is between pruning rate and accuracy. For efficiency purposes, we run the optimization in Problem for a single alternating iteration to identify the rows inÃ and elements ofB that will be pruned, since an exact pruning solution might not be necessary. The algorithm and the parameters setup to solving is left for the appendix. Results. Figure 4 shows the pruning comparison between our tropical approach and the three aforementioned popular pruning schemes on both AlexNet and VGG16 over the different datasets. Our proposed approach can indeed prune out as much as 90% of the parameters of the classifier without sacrificing much of the accuracy. For AlexNet, we achieve much better performance in pruning as compared to other methods. In particular, we are better in AUC by 3%, 3%, and 2% over other pruning methods on SVHN, CIFAR10 and CIFAR100, respectively. This indicates that the decision boundaries can indeed be preserved by preserving the dual subdivision polytope. For VGG16, we perform similarly well on both SVHN and CIFAR10 and slightly worse on CIFAR100. While the performance achieved here is comparable to the other pruning schemes, if not better, we emphasize that our contribution does not lie in outperforming state-of-the-art pruning methods, but rather in giving a new geometry based perspective to network pruning. We conduct more experiments, where only the biases of the network or the biases of the classifier are fine tuned after pruning. Retraining biases can be sufficient as they do not contribute to the orientation of the decision boundaries polytope, thereafter the decision boundaries, but only a translation. Discussion on biases and more are left for the appendix. DNNs are notoriously known to be susceptible to adversarial attacks. In fact, adding small imperceptible noise, referred to as adversarial attacks, at the input of these networks can hinder their performance. Several works investigated the decision boundaries of neural networks in the presence of adversarial attacks. For instance, analyzed high dimensional geometry of adversarial examples by the means of manifold reconstruction. Also, He et al. (2018b) crafted adversarial attacks by estimating the distance to the decision boundaries using random search directions. In this work, we provide a tropical geometric view to this problem. where we show how Theorem 2 can be leveraged to construct a tropical geometric based targeted adversarial attack. Dual View to Adversarial Attacks. For a classifier f: R n → R k and input x 0 that is classified as c, a standard formulation for targeted adversarial attacks flips the classifier prediction to a particular class t and it is usually defined as follows This objective aims at computing the lowest energy input noise η (measured by D) such that the the new sample (x 0 + η) crosses the decision boundaries of f to a new classification region. Here, we present a dual view to adversarial attacks. Instead of designing a sample noise η such that (x 0 + η) belongs to a new decision region, one can instead fix x 0 and perturb the network parameters to move the decision boundaries in a way that x 0 appears in a new classification region. In particular, let A 1 be the first linear layer of f, such that f (x 0) = g(A 1 x 0). One can now perturb A 1 to alter the decision boundaries and relate the perturbation to the input perturbation as follows From this dual view, we observe that traditional adversarial attacks are intimately related to perturbing the parameters of the first linear layer through the linear system: To this end, Theorem 2 provides explicit means to geometrically construct adversarial attacks by means of perturbing decision boundaries. In particular, since the normals to the dual subdivision polytope δ(R(x)) of a given neural network represent the tropical hypersurface set T (R(x)) which is, as per Theorem 2, a superset to the decision boundaries set B, ξ A1 can be designed to in a minimal perturbation to the dual subdivision that is sufficient to change the network prediction of x 0 to the targeted class t. Based on this observation, we formulate the problem as follows The loss is the standard cross-entropy loss. The first row of constraints ensures that the network prediction is the desired target class t when the input x 0 is perturbed by η, and equivalently by perturbing the first linear layer A 1 by ξ A1. This is identical to f 1 as proposed by. Moreover, the third and fourth constraints guarantee that the perturbed input is feasible and that the perturbation is bounded, respectively. The fifth constraint is to limit the maximum perturbation on the first linear layer, while the last constraint enforces the dual equivalence between input perturbation and parameter perturbation. The function D 2 captures the perturbation of the dual subdivision polytope upon perturbing the first linear layer by ξ A1. For a single hidden layer neural network parameterized as (A 1 + ξ A1) ∈ R p×n and B ∈ R 2×p for the 1 st and 2 nd layers respectively, D 2 can capture the perturbations in each of the two zonotopes discussed in Theorem 2. The derivation, discussion, and extension of to multi-class neural networks is left for the appendix. We solve Problem with a penalty method on the linear equality constraints, Motivational Insight to the Dual View. This intuition is presented in Figure 5. We train a single hidden layer neural network where the size of the input is 2 with 50 hidden nodes and 2 outputs on a simple dataset as shown in Figure 5. We then solve Problem 5 for a given x 0 shown in black. We show the decision boundaries for the network with and without the perturbation at the first linear layer ξ A1. Figure 5 shows that indeed perturbing an edge of the dual subdivision polytope, by perturbing the first linear layer, corresponds to perturbing the decision boundaries and in miss-classifying x 0. Interestingly and as expected, perturbing different decision boundaries corresponds to perturbing different edges of the dual subdivision. In particular, one can see from Figure 5 that altering the decision boundaries, by altering the dual subdivision polytope through perturbations in the first linear layer, can in miss-classifying a previously correctly classified input x 0. MNIST Experiment. Here, we design perturbations to misclassify MNIST images. Figure 7 shows several adversarial examples that change the network prediction for digits 8 and 9 to digits 7, 5, and 4, respectively. In some cases, the perturbation η is as small as = 0.1, where x 0 ∈ n. Several other adversarial are left for the appendix. We again emphasize that our approach is not meant to be compared with (or beat) state of the art adversarial attacks, but rather to provide a novel geometrically inspired perspective that can shed new light in this field. In this paper, we leverage tropical geometry to characterize the decision boundaries of neural networks in the form (Affine, ReLU, Affine) and relate it to well-studied geometric objects such as zonotopes and polytopes. We leaverage this representation in providing a tropical perspective to support the lottery ticket hypothesis, network pruning and designing adversarial attacks. One natural extension for this work is a compact derivation for the characterization of the decision boundaries of convolutional neural networks (CNNs) and graphical convolutional networks (GCNs). Diego Ardila, Atilla P. Kiraly, Sujeeth Bharadwaj, Bokyung Choi, Joshua J. Reicher, Lily Peng, Daniel Tse, Mozziyar Etemadi, Wenxing Ye, Greg Corrado, David P. Naidich, and Shravya Shetty. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography.. A PRELIMINARIES AND DEFINITIONS. Fact 1. P+Q = {p + q, ∀p ∈ P and q ∈ Q} is the Minkowski sum between two sets P and Q. Fact 2. Let f be a tropical polynomial and let a ∈ N. Then Let both f and g be tropical polynomials, Then Note that V(P(f)) is the set of vertices of the polytope P(f). Theorem 3. For a bias-free neural network in the form of f (x): R n → R 2 where A ∈ Z p×n and B ∈ Z 2×p, and let • If the decision boundaries of f is given by the set B = {x ∈ R n : f 1 (x) = f 2 (x)}, then we have B ⊆ T (R(x)). Note that A + = max(A, 0) and A − = max(−A, 0) where the max is element-wise. The line segment (B(1, j) is one that has the end points A(j, :) + and A(j, :) − in R n and scaled by the constant B(1, j) Proof. For the first part, recall from Theorem1 that both f 1 and f 2 are tropical rationals and hence, Recall that the tropical hypersurface is defined as the set of x where the maximum is attained by two or more monomials. Therefore, the tropical hypersurface of R(x) is the set of x where the maximum is attained by two or more monomials in (H 1 (x) Q 2 (x)), or attained by two or more monomials in (H 2 (x) Q 1 (x)), or attained by monomials in both of them in the same time, which is the decision boundaries. Hence, we can rewrite that as Therefore, we have that Therefore note that, thus we have that The operator+ indicates a Minkowski sum between sets. Note that ConvexHull A ) is the convexhull between two points which is a line segment in Z n with end points that are is a Minkowski sum of line segments which is is a zonotope. Moreover, note that tropically is given as follows. Thus it is easy to see that δ(Q 2 (x)) is the Minkowski sum of the points {(B − (1, j)−B + (2, j))A − (j, :)}∀j in R n (which is a standard sum) ing in a point. Lastly, it is easy to see that δ(H 1 (x))+δ(Q 2 (x)) is a Minkowski sum between a zonotope and a single point which corresponds to a shifted zonotope. A similar symmetric argument can be applied for the second part δ(H 2 (x))+δ(Q 1 (x)). It is also worthy to mention that the extension to network with multi class output is trivial. In that case all of the analysis can be exactly applied studying the decision boundary between any two classes (i, j) where B = {x ∈ R n : f i (x) = f j (x)} and the rest of the proof will be exactly the same. In this section, we derive the statement of Theorem 2 for the neural network in the form of (Affine, ReLU, Affine) with the consideration of non-zero biases. We show that the presence of biases does not affect the obtained as they only increase the dimension of the space, where the polytopes live, without affecting their shape or edge-orientation. Starting with the first linear layer for x ∈ R n, we have with coordinates, and ∆(Q 1i) is a point in (n + 1) dimensions at (A − (i, :), 0), while under π projection, δ(H 1i) is a point in n dimensions at (A + (i, :)), and δ(Q 1i) is a point in n dimensions at (A − (i, :)). It can be seen that under projection π, the geometrical representation of the output of the first linear layer does not change after adding biases. Looking to the output after adding the ReLU layer, we get, and δ(Q 1i) is the point (A − (i, :)). Again, the biases does not affect the geometry of the output after the ReLU layer, since the line segments now are connecting points in (n + 1) dimensions, but after projecting them using π, they will be identical to the line segments of the network with zero biases. Finally, looking to the output of the second linear layer, we obtain Similar arguments can be given for ∆(Q 3i) and δ(Q 3i). It can be seen that the first part in both expressions is a Minkowski sum of line segments, which will give a zonotope in (n + 1), and n dimensions in the first and second expressions respectively. While the second part in both expressions is a Minkowski sum of bunch of points which gives a single point in (n + 1) and n dimensions for the first and second expression respectively. Note that the last dimension of the aforementioned point in n + 1 dimensions is exactly the i th coordinate of the bias of the second linear layer which is dropped under the π projection. Therefore, the shape of the geometrical representation of the decision boundaries with non-zero biases will not be affected under the projection π, and hence the presence of the biases will not affect any of the of the paper. Proposition 1. Consider p line segments in R n with two arbitrary end points as follows. The zonotope formed by these line segments is equivalent to the zonotope formed be the line segments Proof. Let U j be a matrix with U j (:, i) = u i j, i = 1,..., p, w be a column-vector with w(i) = w i, i = 1,..., p and 1 p is a column-vector of ones of length p. Then, the zonotope Z formed by the Minkowski sum of line segments with arbitrary end points can be defined as Note that the Minkowski sum of any polytope with a point is a translation; thus, the follows directly from Definition 6. A ← arg miñ, where c 1 = ReLU(B(1, :)) + ReLU(−B(2, :)) and c 2 = ReLU(B(2, :)) + ReLU(−B(1, :)). Note that the problem is separable per-row ofÃ. Therefore, the problem reduces to updating rows ofÃ independently and the problem exhibits a closed form solution.. UpdateB + (1, :). Note that C 1 = G 1 − Diag B − (2, :) Ã and where Diag B − (2, :) Ã. Note the problem is separable in the coordinates ofB + (1, :) and a projected gradient descent can be used to solve the problem in such a way as A similar symmetric argument can be used to update the variablesB + (2, :),B + (1, :) andB − (2, :). Note that Theorem 2 describes a superset to the decision boundaries of a binary classifier through the dual subdivision R(x), i.e. δ(R(x)). For a neural network f with k classes, a natural extension for it is to analyze the pair-wise decision boundaries of of all k-classes. Thus, let T (R ij (x)) be the superset to the decision boundaries separating classes i and j. Therefore, a natural extension to the geometric loss in equation 1 is to preserve the polytopes among all pairwise follows The set S is all possible pairwise combinations of the k classes such that S = {[i, j], ∀i = j, i = 1,..., k, j = 1,..., k}. The generator Z (G (i,j) ) is the zonotope with the generator matrix G (i +,j −) = Diag ReLU(B(i, :)) + ReLU(−B(j, :)) Ã. However, such an approach is generally computationally expensive, particularly, when k is very large. To this end, we make the following observation thatG (i +, j −) can be equivalently written as a Minkowski sum between two sets zonotopes with the generators That is to say, ZG. This follows from the associative property of Minkowski sums given as follows: be the set of n line segments. Then we have that S = S 1+...+S n = P+V where the sets P =+ j∈C1 S j and V =+ j∈C2 S j where C 1 and C 2 are any complementary partitions of the set Hence,G (i +,j −) can be seen a concatenation betweenG (i +) andG (j −). Thus, the objective in 10 can be expanded as follows The approximation follows in a similar argument to the binary classifier case where approximating the generators. The last equality follows from a counting argument. We solve the objective for all multi-class networks in the experiments with alternating optimization in a similar fashion to the binary classifier case. Similarly to the binary classification approach, we introduce the 2,1 to enforce sparsity constraints for pruning purposes. Therefore the overall objective has the form For completion, we derive the updates forÃ andB. A = arg miñ Similar to the binary classification, the problem is seprable in the rows ofÃ. and a closed form solution in terms of the proximal operator of 2 norm follows naturally for eachÃ(i, :). UpdateB + (i, :). Note that the problem is separable per coordinates of B + (i, :) and each subproblem is updated as: A similar argument can be used to updateB − (i, :) ∀i. Finally, the parameters of the pruned network will be constructed A ←Ã and B ←B + −B −. Input: In this section, we are going to derive an algorithm for solving the following problem. The function D 2 (ξ A) captures the perturbdation in the dual subdivision polytope such that the dual subdivion of the network with the first linear layer A 1 is similar to the dual subdivion of the network with the first linear layer A 1 + ξ A1. This can be generally formulated as an approximation to the following distance function This can thereafter be extended to multi-class network with k classes as follows. , we take. Therefore, we can write 11 as follows To enforce the linear equality constraints A 1 η − ξ A1 x 0 = 0, we use a penalty method, where each iteration of the penalty method we solve the sub-problem with ADMM updates. That is, we solve the following optimization problem with ADMM with increasing λ such that λ → ∞. For ease of notation, lets denote where The augmented Lagrangian is thus given as follows Thereafter, ADMM updates are given as follows Updating η: Updating w: It is easy to show that the update w is separable in coordinates as follows Updating z: showed that the linearized ADMM converges for some non-convex problems. Therefore, by linearizing L and adding Bergman divergence term η, we can then update z as follows It is worthy to mention that the analysis until this step is inspired by with modifications to adapt our new formulation. Updating ξ A: The previous problem can be solved with proximal gradient method. In this section, we are going to describe the settings and the values of the hyper-parameters that we used in the experiments. Moreover, we will show more since we have limited space in the main paper. We begin by throwing the following question. Why investigating the tropical geometrical perspective of the decision boundaries is more important than investigating the tropical geometrical representation of the functional form of the network? In this section, we show one more experiment that differentiate between these two views. In the following, we can see that variations can happen to the tropical geometrical representation of the functional form (zonotopes in case of single hidden layer neural network), but the shape of the polytope of the decision boundaries is still unchanged and consequently, the decision boundaries. For this purpose, we trained a single hidden layer neural network on a simple dataset like the one in Figure 2, then we do several iteration of pruning, and visualise at each iteration both the polytope of the decision boundaries and the zonotopes of the functional representation of the neural network. It can be easily seen that changes in the zonotopes may not change the shape of the decision boundaries polytope and consequently the decision boundaries of the neural network. And thus it can be clearly seen that our formulation, which is looking at the decision boundaries polytope is more general, precise and indeed more meaningful. Moreover, we conducted the same experiment explained in the main paper of this section on another dataset to have further demonstration on the favour that the lottery ticket initialization has over other initialization when pruning and retraining the pruned model. It is clear that the lottery initializations is the one that preserves the shape of the decision boundaries polytope the most. In the tropical pruning, we have control on two hyper-parameters only, namely the number of iterations and the regularizer coefficient λ which controls the pruning rate. In all of the experiments, we ran the algorithm for 1 iteration only and we increase λ starting from 0.02 linearly with a factor of 0.01 to reach 100% pruning. It is also worthy to mention that the output of the algorithm will be new sparse matricesÃ,B, but the new network parameters will be the elements in the original matrices A, B that have indices correspond to the indices of non-zero elements inÃ,B. By that, the algorithm removes the non-effective line segments that do not contribute to the decision boundaries polytope, without changing the non-deleted segments. Above all, more of pruning of AlexNet and VGG16 on various datasets are shown below. For all of the experiments, {2, λ, η, ρ} had the values {1, 10 −3, 2.5, 1} respectively. the value of 1 was 0.1 when attacking the -fours-images, and 0.2 for the rest of the images. Finally, we show extra of attacking the decision boundaries of synthetic data in R 2 and MNIST images by tropical adversarial attacks. We thank R3 for the time spent reviewing the paper. It is though not clear to the authors the main reason behind the initial score of weak reject as R3 seems to have very generic questions about our work but not a particular criticism of the novelty/contribution that we can address in our rebuttal. We hope that the following response addresses and clarifies some key elements. Moreover, we want to bring to the attention of R3 that we have addressed the comments/typos/suggestions of all reviewers in the revised version and marked them in blue. Q1: What benefit does introducing tropical geometry brings in terms of theoretical analysis? Does using tropical geometry give us the theoretical that traditional analysis can not give us? If so, what is it? I am trying to understand why the authors use this tool. The authors should be explicit in their motivation so that the readers are clear about the contribution of this paper. More specifically, from my perspective, tropical semiring, tropical polynomials and tropical rational functions all can be represented with the standard mathematical tools. Here they are just redefining several concepts. As discussed thoroughly in the introduction (last paragraph of page 1), tropical geometry is the younger twin to algebraic geometry on a particular semiring defined in a way to align with the study of piecewise linear functions. The early definitions stated in the paper (1 to 5) are well known in the TG literature and were restated for the completion of this paper. While it is true that the definitions can be represented with standard mathematical tools; however, this misses the fundamental powerful element TG promises. TG transforms algebraic problems of piecewise linear nature to a combinatoric problem on general polytopes. To that end, Zhang et. al. 2018 (to the best of our knowledge the only work at the intersection between TG and DNNs) rederived classical (upper bound on the number of linear pieces of DNNs) in a much simpler analysis by counting vertices on polytopes. In this work, instead of studying the functional representation of piecewise linear DNNs, we study their decision boundaries using the lens of TG. To wit, the geometric characterization of the decision boundaries of DNNs developed in Theorem 2 cannot be attained using standard mathematical tools. More specifically, Theorem 2 represented a superset to the decision boundaries (the tropical hypersurface T (R(x)), with a geometric structure that is the convex hull between two zonotopes. While this by itself opens doors for a family of new geometrically motivated regualrizers for training DNNs that are in direct correspondence with the behaviour of the decision boundaries, we do not dwell on training beyond this point and leave that for future work. However, this new allowed for re-affirmation to the lottery ticket hypothesis in a new fresh perspective. Moreover, we propose new optimization problems that are geometrically motivated (based on Theorem 2) for several classical problems, i.e. network pruning and adversarial attacks that were not possible before and have provided several new insights and directions. That is we show an intimate relation between network perturbations (through decision boundary polytope perturbations) and the construction of adversarial attacks (input perturbations). In Experiments on Tropical Pruning, the authors mentioned we compare our tropical pruning approach against Class Blind (CB), Class Uniform (CU), and Class Distribution (CD) methods. What is Class Blind, Class Uniform and Class Distribution? There seems to be an error here Figure 5 shows the pruning comparison between our tropical approach..., i think Figure 5 should be Figure 4. We have added the definition of the pruning methods of in the revised version of the paper for completion, and corrected the typo in the Figure reference. Q3: In the adversarial attack part, is the authors proposing a new attack method? If so, then the authors should report the test accuracy under attack. Also, the experimental should not be restricted to MNIST dataset. I am also not sure about the attack settings here, the authors said Instead of designing a sample noise such that (x0 + η) belongs to a new decision region, one can instead fix x0 and perturb the network parameters to move the decision boundaries in a way that x0 appears in a new classification region.. Why use this setting? Are there any intuitions? Since this is different from traditional adversarial attack terminology, the authors should stop using adversarial attacks as in tropical adversarial attacks because it is really misleading. As highlighted in the last sentence in section 6, we are not competing against other attacks, but we rather show how this new geometric view to the decision boundaries provided by the TG analysis in Theorem 2 can be leveraged for the construction of adversarial attacks. We want to emphasize to R3 that the polytope representing the decision boundaries (convex hull of two zonotopes as per Theorem 2) is a function of the network parameters and not of the input space. Thus, it is not initially clear how one can frame the adversarial attacks problem in this new fresh tropical setting since adversarial attacks is the task of perturbing the input space as opposed to the parameters space of the network ing in a flip in the prediction. In the tropical adversarial attacks section, we show that the problem of designing an adversarial attack x 0 +η that flips the network prediction is closely related to the problem of flipping the network prediction by perturbing the network parameters in the first layer A 1 + ζ A1 where both problems are related through a linear system. That is to say, if one finds ζ A1 that perturbs the geometric structure (convex hull between two zonotopes, i.e. decision boundaries) sufficiently enough to flip the network prediction, one can find an equivalent pixel adversarial attack η by solving the linear system A 1 η = ζ A1 x 0 that flips the prediction of the original unperturbed network (see the end of page 7). We thereafter propose Problem incorporating the geometric information from Theorem 2 where the linear system is accounted for in the constraints set. We propose an algorithm to solve the problem (a mix of penalty and ADMM) detailed in Algorithm 1 in the appendix. The solution to problem 5 by applying Algorithm 1 in the construction of adversarial attacks (η) that indeed flip the network prediction over all tested examples on the MNIST dataset. We thank R2 for the time spent reviewing the paper. We also thank R2 for acknowledging our technical and theoretical contributions. Please note that we have addressed the comments/typos/suggestions of all reviewers in the revised version and marked them in blue. Follows our response. Q1: This paper needs to be placed properly among several important missing references on the decision boundary of deep neural networks. In particular, using introduced tropical geometry perspective, how we can obtain the complexity of the decision boundary of a deep neural network? The two works referenced by R2 are not directly related to the body of our work. Below, we summarize both works and state how our work is vastly different from both. The authors of show that under certain assumptions, the decision boundaries of the last fully connected layer converges to an SVM classifier. That is to say, the features learnt in deep neural networks are linearly separable with max margin type linear classifier. On the other hand, the authors of showed that the decision regions of neural networks with width smaller than the input dimension are unbounded. In our work, we use a new type of analysis (tropical geometry) to represent the set of decision boundaries B through its superset T (R(x)) that is the solution set to the tropical polynomial R(x). We then show that this solution set is related to a geometric structure referred to as the decision boundaries polytope (convex hull between two zonotopes), this is analogous to constructing newton polytopes for the solution sets to classical polynomials in algebraic geoemtry. The normals to the edges of this polytope are parallel to the superset of the decision boundaries T (R(x)). That is to say, if one processes the polytope in an way that preserves the direction of the normals, the decision boundaries of the network are preserved. This is the base idea behind all later experiments. In general, this new representation presents a new fresh revisit to the lottery ticket hypothesis and an utterly new view to network pruning and adversarial attacks. We do believe this new representation can be of benefit to other applications and can open doors for a family of new geometrically inspired network regularizers as well. Q3: The second part of Theorem 2 should be explained straightforwardly and clearly as it plays an important role in the subsequent and applications. We have added a "Digesting Theorem 2" paragraph in the revised version and rearranged the structure a bit around Theorem 2. Most of the parameters (memory complexity) are the in fully connected layers. For example, the convolutional part of VGG16 has 14,714,688 parameters, whereas the fully connected layers have 262,000,400 parameters in total which is 17 times larger. Similarly, the convolutional part of AlexNet has 3,747,200 parameters while only the first fully connected layer has 37,752,832 parameters. However, efficiently extending the tropical pruning to convolutional layers is a nontrivial interesting direction. Generally speaking, convolutional layers fit our framework naturally since a convolutional kernel can be represented with a structured topelitz/circulant matrix. However, a question of efficiency still remains as one still needs to construct the underlying structured matrix representing the convolutional kernel. Thereafter, a direction of interest is the tropical formulation of the network pruning problem as a function of the convolutional kernels surpassing the need for the construction of the dense representation of the kernel. We keep this for future work. Q5: For the similarity measure. Comparing the exact decision boundaries between two different architectures can be very difficult in the sense where decision boundaries for a two-class output network f are defined as {x ∈ R n : f 1 (x) = f 2 (x)}. Another approach to compare decision boundaries, which is proposed by our work, is by computing the distance between the the dual subdivision polytope (δ(R(x))) of the tropical polynomials R(x) representing two different architectures. This is since the normals to the edges of the polytope (δ(R(x))) are parallel to a superset of the decision boundaries (see Figure 1). This is exactly the proposed objective where d is a distance function to compare the orientation between two general polytopes. Since finding a good choice for d is generally difficult we instead approximate it by comparing the generators constructing the (δ(R(x))) in Euclidean distance for ease (objective). Experimentally and following prior art in the pruning literature (Han et. al. 2015), to compare the effectiveness of the pruning scheme, we compare the test accuracies across architectures as a function of the pruning ratio. Regardless, the reviewer is right about that similar test accuracies does not imply similar decision boundaries but rather only an indication. In adversarial examples generation, typically for a pre-trained deep neural network model one is interested in generating examples that are misclassified by the model while they resemble real instances. In this setting, we keep the model and thus its decision boundary intact. In this paper, nevertheless, aiming at generating adversarial examples, the decision boundary and thus the (pre-trained) model is altered. By chaining the decision boundary, however, the model's decisions for original real samples might change as well. Therefore, it is not clear to the reviewer how the introduced method is comparable to the well-established adversarial example generation setting. The new approach is definitely comparable to the well-establisehd adversarial example generation setting. Let us explain. The new analysis provided by Theorem 2, allows to present the decision boundaries geometrically as a convex hull between two zonotopes which is a function of only the network parameters and not the input space. Thus, it not clear how one can frame the adversarial attacks problem in this new fresh tropical setting since adversarial attacks is the task of perturbing the input space as opposed to the parameters space of the network ing in a flip in the prediction. In the tropical adversarial attacks section, we show that the problem of designing an adversarial attack x 0 + η that flips the network prediction is closely related to the problem of flipping the network prediction by perturbing the network parameters in the first layer A 1 + ζ A1 where both problems are related through a linear system. That is to say, if one finds ζ A1 (perturbations in the first linear layer) that perturbs the geometric structure (convex hull between two zonotopes, i.e. decision boundaries) sufficiently enough to flip the network prediction, one can find an equivalent pixel adversarial attack η by solving the linear system A 1 η = ζ A1 x 0 that flips the prediction of the original unperturbed network (see the end of page 7). We incorporate this in an overall objective where the linear system is incorporated as a constraint. Upon solving with the proposed Algorithm 1, we attack the original network (unperturbed) with the adversarial attack η. Therefore, this is comparable to the classical adversarial attacks framework. This approach indeed ed into flipping the network prediction over all tested examples on the MNIST dataset. As highlighted at the end of section 6, we do not aim in our approach to outperform the state of the art adversarial attacks, but rather to provide a novel geometrically inspired perspective that can shed new light in this field. Q7: Two previous papers investigated the decision boundary of the deep neural networks in the presence of adversarial examples. Please discuss how the introduced method in this paper is placed among these methods. The work of analyzed the geometry of adversarial examples by means of manifold reconstruction to study the trade off between robustness under different norms. On the other hand, crafted adversarial attacks by estimating the distance to the decision boundaries using random search directions. Both of the papers made a local estimation of the decision boundary around the attacked point to construct the adversarial attack to the input image. In our work, we geometrically characterized the decision boundaries in Theorem 2 where the polytope (convex hull of two zonotopes) is only a function of the network parameters and NOT the input space. We presented a dual view to adversarial attacks in which one can construct adversarial examples by investigating network parameters perturbations that in the largest perturbation to this polytope representing the decision boundaries. The scope of our work, and unlike prior art, is focused towards a new geometric polytope representation of the decision boundary in the network parameter space (not the input space) through a new novel analysis. We have added a discussion of both papers in the adversarial attacks section (Section 6). We have addressed the concerns of R2 and left the changes in blue in the revised version. "On the decision boundary of deep neural networks". SLi, Yu and Richtarik, Peter and Ding, Lizhong and Gao, Xin. We thank R1 for the constructive detailed thorough review of the paper and for acknowledging our contributions and the new insights. Follows our response to R1's concerns. In regards to clarity, exposition and focus. To improve the clarity and exposition, we have added several paragraphs in the revised version of the paper. The revised edits are marked in blue. As in regards to the focus, we found that it is challenging within a reasonable time to carry out this major change in the paper. To that end, we have done our best to further elaborate on several key in the paper. For instance, we have merged the paragraph above the contributions with the contributions paragraph. We have added another paragraph dissecting Theorem 2. We have added some few relevant references that are essential for the context of motivating tropical adversarial attacks. In regards to the suggestions. • Adding the information that the semiring lacks the additive inverse. This has been addressed in the revised version. •Adding tropical quotient to definition 1. This has been addressed in the revised version. • Definition of π and the upper faces. Indeed, π is a projection operator that drops the last coordinate. As for upper faces, the formal definition is given as follows, for a polytope P, F is an upper face of P if x + te / ∈ P for any x ∈ F, t > 0 where e is a canonical vector. That is the faces that can be seen from "above". A good graphical example can be found in Figure 2 from Zhang et. al. 2018. We dropped this definition from the paper as it may add some confusion while it does not play an important role in the later analysis. • Theorem 2 lacks intuitive formulation. This has been addressed in the revised version and we have rearranged the structure of text below Theorem 2. • Regarding the issue of the tropical hypersurface. Indeed, the superset is in terms of set theory. That is to say, the set of decision boundaries B is a supset of the tropical hypersurface set T (R(x)). • Regarding Figure 2. The color map represents the compression percentage. We have added a legend to Figure 2. Note that the second figure in Figure 2 tilted "original polytope" represents the polytope of the dual subdivision (convex hull between two zonotopes). While the polytope seems to have only 4 vertices, there are in fact many other overlapping vertices and thereafter many small edges between all the seemingly overlapping vertices. It is to observe that the normals to only the 4 major edges in the "original polytope" are indeed parallel to the decision boundaries plotted by performing forward passes through the network in the first figure titled "Input Space". Note that despite the fact that more compression is performed, the orientation of the overall polytope is preserved for the lottery ticket initialization and thereafter preserving the main orientation of the decision boundaries. This is unlike the the other types of initialization where the orientation of the polytope is vastly different with different compression ratios ing into a larger change in the orientation of the decision boundaries. • I would suggest to place Figure 1 after stating Theorem 2, since it is only referenced later on. Furthermore, the red structures are somewhat confusing. According to Theorem 2, the decision boundary is a subset of the hypersurface, right? What is the relation of the red structures in the convex hull visualisation? The caption states that they are normals, but as far as I can tell, this has not been formalised anywhere in the paper (it is used later on, though). Correct. The decision boundaries are subsets of the tropical hypersurfaces. However, it has been shown by (Propositon 3.1.6) that the tropical hypersurface T to any d variate tropical polynomial is the (d-1)-skeleton of the polyhedral complex dual to the dual subdivision δ. This implies that the normals to the edges (faces in higher dimensions) are parallel to the tropical hypersurface. If R1 is interested in learning more about this, an excellent starting point to build up this intuition is the work of Erwan Brugall and Kristin Shaw "A bit of tropical geometry". Please refer to section 2.2 "Dual subdivisions" pages 6 and 7. • The discussion about the functional form. We elaborate on this in section F in the appendix. Instead of investigating the decision boundaries polytope of the tropical polynomial representing the decision boundaries R(x), we analyze the 4 different tropical polynomials representing the 2 different classes. Recall each output function of the network is a tropical rational of two tropical polynomials giving rise to 4 different polytopes. Figure 7 shows that the decision boundary polytope δ(R(x)) with the lottery ticket is hardly changed while pruning the network (first column in Figure 7). On the contrary, the zonotopes (dual subdivisions of the 4 different tropical polynomials) vary much more significantly (coloumns 2,3,4,5). This demonstrates that there can exist different tropical polynomials representing the functional form of the network (i.e. H 1,2 and Q 1,2) while having the same structure for the decision boundary polytope the corresponding first figure in the same row of Figure 7. This is a mere observation and worth investigating in future work. • In Section 4, how many experiments of the sort were performed? I find this a highly instructive view so I would love to see more experiments of this sort. Do these claims hold over multiple repetitions and for (slightly) larger architectures as well? We already had one extra experiment (Figure 8) in the appendix for another data. We have also based on the suggestion of R1 added two more experiments (Figure 9) on two other datasets. Note that extracting the decision boundaries polytope for larger architectures is much more difficult for two different reasons. First, for deeper networks beyond the structure Affine-ReLU-Affine, the decision boundaries polytope is generic and no longer enjoys the nice properties zonotopes exhibit. Enumerating their vertices rapidly turns to a computationally intractable problem. Secondly, one can perhaps visualize the polytope for networks with 3 dimensional input but it gets trickier beyond that. • Regarding that the claim that orientations are preserved should be formalised. That is an excellent question. We have investigated several metrics that try to capture information about the orientation of a given polytope. For instance, we have investigated the feature that is the histogram of the oriented normals. That is a histogram of angles for all the normals to edges given a polytope. We have also investigated the Hausdorff distance as a metric between polytopes. However, we have decided to keep this for a future direction as this is by itself a entirely new line of work. That is designing the distance functions d between polytopes that captures orientation information that can be used for tropical pruning (objective 8) or perhaps other applications. Another interesting direction is whether such a distance function can be learnt in a meta-learning fashion. For now, we restrict the experiments to the approximation used in objective which for now only captures the distance between the sets of generators of the zonotopes in Eucledian sense. • Adding link to the definition of minkowski sum in the appendix. This has been addressed in the revised version. • Description to other pruning methods. This has been addressed in the revised version where we have added the definition to all pruning competitors. • About the plots in Figure 4. In the experiments of Figure 4, there is no stochasticity. All base networks (AlexNet and VGG16) are trained before hand to achieve the state-of-art baseline on the respective datasets (SVHN, CIFAR10 and CIFAR100). The networks are then fixed and several pruning schemes, including the tropical approach, are applied with a varying level of pruning ratio. We do not re-train the networks after each pruning step. Thus there exists no source of randomness. However, this is still an excellent observation. This is since we conduct experiments in the appendix where after each pruning ratio we fine tune only the biases of the classifier (Figure 10) or we fine tune the biases of the complete network (Figure 11). In such experiments, due to the fine tuning step, we will consider in the final version to report the averaged over multiple runs. • In Section 6, I find the comment on normals generating a superset to the decision boundaries hard to understand. Indeed, the wording of the sentence was sub optimal. The statement was to reassure what has been established in earlier sections that the normals to the decision boundary polytope δ(R(x)) represent the tropical hypersurface set T (R(x)) which is a super set to the decision boundaries set B as per Theorem 2. We have rephrased the sentence. • Take-away message regarding perturbing the decision boundaries. The proposed approach of perturbing the decision boundaries is a mere dual view for adversarial attacks. That is to say, to flip a network prediction for a sample x 0, one can either adversarially perturb the sample (add noise) to cross the decision boundary to a new classification region or one can perturb the decision boundaries to move closer to the sample x 0 to appear as if it is in a new classification region. Figure 5, demonstrates the later visually through perturbing the decision boundary polytope. One can observe that perturbing the correct edge in the polytope in Figure 5, corresponds to altering a specific decision boundary. Figure 5, indeed as correctly pointed out of R1, is a feasibility study showing that one can perturb decision boundary by perturbing the dual subdiviosn polytope. However, the real take away message is that these two views (perturbing the input or the parameter space) are intimately related through a linear system as discussed in the subsection titled "Dual View to Adversarial Attacks". We propose an objective function (Problem 5) and an algorithm (Algorithm 1) to tackle this problem. The solution to problem provides an INPUT perturbation that in altering the network prediction. That is to say, the new framework allows for incorporating geometrically motivated objective function towards constructing classical adversarial attacks. • Regarding the future extension on CNNs and GCNs. We are currently investigating efficient extension of the current to convolutional layers. Note that while convolutional layers can be represented with a large structured topelitz/circulant matrix, we are interested in extensions that allow for similar analysis but as a function of the convolutional kernel surpassing the need to constructing convolutional matrix. The GCNs is definitely an exciting excellent future direction that we have not yet entertained. • Minor style issues. We have addressed all the style issues in the revised version.

Represent the TLDR for finding the corresponding scientific article

Tropical geometry can be leveraged to represent the decision boundaries of neural networks and bring to light interesting insights.

represent the short summary for finding the corresponding scientific article (long)
------
The provided query could be "We propose a novel approach to solve data-driven model-based optimization problems in both passive and active settings that can scale to high-dimensional input spaces." and the positive "In this work, we aim to solve data-driven optimization problems, where the goal is to find an input that maximizes an unknown score function given access to a dataset of input, score pairs. Inputs may lie on extremely thin manifolds in high-dimensional spaces, making the optimization prone to falling-off the manifold. Further, evaluating the unknown function may be expensive, so the algorithm should be able to exploit static, offline data. We propose model inversion networks (MINs) as an approach to solve such problems. Unlike prior work, MINs scale to extremely high-dimensional input spaces and can efficiently leverage offline logged datasets for optimization in both contextual and non-contextual settings. We show that MINs can also be extended to the active setting, commonly studied in prior work, via a simple, novel and effective scheme for active data collection. Our experiments show that MINs act as powerful optimizers on a range of contextual/non-contextual, static/active problems including optimization over images and protein designs and learning from logged bandit feedback. Data-driven optimization problems arise in a range of domains: from protein design to automated aircraft design , from the design of robots to the design of neural net architectures and learning from logged feedback, such as optimizing user preferences in recommender systems. Such problems require optimizing unknown reward or score functions using previously collected data consisting of pairs of inputs and corresponding score values, without direct access to the score function being optimized. This can be especially challenging when valid inputs lie on a low-dimensional manifold in the space of all inputs, e.g., the space of valid aircraft designs or valid images. Existing methods to solve such problems often use derivative-free optimization (Snoek et al.). Most of these techniques require active data collection where the unknown function is queried at new inputs. However, when function evaluation involves a complex real-world process, such as testing a new aircraft design or evaluating a new protein, such active methods can be very expensive. On the other hand, in many cases there is considerable prior data -existing aircraft and protein designs, and advertisements and user click rates, etc. -that could be leveraged to solve the optimization problem. In this work, our goal is to develop an optimization approach to solve such optimization problems that can readily operate on high-dimensional inputs comprising a narrow, low-dimensional manifold, such as natural images, readily utilize offline static data, and learn with minimal active data collection if needed. We can define this problem setting formally as the optimization problem where the function f (x) is unknown, and we have access to a dataset D = {(x 1, y 1),..., (x N, y N)}, where y i denotes the value f (x i). If no further data collection is possible, we call this the data-driven model-based optimization setting. This can also be extended to the contextual setting, where the aim is to optimize the expected score function value across a context distribution. That is, where π maps contexts c to inputs x, such that the expected score under the context distribution p 0 (c) is optimized. As before, f (c, x) is unknown and we have access to a dataset D = {(c i,, where y i is the value of f (c i, x i). Such contextual problems with logged datasets have been studied in the context of contextual bandits ). A simple way to approach these model-based optimization problems is to train a proxy function f θ (x) or f θ (c, x), with parameters θ, to approximate the true score, using the dataset D. However, directly using f θ (x) in place of the true function f (x) in Equation generally works poorly, because the optimizer will quickly find an input x for which f θ (x) outputs an erroneously large value. This issue is especially severe when the inputs x lie on a narrow manifold in a high-dimensional space, such as the set of natural images . The function f θ (x) is only valid near the training distribution, and can output erroneously large values when queried at points chosen by the optimizer. Prior work has sought to addresses this issue by using uncertainty estimation and Bayesian models for f θ (x), as well as active data collection (Snoek et al.). However, explicit uncertainty estimation is difficult when the function f θ (x) is very complex or when x is high-dimensional. Instead of learning f θ (x), we propose to learn the inverse function, mapping from values y to corresponding inputs x. This inverse mapping is one-to-many, and therefore requires a stochastic mapping, which we can express as f −1 θ (y, z) → x, where z is a random variable. We term such models model inversion networks (MINs). MINs provide us with a number of desirable properties: they can utilize static datasets, handle high-dimensional input spaces such as images, can handle contextual problems, and can accommodate both static datasets and active data collection. We discuss how to design simple active data collection methods for MINs, leverage advances in deep generative modeling (Goodfellow et al.;), and scale to very high-dimensional input spaces. We experimentally demonstrate MINs in a range of settings, showing that they outperform prior methods on high-dimensional input spaces, perform competitively to Bayesian optimization methods on tasks with active data collection and lower-dimensional inputs, and substantially outperform prior methods on contextual optimization from logged data (Swaminathan & Joachims, a). Bayesian optimization. In this paper, we aim to solve data-driven optimization problems. Most prior work aimed at solving such optimization problems has focused on the active setting. This includes algorithms such as the cross entropy method (CEM) and related derivative-free methods; , reward weighted regression Peters & Schaal, Bayesian optimization methods based on Gaussian processes;, and variants that replace GPs with parametric acquisition function approximators such as Bayesian neural networks and latent variable models (; b; a), as well as more recent methods such as CbAS . These methods require the ability to query the true function f (x) at each iteration to iteratively arrive at a near-optimal solution. We show in Section 3.3 that MINs can be applied to such an active setting as well, and in our experiments we show that MINs can perform competitively with these prior methods. Additionally, we show that MINs can be applied to the static setting, where these prior methods are not applicable. Furthermore, most conventional BO methods do not scale favourably to high-dimensional input spaces, such as images, while MINs can handle image inputs effectively. Contextual bandits. Equation 2 captures the class of contextual bandit problems. Prior work on batch contextual bandits has focused on batch learning from bandit feedback (BLBF), where the learner needs to produce the best possible policy that optimizes the score function from logged experience. Existing approaches build on the counterfactual risk minimization (CRM) principle (Swaminathan & Joachims, a;b), and have been extended to work with deep nets . In our comparisons, we find that MINs substantially outperform these prior methods in the batch contextual bandit setting. Deep generative modeling. Recently, deep generative modeling approaches have been very successful at modelling high-dimensional manifolds such as natural images (Goodfellow et al.; Van Den Oord et al.;), speech (van den), text (Yu et al.), alloy composition prediction (Nguyen et al.), etc. MINs combine the strength of such generative models with important algorithmic decisions to solve model-based optimization problems. In our experimental evaluation, we show that these design decisions are important for adapting deep generative models to model-based optimization, and it is difficult to perform effective optimization without them. In this section, we describe our model inversion networks (MINs) method, which can perform both active and passive model-based optimization over high-dimensional input spaces. Problem statement. Our goal is to solve optimization problems of the form x = arg max x f (x), where the function f (x) is not known, but we must instead use a dataset of input-output tuples D = {(x i, y i)}. In the contextual setting described in Equation, each datapoint is also associated with a context c i. For clarity, we present our method in the non-contextual setting, but the contextual setting can be derived analogously by conditioning all functions on the context. In the active setting, which is most often studied in prior work, the algorithm is allowed to actively query f (x) one or more times on each iteration to augment the dataset, while in the static setting, only an initial static dataset is available. The goal is to obtain the best possible x (i.e., the one with highest possible value of f (x)). One naïve way of solving MBO problems is to learn a proxy score function f θ (x), via standard empirical risk minimization. We could then maximize this learned function with respect to x via standard optimization methods. However, naïve applications of such a method would fail for two reasons. First, the proxy function f θ (x) may not be accurate outside the samples on which it is trained, and optimization with respect to it may simply lead to values of x for which f θ (x) makes the largest mistake in the negative direction. The second problem is more subtle. When x lies on a narrow manifold in very high-dimensional space (such as the space of natural images), the optimizer can produce invalid values of x, which in arbitrary outputs when fed into f θ (x). Since the shape of this manifold is unknown, it is difficult to constrain the optimizer to prevent this. This second problem is rarely addressed or discussed in prior work, which typically focuses on optimization over low-dimensional and compact domains with known bounds. Part of the reason for the brittleness of the naïve approach above is that f θ (x) has a high-dimensional input space, making it easy for the optimizer to find inputs x for which the proxy function produces an unreasonable output. Can we instead learn a function with a small input space, which implicitly understands the space of valid, in-distribution values for x? The main idea behind our approach is to model an inverse map that produces a value of x given a score value y, given by f −1 θ: Y → X. The input to the inverse map is a scalar, making it comparatively easy to constrain to valid values, and by directly generating the inputs x, an approximation to the inverse function must implicitly understand which input values are valid. As multiple x values can correspond to the same y, we design f −1 θ as a stochastic map that maps a score value along with a d z -dimensional random vector to a x, f −1 θ: Y × Z → X, where z is distributed according to a prior distribution p 0 (z). To define the inverse map objective, let the data distribution be denoted p D (x, y), let p D (y) be the marginal over y, and let p(y) be an any distribution defined on Y (which could be equal to p D (y)). We can train the proxy inverse map f −1 θ under distribution p(y) by minimizing the following objective: where p f, and D is a measure of divergence between the two distributions. Using the Kullback-Leibler divergence leads to maximum likelihood learning, while Jensen-Shannon divergence motivates a GAN-style training objective. MINs can be adapted to the contextual setting by passing in the context as an input and learning f −1 θ (y i, z, c i). In standard empirical risk minimization, we would choose p(y) to be the data distribution p D (y), such that the expectation be approximated simply by sampling training tuples (x i, y i) from the training set. However, as we will discuss in Section 3.3, a more careful choice for p(y) can lead to better performance. The MIN algorithm is based on training an inverse map, and then using it via the inference procedure in Section 3.2 to infer the x that approximately optimizes f (x). The structure of the MIN algorithm is shown in Algorithm 1. Once the inverse map is trained, the goal of our algorithm is to generate the best possible x, which will maximize the true score function as well as possible under the dataset. Since a score y needs to be provided as input to the inverse map, we must select for which score y to query the inverse map to obtain a near-optimal x. One naïve heuristic is to pick the best y max ∈ D and produce x max ∼ f −1 θ (y * max) as the output. However, the method should be able to extrapolate beyond the best score seen in the dataset, especially in contextual settings, where a good score may not have been observed for all contexts. In order to extrapolate as far as possible, while still staying on the valid data manifold, we need to measure the validity of the generated values of x. One way to do this is to measure the agreement between the learned inverse map and an independently trained forward model f θ: the values of y for which the generated samples x are predicted to have a score similar to y are likely in-distribution, whereas those where the forward model predicts a very different score may be too far outside the training distribution. Since the latent variable z captures the multiple possible outputs of the one-tomany inverse map, we can further optimize over z for a given y to find the best, most trustworthy output x. This can be formalized as the following optimization: y *,z *:= arg max This optimization can be motivated as finding an extrapolated score that corresponds to values of x that lie on the valid input manifold, and for which independently trained forward and inverse maps agree. Although this optimization uses an approximate forward map f θ (x), we show in our experiments in Section 4 that it produces substantially better than optimizing with respect to a forward model alone. The inverse map substantially constraints the search space, requiring an optimization over a 1-dimensional y and a (relatively) low-dimensional z, rather than the full space of inputs. This scheme can be viewed as a special (deterministic) case of a probabilistic optimization procedure described in Appendix A. A naïve implementation of the training objective in Equation samples y from the data distribution p D (y). However, as we are most interested in the inverse map's predictions for high values of y, it is much less important for the inverse map to predict accurate x values for values of y that are far from the optimum. We could consider increasing the weights on datapoints with larger values of y. In the extreme case, we could train only on the best datapoint -either the single datapoint with the largest y or, in the contextual case, the datapoint with the largest y for each context. More generally, we can define the optimal y distribution p * (y), which is simply the delta function centered on the best y, p * (y) = δ y * (y), in the deterministic case. If we instead assume that the observed scores have additive noise (i.e., we observe f (x) + ε, ε ∼ N ), then p * (y) would be a distribution centered around the optimal y. Of course, training on p * (y) is not practical, since it heavily down-weights most of the training data, leading to a very high-variance training objective, and is not even known in general, since the optimal data point is likely not in our training set. In this section, we will propose a better choice for p(y) that trades off the variance due to an overly peaked training distribution and the bias due to training on the "wrong" distribution (i.e., anything other than p * (y)). We can train under a distribution other than the empirical distribution by using importance sampling, such that we sample from p D and assign an importance weight, given by. By bounding the variance and the bias of the gradient of L p (D) estimate, with respect to the reweighted objective without sampling error under y drawn from p * (y), we obtain the following : (Proof in Appendix B) Theorem 3.1 ((Informal) Bias + variance bound in MINs). Let L(p *) be the objective under p * (y) without sampling error: Let N y be the number of datapoints with the particular y value observed in D, For some constants C 1, C 2, C 3, with high confidence, Theorem 3.1 suggests a tradeoff between being close to the optimal distribution p * (y) and reducing variance by covering the full data distribution p D. We observe that the distribution p(y) that minimizes the RHS bound in Theorem 3.1 has the following form: linear function of p * (y) that ensures that the distributions p and p * are close. Theoretically, g(•) is an increasing, piece-wise linear function of •. We can interpret the expression for p(y) as a product of two likelihoods -the optimality of a particular y value and the likelihood of a particular y not being rare in D. We empirically choose an exponential parameteric form for this function, which we describe in Section 3.5. This upweights the samples with higher scores, reduces the weight on rare y-values (i.e., those with low N y), while preventing the weight on common y-values from growing, since Ny Ny+K saturates to 1 for large N y. This is consistent with our intuition: we would like to upweight datapoints with high y-values, provided the number of samples at those values is not too low. Of course, for continuous-valued scores, we rarely see the same score twice. Therefore, we bin the y-values into discrete bins for the purpose of weighting, as we discuss in Section 3.5. While the passive setting requires care in finding the best value of y for the inverse map, the active setting presents a different challenge: choosing a new query point x at each iteration to augment the dataset D and make it possible to find the best possible optimum. Prior work on bandits and Bayesian optimization often uses Thompson sampling (TS) (; ; Srinivas et al.) as the data-collection strategy. TS maintains a posterior distribution over functions p(f t |D 1:t). At each iteration, it samples a function from this distribution and queries the point x t that greedily minimizes this function. TS offers an appealing query mechanism, since it achieves sub-linear Bayesian regret (defined as the expected cumulative difference between the value of the optimal input and the selected input), given by O(where T is the number of queries. Maintaining a posterior over high-dimensional parametric functions is generally intractable. However, we can devise a scheme to approximate Thompson sampling with MINs. To derive this method, first note that sampling f t from the posterior is equivalent to sampling (x, y) pairs consistent with f tgiven sufficiently many (x, y) pairs, there is a unique smooth function f t that satisfies For example, we can infer a quadratic function exactly from three points. For a more formal description, we refer readers to the notion of Eluder dimension (Russo & Van Roy). Thus, instead of maintaining intractable beliefs over the function, we identify a function by the samples it generates, and define a way to sample synthetic (x, y) points such that they implicitly define a unique function sample from the posterior. To apply this idea to MINs, we train the inverse map f −1 θt at each iteration t with an augmented is a dataset of synthetically generated input-score pairs corresponding to unseen y values in D t. Training f −1 θt on D t corresponds to training f −1 θt to be an approximate inverse map for a function f t sampled from p(f t |D 1:t), as the synthetically generated samples S t implicitly induce a model of f t. We can then approximate Thompson sampling by obtaining x t from f −1 θt, labeling it via the true function, and adding it to D t to produce D t+1. Pseudocode for this method, which we call "randomized labeling," is presented in Algorithm 2. In Appendix C, we further derive O(√ T) regret guarantees under mild assumptions. Implementationwise, this method is simple, does not require estimating explicit uncertainty, and works with arbitrary function classes, including deep neural networks. corresponding to unseen data points yi (by randomly pairing noisy observed xi values with unobserved y values.) 4: Train inverse map f −1 t on D t = Dt ∪ St, using reweighting described in Section 3.3. 5: Query function f at xt = f Observe outcome: (xt, f (xt)) and update Dt+1 = Dt ∪ (xt, f (xt)) 7: end for In this section, we describe our instantiation of MINs for high-dimensional inputs with deep neural network models. GANs (Goodfellow et al.) have been successfully used to model the manifold of high-dimensional inputs, without the need for explicit density modelling and are known to produce more realistic samples than other models such as VAEs or Flows . The inverse map in MINs needs to model the manifold of valid x thus making GANs a suitable choice. We can instantiate our inverse map with a GAN by choosing D in Equation 3 to be the Jensen-Shannon divergence measure. Since we generate x conditioned on y, the discriminator is parameterized as Disc(x|y), and trained to output 1 for a valid (x, y) pair (i.e., where y = f (x) and x comes from the data) and 0 otherwise. Thus, we optimize the following objective: This model is similar to a conditional GAN (cGAN), which has been used in the context of modeling distribution of x conditioned on a discrete-valued label . As discussed in Section 3.3, we additionally reweight the data distribution using importance sampling. To that end, we discretize the space Y into B discrete bins b 1, · · ·, b B and, following Section 3.3, weight each bin, where N bi is the number of datapoints in the bin, y * is the maximum score observed, and τ is a hyperparameter. (After discretization, using notation from Section 3.3, for any y that lies in bin b, p In the active setting, we perform active data collection using the synthetic relabelling algorithm described in Section 3.4. In practice, we train two copies of f −1 θ . The first, which we call the exploration model f −1 expl, is trained with data augmented via synthetically generated samples (i.e., D t). The other copy, called the exploitation model f −1 exploit, is trained on only real samples (i.e., D t). This improves stability during training, while still performing data collection as dictated by Algorithm 2. To generate the augmented dataset D t in practice, we sample y values from p * (y) (the distribution over high-scoring ys observed in D t), and add positive-valued noise, thus making the augmented y values higher than those in the dataset which promotes exploration. The corresponding inputs x are simply sampled from the dataset D t or uniformly sampled from the bounded input domain when provided in the problem statement. (for example, benchmark function optimization) After training, we infer best possible x from the trained model using the inference procedure described in Section 3.2. In the active setting, the inference procedure is applied on f −1 exploit, the inverse map that is trained only on real data points. The goal of our empirical evaluation is to answer the following questions. Can MINs successfully solve optimization problems of the form shown in Equations 1 and 2, in static settings and active settings, better than or comparable to prior methods? Can MINs generalize to high dimensional spaces, where valid inputs x lie on a lower-dimensional manifold, such as the space of natural images? Is reweighting the data distribution important for effective data-driven model-based optimization? Does our proposed inference procedure effectively discover valid inputs x with better values than any value seen in the dataset? Does randomized labeling help in active data collection? We first study the data-driven model-based optimization setting. This requires generating points that achieve a better function value than any point in the training set or, in the contextual setting, better than the policy that generated the dataset for each context. We evaluate our method on a batch contextual bandit task proposed in prior work and on a high-dimensional contextual image optimization task. We also evaluate our method on several non-contextual tasks that require optimizing over high-dimensional image inputs to evaluate a semantic score function, including hand-written characters and real-world photographs. Batch contextual bandits. We first study the contextual optimization problem described in Equation 2. The goal is to learn a policy, purely from static data, that predicts the correct bandit arm x for each context c, such that the policy achieves a high overall score f (c, π(c)) on average across contexts drawn from a distribution p 0 (c). We follow the protocol set out by , while the BanditNet column is our implementation; we were unable to replicate the performance from prior work (details in Appendix D). MINs outperform both BanditNet and BanditNet *, both with and without the inference procedure in Section 3.2. MINs w/o reweighting perform at par with full MINs on MNIST, and slightly worse on CIFAR 10, while still outperforming the baseline. which evaluates contextual bandit policies trained on a static dataset for a simulated classification tasks. The data is constructed by selecting images from the (MNIST/CIFAR) dataset as the context c, a random label as the input x, and a binary indicator indicating whether or not the label is correct as the score y. Multiple schemes can be used for selecting random labels for generating the dataset, and we evaluate on two such schemes, as described below. We report the average score on a set of new contexts, which is equal to the average 0-1 accuracy of the learned model on a held out test set of images (contexts). We compare our method to previously proposed techniques, including the BanditNet model proposed by on the MNIST and CIFAR-10 datasets. Note that this task is different from regular classification, in that the observed feedback ((c i, x i, y i) pairs) is partial, i.e. we do not observe the correct label for each context (image) c i, but only whether or not the label in the training tuple is correct or not. We evaluate on two datasets: data generated by selecting random labels x i for each context c i and data where the correct label is used 49% of the time, which matches the protocol in prior work . We compare to BanditNet on identical dataset splits. We report the average 0-1 test accuracy for all methods in Table 1. The show that MINs drastically outperform BanditNet on both MNIST and CIFAR-10 datasets, indicating that MINs can successfully perform contextual model-based optimization in the static (data-driven) setting. The also show that utilizing the inference procedure in Section 3.2 produces an improvement of about 1.5% and 1.0% in test-accuracy on MNIST and CIFAR-10, respectively. Character stroke width optimization. In the next experiment, we study how well MINs optimize over high-dimensional inputs, where valid inputs lie on a lower-dimensional manifold. We constructed an image optimization task out of the MNIST dataset. The goal is to optimize directly over the image pixels, to produce images with the thickest stroke width, such that the image corresponds either (a) to any valid character or (b) a valid instance of a particular character class. A Figure 2: MIN optimization to obtain the youngest faces when trained on faces older than 15 (left) and older than 25 (right). Generated faces (bottom) are obtained via inference in the inverse map at different points during model training. Real faces of varying ages (including ages lower than those used to train the model) are shown in the top rows. We overlay the actual age (negative of the score function) for each face on the real images, and the age obtained from subjective user rankings on the generated faces. The score function being optimized (maximized) in this case is the negative age of the face. successful algorithm will produce the thickest character that is still recognizable. In Figure 1, we observe that MINs generate images x that maximize the respective score functions in each case. We also evaluate on a harder task where the goal is to maximize the number of disconnected blobs of black pixels in an image of a digit. For comparison, we evaluate a method that directly optimizes the image pixels with respect to a forward model, of the form f θ (x). In this case, the solutions are far off the manifold of valid characters. We also compare to MINs without the reweighting scheme and the inference procedure, where y is the maximum possible y in the dataset to demonstrate the benefits of these two aspects. Semantic image optimization. The goal in these tasks is to quantify the ability of MINs to optimize high-level properties that require semantic understanding of images. We consider MBO tasks on the IMDB-Wiki faces (; dataset, where the function f (x) is the negative of the age of the person in the image. Hence, images with younger people have higher scores. We construct two versions of this task: one where the training data consists of all faces older than 15 years, and the other where the model is trained on all faces older than 25 years. This ensures that our model cannot simply copy the youngest face. To obtain ground truth scores for the generated faces, we use subjective judgement from human participants. We perform a study with 13 users. Each user was asked to answer a set of 35 binary-choice questions each asking the user to pick the older image of the two provided alternatives. We then fit an age function to this set of binary preferences, analogously to. Figure 2 shows the images produced by MINs. For comparison, we also present some sample of images from the dataset partitioned by the ground truth score. We find that the most likely age for optimal images produced by training MINs on images of people 15 years or older was 13.6 years, with the best image having an age of 12.2. The model trained on ages 25 and above produced more mixed , with an average age of 26.2, and a minimum age of 23.9. We report these in Table 2. This task is exceptionally difficult, since the model must extrapolate outside of the ages seen in the training set, picking up on patterns in the images that can be used to produce faces that appear younger than any face that the model had seen, while avoiding unrealistic images. We also conducted experiments on contextual image optimization with MINs. We studied contextual optimization over hand-written digits to maximize stroke width, using either the character category as the context c, or the top one-fourth or top half of the image. In the latter case, MINs must learn to complete the image while maximizing for the stroke width. In the case of class-conditioned optimization, MINs attain an average score over the classes of 237.6, while the dataset average is 149.0. In the case where the context is the top half or quarter of the image, MINs obtain average scores of 223.57 and 234.32, respectively, while the dataset average is 149.0 for both tasks. We report these in Table 3. We also conducted a contextual optimization experiment on faces from the Celeb-A dataset, with some example images shown in Figure 3. The context corresponds to the choice for the attributes brown hair, black hair, bangs, or moustache. The optimization score is given by the sum of the attributes wavy hair, eyeglasses, smiling, and no beard. Qualitatively, we can see that MINs successfully optimize the score while obeying the target context, though evaluating the true score is impossible without subjective judgement on this task. We discuss these experiments in more detail in Appendix D.1. Figure 3: Optimized x produced from contextual training on Celeb-A. Context = (brown hair, black hair, bangs, moustache and f (x) = 1(wavy hair, eyeglasses, smiling, no beard). We show the produced x for two contexts. The model optimizes score for both observed contexts such as brown or black hair and extrapolates to unobserved contexts such as brown and black hair. In the active MBO setting, MINs must select which new datapoints to query to improve their estimate of the optimal input. In this setting, we compare to prior model-based optimization methods, and evaluate the exploration technique described in Section 3.4. Global optimization on benchmark functions. We first compare MINs to prior work in Bayesian optimization on standard benchmark problems (DNGO) : the 2D Branin function, and the 6D Hartmann function. As shown in Table 4, MINs reach within ±0.1 units of the global minimum (minimization is performed here, instead of maximization), performing comparably with commonly used Bayesian optimization methods based on Gaussian processes. We do not expect MINs to be as efficient as GP-based methods, since MINs rely on training parametric neural networks with many parameters, which is less efficient than GPs on low-dimensional tasks. Exact Gaussian processes and adaptive Bayesian linear regression outperform MINs in terms of optimization precision and the number of samples queried, but MINs achieve comparable performance with about 4× more samples. We also report the performance of MINs without the random labeling exploration method, instead selecting the next query point by greedily maximizing the current model with some additive noise. We find that the random relabeling method produces substantially better than the greedy data collection approach, indicating the importance of effective exploration methods for MINs. Function Spearmint DNGO MIN MIN + greedy Branin (0.398) 0.398 ± 0.0 0.398 ± 0.0 0.398 ± 0.02 0.4 ± 0.05 Hartmann6 (-3.322) −3.3166 ± 0.02 −3.319 ± 0.00 −3.315 ± 0.05 −3.092 ± 0.12 Protein fluorescence maximization. In the next experiment, we study a high-dimensional active MBO task, previously studied by. This task requires optimizing over protein designs by selecting variable length sequences of codons, where each codon can take on one of 20 values. In order to model discrete values, we use a Gumbel-softmax GAN also previously employed in , and as a baseline in . For backpropagation, we choose a temperature τ = 0.75 for the Gumbel-softmax operation. This is also mentioned in Appendix D. The aim in this task is to produce a protein with maximum fluorescence. Each algorithm is provided with a starting dataset, and then allowed a identical, limited number of score function queries. For each query made by an algorithm, it receives a score value from an oracle. We use the trained oracles released by. These oracles are separately trained forward models, and can potentially be inaccurate, especially for datapoints not observed in the starting static dataset. We compare to CbAS and other baselines, including CEM (Cross Entropy Method), RWR (Reward Weighted Regression) and a method that uses a forward model -GB (Gómez-) reported by. For evaluation, we report the groundtruth score of the output of optimization (max), and the 50th-percentile groundtruth score of all the samples produced via sampling (this is without inference in the MIN case) so as to be comparable to. In Table 5, we show that MINs are comparable to the best performing method on this task, and produce samples with the highest score among all the methods considered. These suggest that MINs can perform competitively with previously proposed model-based optimization methods in the active setting, reaching comparable or better performance when compared both to Bayesian optimization methods and previously proposed methods for a higher-dimensional protein design task. In this work, we presented a novel approach towards model-based optimization (MBO). Instead of learning a proxy forward function f θ (x) from inputs x to scores y, MINs learn a stochastic inverse mapping from scores y to inputs. MINs are resistent to out-of-distribution inputs and can optimize over high dimensional x values where valid inputs lie on a narrow manifold. By using simple and principled design decisions, such as re-weighting the data distribution, MINs can perform effective model-based optimization even from static, previously collected datasets in the data-driven setting without the need for active data collection. We also described ways to perform active data collection if needed. Our experiments showed that MINs are capable of solving MBO optimization tasks in both contextual and non-contextual settings, and are effective over highly semantic score functions such as age of the person in an image. Prior work has usually considered MBO in the active or "onpolicy" setting, where the algorithm actively queries data as it learns. In this work, we introduced the data-driven MBO problem statement and devised a method to perform optimization in such scenarios. This is important in settings where data collection is expensive and where abundant datasets exist, for example, protein design, aircraft design and drug design. Further, MINs define a family of algorithms that show promising on MBO problems on extremely large input spaces. While MINs scale to high-dimensional tasks such as model-based optimization over images, and are performant in both contextual and non-contextual settings, we believe there are a number of interesting open questions for future work. The interaction between active data collection and reweighting should be investigated in more detail, and poses interesting consequences for MBO, bandits and reinforcement learning. Better and more principled inference procedures are also a direction for future work. Another avenue is to study various choices of training objectives in MIN optimization. In this section, we show that the inference scheme described in Equation 4, Section 3.2 emerges as a deterministic relaxation of the probabilistic inference scheme described below. We re-iterate that in Section 3.2, a singleton x * is the output of optimization, however the procedure can be motivated from the perspective of the following probabilistic inference scheme. Let p(x|y) denote a stochastic inverse map, and let p f (y|x) be a probabilistic forward map. Consider the following optimization problem: arg max where p θ (x|y) is the probability distribution induced by the learned inverse map (in our case, this corresponds to the distribution of f −1 θ (y, z) induced due to randomness in z ∼ p 0 (·)), p f (x|y) is the learned forward map, H is Shannon entropy, and D is KL-divergence measure between two distributions. In Equation 4, maximization is carried out over the input y to the inverse-map, and the input z which is captured inp in the above optimization problem, i.e. maximization over z in Equation 4 is equivalent to choosingp subject to the choice of singleton/ Dirac-deltap. The Lagrangian is given by: In order to derive Equation 4, we restrictp to the Dirac-delta distribution generated by querying the learned inverse map f −1 θ at a specific value of z. Now note that the first term in the Lagrangian corresponds to maximizing the "reconstructed"ŷ similarly to the first term in Equation 4. If p f is assumed to be a Gaussian random variable with a fixed variance, then log p f (ŷ|x) = −||ŷ − µ(x)|| Finally, in order to obtain the log p 0 (z) term, note that, D(p(x|y), p θ (x|y)) ≤ D(δ z (·), p 0 (·)) = − log p 0 (z) (by the data processing inequality for KL-divergence). Hence, constraining log p 0 (z) instead of the true divergence gives us a lower bound on L. Maximizing this lower bound (which is the same as Equation 4) hence also maximizes the true Lagrangian L. In this section, we provide details on the bias-variance tradeoff that arises in MIN training. Our analysis is primarily based on analysing the bias and variance in the 2 norm of the gradient in two cases -if we had access to infinte samples of the distribution over optimal ys, p * (y) (this is a Dirac-delta distribution when function f (x) evaluations are deterministic, and a distribution with non-zero variance when the function evaluations are stochastic or are corrupted by noise). Let −1 (y j)) denote the empirical objective that the inverse map is trained with. We first analyze the variance of the gradient estimator in Lemma B.2. In order to analyse this, we will need the expression for variance of the importance sampling estimator, which is captured in the following Lemma. Lemma B.1 (Variance of IS ). Let P and Q be two probability measures on the space (X, F) such that d 2 (P ||Q) < ∞. Let x 1, · · ·, x N be N randomly drawn samples from Q, and f: X → R is a uniformly-bounded function. Then for any δ ∈, with probability atleast 1 − δ, Equipped with Lemma B.1, we are ready to show the variance in the gradient due to reweighting to a distribution for which only a few datapoints are observed. θ. Let N y denote the number of datapoints observed in D with score equal to y, and letL p (D) be as defined, where the expectation is computed with respect to the dataset D. Then, there exist some constants C 1, C 2 such that with a confidence at least 1 − δ, Proof. We first bound the range in which the random variable ∇ θLp (D) can take values as a function of number of samples observed for each y. All the steps follow with high probability, i.e. with probability greater than 1 − δ, is the exponentiated Renyi-divergence between the two distributions p and q, i.e. dy. The first step follows by applying Hoeffding's inequality on each inner term in the sum corresponding to y j and then bounding the variance due to importance sampling ys finally using concentration bounds on variance of importance sampling using Lemma B.1. Thus, the gradient can fluctuate in the entire range of values as defined above with high probability. Thus, with high probability, atleast 1 − δ, The next step is to bound the bias in the gradient that arises due to training on a different distribution than the distribution of optimal ys, p * (y). This can be written as follows: where D TV is the total variation divergence between two distributions p and p *, and L is a constant that depends on the maximum magnitude of the divergence measure D. Combining Lemma B.2 and the above , we prove Theorem 3.1. In this section, we explain in more detail the randomized labeling algorithm described in Section 3.4. We first revisit Thompson sampling, then provide arguments for how our randomized labeling algorithm relates to it, highlight the differences, and then prove a regret bound for this scheme under mild assumptions for this algorithm. Our proof follows commonly available proof strategies for Thompson sampling. 1: Initialize a policy πa: X → R, data so-far D0 = {}, a prior over θ in f θ -P (θ * |D0) 2: for step t in {0, . . ., T-1} do 3: θt ∼ P (θ * |Ft) (Sample θt from the posterior) 4: Query xt = arg maxx E[f θ t (x) | θ = θt] (Query based on the posterior probability xt is optimal) 5: Observe outcome: (xt, f (xt)) 6: Dt+1 = Dt ∪ (xt, f (xt)) 7: end for Notation The TS algorithm queries the true function f at locations (x t) t∈N and observes true function values at these points f (x t). The true function f (x) is one of many possible functions that can be defined over the space R |X |. Instead of representing the true objective function as a point object, it is common to represent a distribution p * over the true function f. This is justified because, often, multiple parameter assignments θ, can give us the same overall function. We parameterize f by a set of parameters θ *. The T period regret over queries x 1, · · ·, x T is given by the random variable Since selection of x t can be a stochastic, we analyse Bayes risk , we define the Bayes risk as the expected regret over randomness in choosing x t, observing f (x t), and over the prior distribution P (θ *). This definition is consistent with. Let π TS be the policy with which Thompson sampling queries new datapoints. We do not make any assumptions on the stochasticity of π TS, therefore, it can be a stochastic policy in general. However, we make 2 assumptions (A1, A2). The same assumptions have been made in.: sup x f (x) − inf x f (x) ≤ 1 (Difference between max and min scores is bounded by 1) -If this is not true, we can scale the function values so that this becomes true. A2: Effective size of X is finite. 1 TS (Alg 3) queries the function value at x based on the posterior probability that x is optimal. More formally, the distribution that TS queries x t from can be written as: π TS t = P (x * = ·|D t). When we use parameters θ to represent the function parameter, and thus this reduces to sampling an input that is optimal with respect to the current posterior at each iteration: MINs (Alg 2) train inverse maps f θ (z, y), where y ∈ R. We call an inverse map optimal if it is uniformly optimal given θ t, i.e. ||f where ε t is controllable (usually the case in supervised learning, errors can be controlled by cross-validation). Now, we are ready to show that the regret incurred the randomized labelling active data collection scheme is bounded by O(√ T). Our proof follows the analysis of Thompson sampling presented in. We first define information ratio and then use it to prove the regret bound. related the expected regret of TS to its expected information gain i.e. the expected reduction in the entropy of the posterior distribution of X *. Information ratio captures this quantity, and is defined as: where I(·, ·) is the mutual information between two random variables and all expectations E t are defined to be conditioned on D t. If the information ratio is small, Thompson sampling can only incur large regret when it is expected to gain a lot of information about which x is optimal. then bounded the expected regret in terms of the maximum amount of information any algorithm could expect to acquire, which they observed is at most the entropy of the prior distribution of the optimal x. Lemma C.1 (Bayes-regret of vanilla TS) ). For any T ∈ N, if Γ t ≤ Γ (i.e. information ratio is bounded above) a.s. for each t ∈ {1, . . ., T}, We refer the readers to the proof of Proposition 1 in. The proof presented in does not rely specifically on the property that the query made by the Thompson sampling algorithm at each iteration x t is posterior optimal, but rather it suffices to have a bound on the maximum value of the information ratio Γ t at each iteration t. Thus, if an algorithm chooses to query the true function at a datapoint x t such that these queries always contribute in learning more about the optimal function, i.e. I(·, ·) appearing in the denominator of Γ is always more than a threshold, then information ratio is lower bounded, and that active data collection algorithm will have a sublinear asymptotic regret. We are interested in the case when the active data collection algorithm queries a datapoint x t at iteration t, such that x t is the optimum for a functionfθ t, wherê θ t is a sample from the posterior distribution over θ t, i.e.θ t lies in the high confidence region of the posterior distribution over θ t given the data D t seen so far. In this case, the mutual information between the optimal datapoint x and the observed (x t, f (x t)) input-score pair is likely to be greater than 0. More formally, The randomized labeling scheme for active data collection in MINs performs this step. The algorithm samples a bunch of (x, y) datapoints, sythetically generated, -for example, in our experiments, we add noise to the values of x, and randomly pair them with unobserved or rarely observed values of y. If the underlying true function f is smooth, then there exist a finite number of points that are sufficient to uniquely describe this function f. One measure to formally characterize this finite number of points that are needed to uniquely identify all functions in a function class is given by Eluder dimension (Russo & Van Roy). By augmenting synthetic datapoints and training the inverse map on this data, the MIN algorithm ensures that the inverse map is implicitly trained to be an accurate inverse for the unique function fθ t that is consistent with the set of points in the dataset D t and the augmented set S t. Which sets of functions can this scheme represent? The functions should be consistent with the data seen so far D t, and can take randomly distributed values outside of the seen datapoints. This can roughly argued to be a sample from the posterior over functions, which Thompson sampling would have maintained given identical history D t. Lemma C.2 (Bounded-error training of the posterior-optimal x t preserves asymptotic Bayes-regret). ∀t ∈ N, letx t be any input such that f (x t) ≥ max x E[f (x)|D t ] − ε t. If MIN chooses to query the true function atx t and if the sequence (ε t) t∈N satisfies T t=0 ε t = O(√ T), then, the regret from querying this ε t -optimalx t which is denoted in general as the policyπ TS is given by E[Regret(T,π Proof. This lemma intuitively shows that if posterior-optimal inputs x t can be "approximately" queried at each iteration, we can still maintain sublinear regret. To see this, note: The second term can be bounded by the absolute value in the worst case, which amounts T t=0 ε t extra Bayesian regret. As Bayesian regret of TS is O( √ T) and Theorem C.3 (Bayesian Regret of randomized labeling active data collection scheme proposed in Section 3.4 is O( √ T)). Regret incurred by the MIN algorithm with randomized labeling is of the order O((ΓH(X *) + C)T ). Proof. Simply put, we will combine the insight about the mutual information I(x, (x t, f (x t))) > 0 and C.2 in this proof. Non-zero mutual information indicates that we can achieve a O(√ T) regret if we query x t s which are optimal corresponding to some implicitly defined forward function lying in the high confidence set of the true posterior given the observed datapoints D t. Lemma C.2 says that if bounded errors are made in fitting the inverse map, the overall regret remains O(√ T). More formally, if ||f and now application of Lemma C.2 gives us the extra regret incurred. (Note that this also provides us a way to choose the number of training steps for the inverse map) Further, note if we sample x t at iteration t from a distribution that shares support with the true posterior over optimal x t (which is used by TS), we still incur sublinear, bounded O(Γ H(A *)T ) regret. In the worst case, the overall bias caused due to the approximations will lead to an additive cumulative increase in the Bayesian regret, and hence, there is a constant Figure 4: Contextual MBO on MNIST. In (a) and (b), top one-half and top one-fourth of the image respectively and in (c) the one-hot encoded label are provided as contexts. The goal is to produce the maximum stroke width character that is valid given the context. In (a) and (b), we show triplets of the groundtruth digit (green), the context passed as input (yellow) and the produced images x from the MIN model (purple). In this set of static dataset experiments, we study contextual MBO tasks on image pixels. Unlike the contextual bandits case, where x corresponds to an image label, here x corresponds to entire images. We construct several tasks. First, we study stroke width optimization on MNIST characters, where the context is the class of the digit we wish to optimize. Results are shown in Figure 4. MINs correctly produce digits of the right class, and achieve an average score over the digit classes of 237.6, whereas the average score of the digits in the dataset is 149.0. The next task is to test the ability of MINs to be able to complete/inpaint unobserved patches of an image given an observed context patch. We use two masks: mask A: only top half and mask B: only top one-fourth parts of the image are visible, to mask out portions of the image and present the masked image as context c to the MIN, with the goal being to produce a valid completion x, while still maximizing score corresponding to the stroke width. We present some The aim is to maximize the score of an image which is given by the sum of attributes: eyeglasses, smiling, wavy hair and no beard. MINs produce optimal x -visually these solutions indeed optimize the score. sample completions in Figure 4. The quantitative are presented in Table 6. We find that MINs are effective as compared completions for the context in the dataset in terms of score while still producing a visibly valid character. We evaluate MINs on a complex semantic optimization task on the CelebA dataset. We choose a subset of attributes and provide their one-hot encoding as context to the model. The score is equal to the 1 norm of the binary indicator vector for a different subset of attributes disjoint from the context. We present our in Figure 3. We observe that MINs produce diverse images consistent with the context, and is also able to effectively infer the score function, and learn features to maximize it. Some of the model produced optimized solutions were presented in Section 4 in Figure 3. In this section, we present the produced generations for some other contexts. Figure 7 shows these . In this section, we present some additional for non-contextual image optimization problems. We also evaluated our contextual optimization procedure on the CelebA dataset in a non-contextual setting. The reward function is the same as that in the contextual setting -the sum of attributes: wavy hair, no beard, smiling and eyeglasses. We find that MINs are able to sucessfully produce solutions in this scenario as well. We show some optimized outputs at different iterations from the model in Figure 5. cGAN baseline. We compare our MIN model to a cGAN baseline on the IMDB-Wiki faces dataset for the semantic age optimization task. In general, we found that the cGAN model learned to ignore the score value passed as input even when trained on the entire dataset (without excluding the youngest faces) and behaved almost like a regular unconditional GAN model when queried to produce images x corresponding to the smallest age. We suspect that this could possibly be due to the fact that age of a person doesn't have enough direct signal to guide the model to utilize it unless other tricks like reweighting proposed in Section 3.3 which explicitly enforce the model attention to datapoints of interest, are used. We present the produced optimized x in Figure 6. In Figure 8, we highlight the quantitative score values for the stroke width score function (defined as the number of pixels which have intensity more than a threshold). Note that MINs achieve the highest value of average score while still resembling a valid digit, that stays inside the manifold of valid digits, unlike a forward model which can get high values of the score function (number of pixels turned on), but doesn't stay on the manifold of valid digits. Figure 6: Optimal x solutions produced by a cGAN for the youngest face optimization task on the IMDB-faces dataset. We note that a cGAN learned to ignore the score value and produced images as an unconditional model, without any noticeable correlation with the score value. The samples produced mostly correspond to the most frequently occurring images in the dataset. Figure 7: Images returned by the MIN optimization for optimization over images. We note that MINs perform successful optimization over the an objective defined by the sum of desired attributes. Moreover, for unseen contexts, such as both brown and black hair, the optimized solutions look aligning with the context reasonably, and optimize for the score as well. In this section, we explain the experimental details and the setup of our model. For our experiments involving MNIST and optimization of benchmark functions task, we used the same architecture as a fully connected GAN -where the generator and discriminator are both fully connected networks. We based our code for this part on the open-source implementation (Linder-Norén). For the forward model experiments in these settings, we used a 3-layer feedforward ReLU network with hidden units of size 256 each in this setting. For all experiments on CelebA and IMDB-Wiki faces, we used the VGAN model and the associated codebase as our starting setup. For experiments on batch contextual bandits, we used a fully connected discriminator and generator for MNIST, and a convolutional generator and Resnet18-like discriminator for CIFAR-10. The prediction in this setting is categorical -1 of 10 labels needs to be predicted, so instead of using reinforce or derivative free optimization to train the inverse map, we used the trick with a temperature τ = 0.75, to be able to use stochastic gradient descent to train the model. For the protein flourescence maximization experiment, we used a 2-layer, 256-unit feed-forward gumbel-softmax inverse map and a 2-layer feed-forward discriminator. We trained models present in open-source implementations of BanditNet (Sachdeva), but were unable to reproduce as reported by. Thus we reported the paper reported numbers from the BanditNet paper in the main text as well. Temperature hyperparameter τ which is used to compute the reweighting distribution is adaptively chosen based on the 90 th percentile score in the dataset. For example, if the difference between y max and y 90 th −percentile is given by α, we choose τ = α. This scheme can adaptively change temperatures in the active setting. In order to select the constant which decides whether the bin corresponding to a particular value of y is small or not, we first convert the expression Ny Ny+λ to use densities rather than absolute counts, that is,p D (y) p D (y)+λ, wherep D (y) is the empirical density of observing y in D, and now we use the same constant λ = 0.003. We did not observe a lot of sensitivity to λ values in the range [0.0001, 0.007], all of which performed reasonably similar. We usually fixed the number of bins to 20 for the purposed of reweighting, however note that the inverse map was still trained on continuous y values, which helps it extrapolate. In the active setting, we train two copies of f −1 jointly side by side. One of them is trained on the augmented datapoints generated out of the randomized labelling procedure, and the other copy is just trained on the real datapoints. This was done so as to prevent instabilities while training inverse maps. Training can also be made more incremental in this manner, and we need to train an inverse map to optimality inside every iteration of the active MIN algorithm, but rather we can train both the inverse maps for a fixed number of gradient steps."

Deep architectures for 3D point clouds that are equivariant to SO(3) rotations, as well as translations and permutations.