Question: Represent the article for finding a single-sentence TLDR

Question: In this paper we investigate the family of functions representable by deep neural networks (DNN) with rectified linear units (ReLU). We give an algorithm to train a ReLU DNN with one hidden layer to {\em global optimality} with runtime polynomial in the data size albeit exponential in the input dimension. Further, we improve on the known lower bounds on size (from exponential to super exponential) for approximating a ReLU deep net function by a shallower ReLU net. Our gap theorems hold for smoothly parametrized families of ``hard'' functions, contrary to countable, discrete families known in the literature. An example consequence of our gap theorems is the following: for every natural number $k$ there exists a function representable by a ReLU DNN with $k^2$ hidden layers and total size $k^3$, such that any ReLU DNN with at most $k$ hidden layers will require at least $\frac12k^{k+1}-1$ total nodes. Finally, for the family of $\R^n\to \R$ DNNs with ReLU activations, we show a new lowerbound on the number of affine pieces, which is larger than previous constructions in certain regimes of the network architecture and most distinctively our lowerbound is demonstrated by an explicit construction of a \emph{smoothly parameterized} family of functions attaining this scaling. Our construction utilizes the theory of zonotopes from polyhedral theory. Deep neural networks (DNNs) provide an excellent family of hypotheses for machine learning tasks such as classification. Neural networks with a single hidden layer of finite size can represent any continuous function on a compact subset of R n arbitrary well. The universal approximation was first given by Cybenko in 1989 for sigmoidal activation function BID4, and later generalized by Hornik to an arbitrary bounded and nonconstant activation function BID15. Furthermore, neural networks have finite VC dimension (depending polynomially on the number of edges in the network), and therefore, are PAC (probably approximately correct) learnable using a sample of size that is polynomial in the size of the networks BID1. However, neural networks based methods were shown to be computationally hard to learn BID1 and had mixed empirical success. Consequently, DNNs fell out of favor by late 90s.to address the issue of efficiently training DNNs. These include heuristics such as dropouts BID39, but also considering alternate deep architectures such as convolutional neural networks BID33, deep belief networks BID14, and deep Boltzmann machines BID31. In addition, deep architectures based on new non-saturating activation functions have been suggested to be more effectively trainable -the most successful and widely popular of these is the rectified linear unit (ReLU) activation, i.e., σ(x) = max{0, x}, which is the focus of study in this paper. In this paper, we formally study deep neural networks with rectified linear units; we refer to these deep architectures as ReLU DNNs. Our work is inspired by these recent attempts to understand the reason behind the successes of deep learning, both in terms of the structure of the functions represented by DNNs, Telgarsky (2015; ; BID17 ; BID36, as well as efforts which have tried to understand the non-convex nature of the training problem of DNNs better BID18 ; BID10 . Our investigation of the function space represented by ReLU DNNs also takes inspiration from the classical theory of circuit complexity; we refer the reader to BID2 ; BID37 ; BID16 ; BID32 ; BID0 for various surveys of this deep and fascinating field. In particular, our gap are inspired by like the ones by BID12, BID27 and BID38 which show a strict separation of complexity classes. We make progress towards similar statements with deep neural nets with ReLU activation. We extend the ReLU activation function to vectors x ∈ R n through entry-wise operation: σ(x) = (max{0, x 1}, max{0, x 2},..., max{0, x n}). For any (m, n) ∈ N, let A n m and L n m denote the class of affine and linear transformations from R m → R n, respectively. [ReLU DNNs, depth, width, size] For any number of hidden layers k ∈ N, input and output dimensions w 0, w k+1 ∈ N, a R w0 → R w k+1 ReLU DNN is given by specifying a sequence of k natural numbers w 1, w 2,..., w k representing widths of the hidden layers, a set of k affine transformations T i: R wi−1 → R wi for i = 1,..., k and a linear transformation T k+1: R w k → R w k+1 corresponding to weights of the hidden layers. Such a ReLU DNN is called a (k + 1)-layer ReLU DNN, and is said to have k hidden layers. The function f: R n1 → R n2 computed or represented by this ReLU DNN is DISPLAYFORM0 where • denotes function composition. The depth of a ReLU DNN is defined as k + 1. The width of a ReLU DNN is max{w 1, . . ., w k}. The size of the ReLU DNN is w 1 + w 2 +... + w k. Definition 2. We denote the class of R w0 → R w k+1 ReLU DNNs with k hidden layers of widths DISPLAYFORM1 Definition 3. [Piecewise linear functions] We say a function f: R n → R is continuous piecewise linear (PWL) if there exists a finite set of polyhedra whose union is R n, and f is affine linear over each polyhedron (note that the definition automatically implies continuity of the function because the affine regions are closed and cover R n, and affine functions are continuous). The number of pieces of f is the number of maximal connected subsets of R n over which f is affine linear (which is finite).Many of our important statements will be phrased in terms of the following simplex. Definition 4. Let M > 0 be any positive real number and p ≥ 1 be any natural number. Define the following set: DISPLAYFORM2 One of the main advantages of DNNs is that they can represent a large family of functions with a relatively small number of parameters. In this section, we give an exact characterization of the functions representable by ReLU DNNs. Moreover, we show how structural properties of ReLU DNNs, specifically their depth and width, affects their expressive power. It is clear from definition that any function from R n → R represented by a ReLU DNN is a continuous piecewise linear (PWL) function. In what follows, we show that the converse is also true, that is any PWL function is representable by a ReLU DNN. In particular, the following theorem establishes a one-to-one correspondence between the class of ReLU DNNs and PWL functions. Theorem 2.1. Every R n → R ReLU DNN represents a piecewise linear function, and every piecewise linear function R n → R can be represented by a ReLU DNN with at most log 2 (n + 1) + 1 depth. Proof Sketch: It is clear that any function represented by a ReLU DNN is a PWL function. To see the converse, we first note that any PWL function can be represented as a linear combination of piecewise linear convex functions. More formally, by Theorem 1 in , for every piecewise linear function f: R n → R, there exists a finite set of affine linear functions 1,..., k and subsets S 1,..., S p ⊆ {1, . . ., k} (not necessarily disjoint) where each S i is of cardinality at most n + 1, such that DISPLAYFORM0 where s j ∈ {−1, +1} for all j = 1,..., p. Since a function of the form max i∈Sj i is a piecewise linear convex function with at most n + 1 pieces (because |S j | ≤ n + 1), Equation (is implementable by a two layer ReLU network and use this construction in an inductive manner to show that maximum of n + 1 numbers can be computed using a ReLU DNN with depth at most log 2 (n + 1).While Theorem 2.1 gives an upper bound on the depth of the networks needed to represent all continuous piecewise linear functions on R n, it does not give any tight bounds on the size of the networks that are needed to represent a given piecewise linear function. For n = 1, we give tight bounds on size as follows: Theorem 2.2. Given any piecewise linear function R → R with p pieces there exists a 2-layer DNN with at most p nodes that can represent f. Moreover, any 2-layer DNN that represents f has size at least p − 1.Finally, the main of this section follows from Theorem 2.1, and well-known facts that the piecewise linear functions are dense in the family of compactly supported continuous functions and the family of compactly supported continuous functions are dense in DISPLAYFORM1 is the space of Lebesgue integrable functions f such that |f | q dµ < ∞, where µ is the Lebesgue measure on R n (see BID29). DISPLAYFORM2 can be arbitrarily well-approximated in the L q norm (which for a function f is given by ||f || q = ( |f | q) 1/q ) by a ReLU DNN function with at most log 2 (n + 1) hidden layers. Moreover, for n = 1, any such L q function can be arbitrarily well-approximated by a 2-layer DNN, with tight bounds on the size of such a DNN in terms of the approximation. Proofs of Theorems 2.2 and 2.3 are provided in Appendix A. We would like to remark that a weaker version of Theorem 2.1 was observed in (, Proposition 4 .1) (with no bound on the depth), along with a universal approximation theorem (, Theorem 4. 3) similar to Theorem 2.3. The authors of BID9 also used a previous of Wang for obtaining their . In a subsequent work Boris Hanin BID11 has, among other things, found a width and depth upper bound for ReLU net representation of positive PWL functions on n. The width upperbound is n+3 for general positive PWL functions and n + 1 for convex positive PWL functions. For convex positive PWL functions his depth upper bound is sharp if we disallow dead ReLUs. Success of deep learning has been largely attributed to the depth of the networks, i.e. number of successive affine transformations followed by nonlinearities, which is shown to be extracting hierarchical features from the data. In contrast, traditional machine learning frameworks including support vector machines, generalized linear models, and kernel machines can be seen as instances of shallow networks, where a linear transformation acts on a single layer of nonlinear feature extraction. In this section, we explore the importance of depth in ReLU DNNs. In particular, in Section 3.1, we provide a smoothly parametrized family of R → R "hard" functions representable by ReLU DNNs, which requires exponentially larger size for a shallower network. Furthermore, in Section 3.2, we construct a continuum of R n → R "hard" functions representable by ReLU DNNs, which to the best of our knowledge is the first explicit construction of ReLU DNN functions whose number of affine pieces grows exponentially with input dimension. The proofs of the theorems in this section are provided in Appendix B. In this section, we are only concerned about R → R ReLU DNNs, i.e. both input and output dimensions are equal to one. The following theorem shows the depth-size trade-off in this setting. Theorem 3.1. For every pair of natural numbers k ≥ 1, w ≥ 2, there exists a family of hard functions representable by a R → R (k + 1)-layer ReLU DNN of width w such that if it is also representable by a (k + 1)-layer ReLU DNN for any k ≤ k, then this (k + 1)-layer ReLU DNN has size at least DISPLAYFORM0 In fact our family of hard functions described above has a very intricate structure as stated below. Theorem 3.2. For every k ≥ 1, w ≥ 2, every member of the family of hard functions in Theorem 3.1 has w k pieces and this family can be parametrized by DISPLAYFORM1 i.e., for every point in the set above, there exists a distinct function with the stated properties. The following is an immediate corollary of Theorem 3.1 by choosing the parameters carefully. Corollary 3.3. For every k ∈ N and > 0, there is a family of functions defined on the real line such that every function f from this family can be represented by a (k 1+) + 1-layer DNN with size k 2+ and if f is represented by a k +1-layer DNN, then this DNN must have size at least DISPLAYFORM2 Moreover, this family can be parametrized as, DISPLAYFORM3 A particularly illuminative special case is obtained by setting = 1 in Corollary 3.3: Corollary 3.4. For every natural number k ∈ N, there is a family of functions parameterized by the set DISPLAYFORM4 such that any f from this family can be represented by a k 2 + 1-layer DNN with k 3 nodes, and every k + 1-layer DNN that represents f needs at least DISPLAYFORM5 We can also get hardness of approximation versions of Theorem 3.1 and Corollaries 3.3 and 3.4, with the same gaps (upto constant terms), using the following theorem. Theorem 3.5. For every k ≥ 1, w ≥ 2, there exists a function f k,w that can be represented by a (k + 1)-layer ReLU DNN with w nodes in each layer, such that for all δ > 0 and k ≤ k the following holds: DISPLAYFORM6 where G k,δ is the family of functions representable by ReLU DNNs with depth at most k + 1, and size at most k DISPLAYFORM7 The depth-size trade-off in Theorems 3.1, and 3.5 extend and improve Telgarsky's theorems from (; in the following three ways:(i) If we use our Theorem 3.5 to the pair of neural nets considered by Telgarsky in Theorem 1.1 in which are at depths k 3 (of size also scaling as k 3) and k then for this purpose of approximation in the 1 −norm we would get a size lower bound for the shallower net which scales as Ω(2 k 2) which is exponentially (in depth) larger than the lower bound of Ω(2 k) that Telgarsky can get for this scenario.(ii) Telgarsky's family of hard functions is parameterized by a single natural number k. In contrast, we show that for every pair of natural numbers w and k, and a point from the set in equation 3.1, there exists a "hard" function which to be represented by a depth k network would need a size of at least w k k k. With the extra flexibility of choosing the parameter w, for the purpose of showing gaps in representation ability of deep nets we can shows size lower bounds which are super-exponential in depth as explained in Corollaries 3.3 and 3.4.(iii) A characteristic feature of the "hard" functions in Boolean circuit complexity is that they are usually a countable family of functions and not a "smooth" family of hard functions. In fact, in the last section of , Telgarsky states this as a "weakness" of the state-of-the-art on "hard" functions for both Boolean circuit complexity and neural nets research. In contrast, we provide a smoothly parameterized family of "hard" functions in Section 3.1 (parametrized by the set in equation 3.1). Such a continuum of hard functions wasn't demonstrated before this work. We point out that Telgarsky's in apply to deep neural nets with a host of different activation functions, whereas, our are specifically for neural nets with rectified linear units. In this sense, Telgarsky's from are more general than our in this paper, but with weaker gap guarantees. Eldan-Shamir BID36 BID7 show that there exists an R n → R function that can be represented by a 3-layer DNN, that takes exponential in n number of nodes to be approximated to within some constant by a 2-layer DNN. While their are not immediately comparable with Telgarsky's or our , it is an interesting open question to extend their to a constant depth hierarchy statement analogous to the recent of Rossman et al BID28. We also note that in last few years, there has been much effort in the community to show size lowerbounds on ReLU DNNs trying to approximate various classes of functions which are themselves not necessarily exactly representable by ReLU DNNs (; BID22 BID30 . One measure of complexity of a family of R n → R "hard" functions represented by ReLU DNNs is the asymptotics of the number of pieces as a function of dimension n, depth k + 1 and size s of the ReLU DNNs. More precisely, suppose one has a family H of functions such that for every n, k, w ∈ N the family contains at least one R n → R function representable by a ReLU DNN with depth at most k + 1 and maximum width at most w. The following definition formalizes a notion of complexity for such a H.Definition 5 (comp H (n, k, w)). The measure comp H (n, k, w) is defined as the maximum number of pieces (see Definition 3) of a R n → R function from H that can be represented by a ReLU DNN with depth at most k + 1 and maximum width at most w. Similar measures have been studied in previous works BID24; BID25; BID26. The best known families H are the ones from Theorem 4 of BID24 ) and a mild generalization of Theorem 1.1 of to k layers of ReLU activations with width w; these constructions achieve (DISPLAYFORM0 At the end of this section we would explain the precise sense in which we improve on these numbers. An analysis of this complexity measure is done using integer programming techniques in BID34 . DISPLAYFORM1 Figure 1: We fix the a vectors for a two hidden layer R → R hard function as DISPLAYFORM2 Left: A specific hard function induced by 1 norm: 0). Note that in this case the function can be seen as a composition of H a 1,a 2 with 1 -norm The set of vertices of DISPLAYFORM3 DISPLAYFORM4 DISPLAYFORM5 The following are well-known in the theory of zonotopes . Theorem 3.6. The following are all true. DISPLAYFORM6. The set of (b 1, . . ., b m) ∈ R n ×... × R n such that this does not hold at equality is a 0 measure set. DISPLAYFORM7 Definition 7 (extremal zonotope set). The set S(n, m) will denote the set of DISPLAYFORM8. S(n, m) is the so-called "extremal zonotope set", which is a subset of R nm, whose complement has zero Lebesgue measure in R nm.Lemma 3.7. Given any b 1,..., b m ∈ R n, there exists a 2-layer ReLU DNN with size 2m which represents the function γ Z(b 1,...,b m) (r).Definition 8. For p ∈ N and a ∈ ∆ p M, we define a function h a: R → R which is piecewise linear over the segments DISPLAYFORM9 DISPLAYFORM10 Proposition 3.8. Given any tuple (b 1, . . ., b m) ∈ S(n, m) and any point DISPLAYFORM11 n−1 w k pieces and it can be represented by a k + 2 layer ReLU DNN with size 2m + wk. Finally, we are ready to state the main of this section. Theorem 3.9. For every tuple of natural numbers n, k, m ≥ 1 and w ≥ 2, there exists a family of R n → R functions, which we call ZONOTOPE n k,w,m with the following properties:(i) Every f ∈ ZONOTOPE n k,w,m is representable by a ReLU DNN of depth k + 2 and size 2m + wk, and has n−1 i=0 m−1 i w k pieces.(ii) Consider any f ∈ ZONOTOPE n k,w,m. If f is represented by a (k + 1)-layer DNN for any k ≤ k, then this (k + 1)-layer DNN has size at least DISPLAYFORM12 The family ZONOTOPE n k,w,m is in one-to-one correspondence with DISPLAYFORM13 Comparison to the in BID24 Firstly we note that the construction in BID24 requires all the hidden layers to have width at least as big as the input dimensionality n. In contrast, we do not impose such restrictions and the network size in our construction is independent of the input dimensionality. Thus our probes networks with bottleneck architectures whose complexity cant be seen from their . Secondly, in terms of our complexity measure, there seem to be regimes where our bound does better. One such regime, for example, is when n ≤ w < 2n and k ∈ Ω(n log(n) ), by setting in our construction m < n. Thirdly, it is not clear to us whether the construction in BID24 gives a smoothly parameterized family of functions other than by introducing small perturbations of the construction in their paper. In contrast, we have a smoothly parameterized family which is in one-to-one correspondence with a well-understood manifold like the higher-dimensional torus. In this section we consider the following empirical risk minimization problem. Given D data points (x i, y i) ∈ R n × R, i = 1,..., D, find the function f represented by 2-layer R n → R ReLU DNNs of width w, that minimizes the following optimization problem DISPLAYFORM0 where: R × R → R is a convex loss function (common loss functions are the squared loss, (y, y) = (y − y) 2, and the hinge loss function given by (y, y) = max{0, 1 − yy}). Our main of this section gives an algorithm to solve the above empirical risk minimization problem to global optimality. Proof Sketch: A full proof of Theorem 4.1 is included in Appendix C. Here we provide a sketch of the proof. When the empirical risk minimization problem is viewed as an optimization problem in the space of weights of the ReLU DNN, it is a nonconvex, quadratic problem. However, one can instead search over the space of functions representable by 2-layer DNNs by writing them in the form similar to (2.1). This breaks the problem into two parts: a combinatorial search and then a convex problem that is essentially linear regression with linear inequality constraints. This enables us to guarantee global optimality. Where DISPLAYFORM0 All possible instantiations of top layer weights 3: DISPLAYFORM1 All possible partitions of data into two parts 4: DISPLAYFORM2 for s ∈ S do 7: DISPLAYFORM3 end for 11: OPT = argmin loss(count) 12: end for 13:return {ã}, {b}, s corresponding to OPT's iterate 14: end function Let T 1 (x) = Ax + b and T 2 (y) = a · y for A ∈ R w×n and b, a ∈ R w. If we denote the i-th row of the matrix A by a i, and write b i, a i to denote the i-th coordinates of the vectors b, a respectively, due to homogeneity of ReLU gates, the network output can be represented as DISPLAYFORM4 whereã i ∈ R n,b i ∈ R and s i ∈ {−1, +1} for all i = 1,..., w. For any hidden node i ∈ {1 . . ., w}, the pair (ã i,b i) induces a partition P i:= (P DISPLAYFORM5 − and a i · x j +b i ≥ 0 ∀j ∈ P i + which are imposed for all i = 1, . . ., w, which is a convex program. Algorithm 1 implements the empirical risk minimization (ERM) rule for training ReLU DNN with one hidden layer. To the best of our knowledge there is no other known algorithm that solves the ERM problem to global optimality. We note that due to known hardness exponential dependence on the input dimension is unavoidable;; Algorithm 1 runs in time polynomial in the number of data points. To the best of our knowledge there is no hardness known which rules out empirical risk minimization of deep nets in time polynomial in circuit size or data size. Thus our training is a step towards resolving this gap in the complexity literature. A related for improperly learning ReLUs has been recently obtained by Goel et al BID8. In contrast, our algorithm returns a ReLU DNN from the class being learned. Another difference is that their considers the notion of reliable learning as opposed to the empirical risk minimization objective considered in (4.1). The running time of the algorithm that we give in this work to find the exact global minima of a two layer ReLU-DNN is exponential in the input dimension n and the number of hidden nodes w. The exponential dependence on n can not be removed unless P = N P; see BID35; BID3; BID6. However, we are not aware of any complexity which would rule out the possibility of an algorithm which trains to global optimality in time that is polynomial in the data size and/or the number of hidden nodes, assuming that the input dimension is a fixed constant. Resolving this dependence on network size would be another step towards clarifying the theoretical complexity of training ReLU DNNs and is a good open question for future research, in our opinion. Perhaps an even better breakthrough would be to get optimal training algorithms for DNNs with two or more hidden layers and this seems like a substantially harder nut to crack. It would also be a significant breakthrough to get gap between consecutive constant depths or between logarithmic and constant depths. We would like to thank Christian Tjandraatmadja for pointing out a subtle error in a previous version of the paper, which affected the complexity for the number of linear regions in our constructions in Section 3.2. Anirbit would like to thank Ramprasad Saptharishi, Piyush Srivastava and Rohit Gurjar for extensive discussions on Boolean and arithmetic circuit complexity. This paper has been immensely influenced by the perspectives gained during those extremely helpful discussions. Amitabh Basu gratefully acknowledges support from the NSF grant CMMI1452820. Raman Arora was supported in part by NSF BIGDATA grant IIS-1546482.Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pp. Proof of Theorem 2.2. Any continuous piecewise linear function R → R which has m pieces can be specified by three pieces of information, s L the slope of the left most piece, the coordinates of the non-differentiable points specified by a (m − 1)−tuple {(a i, b i)} One notes that for any a, r ∈ R, the function DISPLAYFORM0 is equal to sgn(r) max{|r|(x − a), 0}, which can be implemented by a 2-layer ReLU DNN with size 1. Similarly, any function of the form, DISPLAYFORM1 is equal to − sgn(t) max{−|t|(x − a), 0}, which can be implemented by a 2-layer ReLU DNN with size 1. The parameters r, t will be called the slopes of the function, and a will be called the breakpoint of the function. If we can write the given piecewise linear function as a sum of m functions of the form (A.1) and (A.2), then by Lemma D.2 we would be done. It turns out that such a decomposition of any p piece PWL function h: R → R as a sum of p flaps can always be arranged where the breakpoints of the p flaps all are all contained in the p − 1 breakpoints of h. First, observe that adding a constant to a function does not change the complexity of the ReLU DNN expressing it, since this corresponds to a bias on the output node. Thus, we will assume that the value of h at the last break point a m−1 is b m−1 = 0. We now use a single function f of the form (A.1) with slope r and breakpoint a = a m−1, and m − 1 functions g 1,..., g m−1 of the form (A.2) with slopes t 1,..., t m−1 and breakpoints a 1,..., a m−1, respectively. Thus, we wish to express h = f + g 1 +... + g m−1. Such a decomposition of h would be valid if we can find values for r, t 1,..., t m−1 such that the slope of the above sum is = s L for x < a 1, the slope of the above sum is = s R for x > a m−1, and for each i ∈ {1, 2, 3, .., m − 1} we have DISPLAYFORM2 The above corresponds to asking for the existence of a solution to the following set of simultaneous linear equations in r, t 1,..., t m−1: DISPLAYFORM3 It is easy to verify that the above set of simultaneous linear equations has a unique solution. Indeed, r must equal s R, and then one can solve for t 1,..., t m−1 starting from the last equation b m−2 = t m−1 (a m−2 − a m−1) and then back substitute to compute t m−2, t m−3,..., t 1. The lower bound of p − 1 on the size for any 2-layer ReLU DNN that expresses a p piece function follows from Lemma D.6.One can do better in terms of size when the rightmost piece of the given function is flat, i.e., s R = 0. In this case r = 0, which means that f = 0; thus, the decomposition of h above is of size p − 1. A similar construction can be done when s L = 0. This gives the following statement which will be useful for constructing our forthcoming hard functions. Corollary A.1. If the rightmost or leftmost piece of a R → R piecewise linear function has 0 slope, then we can compute such a p piece function using a 2-layer DNN with size p − 1.Proof of theorem 2.3. Since any piecewise linear function R n → R is representable by a ReLU DNN by Corollary 2.1, the proof simply follows from the fact that the family of continuous piecewise linear functions is dense in any L p (R n) space, for 1 ≤ p ≤ ∞. Lemma B.1. For any M > 0, p ∈ N, k ∈ N and a 1,..., a k ∈ ∆ p M, if we compose the functions h a 1, h a 2,..., h a k the ing function is a piecewise linear function with at most (p + 1) k + 2 pieces, i.e., DISPLAYFORM0 is piecewise linear with at most (p + 1) k + 2 pieces, with (p + 1) k of these pieces in the range [0, M] (see Figure 2). Moreover, in each piece in the range [0, M], the function is affine with minimum value 0 and maximum value M.Proof. Simple induction on k. Proof of Theorem 3.2. Given k ≥ 1 and w ≥ 2, choose any point DISPLAYFORM1 By Definition 8, each h a i, i = 1,..., k is a piecewise linear function with w + 1 pieces and the leftmost piece having slope 0. Thus, by Corollary A.1, each h a i, i = 1,..., k can be represented by a 2-layer ReLU DNN with size w. Using Lemma D.1, H a 1,...,a k can be represented by a k + 1 layer DNN with size wk; in fact, each hidden layer has exactly w nodes. inside any triangle of s q, any affine function will incur an 1 error of at least DISPLAYFORM2 Proof of Theorem 4.1. Let: R → R be any convex loss function, and let (x 1, y 1),..., (x D, y D) ∈ R n × R be the given D data points. As stated in (4.1), the problem requires us to find an affine transformation T 1: R n → R w and a linear transformation T 2: R w → R, so as to minimize the empirical loss as stated in (4.1). Note that T 1 is given by a matrix A ∈ R w×n and a vector b ∈ R w so that T (x) = Ax + b for all x ∈ R n. Similarly, T 2 can be represented by a vector a ∈ R w such that T 2 (y) = a · y for all y ∈ R w. If we denote the i-th row of the matrix A by a i, and write b i, a i to denote the i-th coordinates of the vectors b, a respectively, we can write the function represented by this network as DISPLAYFORM0 In other words, the family of functions over which we are searching is of the form DISPLAYFORM1 whereã i ∈ R n, b i ∈ R and s i ∈ {−1, +1} for all i = 1,..., w. We now make the following observation. For a given data point (x j, y j) ifã i · x j +b i ≤ 0, then the i-th term of (C.1) does not contribute to the loss function for this data point (x j, y j). Thus, for every data point (x j, y j), there exists a set S j ⊆ {1, . . ., w} such that f (x j) = i∈Sj s i (ã i · x j +b i). In particular, if we are given the set S j for (x j, y j), then the expression on the right hand side of (C.1) reduces to a linear function ofã i,b i. For any fixed i ∈ {1, . . ., w}, these sets S j induce a partition of the data set into two parts. In particular, we define P i +:= {j : i ∈ S j} and P i −:= {1, . . ., D} \ P i +. Observe now that this partition is also induced by the hyperplane given byã i,b i: DISPLAYFORM2 and DISPLAYFORM3. Our strategy will be to guess the partitions P For a fixed selection of partitions (P i +, P i −), i = 1,..., w and a vector s in {+1, −1} w, the algorithm solves the following convex optimization problem with decision variablesã i ∈ R n,b i ∈ R for i = 1,..., w (thus, we have a total of (n + 1) · w decision variables). The feasible region of the optimization is given by the constraints DISPLAYFORM4 which are imposed for all i = 1,..., w. Thus, we have a total of D · w constraints. Subject to these constraints we minimize the objective DISPLAYFORM5 Assuming the loss function is a convex function in the first argument, the above objective is a convex function. Thus, we have to minize a convex objective subject to the linear inequality constraints from (C.2).We finally have to count how many possible partitions (P n which only holds for n ≥ 2. For n = 1, a similar algorithm can be designed, but one which uses the characterization achieved in Theorem 2.2. Let : R → R be any convex loss function, and let (x 1, y 1),..., (x D, y D) ∈ R 2 be the given D data points. Using Theorem 2.2, to solve problem (4.1) it suffices to find a R → R piecewise linear function f with w pieces that minimizes the total loss. In other words, the optimization problem (4.1) is equivalent to the problem DISPLAYFORM0 f is piecewise linear with w pieces.(C.3)We now use the observation that fitting piecewise linear functions to minimize loss is just a step away from linear regression, which is a special case where the function is contrained to have exactly one affine linear piece. Our algorithm will first guess the optimal partition of the data points such that all points in the same class of the partition correspond to the same affine piece of f, and then do linear regression in each class of the partition. Altenatively, one can think of this as guessing the interval (x i, x i+1) of data points where the w − 1 breakpoints of the piecewise linear function will lie, and then doing linear regression between the breakpoints. More formally, we parametrize piecewise linear functions with w pieces by the w slope-intercept values (a 1, b 1),..., (a 2, b 2),..., (a w, b w) of the w different pieces. This means that between breakpoints j and j + 1, 1 ≤ j ≤ w − 2, the function is given by f (x) = a j+1 x + b j+1, and the first and last pieces are a 1 x + b 1 and a w x + b w, respectively. Define I to be the set of all (w − 1)-tuples (i 1, . . ., i w−1) of natural numbers such that DISPLAYFORM1 Given a fixed tuple I = (i 1, . . ., i w−1) ∈ I, we wish to search through all piecewise linear functions whose breakpoints, in order, appear in the intervals (x i1, x i1+1), (x i2, x i2+1),..., (x iw−1, x iw−1+1). Define also S = {−1, 1} w−1. Any S ∈ S will have the following interpretation: if S j = 1 then a j ≤ a j+1, and if S j = −1 then a j ≥ a j+1. Now for every I ∈ I and S ∈ S, requiring a piecewise linear function that respects the conditions imposed by I and S is easily seen to be equivalent to imposing the following linear inequalities on the parameters (a 1, b 1),..., (a 2, b 2),..., (a w, b w): DISPLAYFORM2 Let the set of piecewise linear functions whose breakpoints satisfy the above be denoted by PWL 1 I,S for I ∈ I, S ∈ S.Given a particular I ∈ I, we define DISPLAYFORM3 The right hand side of the above equation is the problem of minimizing a convex objective subject to linear constraints. Now, to solve (C.3), we need to simply solve the problem (C.5) for all I ∈ I, S ∈ S and pick the minimum. Since |I| = Now we will collect some straightforward observations that will be used often. The following operations preserve the property of being representable by a ReLU DNN. Proof. Follows from (1.1) and the fact that a composition of affine transformations is another affine transformation. Proof. We prove this by induction on k. The base case is k = 1, i.e, we have a 2-layer ReLU DNN. Since every activation node can produce at most one breakpoint in the piecewise linear function, we can get at most w 1 breakpoints, i.e., w 1 + 1 pieces. Now for the induction step, assume that for some k ≥ 1, any R → R ReLU DNN with depth k + 1 and widths w 1,..., w k of the k hidden layers produces at most 2 k−1 · (w 1 + 1) · w 2 ·... · w k pieces. Consider any R → R ReLU DNN with depth k + 2 and widths w 1,..., w k+1 of the k + 1 hidden layers. Observe that the input to any node in the last layer is the output of a R → R ReLU DNN with depth k + 1 and widths w 1,..., w k. By the induction hypothesis, the input to this node in the last layer is a piecewise linear function f with at most 2 k−1 · (w 1 + 1) · w 2 ·... · w k pieces. When we apply the activation, the new function g(x) = max{0, f (x)}, which is the output of this node, may have at most twice the number of pieces as f, because each original piece may be intersected by the x-axis; see Figure 4. Thus, after going through the layer, we take an affine combination of w k+1 functions, each with at most 2 · (2 k−1 · (w 1 + 1) · w 2 ·... · w k ) pieces. In all, we can therefore get at most 2·(2 k−1 ·(w 1 +1)·w 2 ·...·w k )·w k+1 pieces, which is equal to 2 k ·(w 1 +1)·w 2 ·...·w k ·w k+1, and the induction step is completed. Lemma D.5 has the following consequence about the depth and size tradeoffs for expressing functions with agiven number of pieces. Lemma D.6. Let f: R → R be a piecewise linear function with p pieces. If f is represented by a ReLU DNN with depth k + 1, then it must have size at least 1 2 kp 1/k − 1. Conversely, any piecewise linear function f that represented by a ReLU DNN of depth k + 1 and size at most s, can have at most (2s k) k pieces. Proof. Let widths of the k hidden layers be w 1,..., w k. By Lemma D.5, we must have DISPLAYFORM0 By the AM-GM inequality, minimizing the size w 1 + w 2 +... + w k subject to (D.1), means setting w 1 + 1 = w 2 =... = w k. This implies that w 1 + 1 = w 2 =... = w k ≥ 1 2 p 1/k. The first statement follows. The second statement follows using the AM-GM inequality again, this time with a restriction on w 1 + w 2 +... + w k.

represent the tldr for finding the corresponding scientific article Example:
Provided: "Tropical geometry can be leveraged to represent the decision boundaries of neural networks and bring to light interesting insights." Match: "This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piece-wise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to provide a characterization of the decision boundaries of a simple neural network of the form (Affine, ReLU, Affine). Specifically, we show that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes. The generators of the zonotopes are precise functions of the neural network parameters. We utilize this geometric characterization to shed light and new perspective on three tasks. In doing so, we propose a new tropical perspective for the lottery ticket hypothesis, where we see the effect of different initializations on the tropical geometric representation of the decision boundaries. Also, we leverage this characterization as a new set of tropical regularizers, which deal directly with the decision boundaries of a network. We investigate the use of these regularizers in neural network pruning (removing network parameters that do not contribute to the tropical geometric representation of the decision boundaries) and in generating adversarial input attacks (with input perturbations explicitly perturbing the decision boundaries geometry to change the network prediction of the input).. In addition, and in attempt to understand some of the subtle behaviours DNNs exhibit, e.g. the sensitive reaction of DNNs to small input perturbations, several works directly investigated the decision boundaries induced by a DNN used for classification. The work of showed that the smoothness of these decision boundaries and their curvature can play a vital role in network robustness. Moreover, He et al. (2018a) studied the expressiveness of these decision boundaries at perturbed inputs and showed that these boundaries do not resemble the boundaries around benign inputs. showed that under certain assumptions, the decision boundaries of the last fully connected layer of DNNs will converge to a linear SVM. Also, showed that the decision regions of DNNs with width smaller than the input dimension are unbounded. More recently, and due to the popularity of the piecewise linear ReLU as an activation function, there has been a surge in the number of works that study this class of DNNs in particular. x ⊕ y = max{x, y}, x y = x + y, ∀x, y ∈ T. It can be readily shown that −∞ is the additive identity and 0 is the multiplicative identity. Given the previous definition, a tropical power can be formulated as x a = x x · · · x = a.x, for x ∈ T, a ∈ N, where a.x is standard multiplication. Moreover, the tropical quotient can be defined as: x y = x − y where x − y is the standard subtraction. For ease of notation, we write x a as x a. Now, we are in a position to define tropical polynomials, their solution sets and tropical rationals. f (x) = (c 1 x a1) ⊕ (c 2 x a2) ⊕ · · · ⊕ (c n x an), ∀ a i = a j when i = j. We use the more compact vector notation x a = x In this section, we analyze the decision boundaries of a network in the form (Affine, ReLU, Affine) using tropical geometry. For ease, we use ReLUs as the non-linear activation, but any other piecewise linear function can also be used. The functional form of this network is: f (x) = Bmax (Ax + c 1, 0) + c 2, where max is an element-wise operator. The outputs of the network f are the logit scores. Throughout this section, we assume 2 that A ∈ Z p×n, B ∈ Z 2×p, c 1 ∈ R p and c 2 ∈ R 2. For ease of notation, we only consider networks with two outputs, i.e. B 2×p, where the extension to a multi-class output follows naturally and it is discussed in the appendix. Now, since f is a piecewise linear function, each output can be expressed as a tropical rational as per Theorem 1. If f 1 and f 2 refer to the first and second outputs respectively, we have f 1 (x) = H 1 (x) Q 1 (x) and f 2 (x) = H 2 (x) Q 2 (x), where H 1, H 2, Q 1 and Q 2 are tropical polynomials. In what follows and for ease of presentation, we present our main where the network f has no biases, i.e. c 1 = 0 and c 2 = 0, and we leave the generalization to the appendix. Theorem 2. For a bias-free neural network in the form of f (x): R n → R 2 where A ∈ Z p×n and B ∈ Z 2×p, let R(x) = H 1 (x) Q 2 (x) ⊕ H 2 (x) Q 1 (x) be a tropical polynomial. Then: • Let B = {x ∈ R n : f 1 (x) = f 2 (x)} defines the decision boundaries of f, then B ⊆ T (R(x)). • δ (R(x)) = ConvHull (Z G1, Z G2). Z G1 is a zonotope in R n with line segments The proof for Theorem 2 is left for the appendix. Digesting Theorem 2. Theorem 2 can be broken into two major . The first, which is on the algebra side, i.e. finding the solution set to tropical polynomials, states that the decision boundaries linear pieces separating classes C1 and C2. As per Theorem 2, the dual subdivision of this single hidden neural network is the convex hull between the zonotopes Z G 1 and Z G 2. The normals to the dual subdivison δ(R(x)) are in one-to-one correspondence to the tropical hypersurface T (R(x)), which is a superset to the decision boundaries B. Note that some of the normals to δ(R(x)) (in red) are parallel to the decision boundaries. B is a subset of the tropical hypersurface of the tropical polynomial R(x), i.e. T (R(x)). The second , which is on the geometry side, of Theorem 2 relates the tropical polynomial R(x) to the geometric representation of the solution set to R(x), i.e. T (R(x)), referred to as the dual subdivision, i.e. δ(R(x)). In particular, Theorem 2 states that the dual subdivision for a network f is the convex hull of two zonotopes denoted as Z G1 and Z G2. Note that this dual subdivision is a function of only the network parameters A and B. Theorem 2 bridges the gap between the behaviour of the decision boundaries B, through the super-set T (R(x)), and the polytope δ (R(x)), which is the convex hull of two zonotopes. It is worthwhile to mention that discussed a special case of the first part of Theorem 2 for a neural network with a single output and a score function s(x) to classify the output. To the best of our knowledge, this work is the first to propose a tropical geometric formulation of a super-set containing the decision boundaries of a multi-class classification neural network. In particular, the first of Theorem 2 states that one can alter the network, e.g. by pruning network parameters, while preserving the decision boundaries B, if one preserves the tropical hypersurface of R(x) or T (R(x)). While preserving the tropical hypersurfaces can be equally difficult to preserving the decision boundaries directly, the second of Theorem 2 comes in handy. For a bias free network, π becomes an identity mapping with δ(R(x)) = ∆(R(x)), and thus the dual subdivision δ(R(x)), which is the Newton polytope ∆(R(x)) in this case, becomes a well structured geometric object that can be exploited to preserve decision boundaries. (Proposition 3.1.6) showed that the tropical hypersurface is the skeleton of the dual to δ(R(x)), the normal lines to the edges of the polytope δ(R(x)) are in one-to-one correspondence with the tropical hypersurface T (R(x)). Figure 1 details this intimate relation between the decision boundaries, tropical hypersurface T (R(x)), and normals to δ (R(x)). Before any further discussion, we recap the definition of zonotopes. Equivalently, the zonotope can be expressed with respect to the generator matrix U ∈ R p×n, where Another common definition for zonotopes is the Minkowski sum (refer to appendix A for the definition of the Minkowski sum) of a set of line segments that start from the origin with end points u 1,..., u p ∈ R n. It is also well known that the number of vertices of a zonotope is polynomial in the number of line segments. That is to say, While Theorem 2 presents a strong relation between a polytope (convex hull of two zonotopes) and the decision boundaries, it remains unclear how such a polytope can be efficiently constructed. Although the number of vertices of a zonotope is polynomial in the number of its generating line segments, fast algorithms for enumerating these vertices are still restricted to zonotopes with line segments starting at the origin . Since the line segments generating the zonotopes in Theorem 2 have arbitrary end points, we present the next that transforms these line segments into a generator matrix of line segments starting from the origin, as prescribed in Definition 6. This is essential for the efficient computation of the zonotopes in Theorem 2. Proposition 1. Consider p line segments in R n with two arbitrary end points as follows. The zonotope formed by these line segments is equivalent to the zonotope formed by the line segments {[u training dataset, decision boundaries polytope of original network followed by the decision boundaries polytope during several iterations of pruning with different initializations. The proof is left for the appendix. As per Proposition 1, the generator matrices of zonotopes In what follows, we show several applications for Theorem 2. We begin by leveraging the geometric structure to help in reaffirming the behaviour of the lottery ticket hypothesis. The lottery ticket hypothesis was recently proposed by , in which the authors surmise the existence of sparse trainable sub-networks of dense, randomly-initialized, feedforward networks that-when trained in isolation-perform as well as the original network in a similar number of iterations. To find such sub-networks, propose the following simple algorithm: perform standard network pruning, initialize the pruned network with the same initialization that was used in the original training setting, and train with the same number of epochs. They hypothesize that this should in a smaller network with a similar accuracy to the larger dense network. In other words, a subnetwork can have similar decision boundaries to the original network. While in this section we do not provide a theoretical reason for why this proposed pruning algorithm performs favorably, we utilize the geometric structure that arises from Theorem 2 to reaffirm such behaviour. In particular, we show that the orientation of the decision boundaries polytope δ(R(x)), known to be a superset to the decision boundaries T (R(x)), is preserved after pruning with the proposed initialization algorithm of. On the other hand, pruning routines with a different initialization at each pruning iteration will in a severe variation in the orientation of the decision boundaries polytope. This leads to a large change in the orientation of the decision boundaries, which tends to hinder accuracy. To this end, we train a neural network with 2 inputs (n = 2), 2 outputs, and a single hidden layer with 40 nodes (p = 40). We then prune the network by removing the smallest x% of the weights. The pruned network is then trained using different initializations: (i) the same initialization as the original network , (ii) Xavier , (iii) standard Gaussian and (iv) zero mean Gaussian with variance of 0.1. Figure 2 shows the evolution of the decision boundaries polytope, i.e. δ(R(x)), as we perform more pruning (increasing the x%) with different initializations. It is to be observed that the orientation of the polytopes δ(R(x)) vary much more for all different initialization schemes as compared to the lottery ticket initialization. This gives an indication that lottery ticket initialization indeed preserves the decision boundaries throughout the evolution of pruning. Another approach to investigate the lottery ticket could be by observing the polytopes representing the functional form of the network directly, i.e. δ(H {1,2} (x)) and δ(Q {1,2} (x)), in lieu of the decision boundaries polytopes. However, this does not provide conclusive answers to the lottery ticket, since there can exist multiple functional forms, and correspondingly multiple polytopes δ(H {1,2} (x)) and δ(Q {1,2} (x)), for networks with the same decision boundaries. This is why we explicitly focus our analysis on δ(R(x)), which is directly related to the decision boundaries of the network. Further discussions and experiments are left for the appendix. Network pruning has been identified as an effective approach for reducing the computational cost and memory usage during network inference time. While pruning dates back to the work of and , it has recently gained more attention. This is due to the fact that most neural networks over-parameterize commonly used datasets. In network pruning, the task is to find a smaller subset of the network parameters, such that the ing smaller network has similar decision boundaries (and thus supposedly similar accuracy) to the original over-parameterized network. In this section, we show a new geometric approach towards network pruning. In particu- th node, or equivalently removing the two yellow vertices of zonotope ZG 2 does not affect the decision boundaries polytope which will not lead to any change in accuracy. lar, as indicated by Theorem 2, preserving the polytope δ(R(x)) preserves a superset to the decision boundaries T (R(x)), and thus supposedly the decision boundaries themselves. Motivational Insight. For a single hidden layer neural network, the dual subdivision to the decision boundaries is the polytope that is the convex hull of two zonotopes, where each is formed by taking the Minkowski sum of line segments (Theorem 2). Figure 3 shows an example where pruning a neuron in the neural network has no effect on the dual subdivision polytope and equivalently no effect on the accuracy, since the decision boundaries of both networks remain the same. Problem Formulation. Given the motivational insight, a natural question arises: Given an overparameterized binary neural network f (x) = B max (Ax, 0), can one construct a new neural network, parameterized by some sparser weight matricesÃ andB, such that this smaller network has a dual subdivision δ(R(x)) that preserves the decision boundaries of the original network? In order to address this question, we propose the following general optimization problem The function d defines a distance between two geometric objects. Since the generatorsG 1 and G 2 are functions ofÃ andB (as per Theorem 2), this optimization problem can be challenging to solve. However, for pruning purposes, one can observe from Theorem 2 that if the generatorsG 1 andG 2 had fewer number of line segments (rows), this corresponds to a fewer number of rows in the weight matrixÃ (sparser weights). To this end, we observe that ifG 1 ≈ G 1 andG 2 ≈ G 2, thenδ(R(x)) ≈ δ(R(x)), and thus the decision boundaries tend to be preserved as a consequence. Therefore, we propose the following optimization problem as a surrogate to Problem The matrix mixed norm for C ∈ R n×k is defined as C 2,1 = n i=1 C(i, :) 2, which encourages the matrix C to be row sparse, i.e. complete rows of C are zero. Note thatG 1 = Diag[ReLU(B(1, :))+ReLU(−B(2, :))] Ã,G 2 = Diag[ReLU(B(2, :))+ReLU(−B(1, :))]Ã, and Diag(v) rearranges the elements of vector v in a diagonal matrix. We solve the aforementioned problem with alternating optimization over the variablesÃ andB, where each sub-problem is solved in closed form. Details of the optimization and the extension to multi-class case are left for the appendix. Extension to Deeper Networks. For deeper networks, one can still apply the aforementioned optimization for consecutive blocks. In particular, we prune each consecutive block of the form (Affine,ReLU,Affine) starting from the input and ending at the output of the network. Experiments on Tropical Pruning. Here, we evaluate the performance of the proposed pruning approach as compared to several classical approaches on several architectures and datasets. In particular, we compare our tropical pruning approach against Class Blind (CB), Class Uniform (CU) and Class Distribution (CD);. In Class Blind, all the parameters across all nodes of a layer are sorted by magnitude where x% with smallest magnitudes are pruned. Similar to Class Blind, Class Uniform prunes the parameters with smallest x% magnitudes per node in a layer as opposed to sorting all parameters in all nodes as in Class Blind. Lastly, Class Distribution performs pruning of all parameters for each node in the layer, just as in Class Uniform, but the parameters are pruned based on the standard deviation σ c of the magnitude of the parameters per node. Since fully connected layers in deep neural networks tend to have much higher memory complexity than convolutional layers, we restrict our focus to pruning fully connected layers. We train AlexNet and VGG16 on SVHN, CIFAR10, and CIFAR 100 datasets. We observe that we can prune more than 90% of the classifier parameters for both networks without affecting the accuracy. Moreover, we can boost the pruning ratio using our method without affecting the accuracy by simply retraining the network biases only. Setup. We adapt the architectures of AlexNet and VGG16, since they were originally trained on ImageNet , to account for the discrepancy in the input resolution. The fully connected layers of AlexNet and VGG16 have sizes of and, respectively on SVHN and CIFAR100 with the last layer replaced to 100 for CIFAR100. All networks were trained to baseline test accuracy of (92%,74%,43%) for AlexNet on SVHN, CIFAR10 and CIFAR100, respectively and (92%,92%,70%) for VGG16. To evaluate the performance of pruning, following previous works , we report the area under the curve (AUC) of the pruning-accuracy plot. The higher the AUC is, the better the trade-off is between pruning rate and accuracy. For efficiency purposes, we run the optimization in Problem for a single alternating iteration to identify the rows inÃ and elements ofB that will be pruned, since an exact pruning solution might not be necessary. The algorithm and the parameters setup to solving is left for the appendix. Results. Figure 4 shows the pruning comparison between our tropical approach and the three aforementioned popular pruning schemes on both AlexNet and VGG16 over the different datasets. Our proposed approach can indeed prune out as much as 90% of the parameters of the classifier without sacrificing much of the accuracy. For AlexNet, we achieve much better performance in pruning as compared to other methods. In particular, we are better in AUC by 3%, 3%, and 2% over other pruning methods on SVHN, CIFAR10 and CIFAR100, respectively. This indicates that the decision boundaries can indeed be preserved by preserving the dual subdivision polytope. For VGG16, we perform similarly well on both SVHN and CIFAR10 and slightly worse on CIFAR100. While the performance achieved here is comparable to the other pruning schemes, if not better, we emphasize that our contribution does not lie in outperforming state-of-the-art pruning methods, but rather in giving a new geometry based perspective to network pruning. We conduct more experiments, where only the biases of the network or the biases of the classifier are fine tuned after pruning. Retraining biases can be sufficient as they do not contribute to the orientation of the decision boundaries polytope, thereafter the decision boundaries, but only a translation. Discussion on biases and more are left for the appendix. DNNs are notoriously known to be susceptible to adversarial attacks. In fact, adding small imperceptible noise, referred to as adversarial attacks, at the input of these networks can hinder their performance. Several works investigated the decision boundaries of neural networks in the presence of adversarial attacks. For instance, analyzed high dimensional geometry of adversarial examples by the means of manifold reconstruction. Also, He et al. (2018b) crafted adversarial attacks by estimating the distance to the decision boundaries using random search directions. In this work, we provide a tropical geometric view to this problem. where we show how Theorem 2 can be leveraged to construct a tropical geometric based targeted adversarial attack. Dual View to Adversarial Attacks. For a classifier f: R n → R k and input x 0 that is classified as c, a standard formulation for targeted adversarial attacks flips the classifier prediction to a particular class t and it is usually defined as follows This objective aims at computing the lowest energy input noise η (measured by D) such that the the new sample (x 0 + η) crosses the decision boundaries of f to a new classification region. Here, we present a dual view to adversarial attacks. Instead of designing a sample noise η such that (x 0 + η) belongs to a new decision region, one can instead fix x 0 and perturb the network parameters to move the decision boundaries in a way that x 0 appears in a new classification region. In particular, let A 1 be the first linear layer of f, such that f (x 0) = g(A 1 x 0). One can now perturb A 1 to alter the decision boundaries and relate the perturbation to the input perturbation as follows From this dual view, we observe that traditional adversarial attacks are intimately related to perturbing the parameters of the first linear layer through the linear system: To this end, Theorem 2 provides explicit means to geometrically construct adversarial attacks by means of perturbing decision boundaries. In particular, since the normals to the dual subdivision polytope δ(R(x)) of a given neural network represent the tropical hypersurface set T (R(x)) which is, as per Theorem 2, a superset to the decision boundaries set B, ξ A1 can be designed to in a minimal perturbation to the dual subdivision that is sufficient to change the network prediction of x 0 to the targeted class t. Based on this observation, we formulate the problem as follows The loss is the standard cross-entropy loss. The first row of constraints ensures that the network prediction is the desired target class t when the input x 0 is perturbed by η, and equivalently by perturbing the first linear layer A 1 by ξ A1. This is identical to f 1 as proposed by. Moreover, the third and fourth constraints guarantee that the perturbed input is feasible and that the perturbation is bounded, respectively. The fifth constraint is to limit the maximum perturbation on the first linear layer, while the last constraint enforces the dual equivalence between input perturbation and parameter perturbation. The function D 2 captures the perturbation of the dual subdivision polytope upon perturbing the first linear layer by ξ A1. For a single hidden layer neural network parameterized as (A 1 + ξ A1) ∈ R p×n and B ∈ R 2×p for the 1 st and 2 nd layers respectively, D 2 can capture the perturbations in each of the two zonotopes discussed in Theorem 2. The derivation, discussion, and extension of to multi-class neural networks is left for the appendix. We solve Problem with a penalty method on the linear equality constraints, Motivational Insight to the Dual View. This intuition is presented in Figure 5. We train a single hidden layer neural network where the size of the input is 2 with 50 hidden nodes and 2 outputs on a simple dataset as shown in Figure 5. We then solve Problem 5 for a given x 0 shown in black. We show the decision boundaries for the network with and without the perturbation at the first linear layer ξ A1. Figure 5 shows that indeed perturbing an edge of the dual subdivision polytope, by perturbing the first linear layer, corresponds to perturbing the decision boundaries and in miss-classifying x 0. Interestingly and as expected, perturbing different decision boundaries corresponds to perturbing different edges of the dual subdivision. In particular, one can see from Figure 5 that altering the decision boundaries, by altering the dual subdivision polytope through perturbations in the first linear layer, can in miss-classifying a previously correctly classified input x 0. MNIST Experiment. Here, we design perturbations to misclassify MNIST images. Figure 7 shows several adversarial examples that change the network prediction for digits 8 and 9 to digits 7, 5, and 4, respectively. In some cases, the perturbation η is as small as = 0.1, where x 0 ∈ n. Several other adversarial are left for the appendix. We again emphasize that our approach is not meant to be compared with (or beat) state of the art adversarial attacks, but rather to provide a novel geometrically inspired perspective that can shed new light in this field. In this paper, we leverage tropical geometry to characterize the decision boundaries of neural networks in the form (Affine, ReLU, Affine) and relate it to well-studied geometric objects such as zonotopes and polytopes. We leaverage this representation in providing a tropical perspective to support the lottery ticket hypothesis, network pruning and designing adversarial attacks. One natural extension for this work is a compact derivation for the characterization of the decision boundaries of convolutional neural networks (CNNs) and graphical convolutional networks (GCNs). Diego Ardila, Atilla P. Kiraly, Sujeeth Bharadwaj, Bokyung Choi, Joshua J. Reicher, Lily Peng, Daniel Tse, Mozziyar Etemadi, Wenxing Ye, Greg Corrado, David P. Naidich, and Shravya Shetty. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography.. A PRELIMINARIES AND DEFINITIONS. Fact 1. P+Q = {p + q, ∀p ∈ P and q ∈ Q} is the Minkowski sum between two sets P and Q. Fact 2. Let f be a tropical polynomial and let a ∈ N. Then Let both f and g be tropical polynomials, Then Note that V(P(f)) is the set of vertices of the polytope P(f). Theorem 3. For a bias-free neural network in the form of f (x): R n → R 2 where A ∈ Z p×n and B ∈ Z 2×p, and let • If the decision boundaries of f is given by the set B = {x ∈ R n : f 1 (x) = f 2 (x)}, then we have B ⊆ T (R(x)). Note that A + = max(A, 0) and A − = max(−A, 0) where the max is element-wise. The line segment (B(1, j) is one that has the end points A(j, :) + and A(j, :) − in R n and scaled by the constant B(1, j) Proof. For the first part, recall from Theorem1 that both f 1 and f 2 are tropical rationals and hence, Recall that the tropical hypersurface is defined as the set of x where the maximum is attained by two or more monomials. Therefore, the tropical hypersurface of R(x) is the set of x where the maximum is attained by two or more monomials in (H 1 (x) Q 2 (x)), or attained by two or more monomials in (H 2 (x) Q 1 (x)), or attained by monomials in both of them in the same time, which is the decision boundaries. Hence, we can rewrite that as Therefore, we have that Therefore note that, thus we have that The operator+ indicates a Minkowski sum between sets. Note that ConvexHull A ) is the convexhull between two points which is a line segment in Z n with end points that are is a Minkowski sum of line segments which is is a zonotope. Moreover, note that tropically is given as follows. Thus it is easy to see that δ(Q 2 (x)) is the Minkowski sum of the points {(B − (1, j)−B + (2, j))A − (j, :)}∀j in R n (which is a standard sum) ing in a point. Lastly, it is easy to see that δ(H 1 (x))+δ(Q 2 (x)) is a Minkowski sum between a zonotope and a single point which corresponds to a shifted zonotope. A similar symmetric argument can be applied for the second part δ(H 2 (x))+δ(Q 1 (x)). It is also worthy to mention that the extension to network with multi class output is trivial. In that case all of the analysis can be exactly applied studying the decision boundary between any two classes (i, j) where B = {x ∈ R n : f i (x) = f j (x)} and the rest of the proof will be exactly the same. In this section, we derive the statement of Theorem 2 for the neural network in the form of (Affine, ReLU, Affine) with the consideration of non-zero biases. We show that the presence of biases does not affect the obtained as they only increase the dimension of the space, where the polytopes live, without affecting their shape or edge-orientation. Starting with the first linear layer for x ∈ R n, we have with coordinates, and ∆(Q 1i) is a point in (n + 1) dimensions at (A − (i, :), 0), while under π projection, δ(H 1i) is a point in n dimensions at (A + (i, :)), and δ(Q 1i) is a point in n dimensions at (A − (i, :)). It can be seen that under projection π, the geometrical representation of the output of the first linear layer does not change after adding biases. Looking to the output after adding the ReLU layer, we get, and δ(Q 1i) is the point (A − (i, :)). Again, the biases does not affect the geometry of the output after the ReLU layer, since the line segments now are connecting points in (n + 1) dimensions, but after projecting them using π, they will be identical to the line segments of the network with zero biases. Finally, looking to the output of the second linear layer, we obtain Similar arguments can be given for ∆(Q 3i) and δ(Q 3i). It can be seen that the first part in both expressions is a Minkowski sum of line segments, which will give a zonotope in (n + 1), and n dimensions in the first and second expressions respectively. While the second part in both expressions is a Minkowski sum of bunch of points which gives a single point in (n + 1) and n dimensions for the first and second expression respectively. Note that the last dimension of the aforementioned point in n + 1 dimensions is exactly the i th coordinate of the bias of the second linear layer which is dropped under the π projection. Therefore, the shape of the geometrical representation of the decision boundaries with non-zero biases will not be affected under the projection π, and hence the presence of the biases will not affect any of the of the paper. Proposition 1. Consider p line segments in R n with two arbitrary end points as follows. The zonotope formed by these line segments is equivalent to the zonotope formed be the line segments Proof. Let U j be a matrix with U j (:, i) = u i j, i = 1,..., p, w be a column-vector with w(i) = w i, i = 1,..., p and 1 p is a column-vector of ones of length p. Then, the zonotope Z formed by the Minkowski sum of line segments with arbitrary end points can be defined as Note that the Minkowski sum of any polytope with a point is a translation; thus, the follows directly from Definition 6. A ← arg miñ, where c 1 = ReLU(B(1, :)) + ReLU(−B(2, :)) and c 2 = ReLU(B(2, :)) + ReLU(−B(1, :)). Note that the problem is separable per-row ofÃ. Therefore, the problem reduces to updating rows ofÃ independently and the problem exhibits a closed form solution.. UpdateB + (1, :). Note that C 1 = G 1 − Diag B − (2, :) Ã and where Diag B − (2, :) Ã. Note the problem is separable in the coordinates ofB + (1, :) and a projected gradient descent can be used to solve the problem in such a way as A similar symmetric argument can be used to update the variablesB + (2, :),B + (1, :) andB − (2, :). Note that Theorem 2 describes a superset to the decision boundaries of a binary classifier through the dual subdivision R(x), i.e. δ(R(x)). For a neural network f with k classes, a natural extension for it is to analyze the pair-wise decision boundaries of of all k-classes. Thus, let T (R ij (x)) be the superset to the decision boundaries separating classes i and j. Therefore, a natural extension to the geometric loss in equation 1 is to preserve the polytopes among all pairwise follows The set S is all possible pairwise combinations of the k classes such that S = {[i, j], ∀i = j, i = 1,..., k, j = 1,..., k}. The generator Z (G (i,j) ) is the zonotope with the generator matrix G (i +,j −) = Diag ReLU(B(i, :)) + ReLU(−B(j, :)) Ã. However, such an approach is generally computationally expensive, particularly, when k is very large. To this end, we make the following observation thatG (i +, j −) can be equivalently written as a Minkowski sum between two sets zonotopes with the generators That is to say, ZG. This follows from the associative property of Minkowski sums given as follows: be the set of n line segments. Then we have that S = S 1+...+S n = P+V where the sets P =+ j∈C1 S j and V =+ j∈C2 S j where C 1 and C 2 are any complementary partitions of the set Hence,G (i +,j −) can be seen a concatenation betweenG (i +) andG (j −). Thus, the objective in 10 can be expanded as follows The approximation follows in a similar argument to the binary classifier case where approximating the generators. The last equality follows from a counting argument. We solve the objective for all multi-class networks in the experiments with alternating optimization in a similar fashion to the binary classifier case. Similarly to the binary classification approach, we introduce the 2,1 to enforce sparsity constraints for pruning purposes. Therefore the overall objective has the form For completion, we derive the updates forÃ andB. A = arg miñ Similar to the binary classification, the problem is seprable in the rows ofÃ. and a closed form solution in terms of the proximal operator of 2 norm follows naturally for eachÃ(i, :). UpdateB + (i, :). Note that the problem is separable per coordinates of B + (i, :) and each subproblem is updated as: A similar argument can be used to updateB − (i, :) ∀i. Finally, the parameters of the pruned network will be constructed A ←Ã and B ←B + −B −. Input: In this section, we are going to derive an algorithm for solving the following problem. The function D 2 (ξ A) captures the perturbdation in the dual subdivision polytope such that the dual subdivion of the network with the first linear layer A 1 is similar to the dual subdivion of the network with the first linear layer A 1 + ξ A1. This can be generally formulated as an approximation to the following distance function This can thereafter be extended to multi-class network with k classes as follows. , we take. Therefore, we can write 11 as follows To enforce the linear equality constraints A 1 η − ξ A1 x 0 = 0, we use a penalty method, where each iteration of the penalty method we solve the sub-problem with ADMM updates. That is, we solve the following optimization problem with ADMM with increasing λ such that λ → ∞. For ease of notation, lets denote where The augmented Lagrangian is thus given as follows Thereafter, ADMM updates are given as follows Updating η: Updating w: It is easy to show that the update w is separable in coordinates as follows Updating z: showed that the linearized ADMM converges for some non-convex problems. Therefore, by linearizing L and adding Bergman divergence term η, we can then update z as follows It is worthy to mention that the analysis until this step is inspired by with modifications to adapt our new formulation. Updating ξ A: The previous problem can be solved with proximal gradient method. In this section, we are going to describe the settings and the values of the hyper-parameters that we used in the experiments. Moreover, we will show more since we have limited space in the main paper. We begin by throwing the following question. Why investigating the tropical geometrical perspective of the decision boundaries is more important than investigating the tropical geometrical representation of the functional form of the network? In this section, we show one more experiment that differentiate between these two views. In the following, we can see that variations can happen to the tropical geometrical representation of the functional form (zonotopes in case of single hidden layer neural network), but the shape of the polytope of the decision boundaries is still unchanged and consequently, the decision boundaries. For this purpose, we trained a single hidden layer neural network on a simple dataset like the one in Figure 2, then we do several iteration of pruning, and visualise at each iteration both the polytope of the decision boundaries and the zonotopes of the functional representation of the neural network. It can be easily seen that changes in the zonotopes may not change the shape of the decision boundaries polytope and consequently the decision boundaries of the neural network. And thus it can be clearly seen that our formulation, which is looking at the decision boundaries polytope is more general, precise and indeed more meaningful. Moreover, we conducted the same experiment explained in the main paper of this section on another dataset to have further demonstration on the favour that the lottery ticket initialization has over other initialization when pruning and retraining the pruned model. It is clear that the lottery initializations is the one that preserves the shape of the decision boundaries polytope the most. In the tropical pruning, we have control on two hyper-parameters only, namely the number of iterations and the regularizer coefficient λ which controls the pruning rate. In all of the experiments, we ran the algorithm for 1 iteration only and we increase λ starting from 0.02 linearly with a factor of 0.01 to reach 100% pruning. It is also worthy to mention that the output of the algorithm will be new sparse matricesÃ,B, but the new network parameters will be the elements in the original matrices A, B that have indices correspond to the indices of non-zero elements inÃ,B. By that, the algorithm removes the non-effective line segments that do not contribute to the decision boundaries polytope, without changing the non-deleted segments. Above all, more of pruning of AlexNet and VGG16 on various datasets are shown below. For all of the experiments, {2, λ, η, ρ} had the values {1, 10 −3, 2.5, 1} respectively. the value of 1 was 0.1 when attacking the -fours-images, and 0.2 for the rest of the images. Finally, we show extra of attacking the decision boundaries of synthetic data in R 2 and MNIST images by tropical adversarial attacks. We thank R3 for the time spent reviewing the paper. It is though not clear to the authors the main reason behind the initial score of weak reject as R3 seems to have very generic questions about our work but not a particular criticism of the novelty/contribution that we can address in our rebuttal. We hope that the following response addresses and clarifies some key elements. Moreover, we want to bring to the attention of R3 that we have addressed the comments/typos/suggestions of all reviewers in the revised version and marked them in blue. Q1: What benefit does introducing tropical geometry brings in terms of theoretical analysis? Does using tropical geometry give us the theoretical that traditional analysis can not give us? If so, what is it? I am trying to understand why the authors use this tool. The authors should be explicit in their motivation so that the readers are clear about the contribution of this paper. More specifically, from my perspective, tropical semiring, tropical polynomials and tropical rational functions all can be represented with the standard mathematical tools. Here they are just redefining several concepts. As discussed thoroughly in the introduction (last paragraph of page 1), tropical geometry is the younger twin to algebraic geometry on a particular semiring defined in a way to align with the study of piecewise linear functions. The early definitions stated in the paper (1 to 5) are well known in the TG literature and were restated for the completion of this paper. While it is true that the definitions can be represented with standard mathematical tools; however, this misses the fundamental powerful element TG promises. TG transforms algebraic problems of piecewise linear nature to a combinatoric problem on general polytopes. To that end, Zhang et. al. 2018 (to the best of our knowledge the only work at the intersection between TG and DNNs) rederived classical (upper bound on the number of linear pieces of DNNs) in a much simpler analysis by counting vertices on polytopes. In this work, instead of studying the functional representation of piecewise linear DNNs, we study their decision boundaries using the lens of TG. To wit, the geometric characterization of the decision boundaries of DNNs developed in Theorem 2 cannot be attained using standard mathematical tools. More specifically, Theorem 2 represented a superset to the decision boundaries (the tropical hypersurface T (R(x)), with a geometric structure that is the convex hull between two zonotopes. While this by itself opens doors for a family of new geometrically motivated regualrizers for training DNNs that are in direct correspondence with the behaviour of the decision boundaries, we do not dwell on training beyond this point and leave that for future work. However, this new allowed for re-affirmation to the lottery ticket hypothesis in a new fresh perspective. Moreover, we propose new optimization problems that are geometrically motivated (based on Theorem 2) for several classical problems, i.e. network pruning and adversarial attacks that were not possible before and have provided several new insights and directions. That is we show an intimate relation between network perturbations (through decision boundary polytope perturbations) and the construction of adversarial attacks (input perturbations). In Experiments on Tropical Pruning, the authors mentioned we compare our tropical pruning approach against Class Blind (CB), Class Uniform (CU), and Class Distribution (CD) methods. What is Class Blind, Class Uniform and Class Distribution? There seems to be an error here Figure 5 shows the pruning comparison between our tropical approach..., i think Figure 5 should be Figure 4. We have added the definition of the pruning methods of in the revised version of the paper for completion, and corrected the typo in the Figure reference. Q3: In the adversarial attack part, is the authors proposing a new attack method? If so, then the authors should report the test accuracy under attack. Also, the experimental should not be restricted to MNIST dataset. I am also not sure about the attack settings here, the authors said Instead of designing a sample noise such that (x0 + η) belongs to a new decision region, one can instead fix x0 and perturb the network parameters to move the decision boundaries in a way that x0 appears in a new classification region.. Why use this setting? Are there any intuitions? Since this is different from traditional adversarial attack terminology, the authors should stop using adversarial attacks as in tropical adversarial attacks because it is really misleading. As highlighted in the last sentence in section 6, we are not competing against other attacks, but we rather show how this new geometric view to the decision boundaries provided by the TG analysis in Theorem 2 can be leveraged for the construction of adversarial attacks. We want to emphasize to R3 that the polytope representing the decision boundaries (convex hull of two zonotopes as per Theorem 2) is a function of the network parameters and not of the input space. Thus, it is not initially clear how one can frame the adversarial attacks problem in this new fresh tropical setting since adversarial attacks is the task of perturbing the input space as opposed to the parameters space of the network ing in a flip in the prediction. In the tropical adversarial attacks section, we show that the problem of designing an adversarial attack x 0 +η that flips the network prediction is closely related to the problem of flipping the network prediction by perturbing the network parameters in the first layer A 1 + ζ A1 where both problems are related through a linear system. That is to say, if one finds ζ A1 that perturbs the geometric structure (convex hull between two zonotopes, i.e. decision boundaries) sufficiently enough to flip the network prediction, one can find an equivalent pixel adversarial attack η by solving the linear system A 1 η = ζ A1 x 0 that flips the prediction of the original unperturbed network (see the end of page 7). We thereafter propose Problem incorporating the geometric information from Theorem 2 where the linear system is accounted for in the constraints set. We propose an algorithm to solve the problem (a mix of penalty and ADMM) detailed in Algorithm 1 in the appendix. The solution to problem 5 by applying Algorithm 1 in the construction of adversarial attacks (η) that indeed flip the network prediction over all tested examples on the MNIST dataset. We thank R2 for the time spent reviewing the paper. We also thank R2 for acknowledging our technical and theoretical contributions. Please note that we have addressed the comments/typos/suggestions of all reviewers in the revised version and marked them in blue. Follows our response. Q1: This paper needs to be placed properly among several important missing references on the decision boundary of deep neural networks. In particular, using introduced tropical geometry perspective, how we can obtain the complexity of the decision boundary of a deep neural network? The two works referenced by R2 are not directly related to the body of our work. Below, we summarize both works and state how our work is vastly different from both. The authors of show that under certain assumptions, the decision boundaries of the last fully connected layer converges to an SVM classifier. That is to say, the features learnt in deep neural networks are linearly separable with max margin type linear classifier. On the other hand, the authors of showed that the decision regions of neural networks with width smaller than the input dimension are unbounded. In our work, we use a new type of analysis (tropical geometry) to represent the set of decision boundaries B through its superset T (R(x)) that is the solution set to the tropical polynomial R(x). We then show that this solution set is related to a geometric structure referred to as the decision boundaries polytope (convex hull between two zonotopes), this is analogous to constructing newton polytopes for the solution sets to classical polynomials in algebraic geoemtry. The normals to the edges of this polytope are parallel to the superset of the decision boundaries T (R(x)). That is to say, if one processes the polytope in an way that preserves the direction of the normals, the decision boundaries of the network are preserved. This is the base idea behind all later experiments. In general, this new representation presents a new fresh revisit to the lottery ticket hypothesis and an utterly new view to network pruning and adversarial attacks. We do believe this new representation can be of benefit to other applications and can open doors for a family of new geometrically inspired network regularizers as well. Q3: The second part of Theorem 2 should be explained straightforwardly and clearly as it plays an important role in the subsequent and applications. We have added a "Digesting Theorem 2" paragraph in the revised version and rearranged the structure a bit around Theorem 2. Most of the parameters (memory complexity) are the in fully connected layers. For example, the convolutional part of VGG16 has 14,714,688 parameters, whereas the fully connected layers have 262,000,400 parameters in total which is 17 times larger. Similarly, the convolutional part of AlexNet has 3,747,200 parameters while only the first fully connected layer has 37,752,832 parameters. However, efficiently extending the tropical pruning to convolutional layers is a nontrivial interesting direction. Generally speaking, convolutional layers fit our framework naturally since a convolutional kernel can be represented with a structured topelitz/circulant matrix. However, a question of efficiency still remains as one still needs to construct the underlying structured matrix representing the convolutional kernel. Thereafter, a direction of interest is the tropical formulation of the network pruning problem as a function of the convolutional kernels surpassing the need for the construction of the dense representation of the kernel. We keep this for future work. Q5: For the similarity measure. Comparing the exact decision boundaries between two different architectures can be very difficult in the sense where decision boundaries for a two-class output network f are defined as {x ∈ R n : f 1 (x) = f 2 (x)}. Another approach to compare decision boundaries, which is proposed by our work, is by computing the distance between the the dual subdivision polytope (δ(R(x))) of the tropical polynomials R(x) representing two different architectures. This is since the normals to the edges of the polytope (δ(R(x))) are parallel to a superset of the decision boundaries (see Figure 1). This is exactly the proposed objective where d is a distance function to compare the orientation between two general polytopes. Since finding a good choice for d is generally difficult we instead approximate it by comparing the generators constructing the (δ(R(x))) in Euclidean distance for ease (objective). Experimentally and following prior art in the pruning literature (Han et. al. 2015), to compare the effectiveness of the pruning scheme, we compare the test accuracies across architectures as a function of the pruning ratio. Regardless, the reviewer is right about that similar test accuracies does not imply similar decision boundaries but rather only an indication. In adversarial examples generation, typically for a pre-trained deep neural network model one is interested in generating examples that are misclassified by the model while they resemble real instances. In this setting, we keep the model and thus its decision boundary intact. In this paper, nevertheless, aiming at generating adversarial examples, the decision boundary and thus the (pre-trained) model is altered. By chaining the decision boundary, however, the model's decisions for original real samples might change as well. Therefore, it is not clear to the reviewer how the introduced method is comparable to the well-established adversarial example generation setting. The new approach is definitely comparable to the well-establisehd adversarial example generation setting. Let us explain. The new analysis provided by Theorem 2, allows to present the decision boundaries geometrically as a convex hull between two zonotopes which is a function of only the network parameters and not the input space. Thus, it not clear how one can frame the adversarial attacks problem in this new fresh tropical setting since adversarial attacks is the task of perturbing the input space as opposed to the parameters space of the network ing in a flip in the prediction. In the tropical adversarial attacks section, we show that the problem of designing an adversarial attack x 0 + η that flips the network prediction is closely related to the problem of flipping the network prediction by perturbing the network parameters in the first layer A 1 + ζ A1 where both problems are related through a linear system. That is to say, if one finds ζ A1 (perturbations in the first linear layer) that perturbs the geometric structure (convex hull between two zonotopes, i.e. decision boundaries) sufficiently enough to flip the network prediction, one can find an equivalent pixel adversarial attack η by solving the linear system A 1 η = ζ A1 x 0 that flips the prediction of the original unperturbed network (see the end of page 7). We incorporate this in an overall objective where the linear system is incorporated as a constraint. Upon solving with the proposed Algorithm 1, we attack the original network (unperturbed) with the adversarial attack η. Therefore, this is comparable to the classical adversarial attacks framework. This approach indeed ed into flipping the network prediction over all tested examples on the MNIST dataset. As highlighted at the end of section 6, we do not aim in our approach to outperform the state of the art adversarial attacks, but rather to provide a novel geometrically inspired perspective that can shed new light in this field. Q7: Two previous papers investigated the decision boundary of the deep neural networks in the presence of adversarial examples. Please discuss how the introduced method in this paper is placed among these methods. The work of analyzed the geometry of adversarial examples by means of manifold reconstruction to study the trade off between robustness under different norms. On the other hand, crafted adversarial attacks by estimating the distance to the decision boundaries using random search directions. Both of the papers made a local estimation of the decision boundary around the attacked point to construct the adversarial attack to the input image. In our work, we geometrically characterized the decision boundaries in Theorem 2 where the polytope (convex hull of two zonotopes) is only a function of the network parameters and NOT the input space. We presented a dual view to adversarial attacks in which one can construct adversarial examples by investigating network parameters perturbations that in the largest perturbation to this polytope representing the decision boundaries. The scope of our work, and unlike prior art, is focused towards a new geometric polytope representation of the decision boundary in the network parameter space (not the input space) through a new novel analysis. We have added a discussion of both papers in the adversarial attacks section (Section 6). We have addressed the concerns of R2 and left the changes in blue in the revised version. "On the decision boundary of deep neural networks". SLi, Yu and Richtarik, Peter and Ding, Lizhong and Gao, Xin. We thank R1 for the constructive detailed thorough review of the paper and for acknowledging our contributions and the new insights. Follows our response to R1's concerns. In regards to clarity, exposition and focus. To improve the clarity and exposition, we have added several paragraphs in the revised version of the paper. The revised edits are marked in blue. As in regards to the focus, we found that it is challenging within a reasonable time to carry out this major change in the paper. To that end, we have done our best to further elaborate on several key in the paper. For instance, we have merged the paragraph above the contributions with the contributions paragraph. We have added another paragraph dissecting Theorem 2. We have added some few relevant references that are essential for the context of motivating tropical adversarial attacks. In regards to the suggestions. • Adding the information that the semiring lacks the additive inverse. This has been addressed in the revised version. •Adding tropical quotient to definition 1. This has been addressed in the revised version. • Definition of π and the upper faces. Indeed, π is a projection operator that drops the last coordinate. As for upper faces, the formal definition is given as follows, for a polytope P, F is an upper face of P if x + te / ∈ P for any x ∈ F, t > 0 where e is a canonical vector. That is the faces that can be seen from "above". A good graphical example can be found in Figure 2 from Zhang et. al. 2018. We dropped this definition from the paper as it may add some confusion while it does not play an important role in the later analysis. • Theorem 2 lacks intuitive formulation. This has been addressed in the revised version and we have rearranged the structure of text below Theorem 2. • Regarding the issue of the tropical hypersurface. Indeed, the superset is in terms of set theory. That is to say, the set of decision boundaries B is a supset of the tropical hypersurface set T (R(x)). • Regarding Figure 2. The color map represents the compression percentage. We have added a legend to Figure 2. Note that the second figure in Figure 2 tilted "original polytope" represents the polytope of the dual subdivision (convex hull between two zonotopes). While the polytope seems to have only 4 vertices, there are in fact many other overlapping vertices and thereafter many small edges between all the seemingly overlapping vertices. It is to observe that the normals to only the 4 major edges in the "original polytope" are indeed parallel to the decision boundaries plotted by performing forward passes through the network in the first figure titled "Input Space". Note that despite the fact that more compression is performed, the orientation of the overall polytope is preserved for the lottery ticket initialization and thereafter preserving the main orientation of the decision boundaries. This is unlike the the other types of initialization where the orientation of the polytope is vastly different with different compression ratios ing into a larger change in the orientation of the decision boundaries. • I would suggest to place Figure 1 after stating Theorem 2, since it is only referenced later on. Furthermore, the red structures are somewhat confusing. According to Theorem 2, the decision boundary is a subset of the hypersurface, right? What is the relation of the red structures in the convex hull visualisation? The caption states that they are normals, but as far as I can tell, this has not been formalised anywhere in the paper (it is used later on, though). Correct. The decision boundaries are subsets of the tropical hypersurfaces. However, it has been shown by (Propositon 3.1.6) that the tropical hypersurface T to any d variate tropical polynomial is the (d-1)-skeleton of the polyhedral complex dual to the dual subdivision δ. This implies that the normals to the edges (faces in higher dimensions) are parallel to the tropical hypersurface. If R1 is interested in learning more about this, an excellent starting point to build up this intuition is the work of Erwan Brugall and Kristin Shaw "A bit of tropical geometry". Please refer to section 2.2 "Dual subdivisions" pages 6 and 7. • The discussion about the functional form. We elaborate on this in section F in the appendix. Instead of investigating the decision boundaries polytope of the tropical polynomial representing the decision boundaries R(x), we analyze the 4 different tropical polynomials representing the 2 different classes. Recall each output function of the network is a tropical rational of two tropical polynomials giving rise to 4 different polytopes. Figure 7 shows that the decision boundary polytope δ(R(x)) with the lottery ticket is hardly changed while pruning the network (first column in Figure 7). On the contrary, the zonotopes (dual subdivisions of the 4 different tropical polynomials) vary much more significantly (coloumns 2,3,4,5). This demonstrates that there can exist different tropical polynomials representing the functional form of the network (i.e. H 1,2 and Q 1,2) while having the same structure for the decision boundary polytope the corresponding first figure in the same row of Figure 7. This is a mere observation and worth investigating in future work. • In Section 4, how many experiments of the sort were performed? I find this a highly instructive view so I would love to see more experiments of this sort. Do these claims hold over multiple repetitions and for (slightly) larger architectures as well? We already had one extra experiment (Figure 8) in the appendix for another data. We have also based on the suggestion of R1 added two more experiments (Figure 9) on two other datasets. Note that extracting the decision boundaries polytope for larger architectures is much more difficult for two different reasons. First, for deeper networks beyond the structure Affine-ReLU-Affine, the decision boundaries polytope is generic and no longer enjoys the nice properties zonotopes exhibit. Enumerating their vertices rapidly turns to a computationally intractable problem. Secondly, one can perhaps visualize the polytope for networks with 3 dimensional input but it gets trickier beyond that. • Regarding that the claim that orientations are preserved should be formalised. That is an excellent question. We have investigated several metrics that try to capture information about the orientation of a given polytope. For instance, we have investigated the feature that is the histogram of the oriented normals. That is a histogram of angles for all the normals to edges given a polytope. We have also investigated the Hausdorff distance as a metric between polytopes. However, we have decided to keep this for a future direction as this is by itself a entirely new line of work. That is designing the distance functions d between polytopes that captures orientation information that can be used for tropical pruning (objective 8) or perhaps other applications. Another interesting direction is whether such a distance function can be learnt in a meta-learning fashion. For now, we restrict the experiments to the approximation used in objective which for now only captures the distance between the sets of generators of the zonotopes in Eucledian sense. • Adding link to the definition of minkowski sum in the appendix. This has been addressed in the revised version. • Description to other pruning methods. This has been addressed in the revised version where we have added the definition to all pruning competitors. • About the plots in Figure 4. In the experiments of Figure 4, there is no stochasticity. All base networks (AlexNet and VGG16) are trained before hand to achieve the state-of-art baseline on the respective datasets (SVHN, CIFAR10 and CIFAR100). The networks are then fixed and several pruning schemes, including the tropical approach, are applied with a varying level of pruning ratio. We do not re-train the networks after each pruning step. Thus there exists no source of randomness. However, this is still an excellent observation. This is since we conduct experiments in the appendix where after each pruning ratio we fine tune only the biases of the classifier (Figure 10) or we fine tune the biases of the complete network (Figure 11). In such experiments, due to the fine tuning step, we will consider in the final version to report the averaged over multiple runs. • In Section 6, I find the comment on normals generating a superset to the decision boundaries hard to understand. Indeed, the wording of the sentence was sub optimal. The statement was to reassure what has been established in earlier sections that the normals to the decision boundary polytope δ(R(x)) represent the tropical hypersurface set T (R(x)) which is a super set to the decision boundaries set B as per Theorem 2. We have rephrased the sentence. • Take-away message regarding perturbing the decision boundaries. The proposed approach of perturbing the decision boundaries is a mere dual view for adversarial attacks. That is to say, to flip a network prediction for a sample x 0, one can either adversarially perturb the sample (add noise) to cross the decision boundary to a new classification region or one can perturb the decision boundaries to move closer to the sample x 0 to appear as if it is in a new classification region. Figure 5, demonstrates the later visually through perturbing the decision boundary polytope. One can observe that perturbing the correct edge in the polytope in Figure 5, corresponds to altering a specific decision boundary. Figure 5, indeed as correctly pointed out of R1, is a feasibility study showing that one can perturb decision boundary by perturbing the dual subdiviosn polytope. However, the real take away message is that these two views (perturbing the input or the parameter space) are intimately related through a linear system as discussed in the subsection titled "Dual View to Adversarial Attacks". We propose an objective function (Problem 5) and an algorithm (Algorithm 1) to tackle this problem. The solution to problem provides an INPUT perturbation that in altering the network prediction. That is to say, the new framework allows for incorporating geometrically motivated objective function towards constructing classical adversarial attacks. • Regarding the future extension on CNNs and GCNs. We are currently investigating efficient extension of the current to convolutional layers. Note that while convolutional layers can be represented with a large structured topelitz/circulant matrix, we are interested in extensions that allow for similar analysis but as a function of the convolutional kernel surpassing the need to constructing convolutional matrix. The GCNs is definitely an exciting excellent future direction that we have not yet entertained. • Minor style issues. We have addressed all the style issues in the revised version."

This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

This paper identifies classes of problems for which adversarial examples are inescapable, and derives fundamental bounds on the susceptibility of any classifier to adversarial examples.