Question: Represent the scientific article to find a short TLDR

Question: We present a new methodology that constructs a family of \emph{positive definite kernels} from any given dissimilarity measure on structured inputs whose elements are either real-valued time series or discrete structures such as strings, histograms, and graphs. Our approach, which we call D2KE (from Distance to Kernel and Embedding), draws from the literature of Random Features. However, instead of deriving random feature maps from a user-defined kernel to approximate kernel machines, we build a kernel from a random feature map, that we specify given the distance measure. We further propose use of a finite number of random objects to produce a random feature embedding of each instance. We provide a theoretical analysis showing that D2KE enjoys better generalizability than universal Nearest-Neighbor estimates. On one hand, D2KE subsumes the widely-used \emph{representative-set method} as a special case, and relates to the well-known \emph{distance substitution kernel} in a limiting case. On the other hand, D2KE generalizes existing \emph{Random Features methods} applicable only to vector input representations to complex structured inputs of variable sizes. We conduct classification experiments over such disparate domains as time series, strings, and histograms (for texts and images), for which our proposed framework compares favorably to existing distance-based learning methods in terms of both testing accuracy and computational time. In many problem domains, it is easier to specify a reasonable dissimilarity (or similarity) function between instances than to construct a feature representation. This is particularly the case with structured inputs whose elements are either real-valued time series or discrete structures such as strings, histograms, and graphs, where it is typically less than clear how to construct the representation of entire structured inputs with potentially widely varying sizes, even when given a good feature representation of each individual component. Moreover, even for complex structured inputs, there are many well-developed dissimilarity measures, such as the Dynamic Time Warping measure between time series, Edit Distance between strings, Hausdorff distance between sets, and Wasserstein distance between distributions. However, standard machine learning methods are designed for vector representations, and classically there has been far less work on distance-based methods for either classification or regression on structured inputs. The most common distance-based method is Nearest-Neighbor Estimation (NNE), which predicts the outcome for an instance using an average of its nearest neighbors in the input space, with nearness measured by the given dissimilarity measure. Estimation from nearest neighbors, however, is unreliable, specifically having high variance when the neighbors are far apart, which is typically the case when the intrinsic dimension implied by the distance is large. To address this issue, a line of research has focused on developing global distance-based (or similaritybased) machine learning methods BID38 BID16 BID1 BID12, in large part by drawing upon connections to kernel methods BID43 or directly learning with similarity functions BID1 BID12 BID2 BID29; we refer the reader in particular to the survey in BID7. Among these, the most direct approach treats the data similarity matrix (or transformed dissimilarity matrix) as a kernel Gram matrix, and then uses standard kernel-based methods such as Support Vector Machines (SVM) or kernel ridge regression with this Gram matrix. A key caveat with this approach however is that most similarity (or dissimilarity) measures do not provide a positive-definite (PD) kernel, so that the empirical risk minimization problem is not well-defined, and moreover becomes non-convex BID33 BID28.A line of work has therefore focused on estimating a positive-definite (PD) Gram matrix that merely approximates the similarity matrix. This could be achieved for instance by clipping, or flipping, or shifting eigenvalues of the similarity matrix BID36, or explicitly learning a PD approximation of the similarity matrix BID6 BID8. Such modifications of the similarity matrix however often leads to a loss of information; moreover, the enforced PD property is typically guaranteed to hold only on the training data, ing in an inconsistency between the set of testing and training samples BID7 1.Another common approach is to select a subset of training samples as a held-out representative set, and use distances or similarities to structured inputs in the set as the feature function BID20 BID36 ). As we will show, with proper scaling, this approach can be interpreted as a special instance of our framework. Furthermore, our framework provides a more general and richer family of kernels, many of which significantly outperform the representative-set method in a variety of application domains. To address the aforementioned issues, in this paper, we propose a novel general framework that constructs a family of PD kernels from a dissimilarity measure on structured inputs. Our approach, which we call D2KE (from Distance to Kernel and Embedding), draws from the literature of Random Features BID39, but instead of deriving feature maps from an existing kernel for approximating kernel machines, we build novel kernels from a random feature map specifically designed for a given distance measure. The kernel satisfies the property that functions in the corresponding Reproducing Kernel Hilbert Space (RKHS) are Lipschitz-continuous w.r.t. the given distance measure. We also provide a tractable estimator for a function from this RKHS which enjoys much better generalization properties than nearest-neighbor estimation. Our framework produces a feature embedding and consequently a vector representation of each instance that can be employed by any classification and regression models. In classification experiments in such disparate domains as strings, time series, and histograms (for texts and images), our proposed framework compares favorably to existing distance-based learning methods in terms of both testing accuracy and computational time, especially when the number of data samples is large and/or the size of structured inputs is large. We highlight our main contributions as follows:• From the perspective of distance kernel learning, we propose for the first time a methodology that constructs a family of PD kernels via Random Features from a given distance measure for structured inputs, and provide theoretical and empirical justifications for this framework.• From the perspective of Random Features (RF) methods, we generalize existing Random Features methods applied only to vector input representations to complex structured inputs of variable sizes. To the best of our knowledge, this is the first time that a generic RF method has been used to accelerate kernel machines on structured inputs across a broad range of domains such as time-series, strings, and the histograms. Distance-Based Kernel Learning. Existing approaches either require strict conditions on the distance function (e.g. that the distance be isometric to the square of the Euclidean distance) BID22 BID42, or construct empirical PD Gram matrices that do not necessarily generalize to the test samples BID36 BID38 BID34 BID16. BID22 and BID42 provide conditions under which one can obtain a PD kernel through simple transformations of the distance measure, but which are not satisfied for many commonly used dissimilarity measures such as Dynamic Time Warping, Hausdorff distance, and Earth Mover's distance (Haasdonk & Bahlmann, 1A generalization error bound was provided for the similarity-as-kernel approach in BID7, but only for a positive-definite similarity function.). Equivalently, one could also find a Euclidean embedding (also known as dissimilarity representation) approximating the dissimilarity matrix as in Multidimensional Scaling BID36 BID38 BID34 BID16 2. Differently, BID29 presented a theoretical foundation for an SVM solver in Krein spaces and directly evaluated a solution that uses the original (indefinite) similarity measure. There are also some specific approaches dedicated to building a PD kernel on some structured inputs such as text and time-series BID11 BID13, that modify a distance function over sequences to a kernel by replacing the minimization over possible alignments into a summation over all possible alignments. This type of kernel, however, in a diagonal-dominance problem, where the diagonal entries of the kernel Gram matrix are orders of magnitude larger than the off-diagonal entries, due to the summation over a huge number of alignments with a sample itself. Interest in approximating non-linear kernel machines using randomized feature maps has surged in recent years due to a significant reduction in training and testing times for kernel based learning algorithms BID14. There are numerous explicit nonlinear random feature maps that have been constructed for various types of kernels, including Gaussian and Laplacian Kernels BID39 BID48, intersection kernels BID30 ), additive kernels BID47, dot product kernels BID25 BID37, and semigroup kernels BID31. Among them, the Random Fourier Features (RFF) method, which approximates a Gaussian Kernel function by means of multiplying the input with a Gaussian random matrix, and its fruitful variants have been extensively studied both theoretically and empirically BID46 BID17 BID41 BID0 BID10. To accelerate the RFF on input data matrix with high dimensions, a number of methods have been proposed to leverage structured matrices to allow faster matrix computation and less memory consumption BID27 BID23 BID9.However, all the aforementioned RF methods merely consider inputs with vector representations, and compute the RF by a linear transformation that is either a matrix multiplication or an inner product under Euclidean distance metric. In contrast, D2KE takes structured inputs of potentially different sizes and computes the RF with a structured distance metric (typically with dynamic programming or optimal transportation). Another important difference between D2KE and existing RF methods lies in the fact that existing RF work assumes a user-defined kernel and then derives a randomfeature map, while D2KE constructs a new PD kernel through a random feature map and makes it computationally feasible via RF. The table 1 lists the differences between D2KE and existing RF methods. A very recent piece of work BID49 has developed a kernel and a specific algorithm for computing embeddings of single-variable real-valued time-series. However, despite promising , this method cannot be applied on discrete structured inputs such as strings, histograms, and graphs. In contrast, we have an unified framework for various structured inputs beyond the limits of BID49 and provide a general theoretical analysis w.r.t KNN and other generic distance-based kernel methods. We consider the estimation of a target function f: X → R from a collection of samples {( DISPLAYFORM0, where x i ∈ X is the structured input object, and y i ∈ Y is the output observation associated with the target function f (x i). For instance, in a regression problem, y i ∼ f (x i) + ω i ∈ R for some random noise ω i, and in binary classification, we have y i ∈ {0, 1} with P(y i = 1|x i) = f (x i). We are given a dissimilarity measure d: X × X → R between input objects instead of a feature representation of x.2A proof of the equivalence between PD of similarity matrix and Euclidean of dissimilarity matrix can be found in BID4.Note that the size structured inputs x i may vary widely, e.g. strings with variable lengths or graphs with different sizes. For some of the analyses, we require the dissimilarity measure to be a metric as follows. DISPLAYFORM1 An ideal feature representation for the learning task is (i) compact and (ii) such that the target function f (x) is a simple (e.g. linear) function of the ing representation. Similarly, an ideal dissimilarity measure d(x 1, x 2) for learning a target function f (x) should satisfy certain properties. On one hand, a small dissimilarity d(x 1, x 2) between two objects should imply small difference in the function DISPLAYFORM2 On the other hand, we want a small expected distance among samples, so that the data lies in a compact space of small intrinsic dimension. We next build up some definitions to formalize these properties. Assumption 2 (Lipschitz Continuity). For any DISPLAYFORM3 We would prefer the target function to have a small Lipschitz-continuity constant L with respect to the dissimilarity measure d(., .). Such Lipschitz-continuity alone however might not suffice. For example, one can simply set d(x 1, x 2) = ∞ for any x 1 x 2 to satisfy Eq. equation 1. We thus need the following quantity that measures the size of the space implied by a given dissimilarity measure. DISPLAYFORM4 Assuming the input domain X is compact, the covering number N(δ; X, d) measures its size w.r.t. the distance measure d. We show how the two quantities defined above affect the estimation error of a Nearest-Neighbor Estimator. DISPLAYFORM0 We extend the standard analysis of the estimation error of k-nearest-neighbor from finite-dimensional vector spaces to any structured input space X, with an associated distance measure d, and a finite covering number N(δ; X, d), by defining the effective dimension as follows. Assumption 3 (Effective Dimension). Let the effective dimension p X,d > 0 be the minimum p satisfying DISPLAYFORM1 Here we provide an example of effective dimension in case of measuring the space of Multiset. A multiset is a set that allows duplicate elements. Consider two multisets DISPLAYFORM0 be a ground distance that measures the distance between two elements u i, v j ∈ V in a set. The (modified) Hausdorff Distance BID15 DISPLAYFORM1 Let N(δ; V, ∆) be the covering number of V under the ground distance ∆. Let X denote the set of all sets of size bounded by L. By constructing a covering of X containing any set of size less or equal than L with its elements taken from the covering of V, we have N(δ; DISPLAYFORM2 Equipped with the concept of effective dimension, we can obtain the following bound on the estimation error of the k-Nearest-Neighbor estimate of f (x).Theorem 1. Let V ar(y| f (x)) ≤ σ 2, andf n be the k-Nearest Neighbor estimate of the target function f constructed from a training set of size n. Denote p:= p X,d. We have DISPLAYFORM3 for some constant c > 0. For σ > 0, minimizing RHS w.r.t. the parameter k, we have DISPLAYFORM4 Proof. The proof is almost the same to a standard analysis of k-NN's estimation error in, for example, BID21, with the space partition number replaced by the covering number, and dimension replaced by the effective dimension in Assumption 3.When p X,d is reasonably large, the estimation error of k-NN decreases quite slowly with n. Thus, for the estimation error to be bounded by, requires the number of samples to scale exponentially in p X,d. In the following sections, we develop an estimatorf based on a RKHS derived from the distance measure, with a considerably better sample complexity for problems with higher effective dimension. We aim to address the long-standing problem of how to convert a distance measure into a positivedefinite kernel. Here we introduce a simple but effective approach D2KE that constructs a family of positive-definite kernels from a given distance measure. Given an structured input domain X and a distance measure d(., .), we construct a family of kernels as DISPLAYFORM0 where ω ∈ Ω is a random structured object whose elements could be real-valued time-series, strings, and histograms, p(ω) is a distribution over Ω, and φ ω (x) is a feature map derived from the distance of x to all random objects ω ∈ Ω. The kernel is parameterized by both p(ω) and γ. Relationship to Distance Substitution Kernel. An insightful interpretation of the kernel in Equation can be obtained by expressing the kernel in Equation FORMULA12 as DISPLAYFORM1 where the soft minimum function, parameterized by p(ω) and γ, is defined as DISPLAYFORM2 Therefore, the kernel k(x, y) can be interpreted as a soft version of the distance substitution kernel BID22, where instead of substituting d(x, y) into the exponent, it substitutes a soft version of the form DISPLAYFORM3 Note when γ → ∞, the value of Equation FORMULA15 is determined by min ω ∈Ω d(x, ω) + d(ω, y), which equals d(x, y) if X ⊆ Ω, since it cannot be smaller than d(x, y) by the triangle inequality. In other words, when X ⊆ Ω, DISPLAYFORM4 On the other hand, unlike the distance-substituion kernel, our kernel in Equation FORMULA13 is always PD by construction. 1: Draw R samples from p(ω) to get {ω j} R j=1. 2: Set the R-dimensional feature embedding aŝ DISPLAYFORM0 3: Solve the following problem for some µ > 0: DISPLAYFORM1 Random Feature Approximation. The reader might have noticed that the kernel in Equation FORMULA12 cannot be evaluated analytically in general. However, this does not prohibit its use in practice, so long as we can approximate it via Random Features (RF) BID39, which in our case is particularly natural as the kernel itself is defined via a random feature map. Thus, our kernel with the RF approximation can not only be used in small problems but also in large-scale settings with a large number of samples, where standard kernel methods with O(n 2) complexity are no longer efficient enough and approximation methods, such as Random Features, must be employed BID39. Given the RF approximation, one can then directly learn a target function as a linear function of the RF feature map, by minimizing a domain-specific empirical risk. It is worth noting that a recent work BID45 ) that learns to select a set of random features by solving an optimization problem in an supervised setting is orthogonal to our D2KE approach and could be extended to develop a supervised D2KE method. We outline this overall RF based empirical risk minimization for our class of D2KE kernels in Algorithm 1. It is worth pointing out that in line 2 of Algorithm 1 the random feature embeddings are computed by a structured distance measure between the original structured inputs and the generated random structured inputs, followed by the application of the exponent function parameterized by γ. This is in contrast with traditional RF methods that translate the input data matrix into the embedding matrix via a matrix multiplication with random Gaussian matrix followed by a non-linearity. We will provide a detailed analysis of our estimator in Algorithm 1 in Section 5, and contrast its statistical performance to that of K-nearest-neighbor. Relationship to Representative-Set Method. A naive choice of p(ω) relates our approach to the representative-set method (RSM): setting Ω = X, with p(ω) = p(x). This gives us a kernel Equation that depends on the data distribution. One can then obtain a Random-Feature approximation to the kernel in Equation by holding out a part of the training data {x j} R j=1 as samples from p(ω), and creating an R-dimensional feature embedding of the form: DISPLAYFORM2 as in Algorithm 1. This is equivalent to a 1/ √ R-scaled version of the embedding function in the representative-set method (or similarity-as-features method) BID20 BID36 BID38 BID34 BID7 BID16, where one computes each sample's similarity to a set of representatives as its feature representation. However, here by interpreting Equation as a random-feature approximation to the kernel in Equation, we obtain a much nicer generalization error bound even in the case R → ∞. This is in contrast to the analysis of RSM in BID7, where one has to keep the size of the representative set small (of the order O(n)) in order to have reasonable generalization performance. The choice of p(ω) plays an important role in our kernel. Surprisingly, we found that many "close to uniform" choices of p(ω) in a variety of domains give better performance than for instance the choice of the data distribution p(ω) = p(x) (as in the representative-set method). Here are some examples from our experiments: i) In the time-series domain with dissimilarity computed via Dynamic Time Warping (DTW), a distribution p(ω) corresponding to random time series of length uniform in ∈, and with Gaussian-distributed elements, yields much better performance than the Representative-Set Method (RSM); ii) In string classification, with edit distance, a distribution p(ω) corresponding to random strings with elements uniformly drawn from the alphabet Σ yields much better performance than RSM; iii) When classifying sets of vectors with the Hausdorff distance in Equation, a distribution p(ω) corresponding to random sets of size uniform in ∈ with elements drawn uniformly from a unit sphere yields significantly better performance than RSM.We conjecture two potential reasons for the better performance of the chosen distributions p(ω) in these cases, though a formal theoretical treatment is an interesting subject we defer to future work. Firstly, as p(ω) is synthetic, one can generate unlimited number of random features, which in a much better approximation to the exact kernel in Equation. In contrast, RSM requires held-out samples from the data, which could be quite limited for a small data set. Second, in some cases, even with a small or similar number of random features to RSM, the performance of the selected distribution still leads to significantly better . For those cases we conjecture that the selected p(ω) generates objects that capture semantic information more relevant to the estimation of f (x), when coupled with our feature map under the dissimilarity measure d(x, ω). In this section, we analyze the proposed framework from the perspectives of error decomposition. Let H be the RKHS corresponding to the kernel in Equation. Let DISPLAYFORM0 be the population risk minimizer subject to the RKHS norm constraint f H ≤ C. And let DISPLAYFORM1 be the corresponding empirical risk minimizer. In addition, letf R be the estimated function from our random feature approximation (Algorithm 1). Then denote the population and empirical risks as L(f) andL(f) respectively. We have the following risk decomposition DISPLAYFORM2 In the following, we will discuss the three terms from the rightmost to the leftmost. Function Approximation Error. The RKHS implied by the kernel in Equation FORMULA12 is DISPLAYFORM3 which is a smaller function space than the space of Lipschitz-continuous function w.r.t. the distance d(x 1, x 2). As we show, any function f ∈ H is Lipschitz-continous w.r.t. the distance d(., .). where L f = γC.We refer readers to the detailed proof in Appendix A.1. While any f in the RKHS is Lipschitzcontinuous w.r.t. the given distance d(., .), we are interested in imposing additional smoothness via the RKHS norm constraint f H ≤ C, and by the kernel parameter γ. The hope is that the best function f C within this class approximates the true function f well in terms of the approximation error L(f C) − L(f). The stronger assumption made by the RKHS gives us a qualitatively better estimation error, as discussed below. Estimation Error. Define D λ as DISPLAYFORM0 is the eigenvalues of the kernel in Equation and λ is a tuning parameter. It holds that for any λ ≥ D λ /n, with probability at least 1 − δ, L(f n) − L(f C) ≤ c(log 1 δ) 2 C 2 λ for some universal constant c BID50. Here we would like to set λ as small as possible (as a function of n). By using the following kernel-independent bound: D λ ≤ 1/λ, we have λ = 1/ √ n and thus a bound on the estimation error DISPLAYFORM1 The estimation error is quite standard for a RKHS estimator. It has a much better dependency w.r.t. n (i.e. n −1/2) compared to that of k-nearest-neighbor method (i.e. n −2/(2+p X, d) ) especially for higher effective dimension. A more careful analysis might lead to tighter bound on D λ and also a better rate w.r.t. n. However, the analysis of D λ for our kernel in Equation FORMULA12 is much more difficult than that of typical cases as we do not have an analytic form of the kernel. Random Feature Approximation. DenoteL as the empirical risk function. The error from RF approximation DISPLAYFORM2 where the first and third terms can be bounded via the same estimation error bound in Equation FORMULA3, as bothf R andf n have RKHS norm bounded by C. Therefore, in the following, we focus only on the second term of empirical risk. We start by analyzing the approximation error of the kernel DISPLAYFORM3, we have uniform convergence of the form P max DISPLAYFORM4, where p X,d is the effective dimension of X under metric d(., .). In other words, to guarantee |∆ R (x 1, x 2)| ≤ with probability at least 1 − δ, it suffices to have DISPLAYFORM5 We refer readers to the detailed proof in Appendix A.2. Proposition 2 gives an approximation error in terms of kernel evaluation. To get a bound on the empirical riskL(f R) −L(f n), consider the optimal solution of the empirical risk minimization. By the Representer theorem we havê DISPLAYFORM6. Therefore, we have the following corollary. Corollary 1. To guaranteeL(f R) −L(f n) ≤, with probability 1 − δ, it suffices to have DISPLAYFORM7 where M is the Lipschitz-continuous constant of the loss function (., y), and A is a bound on α 1 /n. We refer readers to the detailed proof in Appendix A.3. For most of loss functions, A and M are typically small constants. Therefore, Corollary 1 states that it suffices to have number of Random Features proportional to the effective dimension O(p X,d / 2) to achieve an approximation error. Combining the three error terms, we can show that the proposed framework can achieve -suboptimal performance. Claim 1. Letf R be the estimated function from our random feature approximation based ERM estimator in Algorithm 1, and let f * denote the desired target function. Suppose further that for some absolute constants c 1, c 2 > 0 (up to some logarithmic factor of 1/ and 1/δ):1. The target function f * lies close to the population risk minimizer f C lying in the RKHS spanned by the D2KE kernel: DISPLAYFORM8 We then have that: L(f R) − L(f *) ≤ with probability 1 − δ. We evaluate the proposed method in four different domains involving time-series, strings, texts, and images. First, we discuss the dissimilarity measures and data characteristics for each set of experiments. Then we introduce comparison among different distance-based methods and report corresponding . Distance Measures. We have chosen three well-known dissimilarity measures: 1) Dynamic Time Warping (DTW), for time-series BID3; 2) Edit Distance (Levenshtein distance), for strings BID32 ); 3) Earth Mover's distance BID40 for measuring the semantic distance between two Bags of Words (using pretrained word vectors), for representing documents. 4) (Modified) Hausdorff distance BID24 BID15 for measuring the semantic closeness of two Bags of Visual Words (using SIFT vectors), for representing images. Note that Bag of (Visual) Words in 3) and 4) can also be regarded as a histogram. Since most distance measures are computationally demanding, having quadratic complexity, we adapted or implemented C-MEX programs for them; other codes were written in Matlab. Datasets. For each domain, we selected 4 datasets for our experiments. For time-series data, all are multivariate time-series and the length of each time-series varies from 2 to 205 observations; three are from the UCI Machine Learning repository BID18, the other is generated from the IQ (In-phase and Quadrature components) samples from a wireless line-of-sight communication system from GMU. For string data, the size of alphabet is between 4 and 8 and the length of each string ranges from 34 to 198; two of them are from the UCI Machine Learning repository and the other two from the LibSVM Data Collection BID5. For text data, all are chosen partially overlapped with these in BID26. The length of each document varies from 9.9 to 117. For image data, all of datasets were derived from Kaggle; we computed a set of SIFTdescriptors to represent each image and the size of SIFT feature vectors of each image varies from 1 to 914. We divided each dataset into 70/30 train and test subsets (if there was no predefined train/test split). Properties of these datasets are summarized in TAB5 in Appendix B. Baselines. We compare D2KE against 5 state-of-the-art baselines, including 1) KNN: a simple yet universal method to apply any distance measure to classification tasks; 2) DSK_RBF BID22: distance substitution kernels, a general framework for kernel construction by substituting a problem specific distance measure in ordinary kernel functions. We use a Gaussian RBF kernel; 3) DSK_ND BID22: another class of distance substitution kernels with negative distance; 4) KSVM BID29: learning directly from the similarity (indefinite) matrix followed in the original Krein Space; 5) RSM BID36: building an embedding by computing distances from randomly selected representative samples. Among these baselines, KNN, DSK_RBF, DSK_ND, and KSVM have quadratic complexity O(N 2 L 2) in both the number of data samples and the length of the sequences, while RSM has computational complexity O(N RL 2), linear in the number of data samples but still quadratic in the length of the sequence. These compare to our method, D2KE, which has complexity O(N RL), linear in both the number of data samples and the length of the sequence. For each method, we search for the best parameters on the training set by performing 10-fold cross validation. For our new method D2KE, since we generate random samples from the distribution, we can use as many as needed to achieve performance close to an exact kernel. We report the best number in the range R = (typically the larger R is, the better the accuracy). We employ a linear SVM implemented using LIBLINEAR for all embedding-based methods (RSM and D2KE) and use LIBSVM BID5 for precomputed dissimilairty kernels (DSK_RBF, DSK_ND, and KSVM). More details of experimental setup are provided in Appendix B. TAB1, 4, and 5, D2KE can consistently outperform or match the baseline methods in terms of classification accuracy while requiring far less computation time. There are several observations worth making here. First, D2KE performs much better than KNN, supporting our claim that D2KE can be a strong alternative to KNN across applications. Second, compared to the two distance substitution kernels DSK_RBF and DSK_ND and the KSVM method operating directly on indefinite similarity matrix, our method can achieve much better performance, suggesting that a representation induced from a truly PD kernel makes significantly better use of the data than indefinite kernels. Among all methods, RSM is closest to our method in terms of practical construction of the feature matrix. However, the random objects (time-series, strings, or sets) sampled by D2KE perform significantly better, as we discussed in section 4. More detailed discussions of the experimental for each domain are given in Appendix C. In this work, we have proposed a general framework for deriving a positive-definite kernel and a feature embedding function from a given dissimilarity measure between input objects. The framework is especially useful for structured input domains such as sequences, time-series, and sets, where many well-established dissimilarity measures have been developed. Our framework subsumes at least two existing approaches as special or limiting cases, and opens up what we believe will be a useful new direction for creating embeddings of structured objects based on distance to random objects. A promising direction for extension is to develop such distance-based embeddings within a deep architecture to support use of structured inputs in an end-to-end learning system. DISPLAYFORM0 Proof. Note the function g(t) = exp(−γt) is Lipschitz-continuous with Lipschitz constant γ. Therefore, DISPLAYFORM1 Proof. Our goal is to bound the magnitude of DISPLAYFORM2 Hoefding's inequality, we have DISPLAYFORM3 a given input pair (x 1, x 2). To get a unim bound that holds ∀(x 1, x 2) ∈ X × X, we find an -covering E of X w.r.t. d(., .) of size N(, X, d). Applying union bound over the -covering E for x 1 and x 2, we have P max DISPLAYFORM4 Then by the definition of E we have |d(DISPLAYFORM5 Together with the fact that exp(−γt) is Lipschitz-continuous with parameter γ for t ≥ 0, we have DISPLAYFORM6 for γ chosen to be ≤ 1. This gives us DISPLAYFORM7 Combining equation 13 and equation 14, we have P max DISPLAYFORM8 Choosing = t/6γ yields the . A.3 P C 1Proof. First of all, we have DISPLAYFORM9 by the optimality of {α j} n j=1 w.r.t. the objective using the approximate kernel. Then we havê DISPLAYFORM10 where A is a bound on α 1 /n. Therefore to guaranteê DISPLAYFORM11 Then applying Theorem 2 leads to the . B G E S General Setup. For each method, we search for the best parameters on the training set by performing 10-fold cross validation. Following BID22, we use an exact RBF kernel for DSK_RBF while choosing squared distance for DSK_ND. We use the Matlab implementation provided by BID29 to run experiments for KSVM. Similarly, we adopted a simple method -random selection -to obtain R = data samples as the representative set for RSM BID36 ). For our new method D2KE, since we generate random samples from the distribution, we can use as many as needed to achieve performance close to an exact kernel. We report the best number in the range R = (typically the larger R is, the better the accuracy). We employ a linear SVM implemented using LIBLINEAR for all embedding-based methods (RSM, and D2KE) and use LIBSVM BID5 for precomputed dissimilairty kernels (DSK_RBF, DSK_ND, and KSVM).All datasets are collected from popular public websites for Machine Learning and Data Science research, including the UCI Machine Learning repository BID18, the LibSVM Data Collection BID5, and the Kaggle Datasets, except one time-series dataset IQ that is shared from researchers from George Mason University. TAB5 lists the detailed properties of the datasets from four different domains. All computations were carried out on a DELL dual-socket system with Intel Xeon processors at 2.93GHz for a total of 16 cores and 250 GB of memory, running the SUSE Linux operating system. To accelerate the computation of all methods, we used multithreading with 12 threads total for various distance computations in all experiments. C D E R T -S, S, IC.1 R - For time-series data, we employed the most successful distance measure -DTW -for all methods. For all datasets, a Gaussian distribution was found to be applicable, parameterized by its bandwidth σ. The best values for σ and for the length of random time series were searched in the ranges [1e-3 1e3] and , respectively. TAB1, D2KE can consistently outperform or match all other baselines in terms of classification accuracy while requiring far less computation time for multivariate time-series. The first interesting observation is that our method performs substantially better than KNN, often by a large margin, i.e., D2KE achieves 26.62% higher performance than KNN on IQ_radio. This is because KNN is sensitive to the data noise common in real-world applications like IQ_radio, and has notoriously poor performance for high-dimensional data sets like Auslan. Moreover, compared to the two distance substitution kernels DSK_RBF and DSK_ND, and KSVM operating directly on indefinite similarity matrix, our method can achieve much better performance, suggesting that a representation induced from a truly p.d. kernel makes significantly better use of the data than indefinite kernels. Among all methods, RSM is closest to our method in terms of practical construction of the feature matrix. However, the random time series sampled by D2KE performs significantly better, as we discussed in section 4. First, RSM simply chooses a subset of the original data points and computes the distances between the whole dataset and this representative set; this may suffer significantly from noise or redundant information in the time-series. In contrast, our method samples a short random sequence that could both denoise and find the patterns in the data. Second, the number of data points that can be sampled is limited by the total size of the data while the number of possible random sequences drawn from the distribution is unlimited, making the feature space much more abundant. Third, RSM may incur significant computational cost for long time-series, due to its quadratic complexity in length. Setup. For string data, there are various well-known edit distances. Here, we choose Levenshtein distance as our distance measure since it can capture global alignments of the underlying strings. We first compute the alphabet from the original data and then uniformly sample characters from this alphabet to generate random strings. We search for the best parameters for γ in the range [1e-5 1], and for the length of random strings in the range , respectively. Results. As shown in TAB2, D2KE consistently performs better than or similarly to other distancebased baselines. Unlike the previous experiments where DTW is not a distance metric, Levenshtein distance is indeed a distance metric; this helps improve the performance of our baselines. However, D2KE still offers a clear advantage over baseline. It is interesting to note that the performance of DSK_RBF is quite close to our method's, which may be due to DKS_RBF with Levenshtein distance producing a c.p.d. kernel which can essentially be converted into a p.d. kernel. Notice that on relatively large datasets, our method, D2KE, can achieve better performance, and often with far less computation than other baselines with quadratic complexity in both number and length of data samples. For instance, on mnist-str8 D2KE obtains higher accuracy with an order of magnitude less runtime compared to DSK_RBF and DSK_ND, and two orders of magnitude less than KSVM, due to higher computational costs both for kernel matrix construction and for eigendecomposition. Setup. For text data, following BID26 we use the earth mover's distance as our distance measure between two documents, since this distance has recently demonstrated a strong performance when combining with KNN for document classifications. We first compute the Bag of Words for each document and represent each document as a histogram of word vectors, where google pretrained word vectors with dimension size 300 is used. We generate random documents consisting of each random word vectors uniformly sampled from the unit sphere of the embedding vector space R 300. We search for the best parameters for γ in the range [1e-2 1e1], and for length of random document in the range .Results. As shown in TAB3, D2KE outperforms other baselines on all four datasets. First of all, all distance based kernel methods perform better than KNN, illustrating the effectiveness of SVM over KNN on text data. Interestingly, D2KE also performs significantly better than other baselines by a notiably margin, in large part because document classification mainly associates with "topic" learning where our random documents of short length may fit this task particularly well. For the datasets with large number of documents and longer length of document, D2KE achieves about one order of magnitude speedup compared with other exact kernel/similarity methods, thanks to the use of random features in D2KE. Setup. For image data, following BID36 BID22 we use the modified Hausdorff distance (MHD) BID15 as our distance measure between images, since this distance has shown excellent performance in the literature BID44 BID19. We first applied the open-source OpenCV library to generate a sequence of SIFT-descriptors with dimension 128, then MHD to compute the distance between sets of SIFTdescriptors. We generate random images of each SIFT-descriptor uniformly sampled from the unit sphere of the embedding vector space R 128. We search for the best parameters for γ in the range [1e-3 1e1], and for length of random SIFT-descriptor sequence in the range .Results. As shown in TAB4, D2KE performance outperforms or matches other baselines in all cases. First, D2KE performs best in three cases while DSK_RBF is the best on dataset decor. This may be because the underlying SIFT features are not good enough and thus random features is not effective to find the good patterns quickly in images. Nevertheless, the quadratic complexity of DSK_RBF, DSK_ND, and KSVM in terms of both the number of images and the length of SIFT descriptor sequences makes it hard to scale to large data. Interestingly, D2KE still performs much better than KNN and RSM, which again supports our claim that D2KE can be a strong alternative to KNN and RSM across applications.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

From Distance to Kernel and Embedding via Random Features For Structured Inputs

Represent the short summary for finding the corresponding scientific article (long)

We propose an efficient, provable and data independent method for network compression via neural pruning using coresets of neurons -- a novel construction proposed in this paper.


Question: Represent the scientific article to find a short TLDR:

Question: Deep Convolution Neural Networks (CNNs), rooted by the pioneer work of \cite{Hinton1986,LeCun1985,Alex2012}, and summarized in \cite{LeCunBengioHinton2015}, have been shown to be very useful in a variety of fields. The state-of-the art CNN machines such as image rest net \cite{He_2016_CVPR} are described by real value inputs and kernel convolutions followed by the local and non-linear rectified linear outputs. Understanding the role of these layers, the accuracy and limitations of them, as well as making them more efficient (fewer parameters) are all ongoing research questions. Inspired in quantum theory, we propose the use of complex value kernel functions, followed by the local non-linear absolute (modulus) operator square. We argue that an advantage of quantum inspired complex kernels is robustness to realistic unpredictable scenarios (such as clutter noise, data deformations). We study a concrete problem of shape detection and show that when multiple overlapping shapes are deformed and/or clutter noise is added, a convolution layer with quantum inspired complex kernels outperforms the statistical/classical kernel counterpart and a "Bayesian shape estimator". The superior performance is due to the quantum phenomena of interference, not present in classical CNNs. The convolution process in machine learning maybe summarized as follows. Given an input f L−1 (x) ≥ 0 to a convolution layer L, it produces an output DISPLAYFORM0 From g L (y) a local and non-linear function is applied, f L (y) = f (g L (y)), e.g., f = ReLu (rectified linear units) or f = |.|, the magnitude operator. This output is then the input to the next convolution layer (L+1) or simply the output of the whole process. We can also write a discrete form of these convolutions, as it is implemented in computers. We write g DISPLAYFORM1, where the continuous variables y, x becomes the integers i, j respectively, the kernel function K(y − x) → w ij becomes the weights of the CNN and the integral over dx becomes the sum over j. These kernels are learned from data so that an error (or optimization criteria) is minimized. The kernels used today a real value functions. We show how our understanding of the optimization criteria "dictate" the construction of the quantum inspired complex value kernel. In order to concentrate and study our proposal of quantum inspired kernels, we simplify the problem as much as possible hoping to identify the crux of the limitation of current use of real value kernels. We place known shapes in an image, at any location, and in the presence of deformation and clutter noise. These shapes may have been learned by a CNN. Our main focus is on the feedforward performance, when new inputs are presented. Due to this focus, we are able to construct a Bayesian a posteriori probability model to the problem, which is based on real value prior and likelihood models, and compare it to the quantum inspired kernel method. The main advantage of the quantum inspired method over existing methods is its high resistance to deviations from the model, such as data deformation, multiple objects (shapes) overlapping, clutter noise. The main new factor is the quantum interference phenomenon BID1 BID0, and we argue it is a desired phenomena for building convolution networks. It can be carried out by developing complex value kernels driven by classic data driven optimization criteria. Here we demonstrate its strength on a shape detection problem where we can compare it to state of the art classical convolution techniques. We also can compare to the MAP estimator of the Bayesian model for the shape detection problem. To be clear, we do not provide (yet) a recipe on how to build kernels for the full CNN framework for machine learning, and so the title of this paper reflects that. Here, we plant a seed on the topic of building complex value kernels inspired in quantum theory, by demonstrating that for a given one layer problem of shape detection (where the classic data optimization criteria is well defined), we can build such complex value kernel and demonstrate the relevance of the interference phenomena. To our knowledge such a demonstration is a new contribution to the field. We also speculate on how this process can be generalized. We are given an image I with some known objects to be detected. The data is a set of N feature points, DISPLAYFORM0 Here we focus on 2-dimensional data, so D = 2. An image I may be described by the set of feature points, as shown for example in figure 1. Or, the feature points can be extracted from I, using for example, SIFT features, HOG features, or maximum of wavelet responses (which are convolutions with complex value kernels). It maybe be the first two or so layers of a CNN trained in image recognition tasks. The problem of object detection has been well addressed for example by the SSD machine BID7, using convolution networks. Here as we will demonstrate, given the points, we can construct a ONE layer CNN that solves the problem of shape detection and so we focus on this formulation. It allows us to study in depth its performance (including an analytical study not just empirical). 8, 8), radius 3, and with 100 points, is deformed as follows: for each point x i a random value drawn from a uniform distribution with range (−η i, η i), η i = 0.05, is added along the radius. (b) 1000 points of clutter are added by sampling from a uniform distribution inside a box of size 9 × 9, with corners at points:,,,. We organize the paper as follows. Section 2 presents a general description of shapes, which is easily adapted to any optimization method. Section 3 presents the Bayesian method and the Hough transform method (and a convolution implementation) to the shape detection problem. Section 4 lays out our main proposal of using quantum theory to address the shape detection problem. The theory also leads naturally to a classical statistical method behaving like a voting scheme, and we establish a connection to Hough transforms. Section 5 presents a theoretical and empirical analysis of the quantum method for shape detection and a comparison with the classical statistical method. We demonstrate that for large deformations or clutter noise scenarios the quantum method outperforms the classical statistical method. Section 6 concludes the paper. A shape S may be defined by the set of points x satisfying S Θ (x) = 0, where Θ is a set of parameters describing S. Let µ be a shape's center (in our setting µ = (µ x, µ y)). The choice of µ is in general arbitrary, though frequently there is a natural choice for µ for a given shape, such as its "center of mass": the average position of its coordinates. We consider all the translations of S Θ (x) to represent the same shape, so that a shape is translation invariant. It is then convenient to describe the shapes as S Θ (x − µ), with the parameters Θ not including the parameters of µ. Thus we describe a shape by the set of points X such that DISPLAYFORM0 The more complex a shape is, the larger is the set of parameters required to describe it. For example, to describe a circle, we use three parameters {µ x, µ y, r} representing the center and the radius of the circle, i.e., DISPLAYFORM1 (see figure 1 a.) An ellipse can be described by DISPLAYFORM2, where Θ = {Σ} is the covariance matrix, specified by three independent parameters. We also require that a shape representation be such that if all the values of the parameters in Θ are 0, then the the set of points that belong to the shape "collapses" to just X = {µ}. This is the case for the parameterizations of the two examples above: the circle and the ellipse. Energy Model: Given a shape model we can create an energy model per data point x as DISPLAYFORM3 where the parameter p ≥ 0 defines the L p norm (after the sum over the points is taken and the 1/p root is applied). The smaller E S Θ (x − µ), the more it is likely that the data point x belongs to the shape S Θ with center µ. In this paper, we set p = 1 because of its simplicity and robust properties. To address realistic scenarios we must study the detection of shapes under deformations. When deformations are present, the energy is no longer zero for deformed points associated to the shape S Θ (x − µ). Let each ideal shape data point x S i be deformed by adding η i to its coordinates, so DISPLAYFORM0 Deformations of a shape are only observed in the directions perpendicular to the shape tangents, i.e., along the direction of ∇ x S Θ (x − µ) xi, where ∇ x is the gradient operator. For example, for a (deformed) circle shape, Θ = {r} and S r (x−µ) = 1− DISPLAYFORM1, and so DISPLAYFORM2 r 2 ∝r i, wherer i is a unit vector pointing outwards in the radius direction at point DISPLAYFORM3 Given a set of data points X = {x 1, x 2, ..., x N} in R D originated from a shape S Θ (x i − µ) = 0. We assume that each data point is independently deformed by η i (a random variable, since the direction is along the shape gradient), conditional on being a shape point. Based on the energy model, for p = 1 (for simplicity and robust properties), we can write the likelihood model as DISPLAYFORM0 where C is a normalization constant and λ a constant that scale the errors/energies. The product over all points is a consequence of the conditional independence of the deformations given the shape parameter (Θ, µ). Assuming a prior distributions on the parameters to be uniform, we conclude that the a posteriori distribution is simply the likelihood model up to a normalization, i.e., DISPLAYFORM1 where Z is a normalization constant (does not depend on the parameters). The parameters that maximize the likelihood L(Θ, µ) = log P(Θ, DISPLAYFORM2 A Hough transform cast binary votes from each data point. The votes are for the shape parameter values that are consistent with the data point. More precisely, each vote is given by DISPLAYFORM0 where u(x) is the Heaviside step function, u(x) = 1 if x ≥ 0 and zero otherwise, i.e., u = 1 if DISPLAYFORM1 α and u = 0 otherwise. The parameter α clearly defines the error tolerance for a data point x i to belong to the shape S Θ (x − µ), the larger is α the smaller is the tolerance. One can carry out this Hough transform for center detection as a convolution process. More precisely, create a kernel, DISPLAYFORM2 | for x in a rectangular (or square) shape that includes all x for which u 1 α − |S Θ (x)| = 1. The Hough transform for center detection is then the convolution of the kernel with the input image. The of the convolution at each location is the Hough vote for that location to be the center. (b) The Bayesian method (showing the probability value). The radius is fed to the method. The method mixes all data yielding the highest probability in the wrong place. increasing the parameter p can only improve a little as all data participate in the final estimation. (c) The Hough method with α = 2.769 estimated to include all circle points that have been deformed. The method is resistant to clutter noise. When we have one circle with deformations (e.g., see FIG0), the Bayesian approach is just the "perfect" model. Even if noise distributed uniformly across the image is added (e.g., see figure 1b), the Bayesian method will work very well. However, as one adds clutter noise to the data (noise that is not uniform and "may correspond to clutter" in an image) as shown in figure 2, the Bayesian method mix all the data, has no mechanism to discard any data, and the Hough method outperforms the Bayesian one. Even applying robust measures, decreasing p in the energy model, will have limited effect compared to the Hough method that can discard completely the data. Consider another scenario of two overlapping and deformed circles, shown in FIG2. Again, the Bayesian approach does not capture the complexity of the data, two circles and not just one, and end up yielding the best one circle fit in the "middle", while the Hough method cope with this data by widening the center detection probabilities (bluring the center probabilities) and thus, including both true centers. Still, the Hough method is not able to suggest that there are two circles/two peaks. In summary, the Bayesian model is always the best one, as long as the data follows the exact model generation. However, it is weak at dealing with *real world uncertainty* on the data (clutter data, multiple figures), other scenarios that occur often. The Hough method, modeled after the same true positive event (shape detection) is more robust to these data variations and for the center detection problem can be carried out as a convolution. The radius is fed to the method. The method mixes all data yielding the highest probability approximately in the "middle" of both centers and no suggestion of two peaks/circles/centers exists. (c) The Hough method with α = 2.769 estimated to include all circle points that have been deformed. The method yields a probability that is more diluted and includes the correct centers, but does not suggest two peaks. Quantum theory was developed for system of particles that evolve over time. For us to utilize here the benefits of such a theory for the shape detection problem we invoke a hidden time parameter. We refer to this time parameter as hidden since the input is only one static picture of a shape. A hidden shape dynamics is not a new idea in computer vision, for example, scale space was proposed to describe shape evolution and allows for better shapes comparisons BID11 BID6. Hidden shape dynamics was also employed to describe a time evolution equation to produce shape-skeletons BID9. Since our optimization criteria per point is given by "the energy" of, we refer to classic concept of action, the one that is optimized to produce the optimal path, as DISPLAYFORM0 | where we are adopting for simplicity p = 1. The idea is that a shapes evolve from the center µ = x(t = 0) to the shape point x = x(t = T) in a time interval T. During this evolutions all other parameters also evolve from Θ(t = 0) = 0 to Θ(t = T) = Θ. The evolution is reversible, so we may say equivalently, the shape point x contracts to the center µ in the interval of time T.Following the path integral point of view of quantum theory BID1, we consider the wave propagation to evolve by the integral over all path DISPLAYFORM1 where ψ Θ(t) (x(t)) is the probability amplitude that characterize the state of the shape, P T 0 is a path of shape contraction, from an initial state (x, Θ) = (x, Θ) to a final state (x(T), Θ(T)) = (µ, 0). The integral is over all possible paths that initialize in (x, Θ) = (x, Θ) and end in (x(T), Θ(T)) = (µ, 0). The Kernel K is of the form DISPLAYFORM2 where a new parameter,, is introduced. It has the notation used in quantum mechanics for the reduced Planck's constant, but here it will have its own interpretation and value (see section 5.1.3).We now address the given image described by X = {x 1, x 2, ..., x N} ⊂ R 2 (e.g., see FIG0). We consider an empirical estimation of ψ Θ (x) to be given by a set of impulses at the empirical data set DISPLAYFORM3, where δ(x) is the Dirac delta function. The normalization ensure the probability 1 when integrated everywhere. Note that ψ Θ (x) is a pure state, a superposition of impulses. Then, substituting this state into equation FORMULA18, with the kernel provided by, yields the evolution of the probability amplitude DISPLAYFORM4 where C = e i T |S Θ (x−µ)| dµ. Thus shape points with deformations, x i, are interpreted as evidence of different quantum paths, not just the optimal classical path (which has no deformation). Equation 5 is a convolution of the kernel K(x) = e i T |S Θ (x)| throughout the center candidates, except it is discretized at the locations where data is available. According to quantum theory, the probability associated with this probability amplitude (a pure state) is given by P(Θ) = |ψ Θ (µ)| 2, i.e., DISPLAYFORM5 which can also be expanded as DISPLAYFORM6 It is convenient to define the phase DISPLAYFORM7 Note the interference phenomenon arising from the cosine terms in the probability. More precisely, a pair of data points that belongs to the shape will have a small magnitude difference, |φ ij | 1, and will produce a large cosine term, cos φ ij ≈ 1. Two different data points that belong to the clutter will likely produce different phases, scaled inversely according to, so that small values of will create larger phase difference. Pairs of clutter data points, not belonging to the shape, with large and varying phase differences, will produce quite different cosine terms, positive and/or negative ones. If an image contains a large amount of clutter, the clutter points will end up canceling each other. If an image contains little clutter, the clutter points will not contribute much. This effect can be described by the following property for large numbers: if N 1 then DISPLAYFORM0 N, when each k is a random variable. Figure 4 shows the performance of the quantum method on the same data as shown in FIG1 and FIG2. The accuracy of the detection of the centers and the identification of two centers shows how the quantum inspired method outperforms the classical counterparts. In figure 4a, due to interference, clutter noise cancels out (negative terms on the probability equation 6 balance positive ones), and the center is peaked. We do see effects of the noise inducing some fluctuation. In figure 4b the two circle center peaks outperform both classical methods as depicted in FIG2. A more thorough analysis is carried out in the next section to better understand and compare the performance of these different methods. Note that even though the probability reflects a pair-wise computation as seen in FORMULA23, we evaluate it by taking the magnitude square of the probability amplitude (given by equation FORMULA21), which is computed as a sum of N complex numbers. Thus, the complexity of the computations is linear in the data set size. After all, it is a convolution process.(a) Quantum Probability on figure 2a (b) Quantum Probability on figure 3aFigure 4: Quantum Probability depicted for input shown in FIG1, respectively. The parameters used where T = 1, = 0.12. The quantum method outperform the classical methods, as the center detection shown in (a) is more peaked than in the Hough method and in (b) the two peaks emerge. These are of the interference phenomena, as cancellation of probabilities (negative terms on the probability equation 6) contribute to better resolve the center detection problem. we derive a classical probability from the quantum probability amplitude via the Wick rotation. It is a mathematical technique frequently employed in physics, which transforms quantum physical systems into statistical physical systems and vice-versa. It consists in replacing the term i T by a real parameter α in the probability amplitude. Considering the probability amplitude equation FORMULA21, the Wick rotation yields DISPLAYFORM0 We can interpret this as follows. Each data point x i produces a vote v(Θ, µ|x i) = e − α |S Θ (xi−µ)|, with values between 0 and 1. The parameter α controls the weight decay of the vote. Interestingly, this probability model resembles a Hough transform with each vote FORMULA26 being approximated by the binary vote described by We analyze the quantum method described by the probability, derived from the amplitude, and compare it with the classical statistical method described by and its approximation. This analysis and experiments is carried for a simple example, the circle. We consider the circle shape, S r * (x − µ) = 1 − (x−µ) 2 (r *) 2 of radius r * and its evaluation not only at the true center µ * but also at small displacements from it µ = µ * + δµ where δµ r * < 1 with δµ = |δµ|. The points of an original circle are deformed to create the final "deformed" circle shape. Each point is moved by a random vector η i pointing along the radius, i.e., η i = η ir * i withr * i being the unit vector along the radius. Thus, we may write each point as DISPLAYFORM0 The deformation is assumed to vary independently and uniformly point wise. Thus, η i ∈ (−η, η) and P(η i) = 1 2η. Plugging in the deformations and center displacement into the shape representation, S i = S r * (x i − µ), we get DISPLAYFORM1 DISPLAYFORM2 For the special case of the evaluation of the shape at the true center, δµ = 0 we obtain DISPLAYFORM3 The action for each path is given by |S Θ (x i − µ)| and we have multiple paths samples from the data. Note that when we apply the quantum method, we interpret data derived from shape deformation as evidence of quantum trajectories (paths) that are not optimal while the classical interpretation for such data is a statistical error/deformation. Both are different probabilistic interpretations that lead to different methods of evaluations of the optimal parameters as we analyze next. We interpret the probability amplitude in equation, the sum over i = 1,..., N, as a sum over many independent samples. In general given a function f (|S|), then the sum over all points, DISPLAYFORM0, can be interpreted as N C times the statistical average of the function f over the random variable S i. In this case, the random variable S i (η i, δµ i) represent two independent and uniform random variables, (η i, δµ i), or (a i, b i).Inserting shape equation FORMULA28 into the quantum probability amplitude of equation FORMULA21 DISPLAYFORM1 where I ab (e) = 1 4ab DISPLAYFORM2 2 −2aibi)| and at the true center we get DISPLAYFORM3 The ratio of the probabilities (magnitude square of the probability amplitudes) for the circle is then given by DISPLAYFORM4 These integrals can be evaluated numerically (or via integration of a Taylor series expansions, and then a numerical evaluation of the expansion). Inserting shape equation FORMULA28 into the vote for the Hough transform giving by equation FORMULA14 in DISPLAYFORM0 and interpreting the Hough total vote, V DISPLAYFORM1 as an average over a function of the random variable |S i | multiplied by the number of votes, we get DISPLAYFORM2 where DISPLAYFORM3 and at the true cen- DISPLAYFORM4 The ratio of the votes at the true center and the displaced center is then given by DISPLAYFORM5 We now address the choice of hyper-parameters of the models, namely for the quantum model and α for the classical counterpart so that the detection of the true center is as accurate as possible. In this section, without loss of generality, we set T = 1. In this way we can concentrate on understanding role (and not T). The amplitude probability given by equation FORMULA21 has a parameter where the inverse of scales up the magnitude of the shape values. The smaller is, the more the phase ϕ i = 1 |S Θ (x i − µ)| reaches any point in the unit circle. A large can make the phase ϕ i very small and a small can send each shape point to any point in the unit circle.The parameter large can help in aligning shape points to similar phases. That suggests as large as possible. At the same time, should help in misaligning pair of points where at least one of them does not belong to the shape. That suggests small values of. Similarly, if one is evaluating a shape with the "wrong" set of parameters, we woulud want the shape points to cancel each other. In our example of the circle, we would like that shape points evaluated at a center displacement from the true center to yield some cancellation. That suggests small. One can explore the parameter that maximize the ratio Q C (a, b,) given by equation. We can also attempt to analytically balance both requests (high amplitude at the true center and low amplitude at the displaced center) by choosing values of such that ϕ i = 1 |S r * (x i − µ *)| ≤ π ∀i = 1,..., N C. More precisely, by choosing DISPLAYFORM0 Figure 5 suggests this choice of gives high ratios for Q(a, b,).Now we discuss the estimation of α. we note that for DISPLAYFORM1 2 all shape points will vote for the true center. Thus, choosing α so that largest value of its inverse is DISPLAYFORM2 will guarantee all votes and give a lower vote for shape points evaluated from the displaced center. One could search for higher values of α, smaller inverse values, so that reducing votes at the true center and expecting to reduce them further at the displaced center, i.e., to maximize H(a, b, α) in equation. FIG4 suggests such changes do not improve the Hough transform performance. FIG4 demonstrates that the quantum method outperforms the classical Hough transform on accuracy detection. We can also perform a similar analysis adding noise, to obtain similar . This will require another two pages (a page of analysis and a page of graphs), and if the conference permits, we will be happy to add. Deep Convolution Neural Networks (CNNs), rooted on the pioneer work of BID8; BID4; BID3, and summarized in BID5, have been shown to be very useful in a variety of fields. Inspired in quantum theory, we investigated the use of complex value kernel functions, followed by the local non-linear absolute (modulus) operator square. We studied a concrete problem of. For each of the figures 5a, 5b,5c we vary we vary b = 1 2 a, a, 2a (or center displacements δµ = 0.25, 0.5, 1), respectively. These figures depict ratios Q(a, b,) × (blue) for ∈ (0.047, 0.2802) and H(a, b, α) × ← − α (red) for ← − α ∈ (22.727, 2.769) (The reverse arrow implies the x-axis start at the maximum value and decreases thereafter). All plots have 200 points, with uniform steps in their respective range. Note that our proposed parameter value is = 0.1401, the solution to equation FORMULA42, and indeed gives a high ratio. Also, α = 2.769 is the smallest value to yield all Hough votes in the center. Clearly the quantum ratio outperforms the best classical Hough method, which does not vary much across α values. As the center displacement increases, the quantum method probability, for = 0.1401, decreases much faster than the Hough method probability. Final figure 5d display values of |ψ| 2 (µ *) × (at the true center) in blue, for ∈ (0.047, 0.2802), with 200 uniform steps. In red, V (µ *) × ← − α for ← − α ∈ (22.727, 2.769), with 200 uniform steps. DISPLAYFORM0 shape detection and showed that when multiple overlapping shapes are deformed and/or clutter noise is added, a convolution layer with quantum inspired complex kernels outperforms the statistical/classical kernel counterpart and a "Bayesian shape estimator". It is worth to mention that the Bayesian shape estimator is the best method as long as the data satisfy the model assumptions. Once we add multiple shapes, or add clutter noise (not uniform noise), the Bayesian method breaks down rather easily, but not the quantum method nor the statistical version of it (the Hough method being an approximation to it). An analysis comparing the Quantum method to the Hough method was carried out to demonstrate the superior accuracy performance of the quantum method, due to the quantum phenomena of interference, not present in the classical CNN.We have not focused on the problem of learning the shapes here. Given the proposed quantum kernel method, the standard techniques of gradient descent method should also work to learn the kernels, since complex value kernels are also continuous and differentiable. Each layer of the networks carries twice as many parameters, since complex numbers are a compact notation for two numbers, but the trust of the work is to suggest that they may perform better and reduce the size of the entire network. These are just speculations and more investigation of the details that entice such a construction are needed. Note that many articles in the past have mentioned "quantum" and "neural networks" together. Several of them use Schrödinger equation, a quantum physics modeling of the world. Here in no point we visited a concept in physics (forces, energies), as Schrödinger equation would imply, the only model is the one of shapes (computer vision model). Quantum theory is here used as an alternative statistical method, a purely mathematical construction that can be applied to different models and fields, as long as it brings benefits. Also, in our search, we did not find an article that explores the phenomena of interference and demonstrate its advantage in neural networks. The task of brining quantum ideas to this field must require demonstrations of its utility, and we think we did that here.

A quantum inspired kernel for convolution network, exhibiting interference phenomena,  can be very useful (and compared it with real value  counterpart).

Represent this text to find a much much longer article about it with say hundreds-thousands of words

In this paper, a new method we call Centered Initial Attack (CIA) is provided. It insures by construction the maximum perturbation to be smaller than a threshold fixed beforehand, without the clipping process.


Question: represent the scientific article to find a short tldr

Question: We present an artificial intelligence research platform inspired by the human game genre of MMORPGs (Massively Multiplayer Online Role-Playing Games, a.k.a. MMOs). We demonstrate how this platform can be used to study behavior and learning in large populations of neural agents. Unlike currently popular game environments, our platform supports persistent environments, with variable number of agents, and open-ended task descriptions. The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources. Our platform aims to simulate this setting in microcosm: we conduct a series of experiments to test how large-scale multiagent competition can incentivize the development of skillful behavior. We find that population size magnifies the complexity of the behaviors that emerge and in agents that out-compete agents trained in smaller populations. Life on Earth can be viewed as a massive multiagent competition. The cheetah evolves an aerodynamic profile in order to catch the gazelle, the gazelle develops springy legs to run even faster: species have evolved ever new capabilities in order to outcompete their adversaries. The success of biological evolution has inspired many attempts to emulate it in silico, ranging from genetic algorithms that bear only loose resemblance to natural processes, to full-blown simulations of "artificial life". A recurring question has been: at what level of abstraction should we simulate the competitive game of life?In recent years, the field of deep reinforcement learning (RL) has embraced a related approach: train algorithms by having them compete in simulated games BID16 BID14 BID8. Such games are immediately interpretable and provide easy metrics derived from the game's "score" and win conditions. However, popular game benchmarks are currently still limited: they typically define a narrow, episodic task, with a small fixed number of players. In contrast, life on Earth involves a persistent environment, an unbounded number of players, and a seeming "open-endedness", where ever new and more complex species emerge over time, with no end in sight BID18.Our aim is to develop a simulation platform (see FIG3) that captures important properties of life on Earth, while also borrowing from the interpretability and abstractions of human-designed games. To this end, we turn to the game genre of Massively Multiplayer Online Role-Playing Games (MMORPGs, or MMOs for short). These games involve a large, variable number of players competing to survive and prosper in persistent and far-flung environments. Our platform simulates a "Neural MMO" -an MMO in which each agent is a neural net that learns to survive using RL.We demonstrate the capabilities of this platform through a series of experiments that investigate emergent complexity as a function of the number of agents and species that compete in the simulation. We find that large populations act as competitive pressure that encourages exploration of the environment and the development of skillful behavior. In addition, we find that when agents are organized into species (share policy parameters), each species naturally diverges from the others to occupy its own behavioral niche. Upon publication, we will opensource the platform in full. We alternate between collecting experience across 100 procedurally generated worlds and updating agents' parameters via policy gradients. Test time visualization provides insight into the learned policies through value function estimates, map tile visitation distribution, and agent-agent dependencies. Multiagent Reinforcement Learning Multiagent reinforcement learning has received increased attention in recent years BID11 BID12 BID1 BID9 BID22 BID24 BID14 BID8. Unlike single agent environments often well modeled by classic Markov Decision Processes (MDPs), multiagent interaction often introduces nonstationarity in the transition dynamics of the environment due to the continual learning and co-adaptation of other agents. While previous work has attempted to analyze emergent complexity in interactions of groups of 2-10 agents BID1 BID14 BID8, we focus on large populations of agents in a complex world. BID24 also analyze behavior of many agents, but the task setting is considerably simpler, and agents are directly rewarded for foraging or fighting. In contrast, our work focuses on complexity in a diverse environment where agents are only rewarded for survival; doing so requires maintaining their health through food and water while navigating partially obstructed terrain and with various forms of combat. BID23 also simulate populations scaling to millions of learning agents, but they focus on predator-prey population dynamics in this setting. Artificial Life "Artificial life" projects aim to simulate the complexity of biological life , often framed as a multiagent competition. This setting can lead to the development of skilled behaviors BID21 and capable morphologies BID17. We consider the setting of coevolution BID5 in which agents coadapt alongside others. Our platform can simulate a kind of artificial life, but at a relatively high level of abstraction. While some basic features of our environment (movement, food, water) are similar to those in BID6 BID19, our environment is grounded in the established game genre of MMOs. Unlike most past work in this area, our platform is built around deep reinforcement learning, where each agent is a neural net trained with distributed policy gradients, and includes tools for visualizing properties specific to this setting. Foraging agents learn to efficiently balance their food and water levels while competing with other agents for resources. Center: Combat agents learn the latter while also balancing melee, range, and mage attack styles to engage with and outmaneuver other agents. Right: Graphics key for tiles and agents. Game Platforms for Intelligent Agents The Arcade Learning Environment (ALE) BID2 and Gym Retro BID13 provide 1000+ limited scope arcade games most often used to test individual research ideas or generality across many games. Strictly better performance at a large random subset of games is a reasonable metric of quality, and strikingly divergent performance can motivate further research into a particular mode of learning. For example, Montezumas Revenge in ALE is difficult even with reasonable exploration because of the associative memory required to collect and use keys. However, recent have brought into question the overall complexity each individual environment BID4 and strong performance in such tasks (at least in those not requiring precise reflexes) is not particularly difficult for humans. More recent work has demonstrated success on multiplayer games including Go BID16, the Multiplayer Online Battle Arena (MOBA) DOTA2 (OpenAI), and Quake 3 Capture the Flag BID8. Each of these projects has advanced our understanding of a class of algorithms. However, these games were limited to 2-12 players, are round based on the order of an hour, lack persistence, and lack the game mechanics supporting large persistent populations -there is still a large gap in environment complexity compared to the real world. MMORPGs The game genre we focus on in this paper is MMORPGs, which are role-playing games (RPGs) in which many human players take part. RPGs, such as Pokemon and Final Fantasy, involve reasoning over up to hundreds of hours of persistent gameplay -a much longer time horizon than in MOBAs. Like the real world, RPGs confront the player with problems that have many valid solutions, and choices that have long term consequences. MMOs are the (massively) multiplayer analogs to RPGs. They are typically run across several servers, each of which contains a copy of the environment and supports hundreds to millions of concurrent players. Good MMOs involve a curriculum of challenges that require increasingly clever usage of the game systems. Early game content is accessible to new players, but skills required for late game content are inaccessible (and often incomprehensible) to those not intimately familiar with the game. Players have to acquire resources and "level up" in order to reach more advanced stages of the game. Such a curriculum is present in many game genres, but only MMOs contextualize it within social and economic structures approaching the scale of the real world. We present a persistent and massively multiagent environment that defines foraging and combat systems over procedurally generated maps (see Appendix). The purpose of the platform is to discover game mechanics that support complex behavior and agent populations that can learn to make use of them. We follow the iterative development cycle of human MMOs: developers create balanced mechanics while players maximize their skill in utilizing them. The initial configurations of our systems are the of several iterations of balancing, but are by no means fixed: every numeric parameter presented is editable within a simple configuration file. A map consists of a set of discrete tiles, which are positions an agent can occupy. On each step of the simulations, agents may move one tile North/South/East/West. Agents competes for food tiles while periodically refilling their water supply from infinite water tiles. They may attack each other using any of three attack options, each with different damage values and trade offs. The environment assumes only that agents receive local game state and output a decision. The environment is agnostic to the source of that decision, be it a neural network or a hardcoded algorithm. We have tested up to 100M agent trajectories (lifetimes) on 100 cores in 1 week. The code base already contains additional module code for trade, gathering, crafting, item systems, equipment, communication, and trade to name a few. We are actively balancing and integrating these into the neural API. Input Agents observe local game state-all tiles within a fixed L 1 distance of their current position, including tile terrain types and the visible properties of occupying agents. This is an efficient equivalent representation of what a human sees on the screen without requiring rendering. Output Agents output action choices for the next time step ("game tick"). For the experiments below, actions consist of one movement and one attack. As described in Framework, movement options are: North, South, East, West, Pass (no movement). Attack options are: Melee, Range, Mage. This is purely flavor; for those not familiar with MMOs, each attack option simply applies a preset amount of damage at a preset effective distance. The environment will attempt to execute both actions. Invalid actions, such as moving into a stone wall, are ignored. Our policy architecture is detailed in the Appendix. We provide a simple preprocessor that embeds and flattens this stimulus into a single fixed length environment vector and a list of entity embeddings. We apply a linear layer to the preprocessed embeddings followed by three output heads for movement, attacks. There is also a standard value head which is trained to predict the discounted expected lifetime of the agent. Each head is also a linear layer. New types of action choices can be included by adding additional heads. We train with simple policy gradients plus a value baseline. Agents receive only a stream of reward 1. Rewards are postprocessed by a discounting factor, producing returns equal to a discounted estimate of the agent's time until death. We found it possible to obtain good performance without discounting, but training was less stable. We present an initial series of experiments using our platform to explore multiagent interactions in large populations. We find that agent competence scales with population size. In particular, increasing the maximum number of concurrent players (N ent) magnifies exploration and increasing the maximum number of populations with unshared weights (N pop) magnifies niche formation. Agents are sampled uniformly from a number of "populations"-identical architectures with unshared weights. This is for efficiency-see technical details. Technical details We run each experiment using 100 worlds. We define a constant C over the set of worlds W. For each world w ∈ W, we uniformly sample a c ∈ (1, 2, ...C). We define "spawn cap" such that if world w has a spawn cap c, the number of agents in w cannot exceed c. In each world w, one agent is spawned per game tick provided that doing so would exceed the spawn cap c of w. Ideally, we would fix N ent = N pop, as is the case in standard MMOs (humans are independent networks with unshared weights). However, this incurs sample complexity proportional to number of populations. We therefore share parameters across groups of up to 16 agents for efficiency. We perform four experiments to evaluate the effects on foraging performance of training with larger and more populations. For each experiment, we fix N pop and a spawn cap 1. These are paired from and. We train for a fixed number of trajectories per population 9.Evaluating the influence of these variables is nontrivial. The task difficulty is highly dependent on the size and competence of populations in the environment: mean agent lifetime is not comparable among experiments. Furthermore, there is no analog procedure for evaluating relative player competence among MMO servers. However, MMO servers sometimes undergo merges whereby the player bases from multiple servers are placed within a single server. As such, we propose tournament style evaluation in order to directly compare policies learned in different experiment settings. Tournaments are formed by simply concatenating the player bases of each experiment. Results are shown in Figure 3: we vary the maximum number of agents at test time and find that agents trained in larger settings consistently outperform agents trained in smaller settings. When we introduce the combat module as an additional learnable mode of variation on top of foraging, we observe more interesting policies as a . Agent actions become strongly coupled with the states of other agents. As a sanity, we also confirm that all of the populations trained with combat handily outperform all of the populations trained with only foraging. To better understand theses , we decouple our analysis into two modes of variability: maximum number of concurrent players (N ent) and maximum number of populations with unshared weights (N pop). This allows us to examine the effects of each factor independently. In order to isolate the effects of environment randomization, which also encourages exploration, we perform these experiments on a fixed map. Isolating the effects of these variables produces more immediately obvious , discussed below. We briefly examine the randomized setting in Discussion. In the natural world, competition between animals can incentivize them to spread out in order to avoid conflict. We observe that overall exploration scales with number of concurrent agents, with no other variable factors (see Figure 4 ; the map used is shown in FIG3). Agents learn to explore only because the presence of other agents provides a natural and beneficial curriculum for doing so. We find that, given a sufficiently large and resource rich environment, different populations of agents tend to separate to avoid competing with other populations. Both MMOs and the real world often reward masters of a single craft more than jacks of all trades. From FIG3, specialization to particular regions of the map scales with number of populations. This suggests that the presence of other populations force agents to discover a single advantageous skill or trick. That is, increasing the number of populations in diversification to separable regions of the map. As entities cannot out-compete other agents of their population with shared weights, they tend to seek areas of the map that contain enough resources to sustain their population. Regions that are difficult to get to or otherwise unoccupied are especially desirable; this is revealed by observing value maps over time. Jungle climates produce more biodiversity than deserts. Deserts produce more biodiversity than the tallest mountain peaks. To current knowledge, Earth is the only planet to produce life at all: the initial conditions for formation of intelligent life are of paramount importance. The same holds true in simulation: human MMOs mirror this phenomenon. Some games produce more complex and engaging play than others-those most successful garner large and dedicated playerbases that come to understand the game systems better than the developers. Feedback helps drive development and expansion over a period of years, but the amount of effort required to create that initial seed is large.: Agents learn to depend on other agents. Each square map shows the response of an agent of a particular species, located at the square's center, to the presence of agents at any tile around it. Random: dependence map of random policies. Early: "bulls eye" avoidance maps learned after only a few minutes of training. Additional maps correspond to foraging and combat policies learned with automatic targeting (as in tournament ) and learned targeting (experimental, discussed in Additional Insights). In the learned targeting setting, agents begin to fixate on the presence of other agents within combat range, as denoted by the central square patterns. It is unreasonable to expect pure multiagent competition to produce diverse and interesting behavior if the environment does not support it. This is because multiagent competition is a curriculum magnifier, not a curriculum. The multiagent setting is interesting because learning is responsive to competitive and collaborative pressures of other learning agents-but the environment must support and facillitate such pressures in order for multiagent interaction to drive complexity. There is room for debate among reasonable individuals as to theoretical minimum complexity seed environment required to produce complexity on par with that of the real world. However, this is not our objective, and we do not have a reasonable estimate of this quantity. We have chosen to model our environment after MMOs, even though they may be more complicated than the minimum required environment class, because they are known to support the types of interactions we are interested in while maintaining engineering and implementation feasibility. This is not true of any other class environments we are aware of: exact physical simulations are computationally infeasible, and previously studied genres of human games lack crucial elements of complexity (see Background). While some may see our efforts as cherrypicking environment design, we believe this is precisely the objective: developers cherrypick game design decisions to support complexity commensurate with engaging play at the level of general human intelligence. The player base uses these design decisions to create strategies far beyond the imagination of the developers. The trend of increasing exploration with increasing entity number is clear when training on a single map as seen in Figure 4, 5, but it is more subtle with environment randomization. From FIG4, all population sizes explore adequately. We believe that this is because "exploration" as defined by map coverage is not as difficult a problem as developing robust policies. As demonstrated by the Tournament experiments, smaller populations learn brittle policies that do not generalize to scenarios with more competitive pressure-even against a similar number of agents. We visualize agent-agent dependencies in FIG5. We fix an agent at the center of a hypothetical map crop. For each position visible to that agent, we fake another agent and compute the value function estimate of the ant pair. We find that agents learn policies dependent on those of other agents in both the foraging and combat environments. We briefly detail several miscellaneous investigations and subtle points of interest in FIG6. First, we visualize learned attack patterns of agents. Each time an agent attacks, we splat the attack type to the screen. There are a few valid strategies as per the environment. Melee is intentionally overpowered: learning to utilize it at close range serves as a sanity check. This also cautions agents to keep their distance, as the first to strike wins. From FIG6 and observation of the policies, we find that this behavior is learned when targeting is automated. Our final set of experiments prescribes targeting to the agent with lowest health. Jointly learning attack style selection and targeting requires an attentional mechanism to handle variable number of visible targets. We have experimented with this, but are not yet numerically stable. Solving this is likely to increase policy complexity, but policies learned using automatic targeting are still compelling: agents learn to strafe at the edge of their attack radius, attacking opportunistically. Second, a note on tournaments. We equate number of trajectories trained upon as a fairest possible metric of training progress. We experimented with normalizing batch size but found that larger batch size always leads to more stable performance. Batch size is split among species: larger populations outperform smaller populations even though the latter are easier to train. Finally, a quick note on niche formation. Obtaining clean visuals is dependent on having an environment where interaction with other agents is unfavorable. When this is not the case, niche formation may still occur in another space (e.g., diverse attack policies). This is the importance of niche formation-we expect it to stack well with population-based training BID7 and other such methods that require sample diversity. Figure 9: We procedurally generate maps by thresholding an 8 octave Perlin BID15 ridge fractal. We map tile type to a fixed range of values APPENDIX This section contains full environment details that are useful but not essential to understanding the base experiments. All parameters in the following subsystems are configurable; we provide only sane defaults obtained via balancing. A grid of tiles 2 represents game state. Agents may move one tile North/South/East/West each game tick. They can also choose to Pass (i.e. not make a movement action). Each tile object contains:• Terrain type ∈ {grass, f orest, stone, water, lava}. Lava tiles kill agents upon contact. Stone and water tiles are impassible.• Occupying agents: a reference mapping of agents standing on the tile. The foraging system implements gathering based survival by introducing:• Food: initialized to 32, decremented 1 per tick, incremented 1 by occupying forest tiles, which contain 1 food each. Once the resources of a particular tile are consumed, they regenerate probablistically over time.• Water: decremented once per tick, incremented by standing adjacent to a water tile.• Health: decremented once per tick if either food or water is zero. Incremented if both food and water are above a threshold. A snapshot is shown in FIG1. These definitions of resources impose a carrying capacity. This incurs an arms race of exploration strategies in populations of agents above the carrying capacity: survival is trivial with a single agent, but requires intelligent exploration in the presence of competition attempting to do the same. The combat system implements three different attack "styles":• Melee: Inflicts 10 damage at 1 range FIG0: Agents observe their local environment. The model embeds this observations and computes actions via corresponding value, movement, and attack heads. These are all small fully connected networks with 50-100k parameters.• Ranged: Inflicts 2 damage at 1-2 range • Mage: Inflicts 1 damage at 3 range and freezes the target in place, preventing movement (but not attacks) for two ticks A snapshot is shown in FIG1. These definitions of combat impose trade offs in each style. Melee combat is high risk high return. Ranged combat produces less risky but more prolonged conflicts. Mage combat does little damage, but allows agents to retreat or cut off opponents' escape by freezing them in place. The best strategy is not obvious, again imposing an arms race to discover the best strategy in the presence of many agents attempting to do the same. • Attack range is defined by l1 distance: "1 range" is a 3X3 grid centered on the attacker. Range 2 is a 5X5 grid, etc.• Incentive: Animals hunt each other out of hunger or to protect/seize resources. Agents receive food/water resources equal to the damage they inflict on other agents.• Spawn Killing Agents are immune during their first 15 game ticks alive. This prevents an exploit known as "spawn killing" whereby players are repeatedly attacked immediately upon entering the game. Human games often contain similar mechanism to prevent this strategy, as it in uninteresting play. We provide two APIs for this:Gym Wrapper We provide a minimal extension of the Gym VecEnv API BID3 that adds support for variable numbers of agents per world and at any given time. This API distributes environment computation of observations and centralizes training and inference. While this standardization is convenient, MMOs differ significantly from arcade games, which are easier to standardize under a single wrapper. The Neural MMO setting requires support for a large, variable number of agents that run concurrently, with aggregation across many randomly generated environments. As such, the Gym framework incurs additional communications overhead in our setting that we bypass with a fast native API.Native This interface is simpler to use and pins the environment and agents on it to the same CPU core. Full trajectories run locally on the same core as the environment. Interprocess communication is only required infrequently to synchronize gradients across all environments on a master core. We currently do the backwards pass on the same CPU cores for convenience, but plan on separating this into an asynchronous experience queue for GPU workers. We also provide a front end visualizer packed with research tools for analyzing observe agent policies and relevant statistics. We leave detailed documentation of this to the source release, but it includes:• 2D game renderer: shown in FIG1 • 3D game renderer in beta • Value map "ghosting" (See FIG1 • Exploration maps • Interagent dependence • Attack maps (See Experiments)Preprocessor:• Embed indicies corresponding to each tile into a 7D vector. Also concatenates with the number of occupying entities.• Project visible attributes of nearby entities to 32D• Flatten the tile embeddings • Max pool over entity embeddings to handle variable number of observations • Concatenate the two • This produces an embedding, which is computed separately for each head. • For foraging experiments, the attack network is still present for convenience, but the chosen actions are ignored.• It then concatenates the ant vector with the (unlearned) features of the current entity. These include visible properties such as health, food, water, and position.• In order to handle the variable number of visible entities, a linear layer is applied to each followed by 1D max pooling. Attention BID0 may appear the more conventional approach, but recently OpenAI demonstrated that simpler and more efficient max pooling suffices.• The pooled entity vector is then concatenated with the tile and self embeddings to produce a single vector and followed by a final linear layer as shown in FIG0.

Represent the short summary for finding the corresponding scientific article (long)

An MMO-inspired research game platform for studying emergent behaviors of large populations in a complex environment

Represent the TLDR for finding the corresponding scientific article

Techniques for combining generalized policies with search algorithms to exploit the strengths and overcome the weaknesses of each when solving probabilistic planning problems