Question: Represent the text to find a very short summary (1-2 sentences max)

Question: In this paper, we propose a novel kind of kernel, random forest kernel, to enhance the empirical performance of MMD GAN. Different from common forests with deterministic routings, a probabilistic routing variant is used in our innovated random-forest kernel, which is possible to merge with the CNN frameworks. Our proposed random-forest kernel has the following advantages: From the perspective of random forest, the output of GAN discriminator can be viewed as feature inputs to the forest, where each tree gets access to merely a fraction of the features, and thus the entire forest benefits from ensemble learning. In the aspect of kernel method, random-forest kernel is proved to be characteristic, and therefore suitable for the MMD structure. Besides, being an asymmetric kernel, our random-forest kernel is much more flexible, in terms of capturing the differences between distributions. Sharing the advantages of CNN, kernel method, and ensemble learning, our random-forest kernel based MMD GAN obtains desirable empirical performances on CIFAR-10, CelebA and LSUN bedroom data sets. Furthermore, for the sake of completeness, we also put forward comprehensive theoretical analysis to support our experimental . Generative adversarial nets (GANs;) are well-known generative models, which largely attribute to the sophisticated design of a generator and a discriminator which are trained jointly in an adversarial fashion. Nowadays GANs are intensely used in a variety of practical tasks, such as image-to-image translation ; 3D reconstruction ; video prediction ; text-to-image generation ; just to name a few. However, it's well-known that the training of GANs is a little tricky, see e.g. . One reason of instability of GAN training lies in the distance used in discriminator to measure the divergence between the generated distribution and the target distribution. For instance, concerning with the Jensen-Shannon divergence based GANs proposed in , points out that if the generated distribution and the target distribution are supported on manifolds where the measure of intersection is zero, Jensen-Shannon divergence will be constant and the KL divergences be infinite. Consequently, the generator fails to obtain enough useful gradient to update, which undermines GAN training. Moreover, two non-overlapping distributions may be judged to be quite different by the Jensen-Shannon divergence, even if they are nearby with high probability. As a , to better measure the difference between two distributions, Integral Probability Metrics (IPM) based GANs have been proposed. For instance, utilizes Wasserstein distance in GAN discriminator, while adopts maximum mean discrepancy (MMD), managing to project and discriminate data in reproducing kernel Hilbert space (RKHS). To mention, the RKHS with characteristic kernels including Gaussian RBF kernel and rational quadratic kernel (Bińkowski et al., 2018) has strong power in the discrimination of two distributions, see e.g. . In this paper, inspired by non-linear discriminating power of decision forests, we propose a new type of kernel named random-forest kernel to improve the performance of MMD GAN discriminator. In order to fit with back-propagation training procedure, we borrow the decision forest model with stochastic and differentiable decision trees from in our random-forest kernel. To be specific, each dimension of the GAN discriminator outputs is randomly connected to one internal node of a soft decision forest, serving as the candidate to-be-split dimension. Then, the tree is split with a soft decision function through a probabilistic routing. Other than the typical decision forest used in classification tasks where the value of each leaf node is a label, the leaf value of our random forest is the probability of a sample x i falling into a certain leaf node of the forest. If the output of the discriminator is denoted as h θ N (x i) and the probability output of the t-th tree is denoted as µ t (h θ N (x i); θ F ), the random forest kernel k RF can be formulated as where T is the total number of trees in the forest, θ N and θ F denote the parameters of the GAN discriminator and the random forest respectively. Recall that random forest and deep neural networks are first combined in , where differentiable decision tree model and deep convolutional networks are trained together in an end-to-end manner to solve classification tasks. extends the idea to label distribution learning, and makes further extensions in regression regime. , and also introduce deep decision forests. Apart from the typical ensemble method that averages the across trees, they aggregate the by multiplication. As for the combination of random forest and introduce forests structure in GAN discriminator, combining CNN network and forest as a composited classifier, while uses forest structure as one of non-linear mapping functions in regularization part. On the other hand, in the aspect of relationship between random forest and kernel method, initiates the literature concerning the link. He shows the fact that a purely random tree partition is equivalent to a kernel acting on the true margin, of which form can be viewed as the probability of two samples falling into the same terminal node. proves that random forest kernel is characteristic. Some more theoretical analysis can be found in , ,. However, despite their theoretical breakthroughs, forest decision functions used in these forest kernels are non-differentiable hard margins rather than differentiable soft ones, and thus cannot be directly used in back propagation regime. To the best of our knowledge, MMD GAN with our proposed random-forest kernel is the first to combine random forest with deep neural network in the form of kernel MMD GAN. Through theoretical analysis and numerical experiments, we evaluate the effectiveness of MMD GAN with our random-forest kernel. From the theoretical point of view, our random-forest kernel enjoys the property of being characteristic, and the gradient estimators used in the training process of random-forest kernel GAN are unbiased. In numerical experiments, we evaluate our random-forest kernel under the setting of both the original MMD GAN and the one with repulsive loss . Besides, we also compare our random-forest kernel with Gaussian RBF kernel , rational quadratic kernel (Bińkowski et al., 2018), and bounded RBF kernel . As a , MMD GAN with our random-forest kernel outperforms its counterparts with respect to both accuracy and training stability. This paper is organized as follows. First of all, we introduce some preliminaries of MMD GAN in Section 2. Then we review the concept of deep random forest and show how it is embedded within a CNN in 3.1. After that, random-forest kernels and MMD GAN with random-forest kernels are proposed in 3.2 and 3.3 respectively. Besides, the training techniques of MMD GAN with random-forest kernel are demonstrated in Section 3.4 and the theoretical are shown in Section 3.5. Eventually, Section 4 presents the experimental setups and , including the comparison between our proposed random-forest kernel and other kernels. In addition, all detailed theoretical proofs are included in the Appendices. The generative model captures the data distribution P X, by building a mapping function G: Z → X from a prior noise distribution P Z to data space. While the discriminative model D: X → R is used to distinguish generated distribution P Y from real data distribution P X. Taking X, X ∼ P X and Y, Y ∼ P Y:= P G (Z) where Y:= G(Z) and Y:= G(Z), the squared MMD is expressed as The loss of generator and discriminator in MMD GAN proposed in is: proposed MMD GAN with repulsive loss, where the objective functions for G and D are: we can write an unbiased estimator of the squared MMD in terms of k as When k is a characteristic kernel, we have MMD 2 [P X, P Y] ≥ 0 with equality applies if and only if P X = P Y. The best-known characteristic kernels are gaussian RBF kernel and rational quadratic kernel (Bińkowski et al., 2018). In this section, we review a stochastic and differentiable variant of random forest and how it is embedded within a deep convolutional neural network proposed in. Then we propose random-forest kernel and we apply it in MMD GAN. We illustrate the advantages of our random-forest kernel, show the training technique of MMD GAN with random-forest kernel, and study its theoretical properties. Suppose that a random forest consists of T ∈ N random trees. For the t-th tree in the forest, t ∈ {1, . . ., T}, we denote N t:= {d t j} |Nt| j=1 as the set of its internal nodes and if T trees have the same structure, then we have |N t | = |N |, see Figure 1. Furthermore, we denote L t as the set of its leaf nodes and θ t F as the parameters of the t-th tree. Here we introduce the routing function µ t (x; θ t F) which indicates the probability of the sample x falling into the -th leaf node of the t-th tree. In order to provide an explicit form for the routing function µ t (x; θ t F), e.g. the thick black line in Figure 1, we introduce the following binary relations that depend on the tree structure: d t j is true, if belongs to the left subtree of node d ) be the decision function of the j-th internal node in the t-th tree, that is the probability of the sample x falling into the left child of node d t j in the t-th tree. Then, µ t can be expressed by these relations as where R t denotes the unique path from node 1 to node of the t-th tree. Figure 1: Example of a random forest with T trees: blue nodes denote the internal nodes N:= {d1, ..., d7} while purple nodes are the leaf nodes L:= {1, · · ·, 8}. The black thick path illustrates the route of a sample x falling into the 6 leaf node of the t-th tree. Now, let us derive the explicit form of the decision function p t j (x; θ t F). Here, to utilize the power of deep learning, we consider using the convolutional neural network to construct decision functions of random forest. To be specific, given the parameter θ N trained from a CNN network, we denote h(·; θ N) as the d -dimension output of a convolutional neural network, which is the unit of the last fullyconnected layer in the CNN, and h i (·; θ N) is the i-th element of the CNN output. We denote C: {1, . . ., T |N |} → {1, . . ., d} as the connection function, which represents the connection between the internal node d t j and the former CNN output h i. Note that during the whole training process, the form of the connection function C is not changed and every internal node d t j is randomly assigned to an element h C(T (t−1)+j) (·; θ). If we choose the sigmoid function σ(x) = (1+e −x) −1 as the decision function, and let the parameters of the t-th tree be θ For example, we have the probability p Every leaf node in each tree has a unique road R t from node 1 to node with length |R t | = log 2 (|L t |). Then, for the every leaf node of the t-th tree, we have where Lf denotes the set of all left son nodes of its father node. Here, we propose the random-forest kernel as follows: Definition 1 (Random-Forest Kernel) Let x, y be a pair of kernel input, let θ t F = (w t, b t) denotes the weights and bias of the t-th tree of the random forest, and θ F:= (θ t F) T t=1. The random-forest kernel can be defined as where L t denotes the set of leaf nodes in the t-th tree, We write and introduce the objective functions of MMD GAN with random-forest kernel by where y = G ψ (z), z is noise vector, and R is the regularizer of random-forest kernel (the detail is shown in Section 3.4). In addition, the objective functions of MMD GAN with repulsive loss are Random-forest kernel MMD GAN enjoys the following advantages: • Our proposed random-forest kernel used in MMD GAN benefits from ensemble learning. From the perspective of random forest, the output of MMD GAN discriminator h(·; θ N) can be viewed as feature inputs to the forest. To mention, each tree only gets access to merely a fraction of the features by random connection functions, and thus the entire forest benefits from ensemble learning. • Our random-forest kernel MMD GAN enjoys the advantages of three powerful discriminative methods, which are CNN, kernel method, and ensemble learning. To be specific, CNN is good at extracting useful features from images; Kernel method utilize RKHS for discrimination; Ensemble learning utilizes the power of randomness and ensemble. • Our proposed random-forest kernel has some good theoretical properties. In one aspect, random-forest kernel is proved to be characteristic in. In another, in Section 3.5, the unbiasedness of the gradients of MMD GAN with random-forest kernel is proved. , the authors mention that the tree may get stuck on plateaus if internal nodes always assign the most of probability to one of its subtree. The gradients will vanish because the gradients of the logistic-type decision function will be very closed to zero. In order to stabilize the training of random-forest kernel and avoid the stuck on bad solutions, we add penalty that encourage each internal node to split in a balanced style as does, that is, we penalize the cross entropy between the desired 0.5, 0.5 average probabilities of falling into two subtrees and the actual average probability α, 1 − α. The actual average probability of the i-th internal node α i is, where P i (x) is the routing probability of x from root node to internal node i, p i (x) is the probability of x falling into the left subtree of the i-th internal node, and Ω is the collection of mini-batch samples. Then, the formulation of the regularizer is: where λ is exponentially decayed with the depth of d of the internal node by multiplying the coefficient 2 −d, for the intuition that less balanced split in deeper internal node may increase the non-linear discrimination power. When training random-forest kernel, a mini-batch of real samples X and generated pictures Y are both fed into the discriminator, and then k(X, X), k(X, Y) and k(Y, Y) are calculated, where k:= k RF • h(·; θ N). Here, to notify, we find that the Ω in the regularizer formulation does matter in forest-kernel setting. It's better to calculate α i and R(Ω) in the case of Ω = X, Ω = Y, Ω = X ∪ Y respectively, and then sum up three parts of regularizer as final regularizer R. Therefore, the formulation of regularizer R added in the training of random-forest kernel is In this subsection, we present our main theoretical . Theorem 2 (Unbiasedness) Let X be the true data on X with the distribution P X and Z be the noise on Z with the distribution P Z satisfying E P X X α < ∞ and E P Z Z α < ∞ for some α ≥ 1. Moreover, let G ψ: Z → X be a generator network, h θ N: X → R d be a discriminator network, k RF be the random-forest kernel, and θ D:= (θ N, θ F) be the parameters of the GAN discriminator. Then, for µ-almost all θ D ∈ R |θ D | and ψ ∈ R |ψ|, there holds In other words, during the training process of Random-Forest Kernel MMD GAN, the estimated gradients of MMD with respect to the parameters ψ and θ D are unbiased, that is, the expectation and the differential are exchangeable. In this section, we evaluate our proposed random-forest kernel in the setting of MMD GAN in and the MMD GAN with repulsive loss . To illustrate the efficacy of our random-forest kernel, we compare our random-forest kernel with Gaussian kernel , rational quadratic kernel (Bińkowski et al., 2018) As is shown in Figure 3, the shapes of the Gaussian RBF kernel and the rational quadratic kernel are both symmetric. However, the local structure of random-forest kernel (w.r.t reference points except 70-dimensional zero vector) is asymmetric and very complex. The asymmetry and complexity of random-forest kernel may be helpful to discriminate two distributions in MMD GAN training. For dataset Cifar10 and dataset LSUN bedroom, DCGAN architecture with hyper parameters from is used for both generator and discriminator; and for dataset CelebA, we use a 5-layer DCGAN discriminator and a 10-layer ResNet generator. Further details of the network architecture are given in Appendix A.3. We mention that in all experiments, batch normalization is used in the generator and spectral normalization is used in the discriminator. The hyper-parameter details of kernels used in the discriminator are shown in Appendix A.1. For the sake of comparison with forest kernel, the dimension of discriminator output layer s is set to be 70 for random-forest kernel and to be 16 for other kernels following the previous setting of Bińkowski et al.;. We set the initial learning rate 10 −4 and decrease the learning rate by coefficient 0.8 in iteration 30000, 60000, 90000, and 120000. Adam optimizer is used with momentum parameters β 1 = 0.5 and β 2 = 0.999. The batch size of each model is 64. All models were trained for 150000 iterations on CIFAR-10, CelebA, and LSUN bedroom datasets, with five discriminator updates per generator update. The following three metrics are used for quantitative evaluation: Inception score (IS) , Fréchet inception distance (FID) , and Kernel inception distance (KID) (Bińkowski et al., 2018). In general, higher IS and Lower FID, KID means better quality. However, outside the dataset Imagenet, the metric IS has some problem, especially for datasets celebA and LSUN bedroom. Therefore, for inception score, we only report the inception score of CIFAR-10. Quantitative scores are calculated based on 50000 generator samples and 50000 real samples. We compare our proposed random-forest kernel with mix-rbf kernel and mix-rq kernel in the setting of the MMD GAN loss, and compare our proposed random-forest kernel with rbf-b kernel in the setting with MMD GAN repulsive loss. The Inception Score, the Fréchet Inception Distance and the Kernel Inception Distance of applying different kernels and different loss functions on three benchmark datasets are shown in table 1. We find that, in the perspective of the original MMD GAN loss, our newly proposed random-forest kernel shows better performance than the mix-rbf kernel and the mix-rq kernel in CIFAR-10 dataset and LSUN bedroom dataset; and in the perspective of the repulsive loss, the performance of our newly proposed random-forest kernel is comparable or better than the rbf-b kernel. The efficacy of our newly proposed random-forest kernel is shown under the setting of both MMD GAN loss and MMD GAN repulsive loss. Some randomly generated pictures of model learned with various kernels and two different loss functions are visualized in Appendix D. The formulation of our proposed random-forest kernel is In experiments, we take the number of trees in the forest T = 10 and the depth of trees dep = 3. Thus, the total number of internal nodes is 70. To notify, in general, the parameters are trainable, where and N is the set of every internal nodes of a tree. However, for the sake of experimental simplification, we fix each w where Σ = {2, 5, 10, 20, 40, 80}, and rational quadratic kernel (Bińkowski et al., 2018) with mixture of kernel scale α, that is, where A = {0.2, 0.5, 1, 2, 5}. In the setting of the MMD GAN with repulsive loss proposed in , we compare our forest kernel with bounded RBF kernel , that is, with σ = 1, b l = 0.25 and b u = 4. In figure 3, we compare the contours of three different kernels (the detail of kernels is shown in A.1). We directly plot the filled contours of 2-dimensional Gaussian kernel and rational quadratic kernel with reference to; As for random-forest kernel with T = 10 and dep = 3, where the input dimension is 70, the steps of a 2-dimensional visualization are as follows: 1) We randomly generate 25000 points from the uniform distribution U[−1, 1] 70 and set (0.5, . . ., 0.5) ∈ R 70 as the reference point. To notify, if the reference point is 70-dimensional zero vector, the values of random-forest kernel will be constant; 2) We calculate the 25000 output values of random-forest kernel; 3) We transform 70-dimensional 25000 randomly generated points and reference point together by t-SNE to 2-dimensional points; 4) We show the filled contour of the neighborhood of transformed reference point. We try to visualize the local structure of random-forest kernel. In the experiments of the CIFAR-10 and LSUN bedroom datasets, we use the DCGAN architecture following , and for the experiments of the CelebA dataset, we use a 5-layer DCGAN discriminator and a 10-layer ResNet generator as in Bińkowski et al.. The first few layers of the ResNet generator consist of a linear layer and 4 residual blocks as in. The network architecture details are shown in Table 2 and 3. Table 2: Network architecture used in the image generation on CIFAR-10 dataset and LSUN bedroom dataset. In terms of the shape parameter h and w in the table, we take h = w = 4 for CIFAR-10 and h = w = 8 for LSUN bedroom. As for the output dimension of discriminator s, we take s = 70 for random-forest kernel and s = 16, 70 for other kernels. To notify, 16 is the setting in Bińkowski et al. In Section B, we will show the main propositions used to prove the Theorem 2. To be specific, in Section B.1, we represent neural networks as computation graphs. In Section B.2, we consider a general class of piecewise analytic functions as the non-linear part of neural networks. In Section B.3, we prove the Lipschitz property of the whole discriminators. In Section B.4, we discover that for P X almost surely, the network is not differential for its parameters θ N and ψ. Fortunately, we prove that the measure of bad parameters set is zero. In Section C, we will show the explicit proof of main propositions in Section B and Theorem 2. Historical attempts to scale up GANs using CNNs to model images have been unsuccessful. The original CNN architecture is made up of convolution, non-linear and pooling. Now for our model, we adopt the deconvolution net to generate the new data with spatial upsampling. Moreover, batch normalization is a regular method which stabilizes learning by normalizing the input to each unit to have zero mean and unit variance. Furthermore, relu functions are used both in generator and discriminator networks as non-linear part. Here we avoid spatial pooling such as max-pooling and global average pooling. Throughout this paper, we always denote by the output of a fully connected layer, where d represents the number of neurons in the output. The general feed-forward networks including CNN and FC can be formulated as a directed acyclic computation graph G consisting of L + 1 layers, with a root node i = 0 and a leaf node i = L. For a fixed node i, we use the following notations: π(i): the set of parent nodes of i; j < i: j is a parent node of i. Each node i > 0 corresponds to a function f i that receives a R d π(i) -valued input vector, which is the concatenation of the outputs of each layer in π(i), and outputs a R di -valued vector, where According to the construction of the graph G, the feed-forward network that factorizes with functions f i recursively can therefore be defined by h 0 = X, and for all 0 < i ≤ L, where h π(i) is the concatenation of the vectors h j, j ∈ π(i). Here, the functions f i can be of the following different types: ) is a linear operator on the weights W i, e.g. convolutions, then the functions f i are of the linear form (ii) Non-linear: Such functions f i including ReLU, max-pooling and ELU, have no learnable weights, can potentially be non-differentiable, and only satisfy some weak regularization conditions, see Definition 3 and the related Examples 2, 3, and 4. In the following, we denote by I the set of nodes i such that f i is non-linear, and its complement by I c:= {1, . . ., L} \ I, that is, the set of all of all linear modules. We write θ N as the concatenation of parameters where |θ N | denotes the total number of parameters of θ N. Moreover, the feature vector of the network corresponds to the output neurons G L of the last layer L and will be denoted by where the subscript θ stands for the parameters of the network. If X is random, we will use h θ N (X) to denote explicit dependence on X, and otherwise when X is fixed, it will be omitted. Throughout this paper, in the construction of neural networks, we only consider activation functions that are piecewise analytic which can be defined as follows: Definition 3 (Piecewise Analytic Functions) Let {f i} i∈I be non-linear layers in neural networks, then f i is said to be an piecewise analytic function if there exists a partition of R d π(i) with J i pieces, that is, The sets D The following examples show that in practice, the vast majority of deep networks satisfy the conditions in the above definition, and therefore are piecewise analytic. Example 1 (Sigmoid) Let f i outputs the sigmoid activation function on the inputs. Then we need no partition, and hence there exist corresponding to the whole space; (ii) J i = 1 real analytic function f Here, we mention that the case in Example 1 corresponds to most of the differentiable activation functions used in deep learning, more examples include the softmax, hyperbolic tangent, and batch normalization functions. On the other hand, in the case that the functions are not differentiable, as is shown in the following examples, many commonly used activation functions including the ReLU activation function are at least piecewise analytic. Besides the ReLU activation function, other activation functions, such as the Max-Pooling and the ELU also satisfy the form of Definition 3 are therefore piecewise analytic. In this section, we investigate the Lipschitz property of the proposed discriminative networks, which is formulated as follows: Proposition 4 Let θ D be the parameters of discriminators and B r (θ D) ⊂ R |θ D | be the ball with center θ D and radius r ∈ (0, ∞). Then, for all θ D ∈ B r (θ D) and all x ∈ R d, there exists a regular function c(x) with In the analysis of section B.3, we intentionally ignore the situation when samples fall into the "bad sets", where the network is not differentiable for data sets x with nonzero measure. And in proposition 5, we show that the measure of the so called "bad sets" is zero. To better illustrate this proposition, we first denote some notations as follows: as the set of input vectors x ∈ R d such that h θ N (x) is not differentiable with respect to θ N at the point θ N,0. Then we call the set of critical parameters, where the network is not differentiable for data sets x with nonzero measure. Proposition 5 Let the set Θ P X be as in equation 8. Then, for any distribution P X, we have µ(Θ P X) = 0. To prove Proposition 4, we first introduce Lemma 6 and Lemma 7. Lemma 6 describes the growth and Lipschitz properties of general networks including convolutional neural networks and fully connected neural networks introduced in Section B.1. Lemma 6 Let h θ N be the neural networks defined as in Section B.1. Then there exist continuous functions a, b: R |θ N | → R and α, β: Proof [of Lemma 6] We proceed the proof by induction on the nodes of the network. Obviously, for i = 0, the inequalities hold with b 0 = 0, a 0 = 1, β 0 = 0 and α 0 = 0. Then, for the induction step, let us fix an index i. Assume that for all x ∈ R d and all where are some continuous functions. Moreover, concerning with the Lipschitz property, we have where we used the notations (ii) If i ∈ I c, that is, i is not a linear layer, here we only consider the sigmoid and ReLU functions. We first show that both of them are Lipschitz continuous. Concerning the sigmoid function, σ(x) = 1 1 + e −x we obviously have for all x ∈ R. Consequently, the sigmoid function is Lipschitz continuous with Lipschitz constant |σ| 1:= 1/4. Next, for the ReLU function, we have for all x ∈ R, Therefore, the ReLU function is Lipschitz continuous with Lipschitz constant |σ| 1:= 1. Thus, non-linear layer f i is Lipschitz continuous with Lipschitz constant M:= |f i | 1. Consequently, by recursion, we obtain continuous functions Next, we investigate the growth conditions and Lipschitz property of the random forest. For the ease of notations, we write the function µ (T) as Moreover, it is easily seen that T · |N | equals the number of internal nodes in the random forest. Lemma 7 Let h(x; θ N) be the input vector of random trees and θ F:= (w, b) where w and b denote the weights and bias of the random forests, respectively. Then, for all h ∈ R d and θ F, θ F ∈ R 2T |N |, there exist continuous functions c 1, c 2, and constants c 3, c 4, c 5 such that µ ≤ 1 Taking the summation on both sides of the above inequality with respect to all of the nodes in the random forest, that is, w.r.t. ∈ L, we obtain where the second inequality follows from the Cauchy-Schwartz inequality and the third inequality is due to the fact that the number that the nodes p i in the random forest assigned to the corresponding node h j equals T |N |/d or T |N |/d + 1, which are less than |L|/d. Now, we show the Lipschitz properties equation 12 and equation 13 of the random forest. From the equivalent form equation 3 concerning the value of the leaf node, we easily see that µ can be written as a product of probability functions p i or 1 − p i. Therefore, without loss of generality, we can assume µ are of the product form For a fixed t = 1,..., T, recall that T t denotes the connection function of the t-th random tree. Then, the Lipschitz property of the sigmoid function and the continuously differentiability of the linear transform yield Then, equation 14 together with equation 15 implies where the last inequality follows from the Cauchy-Schwartz inequality. Consequently, concerning the random forest, we obtain where the second inequality is again due to then Cauchy-Schwartz inequality. Analogously, for a fixed t = 1,..., T and any i, we have and consequently we obtain and for the random forest, there holds which completes the proof. The next proposition presents the growth condition and the Lipschitz property of the composition of the neural network and the random forest. Lemma 8 Let B r (θ D) ⊂ R |θ D | be the ball with center θ D and radius r ∈ (0, ∞). Then, for all θ D ∈ B r (θ D), all x ∈ R d, there exist continuous functions c 6, c 7, c 8, and c 9 such that Proof [of Lemma 8] Combining equation 9 in Lemma 6 with equation 11 in Lemma 7, we obtain the growth condition of the form Concerning the Lipschitz property, using equation 12 and equation 13, we get where the last inequality follows from equation 9 and equation 10 established in Proposition 6. With the concatenation θ D:= (θ N, θ F):= (θ N, w, b) we obtain and thus the assertion is proved. Proof [of Proposition 4] First we give the growth conditions for the linear kernel. Let k be the linear kernel k L (x, y):= x, y, then we have If we denote u:= (x, y), then the above linear kernel k L (x, y) can be written a as a univariate function Therefore we have K is continuously differentiable satisfying the following growth conditions: Let us define the function f: Then we have f = K(v) and f = K(u). Moreover, f is differentiable with derivative The growth condition equation 18 implies Using the mean value theorem, we obtain that for all u, v ∈ R L, there holds Proposition 8 tells us that holds for some continuous functions c 6, c 7, c 8, and c 9, which are also bounded by certain constant B > 0 on the ball B r (θ D). Some elementary algebra shows that Since t → (1+t 1/2) 2 is concave on t ≥ 0, Jensen's inequality together with the moment assumption E x 2 < ∞ implies and also E(1 + x) < ∞ by the moment assumption. Therefore, the regular function c(x) defined by c(is integrable and thus our assertion is proved. To reach the proof of Proposition 5, we first introduce Lemma 9 and Lemma 11. Let i be a fixed node. To describe paths through the network's computational graph, we need to introduce the following notations: A(i):= {i | i is an ancestor of i}; Obviously, we always have ∂i ⊆ ¬i and ¬i = ∂i ∪ ¬π(i). We define a backward trajectory starting from node i by The set of all backward trajectories for node i will be denoted by Q(i). Lemma 9 Let i be a fixed node in the network graph. If θ N ∈ R |θ N | \ ∂S ¬i, then there exist a constant η > 0 and a trajectory q ∈ Q(i) such that for all θ N ∈ B η (θ N), there holds where f q is the real analytic function on R |θ N | with the same structure as h θ N, only replacing each nonlinear f i with the analytic function f Proof [of Lemma 9] We proceed by induction on the nodes of the network. If i = 0, we obviously have h 0 θ N = x, which is real analytic on R |θ N |. For the induction step we assume that the assertion holds for ¬π(i) and let θ N ∈ R |θ N | \ ∂S ¬i. Then there exist a constant η > 0 and a trajectory q ∈ Q(π(i)) such that for all θ N ∈ B η (θ N), there holds with f q: R |θ N | → R being a real analytic function. (i) If θ N ∈ S ∂i, then there exists a sufficiently small constant η > 0 such that B η (θ N) ∩ S ∂i = ∅. Therefore, there exists a j ∈ {1, . . ., J i} such that for all θ N ∈ B η (θ N), there holds where f j i is one of the real analytic functions in the definition of f i. Then, equation 19 implies that for all θ N ∈ B min{η,η} (θ N), there holds (ii) Consider the case θ N ∈ S ∂i. By assumption, we have θ N ∈ ∂S ∂i. Then there exists a small enough constant η > 0 such that B η (θ N) ⊂ S ∂i. If we denote Now we show by contradiction that for η small enough, there holds To this end, we assume that there exists a sequence of parameter and index pairs (θ N,n, p n) such that p n ∈ A c, θ N,n ∈ S pn, and θ N,n → θ N as n → ∞. Since A c is finite, there exists a constant subsequence {p ni} ⊂ {p n} and some constant p 0 ∈ A c with p ni = p 0 for all i. Then the continuity of the network and g p0 imply that S p0 is a closed set and consequently we obtain θ N ∈ S p0 by taking the limit, which contradicts the fact that θ N ∈ p∈A c S p. Therefore, for η small enough, there holds B η (θ N) ⊆ p∈A S p, which contradicts the assumption θ N ∈ ∂S ∂i. Consequently, there exists a j ∈ {1, . . ., J i} satisfying equation 20. By setting, where ⊕ denotes concatenation, then for for all θ N ∈ B min{η,η} (θ N), there holds and the assertion is proved. Now, for a fixed p = (i, j, s) ∈ P, we denote the set of network parameters θ N that lie on the boundary of p by where the functions g j i,s are as in Definition 3. As usual, the boundary of the set S p is denoted by ∂S p and the set of the boundaries is denoted by Finally, for the ease of notations, if P ⊂ P, we write Obviously, we have θ N ∈ R m \ ∂S ¬π(i). To prove Lemma 11, we need the following lemma which follows directly from and hence we omit the proof. Lemma 10 Let θ N → F (θ N): R |θ N | → R be a real analytic function and define Then we have either µ(M) = 0 or F = 0. Lemma 11 Let the set of boundaries ∂S P be as in equation 21. Then we have Proof [of Lemma 11] We proceed the proof by induction. For i = 0, we obviously have ∂S ¬0 = ∅ and therefore µ(∂S ¬0) = 0. For the induction step let us assume that For s = (p, q), the pair of an index p ∈ ∂i and a trajectory q ∈ Q(i), we define where the analytic function f q is defined as in Proposition 9. Then we prove by contradiction that for any θ N ∈ ∂S ∂i \ ∂S ¬π(i), there exists an s ∈ ∂i × Q(i) such that θ N ∈ M s and µ(M s) = 0. To this end, let θ N ∈ ∂S ∂i \ ∂S ¬π(i), then for small enough η > 0, there holds. By Proposition 9, there exists a trajectory q ∈ Q(π(i)) such that for all θ N ∈ B η (θ N), there holds Moreover, since θ N ∈ ∂S ∂i, there exists an index p ∈ ∂i such that g p (h This means that for s = (p, q), we have θ N ∈ M s. Therefore, we have where Combing equation 24 and equation 25, we obtain By the assumption µ(∂S ¬π(i) ) = 0, we conclude that µ(∂S ¬i) = 0. Since for the last node L, we have ¬L = P and therefore µ(∂S P) = 0. Note that the random forest µ (T) (·) can be considered as the composition of affine transformations and sigmoid functions and hence are always continuously differentiable with respect to θ F, we only need to investigate whether the neural network h θ N (x) is differentiable with respect to θ N. For a fixed x ∈ R d, we write as the set of parameters for which the network is not differentiable. Lemma 12 Let the set Θ x be as in equation 26. Then, for any x ∈ R d, we have Proof [of Lemma 12] Let the boundary set ∂S P be defined as in equation 22 and θ N,0 ∈ Θ x. Obviously, we have θ 0 ∈ S P. We proceed the proof of the inclusion Θ x ⊆ ∂S P by contradiction. To this end, we assume that θ D,0 ∈ ∂S P. When Lemma 9 applied to the output layer, there exist some η > 0 and a real analytic function f (θ N) such that for all θ N ∈ B η (θ N,0), there holds Consequently, the network is differentiable at θ N,0, contradicting the fact that θ N,0 ∈ Θ x. Therefore, we have Θ x ⊆ ∂S P and hence µ(Θ x) = 0 since µ(∂S P) = 0 according to Lemma 11. Proof [of Proposition 5] Let the sets N (θ N), Θ P X, and Θ x be as in equation 7, equation 8, and equation 26, respectively. Consider the sets S 1 = {(θ N, x) ∈ R |θ N | × R d | θ N ∈ Θ P X and x ∈ N (θ N)}, Since the network is continuous and not differentiable, Theorem I in implies that the sets S 1 and S 2 are measurable. Obviously, we have S 1 ⊂ S 2 and therefore we obtain ν(S 1) ≤ ν(S 2), where ν:= P X ⊗ µ. On the one hand, Fubini's theorem implies By Lemma 12, we have µ(Θ x) = 0 and therefore ν(S 2) = 0 and hence ν(S 1) = 0. On the other hand, Fubini's theorem again yields By the definition of Θ P X we have P X (N (θ N)) > 0 for all θ N ∈ Θ P X. Therefore, ν(S 1) = 0 implies that µ(Θ P X) = 0. C.3 PROOFS OF SECTION 3.5 To prove Theorem 2, we also need the following lemma. Lemma 13 Let (θ n) n∈N be a sequence in R m converging towards θ 0, i.e., θ n = θ 0 as n → ∞. Moreover, let f: R m → R be a function and g be a vector in R m. If holds for all sequences (θ n) n∈N, then f is differentiable at θ 0 with differential g. Proof [of Lemma 13] The definition of a differential tell us that g is the differential of f at θ 0, if By the sequential characterization of limits, we immediately obtain the assertion. Proof [of Theorem 2] Consider the following augmented networks: (θ D,ψ) (Z, Z) = (µ Without loss of generality, in the following, we only consider the network h (θ D,ψ) (X, Z) with inputs from P X ⊗ P Z, which satisfies E P X ⊗P Z (X, Z) 2 < ∞. By the expression of definition 1, we have where we denote µ (T) θ D (x):= µ (T) (h θ N (x); θ F ). Due to the linear kernel k L (x, y):= x, y, there holds )) is differentiable at λ 0 for P X -almost all x and P Z -almost all z. Then, according to Proposition 5, this statement holds for µ-almost all θ D ∈ R |θ D | and all ψ ∈ R |ψ|. Consider a sequence (λ n) n∈N that converges to λ 0, then there exists a δ > 0 such that λ n − λ 0 < δ for all n ∈ N. For a fixed u = (x, z) ∈ R d × R |z|, Proposition 4 states that there exists a regular function c(u) with E P X c(X) < ∞ such that |K(h λn (u)) − K(h λ0 (u))| ≤ c(u) λ n − λ 0 and consequently we have |∂ λ K(h λ0 (u))| ≤ c(u) for P X ⊗ P Z -almost all u ∈ R |u|. For n ∈ N, we define the sequence g n (x) by g n (u) = |K(h λn (u)) − K(h λ0 (u)) − ∂ λ K(h λ0 (u))(λ n − λ 0)| λ n − λ 0. Obviously, the sequence g n (x) converges pointwise to 0 and is bounded by the integrable function 2c(u). By the dominated convergence theorem, see e.g., Theorem 2.24 in , we have Moreover, for n ∈ N, we define the sequenceg n (x) bỹ Clearly, the sequenceg n (x) is upper bounded by E P X ⊗P Z g n (u) and therefore converges to 0. By Lemma 13, E P X ⊗P Z [K(h λ (u))] is differentiable at λ 0 with differential Since similar as above hold also for the networks h and h, and Lemma 6 in states that MMD 2 u is unbiased, our assertion follows then from the linearity of the form of MMD D SAMPLES OF GENERATED PICTURES Generated samples on the datasets CIFAR-10, CelebA, and LSUN bedroom are shown in Figure 4, 5, and 6, respectively.

represent this text to find a much much longer article about it with say hundreds-thousands of words

Equip MMD GANs with a new random-forest kernel.

The paper designs two algorithms for the stochastic AUC maximization problem with state-of-the-art complexities when using deep neural network as predictive model, which are also verified by empirical studies.


Question: Represent the article for finding a single-sentence TLDR

Question: Reinforcement learning in an actor-critic setting relies on accurate value estimates of the critic. However, the combination of function approximation, temporal difference (TD) learning and off-policy training can lead to an overestimating value function. A solution is to use Clipped Double Q-learning (CDQ), which is used in the TD3 algorithm and computes the minimum of two critics in the TD-target. We show that CDQ induces an underestimation bias and propose a new algorithm that accounts for this by using a weighted average of the target from CDQ and the target coming from a single critic. The weighting parameter is adjusted during training such that the value estimates match the actual discounted return on the most recent episodes and by that it balances over- and underestimation. Empirically, we obtain more accurate value estimates and demonstrate state of the art on several OpenAI gym tasks. In recent years it was shown that reinforcement learning algorithms are capable of solving very complex tasks, surpassing human expert performance in games like Go, Starcraft (DeepMind) or Dota (OpenAI). However, usually a large amount of training time is needed to achieve these (e.g. 45,000 years of gameplay for Dota). For many important problems (e.g. in robotics) it is prohibitively expensive for the reinforcement learning agent to interact with its environment that much. This makes it difficult to apply such algorithms in the real world. Off-policy reinforcement learning holds the promise of being more data-efficient than on-policy methods as old experience can be reused several times for training. Unfortunately, the combination of temporal-difference (TD) learning, function approximation and off-policy training can be unstable, which is why it has been called the deadly triad (; van). If the action space is discrete, solutions like Double DQN are very effective at preventing divergence of the value estimates by eliminating an otherwise prevailing overestimation bias. For continuous action spaces, which characterize many tasks, it was shown that Double DQN can not solve the overestimation problem. In an actor-critic setting it is important that the value estimates of the critic are accurate in order for the actor to learn a policy from the critic. The TD3 algorithm uses Clipped Double Q-learning (CDQ) to produce a critic without an overestimation bias, which greatly improved the performance of the algorithm. In CDQ two critics are trained at the same time and the TD target for both of them is the minimum over the two single TD targets. While the authors note that the CDQ critic update tends to underestimate the true values, this is not further examined. We show that this underestimation bias occurs in practice and propose a method that accounts for over-and underestimation of the critic at the same time. Similarly to CDQ we train two function approximators for the Q-values, but we regress them not on the same quantity. The TD target for each of the two critics is a weighted average of the single TD target for that critic and the TD target from CDQ. The weighting parameter is learned by comparing the value estimates for the most recent state-action pairs with the observed discounted returns for these pairs. As the one term of the average has an underestimation bias while the other one has an overestimation bias, the weighted average balances these biases and we show empirically that this method obtains much more accurate estimates of the Q-values. We verify that the more accurate critics improve the performance of the reinforcement learning agent as our method achieves state of the art on a range of continuous control tasks from OpenAi gym. To guarantee reproducibility we open source our code which is easy to execute and evaluate our algorithm on a large number of different random seeds. The Deterministic Policy Gradient algorithm (DPG) learns a deterministic policy in an actor-critic setting. This work was extended to the Deep Deterministic Policy Gradient algorithm by using multi-layer neural networks as function approximators. The Twin Delayed Deep Deterministic policy gradient algorithm (TD3) adds three more components to DDPG and achieves state of the art . First, the actor is updated less frequently than the critic, to allow for more accurate critic estimates before they are used for the actor. Second, in the critic update noise is added to the actions proposed by the actor. While these two extensions are introduced to decrease the variance in the policy gradient, the third one, Clipped Double Q-learning, aims at preventing an overestimation bias. The use of two Q-estimators was first proposed in the Double Q-learning algorithm. The two estimates are combined in the TD target such that determining the maximizing action is decoupled from computing the value for that action. Later it was proposed to use the target network (whose parameters are periodically set to the current parameters or are an exponentially weighted moving average of them) for one of the two value estimates. This eliminates the need to train two networks. While this works well for discrete actions, versions of double Qlearning adapted to the actor-critic setting were shown to still suffer from an overestimation bias. Other approaches that aim at preventing overestimation bias in Q-learning have averaged the Q-estimates obtained from snapshots of the parameters from different training steps or used a bias correction term. Balancing between over-and underestimating terms in the Q-estimates has been done for a discrete action space. The work investigates multi-armed bandit problems and an underestimation bias is reported for Double Q-learning, while Q-learning with a single estimator is reported to overestimate the true values. Similarly to our approach a weighting is introduced in the TD target. Different to us, the weighting parameter is not learned by taking actual samples for the value estimator into account, but the parameter is set individually for each state-action pair used to train the Q-network according to a function that computes the minimum and maximum Q-value over all actions for the given state. Finding these optimal actions for every transition on which the Q-networks are trained becomes infeasible for continuous action spaces. Divergence of Q-values has been investigated in several recent works van. Of them only in the case of a continuous action space is considered. In their analysis it is investigated under which conditions a certain approximation of the Q-value updates is a contraction in the sup norm. From that an algorithm is derived that does not need multiple critics or target networks. The downside is that it is very compute intensive. We consider model-free reinforcement learning for episodic tasks with continuous action spaces. An agent interacts with its environment by selecting an action a t ∈ A in state s t ∈ S for every discrete time step t. The agent receives a scalar reward r t and observes the new state s t+1. The goal is to learn a policy π: S → A that selects the agents actions in order to maximize the sum of future discounted rewards For a given state-action pair (s, a) the value function is defined as Q π (s, a):= E si∼pπ,ai∼π [R t |s, a], which is the expected return when executing action a in state s and following π afterwards. We write π φ for the policy with parameters φ, that we learn in order to maximize the expected return J(φ) = E si∼pπ,ai∼π [R 0]. The parameters can be optimized with the gradient of J w.r.t. the policy parameters φ. The deterministic policy gradient is given by In practice the value function Q π is not given and has to be approximated. This setting is called actor-critic, the policy is the actor and the learned value function has the role of a critic. The Q-learning algorithm tries to learn the value function with TD learning , which is an incremental update rule and aims at satisfying the Bellman equation. is a variant of this algorithm and can be used to learn the parameters θ of an artificial neural network Q θ: S × A → R that approximates the value function. The network is updated by regressing its value at (s t, a t) to its 1-step TD targets where Qθ, πφ are the target networks of Q φ, π θ and the corresponding parametersφ andθ are updated according to a exponential moving average:θ ← τ θ + (1 − τ)θ, similarly for φ. If a learned critic Q θ is used in the deterministic policy gradient (eq. 1), the actor π φ is updated through Q θ, which in turn is learned with the rewards obtained from the environment. This means that the actor requires a good critic to be able to learn a well performing policy. Recently, it was shown, that using Q-learning in such an actor-critic setting can lead to an overestimation of the. This is problematic if the overestimation in the critic occurs for actions that lead to low returns. To avoid this, Clipped Double Q-learning was proposed. In this approach two Q-networks (Q θ1, Q θ2) are learned in parallel. They are trained on the same TD target, which is defined via the minimum of the two Q-networks The authors note that this can lead to an underestimation bias for the Q-values, but argue that this is not as bad as overestimating. A big advantage of CDQ is that the Q-estimates do not explode, which otherwise can sometimes happen and is usually followed by a breakdown in performance. Apart from that, an over-or underestimation bias would not be problematic if all values are biased by the same constant value. It becomes a problem if the bias of the value estimates for different state-action pairs differs. Then the critic might reinforces the wrong actions. If this happens and in a given state an action is erroneously given a high value by the critic, the actor is reinforced to choose the corresponding action. This increases the probability that the agent selects that action the next time when it is in that (or a similar) state. The agent will receive a low reward, which leads to a decrease of performance. But the critic can correct itself on the new experience which will eventually be propagated through to the actor. If on the other hand, the critic underestimates the Q-value of a good action, the actor is trained to never try this action. In this case the critic might never be corrected as experience opposing the critics believe is never encountered. While this is a simplistic picture of the ongoing learning dynamics, it can give a good intuition, why both cases should be prevented if possible. It is obvious that taking the minimum over two estimates can lead to an underestimation bias. To check if this also occurs in practice, we conducted an experiment, where we examined the Q-value estimates of different agents. We trained an TD3 agent that uses CDQ as defined in eq. 3 and one TD3 agent that uses instead of CDQ the critic updates of as defined in eq. 2. We trained on three different environments from OpenAi gym. Periodically, we sampled 1000 state-action pairs from the replay-buffer and computed the value estimate of the critic. We approximated the true values for each state-action pair by rolling out the current policy 50 times from that pair onwards and averaged the observed discounted return. The for the average value of each time step are shown in the first row of Figure 1. Similarly to previous work , we observe that the DDPG-style updates of the Q-network lead to an overestimation bias. For CDQ we indeed observe an underestimation bias as the value estimates are significantly lower than the true values. We propose Balanced Clipped Double Q-learning (BCDQ), a new algorithm to learn the critic with the goal of reducing the bias. We adopt the idea of two Q-networks, but train them on different TD Figure 1: Measuring estimation bias in the Q-value estimates of DDPG, CDQ and Balanced Clipped Double Q-learning (BCDQ) on three different OpenAI gym environments. The first row shows the estimates of DDPG and CDQ and it can be seen that DDPG leads to an overestimation bias, while CDQ leads to an underestimation bias. In the second row the value estimates of BCDQ are shown. It can be observe that the BCDQ estimates are more accurate and do not exhibit a clear bias in any direction. Hopper-v3 HalfCheetah-v3 Figure 2: The plots show the average over 10 runs of the weighting parameter β for three OpenAI gym environments. targets. The TD target y k for the k-th Q-network Q θ k, k ∈ {1, 2} is defined as a weighted average of the network itself and the minimum of both networks where β ∈. The first term corresponds to the TD target according to DDPG and second term corresponds to the TD target of CDQ. While the first term tends to overestimate, the second term tends to underestimate the true Q-values. Correctly weighting between them can correct for this bias. However, setting β manually is difficult. The perfect β that maximally reduces bias may change from environment to environment and also over the time of the training process. Consequently, we adjust β over the course of the training. As the goal is to minimize bias and since β controls in which direction more bias is introduced, we use samples of the Q-values to learn β. After every episode we compute for every seen state-action pair (s t, a t) the actual discounted future return from that pair onwards R t = T i=t γ i−t r i, which is a sample for the quantity the Q-networks Q θ k (s t, a t) try to estimate. If the Q-estimates are higher than R t, they overestimated and β should be decreased to give the "min" term in eq. 4 more weight. If on the other hand the Q-estimates are lower, we observe the case of underestimation and β should be increased. This behaviour can be achieved by Initialize critic networks Q θ1, Q θ2, and actor network π φ with random parameters θ 1, θ 2, φ Initialize target networksθ 1 ← θ 1,θ 2 ← θ 2,φ ← φ, set k = 0 and β ∈ Initialize replay buffer B for t = 1 to total timesteps do Select action with exploration noise a ∼ π φ (s) +, ∼ N (0, σ) and observe reward r, new state s and binary value d indicating if the episode ended Store transition tuple (s, a, r, s, d) in B and set k ← k + 1 if k ≥ beta update rate and if t mod actor delay rate = 0 then // Update φ by the deterministic policy gradient: minimizing the following objective w.r.t. β: where we restrict β to be in the interval, E is the number of episodes we optimize over, s (j) t is the t-th state in the j-th considered episode (similarly for the actions a (j) t ), T j is the number of time steps in episode j and R (j) t are the future discounted returns. The parameter β is updated every time the sum over all time steps in the episodes since the last update, E j=1 T j, exceeds a fixed threshold. We set this threshold to be the maximum number of episode steps that are possible. To optimize β we use stochastic gradient descent. We note that learning β increases the computational complexity only minimal, as it is just one parameter that has to be optimized. To evaluate the objective in eq. 5, a further forward path through the Q-network is performed, but no backward path is needed in the training. We evaluated the accuracy of the value estimates of BCDQ and report the in the second row of Figure 1. It can be seen, that compared to the other methods BCDQ approximates the true Qvalues much better. This indicates that the weighting parameter β can indeed be adjusted over the course of the training such that the two opposing biases cancel each other out. The behaviour of β is visualized in Figure 2 as an average over 10 runs per environment. For the Hopper task it can be observed that after some time the parameter β gets very close to zero, which corresponds to using only CDQ to update the critic. For HalfCheetah, the CDQ term is not weighted very high. This is explainable as in Figure 1 it can be seen that CDQ induces a large bias on this -2.66 ± 0.17 -2.68 ± 0.16 -6.75 ± 0.48 -3.61 ± 0.23 task. Adjusting β over time allows to put more weight on the term that currently gives a more accurate estimate. This prevents the accumulation of errors introduced by bootstrapping in the TD target. From the plots it can also be seen that treating β as an hyperparameter might be difficult, as it would have to be tuned for every environment. Furthermore, leaving β fixed could not account for the changing learning dynamics over the course of training. In Figure 2 it can be seen that different drifts exist in each environment. For example in Walker2d the learned β decreases on average after an initial stabilization period. Since the two Q-networks are not trained on the same target as it is the case for CDQ, the difference between the predictions of the two Q-networks will be higher. This suggest that -similarly to ensemble methods -the average of the two predictions might be an even better estimator. Following that rationale, in our algorithm the critic that teaches the actor is the average of the predictions of the two Q-networks. As a of the above discussion, we propose the Balanced Twin Delayed Deep Deterministic policy gradient algorithm (BTD3), which builds on TD3. Differently to TD3, our algorithm uses BTDQ instead of CDQ to update the critics. For the learning of the actor the predictions of the two critics are averaged instead of using only the first critic. The BTD3 algorithm is shown in Algorithm 1. We evaluate our algorithm on a range of challenging continuous control tasks from , which makes use of the physics engine (version 2.0). To guarantee an apples-to-apples comparison with TD3, we extended the original source code of TD3 with our method and evaluate our algorithm with the default hyperparameters of TD3 for all tasks except for Humanoid-v3. We observed that TD3 does not learn a successful policy on Humanoid-v3 with the default learning rate of 0.001, but we found that TD3 does learn if the learning rate for both actor and critic is reduced. Consequently, we set it to 0.0001 for this task and did the same for BTD3. We set the learning rate for the weighting parameter β to 0.05 and initialize β = 0.5 at the beginning of the training for all environments. As is done in TD3, we reduce the dependency on the initial parameters by using a random policy for the first 10000 steps for HalfCheetah-v3, Ant-v3, Humanoid-v3 and the first 1000 steps for the remaining environments. After that period we add Gaussian noise N (0, 0.1) to each action in order to ensure enough exploration. During training the policy is evaluated every 5000 environment steps by taking the average over the episode reward obtained by rolling out the current policy without exploration noise 10 times. For each task and algorithm we average the of 10 trials each with a different random seed, except for Humanoid-v3, where we used 5 trials. We compare our algorithm to the state of the art continuous control methods SAC Haarnoja et al. (2018a) (with learned temperature parameter Haarnoja et al. (2018b) ), TD3 and to. For both, SAC and TD3, we used the source code published by Hopper-v3 Ant-v3 Humanoid-v3 Reacher-v2 The learning curves are shown in Figure 3. For all tasks BTD3 matches or outperforms TD3. Furthermore, it performs significantly better than SAC and DDPG. In Table 1 the are presented in terms of the average maximum episode reward. In order to compute that statistic, for each trial we computed the maximum over the evaluations that were executed all 5000 time steps, where the evaluations are itself the average over 10 rollouts of the current policy. Afterwards, we computed the average of this value over the different trials. The show that the best policies of BTD3 achieve significantly higher episode rewards than the best policies of the other methods. To further understand the influence of the dynamic weighting scheme we trained BTD3 with a fixed value for β. We evaluated for the values β ∈ {0.00, 0.25, 0.50, 0.75, 1.00}, where β = 0.00 corresponds to TD3 and β = 1.00 corresponds to DDPG. The averaged over 10 runs are shown in Figure 4. From the plots we can make two observations. First, it is essential that β is adjusted during the training. For any of the considered values of β leaving it fixed leads to a worse performance compared to BTD3 and in most cases also worse than TD3. In Figure 2 it was shown that the adjusted weighting parameter is on average over many runs attracted to different values depending not only on the environment but also on the timestep during training. The dynamic adjustment to prevent accumulating errors is not possible when β is fixed. Second, it is surprising that fixed values for β that would seem promising to try from Figure 2 can perform worse than other fixed values. For example inspecting the plots in Figure 2 the value β = 0.75 seems a good fit for the HalfCheetah environment. But the evaluation shows that β = 0.25 and β = 0.50 perform better. This further supports the hypothesis that the most important part about BCDQ is the dynamic adjustment of the weighting parameter. Figure 4: Learning curves for four different continuous control tasks from OpenAi gym over 10 random seeds each. The show algorithms are BTD3 and versions of it with a fixed value of β. For each algorithm the curves show the mean over 10 runs with different random seeds and are filtered with a uniform filter of size 15. We showed that Clipped Double Q-learning (CDQ) induces an underestimation bias in the critic, while an overestimation bias occurs if just one Q-network is used. From that we derived the Balanced Clipped Double Q-learning algorithm (BCDQ) that updates the critic through a weighted average of the two mentioned update mechanisms. The weighting parameter is adjusted over the course of training by comparing the Q-values of recently visited state-action pairs with the actual discounted return observed from that pair onwards. It was shown that BCDQ achieves much more accurate value estimates by adjusting the weighting parameter. Replacing CDQ with BCDQ leads to the Balanced Twin Delayed Deep Deterministic policy gradient algorithm (BTD3). Our method achieves state of the art performance on a range of continuous control tasks. Furthermore, BCDQ can be added to any other actor-critic algorithm while it only minimally increases the computational complexity compared to CDQ. It is also be possible to use BCDQ for discrete action spaces. Evaluating that approach is an interesting area for future research.

Represent the short summary for finding the corresponding scientific article (long)

A method for more accurate critic estimates in reinforcement learning.

represent this text to find a much much longer article about it with say hundreds-thousands of words

We augment the Q-value estimates with a count-based bonus that ensures optimism during action selection and bootstrapping, even if the Q-value estimates are pessimistic.


Question: Represent the scientific article to find a short TLDR:

Question: We introduce a systematic framework for quantifying the robustness of classifiers to naturally occurring perturbations of images found in videos. As part of this framework, we construct ImageNet-Vid-Robust, a human-expert--reviewed dataset of 22,668 images grouped into 1,145 sets of perceptually similar images derived from frames in the ImageNet Video Object Detection dataset. We evaluate a diverse array of classifiers trained on ImageNet, including models trained for robustness, and show a median classification accuracy drop of 16\%. Additionally, we evaluate the Faster R-CNN and R-FCN models for detection, and show that natural perturbations induce both classification as well as localization errors, leading to a median drop in detection mAP of 14 points. Our analysis shows that natural perturbations in the real world are heavily problematic for current CNNs, posing a significant challenge to their deployment in safety-critical environments that require reliable, low-latency predictions. Despite their strong performance on various computer vision benchmarks, convolutional neural networks (CNNs) still have many troubling failure modes. At one extreme,`padversarial examples can cause large drops in accuracy for state of the art models with visually imperceptible changes to the input image BID4. But since carefully crafted`pperturbations are unlikely to occur naturally in the real world, they usually do not pose a problem outside a fully adversarial context. To study more realistic failure modes, researchers have investigated benign image perturbations such as rotations & translations, colorspace changes, and various image corruptions. However, it is still unclear whether these perturbations reflect the robustness challenges commonly arising in real data since the perturbations also rely on synthetic image modifications. Recent work has therefore turned to videos as a source of naturally occurring perturbations of images [6, BID0 . In contrast to other failure modes, the perturbed images are taken from existing image data without further modifications that make the task more difficult. As a , robustness to such perturbations directly corresponds to performance improvements on real data. However, it is currently unclear to what extent such video perturbations pose a significant robustness challenge. Azulay and Weiss BID0 only provide anecdotal evidence from a small number of videos. While work with a larger video dataset to obtain accuracy estimates, they only observe a small drop in accuracy of around 2.7% on videoperturbed images, suggesting that small perturbations in videos may not actually reduce the accuracy of current CNNs significantly. We address this question by conducting a thorough evaluation of robustness to natural perturbations arising in videos. As a cornerstone of our investigation, we introduce ImageNet-Vid-Robust, a carefully curated subset of ImageNet-Vid. In contrast to earlier work, all images in ImageNet-Vid-Robust were screened by a set of expert labelers to ensure a high annotation quality and to minimize selection biases that arise when filtering with CNNs. Overall, ImageNet-Vid-Robust contains 22,668 images grouped into 1,145 sets of temporally adjacent and visually similar images of a total of 30 classes. We then utilize ImageNet-Vid-Robust to measure the accuracy of current CNNs to small, naturally occurring perturbations. Our testbed contains over 40 different model types, varying both architecture and training methodology (adversarial training, data augmentation, etc). We find that natural perturbations from ImageNet-Vid-Robust induce a median 16% accuracy drop for classification tasks and a median 14% drop in mAP for detection tasks. Even for the best-performing model, we observe an accuracy drop of 14% -significantly larger than the 2.7% drop in over the same time horizon in the video. Our show that robustness to natural perturbations in videos is indeed a significant challenge for current CNNs. As these models are increasingly deployed in safety-critical environments that require both high accuracy and low latency (e.g., autonomous vehicles), ensuring reliable predictions on every frame of a video is an important direction for future work. The ImageNet-Vid-Robust dataset is sourced from videos contained in the ImageNet-Vid dataset, we provide more details about ImageNet-Vid in the supplementary. Next, we describe how we extracted neighboring sets of naturally perturbed frames from ImageNet-Vid to create ImageNet-Vid-Robust. A straightforward approach is to select a set of anchor frames and use nearby frames in the video with the assumption that such frames contain only small perturbations from the anchor frame. However, as FIG0 in the supplementary illustrates, this assumption is frequently broken, especially in the presence of fast camera or object motion. Instead, we collect a preliminary dataset of natural perturbations and then we manually review each of the frame sets. For each video, we first randomly sample an anchor frame and then take k = 10 frames before and after the anchor frame as candidate perturbation images. This in a dataset containing 1 anchor frame each from 1,314 videos, with approximately 20 candidate perturbation frames each BID0.Next, we curate the dataset with the help of four expert human annotators. The goal of the curation step is to ensure that each anchor frame and nearby frame is correctly labeled with the same ground truth class and that the anchor frame and the nearby frame are visually similar. For each pair of anchor and candidate perturbation frame, an expert human annotator labels whether the pair is correctly labeled in the dataset, whether the pair is similar. Asking human annotators to label whether a pair of frames is similar can be highly subjective. We took several steps to mitigate this issue and ensure high annotation quality. First, we trained reviewers to mark frames as dissimilar if the scene undergoes any of the following transformations: significant motion, significant change, or significant blur change, and additionally asked reviewers to mark each of the dissimilar frames with one of these transformations, or "other". Second, as presenting videos or groups of frames to reviewers could cause them to miss potentially large changes due to the well-studied phenomenon of change blindness, we present only a single pair of frames at a time to reviewers. Finally, to increase consistency in annotation, human annotators proceed using two rounds of review. In the first round, all annotators were given identical labeling instructions, and then individually reviewed 6500 images pairs. We instructed annotators to err on the side of marking a pair of images as dissimilar if a BID0 Note that some anchor frames may have less than 20 candidate frames if the anchor frame is near the start or end of the video. distinctive feature of the object is only visible in one of the two frames (such as the face of a dog). If an annotator was unsure about a pair he or she could mark the pair as "don't know".For the second round of review, all annotators jointly reviewed all frames marked as dissimilar, "don't know" or "incorrect". A frame was only considered similar if a strict majority of the annotators marked the pair of as "similar".After the reviewing was complete, we discarded all anchor frames and candidate perturbations that annotators marked as dissimilar or incorrectly labeled. Our final dataset contains 1,145 anchor frames with a minimum of 1, maximum of 20 and median of 20 similar frames. Given the dataset above, we would like to measure a model's robustness to natural perturbations. In particular, let A = {a 1, ..., a n} be the set of valid anchor frames in our dataset. Let Y = {y 1, ..., y n} be the set of labels for A. We let N k (a i) be the set of frames marked as similar to anchor frame a i. In our setting N k is a subset of the 2k temporally adjacent frames (plus/minus k frames from anchor).The pm-k analogues of the standard metrics for detection and classification evaluate only on the worst-case frame in the set of N k. We formally define the pm-k analogues for the standard metrics for classification and detection (acc pmk and mAP pmk) in the supplementary. We evaluate a testbed of 50 classification models and 3 state of the art detection models on ImageNet-Vid-Robust. We first discuss the various types of classification models evaluated with pm-k classification metric. We then study the per-class accuracies to study whether our perturbations exploits a few "hard" classes or affects performance uniformly across classes. Second we use the bounding box annotations inherited from ImageNet-VID to study the effect of detection models evaluated on ImageNet-Vid-Robust using the pm-k metric. We then analyze the errors made on the detection adversarial examples to isolate the effects of localization errors vs classification errors. In FIG1, we plot acc orig versus acc pmk for all classification models in our test bed and find that there is a surprisingly linear relationship between acc orig and acc pmk across all 48 models in our test bed. We note the similarity of this plot to FIG0 in BID12.1578 out 22668 frames in ImageNet-Vid-Robust have multiple correct classification labels, due to multiple objects in the frame. To handle this in a classification set- Each data point corresponds to one model in our testbed (shown with 95% Clopper-Pearson confidence intervals). Each "perturbed" frame was taken from a neighborhood of a maximum 10 adjacent frames to the original frame in a 30 FPS video. This allows the scene to change for roughly 0.3s. All frames were reviewed by humans to confirm visual similarity to the original frames.ting, we opt for the most conservative approach: we count a prediction as correct if the model predicts any of the classes for a frame. We note that this is a problem that plagues many classification datasets, where objects of multiple classes can be in an image BID12 but there is only one true label. We considered 5 models types of increasing levels of supervision. We present our full table of classification accuracies in the supplementary material, and for representative models from each model type in Table 1.ILSVRC Trained As mentioned in??, leveraging the WordNet hierarchy enables evaluating models available from trained on the 1000 class ILSVRC challenge on images in ImageNet-Vid-Robust directly. We exploit this to evaluate a wide array of model architectures against our natural perturbations. We note that this test set is a substantial distribution shift from the original ILSVRC validation set that these models are tuned for. Thus we will expect the benign accuracy acc orig to be lower than the comparable accuracy on the ILSVRC validation set. However the quantity of interest for this experiment is the difference between the original and perturbed accuracies accuracies acc origacc pmk, which should be less sensitive to an absolute drop in acc orig.ILSVRC Trained with Noisy Augmentation One hypothesis for the accuracy drop is that subtle artifacts and corruptions introduced by video compression schemes could introduce a large accuracy drop when evaluated on these corrupted frames. The worst-case nature of the pm-k metric could be biasing evaluation towards these corrupt frames. One model for these corruptions are the perturbations introduced in. To test this hypothesis we evaluate models augmented with a subset of the perturbations (Gaussian noise Gaussian blur, shot noise, contrast change, impulse noise, JPEG compression) found in. We found that this augmentation scheme did little to help robustness against our perturbations. ILSVRC Trained for L 2 /L 1 Robustness We evaluate the best performing robust model against the very strong L 2 /L 1 attacks. We find that this model does have a slightly smaller performance drop than both ILSVRC and ILSVRC trained with noise augmentation but the difference is well within the error bars induced by the small size of our evaluations set. We also note that this robust model gets significantly lower original and perturbed accuracy than examples from either of the model types above. ILSVRC Trained + Finetuned on ImageNet-VID To adapt to the 30 class problem and the different domain of videos we fine tune several network architectures on the training set in ImageNet VID. We start with a base learning rate of 1e 4 and train with the SGD optimizer until the validation accuracy plateaus. We trained using cross entropy loss using the largest object in the scene as the label during training, as we found this performed better than training using a multi-label loss function. After training for 10 epochs we evaluate on ImageNet-Vid-Robust. These models do improve in absolute accuracy over their ILSVRC pretrained counterparts (12% for a ResNet50). However, this improvement in absolute accuracy does not significantly decrease the accuracy drop induced by natural perturbations. Finally, we analyze whether additional supervision, in the form of bounding box annotations, improves robustness. To this end, we train the Faster R-CNN detection model with a ResNet 50 backbone on ImageNet Vid. Following standard practice, the detection backbone is pre-trained on ILSVRC. To evaluate this detector for classification, we assign the score for each label for an image as the score of the most confident bounding box for that label. We find that this transformation reduces accuracy compared to the To analyze the generalizability of natural perturbations to other tasks, we next analyze their impact on the object localization and detection tasks. We report for two related tasks: object localization and detection. Object detection is the standard computer vision task of correctly classifying an object and regressing the coordinates of a tight bounding box containing the object. " Object localization", meanwhile, refers to the only the subtask of regressing to the bounding box, without attempting to correctly classify the object. This is an important problem from a practical perspective (for example, the size and location of an obstacle may be more important for navigation than the category), as well as from an analytical perspective, as it allows analyzing mistakes orthogonal to classification errors. For example, it may be the case that natural perturbations cause misclassification errors frequently, as it may be natural to mistake a cat for a fox, but cause few localization errors. We present our using the popular Faster R-CNN and R-FCN architectures for object detection and localization in TAB2. We first note the significant drop in mAP of 12 15% for object detection due to perturbed frames for both the Faster R-CNN and R-FCN architectures. Next, we show that localization is indeed easier than detection, as the mAP increases significantly (e.g., from 61.8 to 75.5 for Faster R-CNN with ResNet 50 backbone). Perhaps surprisingly, however, switching to the localization task does not improve the delta between original and perturbed frames, indicating that natural perturbations induce both classification and localization errors. An advantage of using the ImageNet-Vid dataset as the source of our dataset is that all 30 object Anchor frame Discarded frame Anchor frame Anchor frame Discarded frame Discarded frame FIG0: Temporally adjacent frames may not be visually similar. We visualize three randomly sampled frame pairs where the nearby frame was marked during human review as "dissimilar" to the anchor frame and discarded from our dataset. Classification accuracy is defined as: DISPLAYFORM0 Submitted to 33rd Conference on Neural Information Processing Systems (NeurIPS 2019). Do not distribute. as: DISPLAYFORM1 Which simply corresponds to picking the worst frame from the each N k (a i) set before computing 20 misclassification accuracy. Detection The standard metric for detection is mean average precision of the predictions at a fixed ). We define the pm-k analog of mAP by replacing each anchor frame in the dataset with a nearby 30 frame that minimizes the per-image average precision. Note that as the category-specific average 31 precision is undefined for categories not present in an image, we minimize the average precision 32 across categories for each frame rather than the mAP. We then define the pm-k mAP as follows, with 33 a slight abuse of notation to denote y b as the label for frame b: DISPLAYFORM0 In FIG1, we plot the relationship between perturbed accuracy and and perturbation distance (i.e the 36 k in the pm-k metric described in Section 3). We note that the entire x-axis in FIG1 corresponds 37 to a temporal distance of 0s to 0.3s between the original and perturbed frames. We study the effect of our peturbations on the 30 classes found in ImageNet-Vid-Robust to 40 determine whethre our performance drop was concentrated in a few "hard" classes. "artificial" nature of L p attacks, recent work has proposed more realistic modifications to images. Engstrom et. al. BID4 study an adversary that performs minor rotations and translations of the input, Hosseni et. al. BID12 1e-5 for all models. We additionally detail hyperparameters for detection models in

Represent the TLDR for finding the corresponding scientific article

We introduce a systematic framework for quantifying the robustness of classifiers to naturally occurring perturbations of images found in videos.

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: Represent the scientific article to find a short TLDR

Question: Structured tabular data is the most commonly used form of data in industry according to a Kaggle ML and DS Survey. Gradient Boosting Trees, Support Vector Machine, Random Forest, and Logistic Regression are typically used for classification tasks on tabular data. The recent work of Super Characters method using two-dimensional word embedding achieved state-of-the-art in text classification tasks, showcasing the promise of this new approach. In this paper, we propose the SuperTML method, which borrows the idea of Super Characters method and two-dimensional embedding to address the problem of classification on tabular data. For each input of tabular data, the features are first projected into two-dimensional embedding like an image, and then this image is fed into fine-tuned ImageNet CNN models for classification. Experimental have shown that the proposed SuperTML method have achieved state-of-the-art on both large and small datasets. In data science, data is categorized into structured data and unstructured data. Structured data is also known as tabular data, and the terms will be used interchangeably. Anthony Goldbloom, the founder and CEO of Kaggle observed that winning techniques have been divided by whether the data was structured or unstructured BID12. Currently, DNN models are widely applied for usage on unstructured data such as image, speech, and text. According to Anthony, "When the data is unstructured, its definitely CNNs and RNNs that are carrying the day" BID12. The successful CNN model in the ImageNet competition BID8 has outperformed human Preliminary work. Under review by the International Conference on Machine Learning (ICML). Do not distribute.for image classification task by ResNet BID6 since 2015.On the other side of the spectrum, machine learning models such as Support Vector Machine (SVM), Gradient Boosting Trees (GBT), Random Forest, and Logistic Regression, have been used to process structured data. According to a recent survey of 14,000 data scientists by , a subdivision of structured data known as relational data is reported as the most popular type of data in industry, with at least 65% working daily with relational data. Regarding structured data competitions, Anthony says that currently XGBoost is winning practically every competition in the structured data category BID4. XGBoost BID2 is one popular package implementing the Gradient Boosting method. Recent research has tried using one-dimensional embedding and implementing RNNs or one-dimensional CNNs to address the TML (Tabular Machine Learning) tasks, or tasks that deal with structured data processing BID7 BID11, and also categorical embedding for tabular data with categorical features BID5. However, this reliance upon onedimensional embeddings may soon come to change. Recent NLP research has shown that the two-dimensional embedding of the Super Characters method BID9 is capable of achieving state-of-the-art on large dataset benchmarks. The Super Characters method is a two-step method that was initially designed for text classification problems. In the first step, the characters of the input text are drawn onto a blank image. In the second step, the image is fed into two-dimensional CNN models for classification. The two-dimensional CNN models are trained by fine-tuning from pretrained models on large image dataset, e.g. ImageNet. In this paper, we propose the SuperTML method, which borrows the concept of the Super Characters method to address TML problems. For each input, tabular features are first projected onto a two-dimensional embedding and fed into fine-tuned two-dimensional CNN models for classification. The proposed SuperTML method handles the categorical type and missing values in tabular data automatically, without need for explicit conversion into numerical type values. The SuperTML method is motivated by the analogy between TML problems and text classification tasks. For any sample given in tabular form, if its features are treated like stringified tokens of data, then each sample can be represented as a concatenation of tokenized features. By applying this paradigm of a tabular sample, the existing CNN models used in Super Characters method could be extended to be applicable to TML problems. As mentioned in the introduction, the combination of twodimensional embedding (a core competency of the Super Characters methodology) and pre-trained CNN models has achieved state-of-the-art on text classification tasks. However, unlike the text classification problems studied in BID9, tabular data has features in separate dimensions. Hence, generated images of tabular data should reserve some gap between features in different dimensions in order to guarantee that features will not overlap in the generated image. SuperTML is composed of two steps, the first of which is two-dimensional embedding. This step projects features in the tabular data onto the generated images, which will be called the SuperTML images in this paper. The conversion of tabular training data to SuperTML image is illustrated in Figure 1, where a collection of samples containing four tabular features is being sorted. The second step is using pretrained CNN models to finetune on the generated SuperTML images. Figure 1 only shows the generation of SuperTML images for the training data. It should be noted that for inference, each instance of testing data goes through the same preprocessing to generate a SuperTML image (all of which use the same configuration of two-dimensional embedding) before getting fed into the CNN classification model. the generated SuperTML images. 9: return the trained CNN model on the tabular data known as SuperTML VF, is described in Algorithm 1.To make the SuperTML more autonomous and remove the dependency on feature importance calculation done in Algorithm 1, the SuperTML EF method is introduced in Algorithm 2. It allocates the same size to every feature, and thus tabular data can be directly embedded into SuperTML images without the need for calculating feature importance. This algorithm shows even better than 1, which will be described more in depth later in the experimental section. The data statistics from UCI Machine Learning Repository is shown in TAB2. " This is perhaps the best known database to be found in the pattern recognition literature"1. The Iris dataset is widely used in machine learning courses and tutorials. FIG1 shows an example of a generated SuperTML image, created using Iris data. The experimental of using SEnet-154 shown in Table 2 for each feature of the sample do 3:Draw the feature in the same font size without overlapping, such that the total features of the sample will occupy the image size as much as possible. For this dataset 2, we use SuperTML VF, which gives features different sizes on the SupterTML image according to their importance score. The feature importance score is obtained using the XGBoost package BID2. One example of a SuperTML image created using data from this dataset is shown in FIG1. The in Table 2 shows that the SuperTML method obtained a slightly better accuracy than XGBoost on this dataset. The task of this Adult dataset 3 is to predict whether a persons income is larger or smaller than 50,000 dollars per year based on a collection of surveyed data. For categorical features that are represented by strings, the Squared English Word (SEW) method BID10 Figure 3. SuperTML VF image example from Adult dataset. This sample has age = 59, capital gain = 0, capital loss = 0, hours per week = 40, fnlweight = 372020, education number = 13, occupation = "?" (missing value), marital status = "Married-civ-spouse", relationship = "Husband", workclass = "?" (missing value), education = "Bachelors", sex = "Male", race = "White", native country = "United-States".is used. One example of a generated SuperTML image is given in Figure 3. Table 2 shows the on Adult dataset. We can see that on this dataset, the SuperTML method still has a higher accuracy than the fine-tuned XGBoost model, outperforming it by 0.32% points of accuracy. The Higgs Boson Machine Learning Challenge involved a binary classification task to classify quantum events as signal or . It was hosted by Kaggle, and though the contest is over, the challenge data is available on opendata BID1. It has 25,000 training samples, and 55,000 testing samples. Each example has 30 features, each of which is stored as a real number value. In this challenge, AMS score ( is used as the performance metric. FIG3 shows two examples of generated SuperTML images. TAB3 shows the comparison of different algorithms. The DNN method and XGBoost used in the first two rows are using the numerical values of the features as input to the models, which is different from the SuperTML method of using two-dimensional embeddings. It shows that SuperTML EF method gives the best AMS score of 3.979. In addition, the SuperTML EF gives better than SuperTME VF , which indicates SuperTML method can work well without the calculation of the importance scores. The proposed SuperTML method borrows the idea of twodimensional embedding from Super Characters and transfers the knowledge learned from computer vision to the structured tabular data. Experimental shows that the proposed SuperTML method has achieved state-of-the-art on both large and small tabular dataset TAB2

represent the tldr for finding the corresponding scientific article

Deep learning for structured tabular data machine learning using pre-trained CNN model from ImageNet.

Represent the short summary for finding the corresponding scientific article (long)