Question: Represent the article for finding a single-sentence TLDR

Question: GANs provide a framework for training generative models which mimic a data distribution. However, in many cases we wish to train a generative model to optimize some auxiliary objective function within the data it generates, such as making more aesthetically pleasing images. In some cases, these objective functions are difficult to evaluate, e.g. they may require human interaction. Here, we develop a system for efficiently training a GAN to increase a generic rate of positive user interactions, for example aesthetic ratings. To do this, we build a model of human behavior in the targeted domain from a relatively small set of interactions, and then use this behavioral model as an auxiliary loss function to improve the generative model. As a proof of concept, we demonstrate that this system is successful at improving positive interaction rates simulated from a variety of objectives, and characterize s Generative image models have improved rapidly in the past few years, in part because of the success of Generative Adversarial Networks, or GANs BID2. GANs attempt to train a "generator" to create images which mimic real images, by training it to fool an adversarial "discriminator," which attempts to discern whether images are real or fake. This is one solution to the difficult problem of learning when we don't know how to write down an objective function for image quality: take an empirical distribution of "good" images, and try to match it. Often, we want to impose additional constraints on our goal distribution besides simply matching empirical data. If we can write down an objective which reflects our goals (even approximately), we can often simply incorporate this into the loss function to achieve our goals. For example, when trying to generate art, we would like our network to be creative and innovative rather than just imitating previous styles, and including a penalty in the loss for producing recognized styles appears to make GANs more creative BID1. Conditioning on image content class, training the discriminator to classify image content as well as making real/fake judgements, and including a loss term for fooling the discriminator on class both allows for targeted image generation and improves overall performance BID9.However, sometimes it is not easy to write an explicit objective that reflects our goals. Often the only effective way to evaluate machine learning systems on complex tasks is by asking humans to determine the quality of their (, e.g.) or by actually trying them out in the real world. Can we incorporate this kind of feedback to efficiently guide a generative model toward producing better ? Can we do so without a prohibitively expensive and slow amount of data collection? In this paper, we tackle a specific problem of this kind: generating images that cause more positive user interactions. We imagine interactions are measured by a generic Positive Interaction Rate (PIR), which could come from a wide variety of sources. For example, users might be asked to rate how aesthetically pleasing an image is from 1 to 5 stars. The PIR could be computed as a weighted sum of how frequently different ratings were chosen. Alternatively, these images could be used in the of web pages. We can assess user interactions with a webpage in a variety of ways (time on page, clicks, shares, etc.), and summarize these interactions as the PIR. In both of these tasks, we don't know exactly what features will affect the PIR, and we certainly don't know how to explicitly compute the PIR for an image. However, we can empirically determine the quality of an image by actually showing it to users, and in this paper we show how to use a small amount of this data ( on 1000 images) to efficiently tune a generative model to produce images which increase PIR. In this work we focus on simulated PIR values as a proof of concept, but in future work we will investigate PIR values from real interactions. The most straight-forward way to improve an image GAN might be to evaluate the images the model produces with real users at each training step. However, this process is far too slow. Instead, we want to be able to collect a batch of PIR data on a batch of images, and then use this batch of data to improve the generative model for many gradient steps; we want to do this despite the fact that the images the generator is producing may evolve to be very different from the original images we collected PIR data on. In order to do this, we use the batch of image and PIR data to train a "PIR Estimator Model" which predicts PIRs on images. We then use these estimated PIRs at each step as a loss. Our approach is inspired by the work of Christiano and colleagues BID0, who integrated human preference ratings between action sequences into training of a reinforcement learning model by using the preference data to estimate a reward function. However, our problem and approach differ in several key ways. First, we are optimizing a generative image model rather than a RL model. This is more difficult in some ways, since the output space is much higher-dimensional than typical RL problems, which means that scalar feedback (like a PIR) may be harder for the system to learn from. This difficulty is partially offset by the fact that we assume we get "reward" (PIR) information for an image when we evaluate, instead of just getting preferences which we have to map to rewards. Perhaps most importantly, we use our PIR estimation model as a fully-differentiable loss function for training, instead of just using its estimated rewards. This allows us to more effectively exploit its knowledge of the objective function (but risks overfitting).Our system consists of three components: A generative image model, users who interact with the generated images in some way, and a PIR estimator that models user interactions given an image. See FIG0 for a diagram of the system's general operation. The generative model produces images, which are served to users. Using interaction data from these users, we train the PIR estimator model, which predicts PIRs given a image, and then incorporate this estimated PIR into the loss of the generative model to tune it to produce higher quality images. Below, we discuss each of these components in more detail. We begin with a GAN 1 BID2 which we pre-trained to produce images from a target distribution (specifically, landscapes of mountains and coasts). Let D source be the source estimated by the discriminator, G the generator, and z a noise input to the generator sampled from a multivariate standard normal distribution N (0, I), and I be the set of real images shown to the discriminator. Define: DISPLAYFORM0 Then the discriminator and generator are trained to maximize the following losses (respectively), where the w * are weights set as hyperparameters: DISPLAYFORM1 Note that there is a difference between these losses and the standard GAN formulation given in BID2 ) -we maximize L fake image fools = log P (classified real) rather than minimizing log (1 − P (classified real)). This seems to in the generation of slightly better images in practice. This GAN was trained on a dataset consisting of landscape images of mountains and coastlines (see Appendix C.2 for details of the architecture and training). It is worth noting that this generative model is not photorealistic (see FIG2 for some samples). Its expressive capacity is limited, and it has clear output modes with limited intra-mode variability. However, for our purposes this may not matter. Indeed, it is in some ways more interesting if we can tweak this model to optimize for many objective functions, since its limited expressive capacity will make it more difficult for us to estimate and pursue the real objective -a limited set of images will effectively give us fewer points to estimate the PIR function from, and will reduce the space in which the model can easily produce images, thus reducing the possibility of getting very optimal images from the model. For example, a model which produces images of birds may not produce data points which provide good estimates of a PIR based on how much the image looks like a car, and even if it could, it may not be able to produce images which are "more car-like." If we are able to succeed in improving PIRs with this generative model, it is likely that a better generative model would yield even better . We will show these images to users in a variety of ways, depending on our target domain. For the purposes of this paper, however, we will use simulated interaction data (see Section 3 for details). Of course, since showing images to users is an expensive prospect, we wanted to limit the size of the datasets we used to train the model. Typical datasets used to train vision models are on the order of millions of images, (e.g. ImageNet BID11), but it is completely infeasible to collect user data on this number of images. We estimated that we could show 1000 images each 1000 times to generate our datasets. We used these dataset sizes and number of impressions for all experiments discussed here, and added noise to the PIRs that was binomially distributed according to the number of times each image was shown and the "true" PIR simulate from the objective. The final component of our system is the PIR estimator model, which learns to predict PIR from a image. We denote this model by R: image →. We parameterize this model as a deep neural network. Specifically, we take the Inception v2 architecture BID13, remove the output layer, and replace it with a fully-connected layer to PIR estimates. We initialize the Inception v2 parameters from a version of the model trained on [dataset redacted for blind review]. See Appendix C.3 for more details. Why did we not make estimated PIR simply another auxiliary output from the discriminator, like class in the ACGAN (Appendix C.1)? Because the PIR estimator needs to be held constant in order to provide an accurate training objective. If the PIR estimates were produced by the discriminator, then as the discriminator changed to accurately discriminate the evolving generator images, the PIR estimates would tend to drift without a ground-truth to train them on. Separating the discriminator and the PIR estimator allows us to freeze the PIR estimator while still letting the discriminator adapt. Once we have trained a PIR estimator model, we have to use it to improve the GAN. We do this as follows. Let R denote the PIR estimator model, as above. Define L PIR to be the expectation of the estimated PIR produced over images sampled from the generator: DISPLAYFORM0 Then we simply supplement the generator loss by adding this term times a weight w P IR, set as a hyperparameter: DISPLAYFORM1 We set w P IR = 1000 as this made the magnitude of the PIR loss and the other loss terms roughly comparable. Otherwise we used the same parameters as in the GAN training above, except that we reduced the learning rate to 10 −6 to allow the system to adapt more smoothly to the multiple objectives, and we trained for 50,000 steps. In this paper, we use simulated interaction data as proof of concept. This raises an issue: what functions should we use to simulate user interactions? Human behavior is complex, and if we already knew precisely what guided user interactions, there would be no need to actually collect human behavioral data at all. Since we don't know what features will guide human behavior, the next best thing we can do is to ensure that our system is able to alter the image generation model in a broad variety of ways, ranging from low level features (like making the images more colorful) to altering complex semantic features (such as including more plants in outdoor scenery). We also want to avoid hand-engineering tasks to the greatest extent possible. We present an overview of our approaches to simulating PIR data below, see Appendix C.4 for more details. The first approach we took to evaluating our system's ability to train for different features was to use activity from hidden layers of a computer vision model, specifically VGG 16 BID12 ) trained on ImageNet BID11. In particular, we took the activity of a single filter in a layer of VGG relative to the overall activity of that layer. This approach to simulating PIRs has several benefits. First, it gives a wide variety of complex objectives that can nevertheless be easily computed to simulate data. Second, models like VGG exhibit hierarchical organization, where lower levels generally respond to lower-level features such as edges and colors, while higher levels respond to higher-level semantic features such as faces BID15, and the represented features relate to those in human and macaque visual cortex BID14. Thus VGG features give a wide range of objectives which we may relate to the human perception we wish to target. There are some caveats to this approach, however. First, although the higher layers of CNNs are somewhat selective for "abstract" object categories, they are also fooled by adversarial images that humans would not be, and directly optimizing inputs for these high level features does not actually produce semantically meaningful images BID6. Thus, even if our system succeeds in increasing activity in a targeted layer which is semantically selective, it will likely do so by adversarially exploiting particulars of VGG 16's parameterization of the classification problem (although the fact that we are not backpropagating through the true objective will make this harder). It is not necessarily a failure of the system if it exploits simple features of the objective it is given to increase PIRs -indeed, it should be seen as a success, as long as it is generalizes to novel images. However, success on this task does not necessarily guarantee success on modifying semantic content when interacting with actual humans. It may be easier for the PIR estimator model (which is based on a CNN) to learn objectives which come from another CNN than more general possible objectives. The fact that adversarial examples can sometimes transfer between networks with different architectures BID5 suggests that the computations being performed by these networks are somewhat architecture invariant. Thus CNN objectives may be easier for our estimator than human ones. We have tried to minimize these problems to the greatest extent possible by using different network architectures (Inception V2 and VGG 16, respectively) trained on different datasets ([hidden] and ImageNet BID11, respectively) for the estimator and the objective. However, we cannot be certain that the network is not "cheating" in some way on the VGG 16 tasks, so our must be considered with this qualification in mind. Despite this, we think that evaluating our system's ability to optimize for objectives generated from various layers of VGG will show its ability to optimize for a variety of complex objectives, and thus will serve as a useful indicator of its potential to improve PIRs from real users. After using our system on the tasks above, we noted that its performance was quite poor at layers 5, 6, and 7 of VGG compared to other tasks (see FIG1). This could suggest that our system was unable to capture the complex features represented at the higher levels of VGG. However, we also noticed that the feature representations at these layers tended to be quite sparse, so many of the simulated PIRs we generated were actually zero to within the bin width of our PIR estimator (see Appendix B FIG7 for a plot of how this affected learning). In order to evaluate whether the poor performance of our system at the higher layers of VGG was due to the number of zeros or to the complexity of the features, we created less sparse features from these layers by simply targeting a set of k filters sampled without replacement from the layer, rather than a single filter. The single filter cases above can be thought of as a special case of this, where k = 1. To complement these, we also tried k = 20.This can also be thought of as perhaps a more realistic simulation of human behavior, in the sense that it is highly unlikely that there is a single feature which influences human PIRs. Rather, there are probably many related features which influence PIR in various ways. Thus it is important to evaluate our system's ability to target these types of features as well. Finally, we also considered some simpler objectives based on targeting specific colors in the output images, or targeting vertical bands of two different colors, one in each half of the image, or three colors, one in each third of the image. These objectives provide a useful complement to the VGG objectives above. Although the single color objectives may be relevant to the classification task VGG 16 performs, the split color tasks are less likely to be relevant to classification. Note that it is important that we split the images along the width instead of the height dimension, as there may well be semantically relevant features corresponding to color divisions along the height dimension, e.g. a blue upper half and green lower half likely correlates with outdoor images, which would provide useful class information. By contrast, it is harder to imagine circumstances where different colors on the left and right halves of the image are semantically predictive, especially since flipping left to right is usually included in the data augmentation for computer vision systems. Thus success on optimizing for these objectives would increase our confidence in the generality of our system. We present our in terms of the change in mean PIR from 1000 images produced by the GAN before tuning to 1000 images produced after tuning, or in terms of the effect size of this change (Cohen's d, i.e. the change in mean PIR standardized by the standard deviation of the PIRs in the pre-and post-tuning image sets). We assess whether these changes are significant by performing a Welch's t-test (with a significance threshold of α = 0.001) between the pre-and post-tuning PIRs. Overall, our system was quite successful at improving PIRs across a range of simulated objective functions (see FIG1). Below, we discuss these in more detail. Our system largely succeed at increasing PIRs on a variety of VGG objectives (see FIG1). However, there are several interesting patterns to note. First, the system is not particularly successful at targeting single filters from the pool5, fc6, and fc7 layers. However, we believe this is due to the fact that filters in these layers produce relatively sparse activation, see section 3.2. Indeed, the performance of the system seems much more consistent when it is optimizing for sets of 20 filters than for single filters. Even when using 20 filters, however, there is a noticeable decline in the effect size of the improvement the system is able to make at higher layers of VGG (β = −0.13 per layer, t = −7.8, p < 10 −10 in a linear model controlling for initial standard deviation and percent zeros). This suggests that as the objectives grow more complex, the system may be finding less accurate approximations to them. However, the system's continuing (if diminished) success at the higher layers of VGG suggests that our model is capable of at least partially capturing complex objective functions. Overall, the system performed quite well at optimizing for the color objectives, particularly the single and two-color (see FIG1). It had more difficulty optimizing for the three-color , and indeed had produced only very small improvements after the usual 50,000 tuning steps for the generative model, but after 500,000 steps it was able to produce significant improvements for two out of the three objectives (these longer training are the ones included here).Because the color objectives are easiest to assess visually, we have included for a variety of these objectives in FIG2. For the single color objectives, the improvement is quite clear, for example the images in FIG2 appear much more blue than the pre-training ones. For the two color objectives, it appears that the system found the "trick" of reducing the third color, for example the red-green split images in FIG2 appear much less blue than the pre-training images. Even on the three-color images where the system struggled, there are some visible signs of improvement, for example on the green-blue-red task the system has started producing a number of images with a blue streak in the middle. We also conducted several supplemental analyses which can be found in detail Appendix A. In summary, the initial variability in the PIR of the images used to train the system is strongly correlated with the amount of improvement the system makes in the PIR, the system fairly consistently underestimates the performance it achieves (because of a detail of training procedure, see the Appendix), and iterating the process of improving PIRs yields better for objectives from a lower layer of VGG but not a higher. Overall, our system appears to be relatively successful. It can optimize a generative model to produce images which target a wide variety of objectives, ranging from low-level visual features such as colors and early features of VGG to features computed at the top layers of VGG. This success across a wide variety of objective functions allows us to be somewhat confident that our system will be able to achieve success in optimizing for real human interactions. Furthermore, the system did not require an inordinate amount of training data. In fact, we were able to successfully estimate many different objective functions from only 1000 images, several orders of magnitude fewer than is typically used to train CNNs for vision tasks. Furthermore, these images came from a very biased and narrow distribution (samples from our generative model) which is reflective of neither the images that were used to pre-train the Inception model in the PIR estimator, nor the images the VGG model (which produced the simulated objectives) was trained on. Our success from this small amount of data suggests that not only will our system be able to optimize for real human interactions, it will be able to do so from a feasible number of training points. These are exciting -the model is able to approximate apparently complex objective functions from a small amount of data, even though this data comes from a very biased distribution that is unrelated to most the objectives in question. But what is really being learned? In the case of the color images, it's clear that the model is doing something close to correct. However, for the objectives derived from VGG we have no way to really assess whether the model is making the images better or just more adversarial. For instance, when we are optimizing for the logit for "magpie," it's almost certainly the case that the of this optimization will not look more like a magpie to a human, even if VGG does rate the images as more "magpie-like." On the other hand, this is not necessarily a failure of the system -it is accurately capturing the objective function it is given. What remains to be seen is whether it can capture how images influence human behavior as well as it can capture the vagaries of deep vision architectures. We believe there are many domains where a system similar to ours could be useful. We mentioned producing better webpage s and making more aesthetic images above, but there are many potential applications for improving GANs with a limited amount of human feedback. For example, a model could be trained to produce better music (e.g. song skip rates on streaming generated music could be treated as inverse PIRs). When tuning the GAN, the decrease in the PIR loss is usually accompanied by an increase in the generator loss, and often by a partial collapse of the generator output (for example, the optimized images generally seem to have fewer output modes than the pre-training images in FIG2). This is not especially surprising -because we weighted the PIR loss very highly, the model is rewarded for trading some image diversity for image optimality. Depending on the desired application, the weight on the PIR loss could be adjusted as necessary to trade off between producing images close to the data distribution and optimizing PIR. At its most extreme, one could down-weight the generator loss entirely, and train until the model just produces a single optimal image. However, the generator likely provides some regularization by constraining the images to be somewhat close to the real images, which will reduce overfitting to an imperfect estimate of the PIR function. Furthermore, in many settings we will want to generate a variety of images (e.g. s for different websites). For these reasons, we chose to keep the generator loss when tuning the GAN. There are a number of future directions suggested by this work. A number of possible improvements are discussed in Appendix C.5. However, we also think this work has potential applications from the perspective of distillation or imitation approaches, which attempt to train one network to emulate another BID3 , e.g ), as well as from the perspective of understanding the computations that these vision architectures perform. As far as we are aware, these are the first to show that a deep vision model can be tuned rapidly from relatively little data to produce outputs which accurately emulate the behavior of hidden layers of another deep vision architecture trained on a different dataset. This suggests both that the inductive biases shared among these architectures are causing them to find similar solutions (which is also supported by work on transferable adversarial examples (, e.g.)), and that these networks final layers represent the computations of earlier hidden layers in a way that is somewhat accessible. It's possible that using our system with objectives from CNN layers as we did here might help to understand the features those layers are attending to, by analyzing the distribution of images that are produced. In this sense, our system can be thought of as offering a new approach to multifaceted feature visualization BID7, because our system attempts to optimize a distribution of images for an objective and encourages diversity in the distribution produced, rather than just optimizing a single image. We have described a system for efficiently tuning a generative image model according to a slow-toevaluate objective function. We have demonstrated the success of this system at targeting a variety of objective functions simulated from different layers of a deep vision model, as well as from low-level visual features of the images, and have shown that it can do so from a small amount of data. We have quantified some of the features that affect its performance, including the variability of the training PIR data and the number of zeros it contains. Our system's success on a wide variety of objectives suggests that it will be able to improve real user interactions, or other objectives which are slow and expensive to evaluate. This may have many exciting applications, such as improving machine-generated images, music, or art. A OTHER ANALYSES Because L P IR is just the expected value of the PIR, by looking at L P IR before and after tuning the generative model, we can tell how well the system thinks it is doing, i.e. how much it estimates that it improved PIR. This comparison reveals the interesting pattern that the system is overly pessimistic about its performance. In fact, it tends to underestimate its performance by a factor of more than 1.5 (β = 1.67 when regressing change in mean PIR on predicted change in mean PIR, see FIG3). However, it does so fairly consistently. This effect appears to be driven by the system consistently underestimating the (absolute) PIRs, which is probably caused by our change in the softmax temperature between training the PIR estimator and tuning the generative model (which we empirically found improves performance, as noted above). This is in contrast to the possible a priori expectation that the model would systematically overestimate its performance, because it is overfitting to an imperfectly estimated objective function. Although decreasing the softmax temperature between training and using the PIR obscures this effect, we do see some evidence of this; the more complex objectives (which the system produced lower effect sizes on) seem to both have lower estimated changes in mean PIR and true changes in PIR which are even lower than the estimated ones (see FIG3). Thus although the system is somewhat aware of its reduced effectiveness with these objectives (as evidenced by the lower estimates of change in mean PIR), it is not reducing its estimates sufficiently to account for the true difficulty of the objectives (as evidenced by the fact that the true change in PIR is even lower than the estimates). However, the system was generally still able to obtain positive on these objectives (see FIG1). There is a general trend (see FIG4) that the variability in PIR in the initial dataset is strongly positively related with the change in PIR the system is able to produce (β = 0.99, t = 10.6, p < 10 −10, in a linear model controlling for initial mean PIR and initial percent of values that are zero). In fact, initial standard deviation explains about 50% of the variance in the change in mean PIR. This is perhaps not too surprising -more variability means that the generative model has capacity to produce higher PIR images without too much tweaking, and that the PIR estimator model gets a wider range of values to Figure 6: Effect size evolution over two iterations learn from. Still, when attempting to use this system in practice, it is important to keep in mind that starting with a sufficiently expressive generative model will more likely produce better than starting with a more limited model. Given that our model improves PIRs, an obvious question is whether we can iterate the process. Once we have increased PIRs, can we train a new PIR estimator on samples from our new generative model, and use that to increase PIRs again? If we could iterate this many times, we might be able to create much larger improvements in PIR than we can on a single step. On the other hand, it is possible that after a single step of optimization we will effectively have saturated the easily achievable improvement in the model, and further steps will not in much improvement. To evaluate this, we took the subset of models trained on single filters from VGG layers pool2 and fc8, and used the set of images generated from the post-tuning model along with the original set of images to tune their PIR estimators for another 25,000 steps, and then tuned the generative model using this updated PIR estimator for another 50,000 steps. We then evaluated them as before, see C DETAILED METHODS C.1 ACGAN We describe our using the standard GAN framework for clarity, but we actually used an ACGAN BID9, which allows for conditioning for various user-specific features. This requires the following adjustments. Defining C to be the set of possible classes and I c to be the set of real images corresponding to a class c ∈ C: DISPLAYFORM0 Then the discriminator and generator losses are modified as follows (letting w f akeclass = 1.5 and w realclass = 1.0): DISPLAYFORM1 Note that unlike the standard ACGAN formulation given in BID9, we do not include L fake class in the discriminator's loss, to keep the discriminator from cooperating with the generator on the classification task. We modified the generator network by adding one-hot class inputs, and the discriminator by adding class outputs alongside the source output, as in BID9. We parameterized the generator as a deep neural network, which begins with a fully-connected mapping from the latent (noise) space to a 4 × 4 × 512 dimensional image, and then successively upsampled (a factor of 2 by nearest neighbor), padded and applied a convolution (3 × 3 kernel, stride of 1) and a leaky ReLU (α = 0.2) nonlinearity repeatedly. We repeated this process 5 times (except with no upsampling on the first step, and a tanh nonlinearity on the last), while stepping the image depth down as follows: 512, 512, 256, 128, 64, and finally 3 (RGB) for the output image. This means that the final output images were 64 × 64. We parameterized the discriminator as a convolutional network with 7 layers, 6 convolutions (kernels all 3 × 3; strides 2, 1, 2, 1, 2, 1; dropout after the 1st, 3rd, and 5th layers; filter depth 16, 32, 64, 128, 256, 512 ; batch normalization after each layer) and a fully connected layer to a single output for real/fake. We used a leaky ReLU (α = 0.2) nonlinearity after each layer, except the final layer, where we used a tanh. This GAN was trained on a dataset consisting of landscape images of mountains and coastlines obtained from the web. The generator was trained with the Adam optimizer BID4, and the discriminator with RMSProp. The learning rates for both were set to 10 −5, and for Adam we set β 1 = 0.5. We used a latent size of 64 units. The model was trained for 1.1 × 10 6 gradient steps, when the generated images appeared to stop improving. Instead of predicting PIR as a scalar directly, we predict it by classifying into 100 bins via a softmax, which performs better empirically. This choice was motivated by noting that the scalar version was having trouble fitting some highly multi-modal distributions that appear in the data. We trained the PIR estimator with the Adam optimizer (learning rate 5 · 10 −4). When evaluating and when using this model for improving the GAN we froze the weights of the PIR estimator. We also reduced the output softmax's temperature to 0.01, so it was behaving almost like a max, which empirically improved . Intuitively, a low softmax temperature in training allows the system to rapidly get gradients from many different output bins and adjust the distribution appropriately, whereas when actually using the system we want to be conservative with our estimates and not be too biased by low probability bins far from the modal estimate. The first approach we took to evaluating our system's ability to train for different features was to use activity from hidden layers of a computer vision model, specifically VGG 16 BID12 ) trained on ImageNet BID11. In particular, we took the 2 norm of the activity of one filter within a layer, and normalized it by the 2 norm of the total layer's activity, i.e. letting VGG l,f (i) be the vector of unit activations in filter f of layer l of VGG 16 on image i, we computed PIR for that image and a given layer and filter l *, f * as: DISPLAYFORM0 (Note that if we did not normalize by the activity in the whole layer, the system might be able to "cheat" to improve the PIR by just increasing the contrast of the images, which will likely increase overall network activity.) As noted above, we also added binomially distributed noise to these PIRs. After using our system on the tasks above, we noted that its performance was quite poor at layers 5, 6, and 7 of VGG compared to other tasks (see FIG1). This could suggest that our system was unable to capture the complex features represented at the higher levels of VGG. However, we also noticed that the feature representations at these layers tended to be quite sparse, so many of the simulated PIRs we generated were actually zero to within the bin width of our PIR estimator. Respectively, these layers had only 20%, 25%, and 24% non-zero PIRs (collapsing across filters), and around half the filters in each (resp. 6, 4, and 5) were producing > 90% zero PIRs. (By contrast, the layer with the next greatest number of zero PIRs, fc8, still had 69% nonzero PIRs overall, and had no filters in which 90% or more of the PIRs were zero.) In a few cases on layers 5, 6, and 7, all of the generated PIRs were zero. This clearly makes learning infeasible, and indeed we noted that there was a strong relationship between number of non-zero simulated PIRs in the training dataset and the ability of our system to improve PIR (see FIG7). This is somewhat troubling, since probably most images in the real world will not produce a PIR that is truly zero.in order to evaluate whether the poor performance of our system at the higher layers of VGG was due to the number of zeros or to the complexity of the features, we created less sparse features from these layers by simply targeting a set of k filters sampled without replacement from the layer, rather than a single filter. We did this by taking the norm across the k target filters, or equivalently by summing the squared norms of the k filters before taking the square root, and then normalizing by the activity in the layer as before. Formally, letting a 1,..., a k be a set of k filter indices sampled without replacement from {0, ..., number of filters in layer}, we computed the PIR for an image as: DISPLAYFORM0 The single filter cases above can be thought of as a special case of this, where k = 1. To complement these, we also tried k = 20. As above, we also added binomially distributed noise to these PIRs. This can also be thought of as perhaps a more realistic simulation of human behavior, in the sense that it is highly unlikely that there is a single feature which influences human PIRs. Rather, there are probably many related features which influence PIR in various ways. Thus it is important to evaluate our system's ability to target these types of features as well. Finally, we also considered some simpler objectives based on targeting specific colors in the output images. Analogously to the VGG features, we computed the PIRs from the vector norm of a given image in the targeted color, normalized by the total image value. We considered several objectives of this type: Three color: Similar to two color, but split the image into thirds, and computed PIR from a different color in each third.(As above, we also added binomially distributed noise to these PIRs.) These objectives provide a useful complement to the VGG objectives discussed in section 3.1. Although the single color objectives may be relevant to the classification task VGG 16 performs, the split color tasks are less likely to be relevant to classification. Note that it is important that we split the images along the width instead of the height dimension, as there may well be semantically relevant features corresponding to color divisions along the height dimension, e.g. a blue upper half and green lower half likely correlates with outdoor images, which would provide useful class information. By contrast, it is harder to imagine circumstances where different colors on the left and right halves of the image are semantically predictive, especially since flipping left to right is usually included in the data augmentation for computer vision systems. Thus success on optimizing for these objectives would increase our confidence in the generality of our system. There are a number of techniques that could be explored to improve our system. As we mentioned above, iterating for multiple steps of PIR collection and generative model tuning is worth exploring further. Also, some form of data scaling might allow the system to perform better on tasks with low variance. We briefly tried normalizing all data for an objective to have mean 0.5 and standard deviation 0.25, but did not achieve particularly good from this, possibly because there were many outliers getting clipped to 0 or 1. Still, there are many other possibilities for scaling data that could potentially in some improvement in performance. Also, one alternative approach to training a GAN to produce high-PIR images would be to use the PIR estimator objective in the Plug & Play Generative Networks framework BID8 instead of using it to tune the GAN. This could be an interesting direction to explore, but its success would probably depend on expressiveness of the initial generative model. With the mediocre model we started with, it's probably better to actually tune the model itself, which may allow it to explore parts of image space which it had not previously.

We describe how to improve an image generative model according to a slow- or difficult-to-evaluate objective, such as human feedback, which could have many applications, like making more aesthetic images.

Represent the TLDR for finding the corresponding scientific article

Our proposed algorithm does not use all of the unlabeled data for the training, and it rather uses them selectively.


Question: Represent the scientific article to find a short TLDR:

Question: Semi-supervised learning (SSL) is a study that efficiently exploits a large amount of unlabeled data to improve performance in conditions of limited labeled data. Most of the conventional SSL methods assume that the classes of unlabeled data are included in the set of classes of labeled data. In addition, these methods do not sort out useless unlabeled samples and use all the unlabeled data for learning, which is not suitable for realistic situations. In this paper, we propose an SSL method called selective self-training (SST), which selectively decides whether to include each unlabeled sample in the training process. It is also designed to be applied to a more real situation where classes of unlabeled data are different from the ones of the labeled data. For the conventional SSL problems which deal with data where both the labeled and unlabeled samples share the same class categories, the proposed method not only performs comparable to other conventional SSL algorithms but also can be combined with other SSL algorithms. While the conventional methods cannot be applied to the new SSL problems where the separated data do not share the classes, our method does not show any performance degradation even if the classes of unlabeled data are different from those of the labeled data. Recently, machine learning has achieved a lot of success in various fields and well-refined datasets are considered to be one of the most important factors (; ; BID6 . Since we cannot discover the underlying real distribution of data, we need a lot of samples to estimate it correctly . However, creating a large amount of dataset requires a huge amount of time, cost and manpower BID3.Semi-supervised learning (SSL) is a method relieving the inefficiencies in data collection and annotation process, which lies between the supervised learning and unsupervised learning in that both labeled and unlabeled data are used in the learning process (; BID3 . It can efficiently learn a model from fewer labeled data using a large amount of unlabeled data BID15 . Accordingly, the significance of SSL has been studied extensively in the previous literatures BID18 BID5 ; BID4 BID2 . These suggest that SSL can be a useful approach in cases where the amount of annotated data is insufficient. However, there is a recent research discussing the limitations of conventional SSL methods BID3 . They have pointed out that conventional SSL algorithms are difficult to be applied to real applications. Especially, the conventional methods assume that all the unlabeled data belong to one of the classes of the training labeled data. Training with unlabeled samples whose class distribution is significantly different from that of the labeled data may degrade the performance of traditional SSL methods. Furthermore, whenever a new set of data is available, they should be trained from the scratch using all the data including out-of-class 1 data. In this paper, we focus on the classification task and propose a deep neural network based approach named as selective self-training (SST) to solve the limitation mentioned above. Unlike the conventional self-training methods in , our algorithm selectively utilizes the unlabeled data for the training. To enable learning to select unlabeled data, we propose a selection network, which is based on the deep neural network, that decides whether each sample is to be added or not. Different from BID12, SST does not use the classification for the data selection. Also, we adopt an ensemble approach which is similar to the co-training method BID0 ) that utilizes outputs of multiple classifiers to iteratively build a new training dataset. In our case, instead of using multiple classifiers, we apply a temporal ensemble method to the selection network. For each unlabeled instance, two consecutive outputs of the selection network are compared to keep our training data clean. In addition, we have found that the balance between the number of samples per class is quite important for the performance of our network. We suggest a simple heuristics to balance the number of selected samples among the classes. By the proposed selection method, reliable samples can be added to the training set and uncertain samples including out-of-class data can be excluded. SST is a self-training framework, which iteratively adopts the newly annotated training data (details in Section 2.1). SST is also suitable for the incremental learning which is frequently used in many real applications when we need to handle gradually incoming data. In addition, the proposed SST is suitable for lifelong learning which makes use of more knowledge from previously acquired knowledge BID10; ). Since SSL can be learned with labeled and unlabeled data, any algorithm for SSL may seem appropriate for lifelong learning. However, conventional SSL algorithms are inefficient when out-of-class samples are included in the additional data. SST only add samples having high relevance in-class data and is suitable for lifelong learning. The main contributions of the proposed method can be summarized as follows:• For the conventional SSL problems, the proposed SST method not only performs comparable to other conventional SSL algorithms but also can be combined with other algorithms.• For the new SSL problems, the proposed SST does not show any performance degradation even with the out-of-class data.• SST requires few hyper-parameters and can be easily implemented.• SST is more suitable for lifelong learning compared to other SSL algorithms. To prove the effectiveness of our proposed method, first, we conduct experiments comparing the classification errors of SST and several other state-of-the-art SSL methods (; BID9 ;) in conventional SSL settings. Second, we propose a new experimental setup to investigate whether our method is more applicable to realworld situations. The experimental setup in BID3 samples classes among in-classes and out-classes. In the experimental setting in this paper, we sample unlabeled instances evenly in all classes. (details in Section 6.6 of the supplementary material). We evaluate the performance of the proposed SST using three public benchmark datasets: CIFAR-10, CIFAR-100 BID8 ), and SVHN . In this section, we introduce the of our research. First, we introduce some methods of self-training (; BID16 BID17) on which our work is based. Then we describe consistency regularization-based algorithms such as temporal ensembling . Self-training method has long been used for semi-supervised learning (; BID5 BID16 BID17 . It is a resampling technique that repeatedly labels unlabeled training samples based on the confidence scores and retrains itself with the selected pseudo-annotated data. Our proposed method can also be categorized as a self-training method. FIG0 shows an overview of our SSL system. Since our proposed algorithm is based on the selftraining, we follow its learning process. This process can be formalized as follows. However, most self-training methods assume that the labeled and unlabeled data are generated from the identical distribution. Therefore, in real-world scenarios, some instances with low likelihood according to the distribution of the labeled data are likely to be misclassified inevitably. Consequently, these erroneous samples significantly lead to worse in the next training step. To alleviate this problem, we adopt the ensemble and balancing methods to select reliable samples. Consistency regularization is one of the popular SSL methods and has been referred to many recent researches (; ; BID9 . Among them, Π model and temporal ensembling are widely used . They have defined new loss functions for unlabeled data. The Π model outputs f (x) andf (x) for the same input x by perturbing the input with different random noise and using dropout BID8, and then minimizes DISPLAYFORM0 2 ) between these output values. Temporal ensembling does not make different predictions f (x) andf (x), but minimizes the difference (DISPLAYFORM1 2) between the outputs of two consecutive iterations for computational efficiency. In spite of the improvement in performance, they require lots of things to consider for training. These methods have various hyperparameters such as'ramp up','ramp down','unsupervised loss weight' and so on. In addition, customized settings for training such as ZCA preprocessing and mean-only batch normalization BID7 are also very important aspects for improving the performance BID3.Algorithm 1 Training procedure of the proposed SST Require: x i, y i: training data and label Require: L, U: labeled and unlabeled datasets Require: I U: set of unlabeled sample indices Require: f n (·; θ n), f cl (·; θ c) and f sel (·; θ s): trainable SST model Require: α,, K, K re: hyper-parameters, 0 ≤ α < 1, 0 ≤ < 1 1: randomly initialize θ n, θ c, θ s 2: train f n (·; θ n), f cl (·; θ c) and f sel (·; θ s) for K epochs using L 3: repeat 4:initialize r DISPLAYFORM2 for each i ∈ I U do 6: DISPLAYFORM3 end if 10: DISPLAYFORM4 if z i > 1 − then I S ← I S ∪ {i} DISPLAYFORM0 retrain f n (·; θ n), f cl (·; θ c) and f sel (·; θ s) for K re epochs using T 19: until stopping criterion is true In this section, we introduce our selective self-training (SST) method. The proposed model consists of three networks as shown in the bottom part of FIG0. The output of the backbone network is fed into two sibling fully-connected layers -a classification network f cl (·; θ c) and a selection network f sel (·; θ s), where θ c and θ s are learnable parameters for each of them. In this paper, we define the classification and the selection score as r i = f cl (f n (x i ; θ n); θ c ) and DISPLAYFORM0 respectively, where f n (·; θ n) denotes the backbone network with learnable parameters θ n. Note that we define r i as the ant label and it belongs to one of the class labels r i ∈ Y = {1, 2, · · ·, C}. The network architecture of the proposed model is detailed in Section 6.2 in the supplementary material. As shown in FIG0, the proposed SST method can be represented in the following four steps. First, SST trains the network using a set of the labeled data L = {(x i, y i) | i = 1, · · ·, L}, where x i and y i ∈ {1, 2, · · ·, C} denote the data and the ground truth label respectively, which is a standard supervised learning method. The next step is to predict all the unlabeled data U = {x i | i = L + 1, · · ·, N} and select a subset of the unlabeled data {x i |i ∈ I S} whose data have high selection scores with the current trained model, where I S denotes a set of selected sample indices from I U = {L + 1, · · ·, N}. Then, we annotate the selected samples with the pseudo-categories evaluated by the classification network and construct a new training dataset T composed of L and U S = {(x i,ŷ i)|i ∈ I S }. After that, we retrain the model with T and repeat this process iteratively. The overall process of the SST is described in Algorithm 1 and the details of each of the four steps will be described later. The SST algorithm first trains a model with supervised learning. At this time, the entire model (all three networks) is trained simultaneously. The classification network is trained using the softmax function and the cross-entropy loss as in the ordinary supervised classification learning task. In case of the selection network, the training labels are motivated by discriminator of generative adversarial networks (GAN) (; BID13 . When i-th sample x i with the class label y i is fed into the network, the target for the selection network is set as: DISPLAYFORM0 where I L = {1, · · ·, L} represents a set of labeled sample indices. The selection network is trained with the generated target g i. Especially, we use the sigmoid function for the final activation and the binary cross-entropy loss to train the selection network. Our selection network does not utilize the softmax function because it produces a relative value and it can induce a high value even for an out-of-class sample. Instead, our selection network is designed to estimate an absolute confidence score using the sigmoid activation function. Consequently, our final loss function is a sum of the classification loss L cl and the selection loss L sel: DISPLAYFORM1 After learning the model in a supervised manner, SST takes all instances of the unlabeled set U as input and predicts classification r i and the selection score s i, for all i ∈ I U. We utilize the classification and selection score (r i and s i) to annotate and choose unlabeled samples, respectively. In the context of self-training, removing erroneously annotated samples is one of the most important things for the new training dataset. Thus, we adopt temporal co-training and ensemble methods for selection score in order to keep our training set from contamination. First, let r t i and r t−1 i be the classification of the current and the previous iterations respectively and we utilize the temporal consistency of these values. If these values are different, we set the ensemble score z i = 0 to reduce uncertainty in selecting unlabeled samples. Second, inspired by , we also utilize multiple previous network evaluations of unlabeled instances by updating the ensemble score z i = αz i + (1 − α)s i, where α is a momentum weight for the moving average of ensemble scores. However, the aim of our ensembling approach is different from . They want to alleviate different predictions for the same input, which are ed from different augmentation and noise to the input. However, our aim differs from theirs in that we are interested in selecting reliable (pseudo-)labeled samples. After that, we select unlabeled samples with high ensemble score z i. It is very important to set an appropriate threshold because it decides the quality of the added unlabeled samples for the next training. If the classification network is trained well on the labeled data, the training accuracy would be very high. Since the selection network is trained with the target g i generated from the classification score r i, the selection score s i will be close to 1.0. We set the threshold to 1 − and control it by changing. In this case, if the ensemble score z i exceeds 1 −, the pseudo-label of the unlabeled sampleŷ i is set to the classification r i. When we construct a new training dataset, we keep the number of samples of each class the same. The reason is that if one class dominates the others, the classification performance is degraded by the imbalanced distribution (FernáNdez et al., 2013). We also empirically found that naively creating a new training dataset fails to yield good performance. In order to fairly transfer the selected samples to the new training set, the amount of migration in each class should not exceed the number of the class having the least selected samples. We take arbitrary samples in every class as much as the maximum number satisfying this condition. The new training set T is composed of both a set of labeled samples L and a set of selected unlabeled samples U S. The number of selected unlabeled samples is the same for all classes. After combining the labeled and selected pseudo-labeled data, the model is retrained with the new dataset for K re epochs. In this step, the label for the selection network is obtained by a process similar to Eq.. Above steps (except for Section 3.1) are repeated for M iterations until (near-) convergence. Table 1: Ablation study with 5 runs on the CIFAR-10 dataset.'balance' denotes the usage of data balancing scheme during data addition as described in Sec. 3.3,'ensemble' is for the usage of previous selection scores as in the 10th line of Algorithm 1, and'multiplication' is the scheme of multiplying top-1 softmax output of the classifier network to the selection score and use it as a new selection score. To evaluate our proposed SST algorithm, we conduct two types of experiments. First, we evaluate the proposed SST algorithm for the conventional SSL problem where all unlabeled data are in-class. Then, SST is evaluated with the new SSL problem where some of the unlabeled data are out-of-class. In the case of in-class data, gradually gathering highly confident samples in U can help improve the performance. On the other hand, in the case of out-of-class data, a strict threshold is preferred to prevent uncertain out-of-class data from being involved in the new training set. Therefore, we have experimented with decay mode that decreases the threshold in log-scale and fixed mode that fixes the threshold in the way described in Section 4.2. We have experimented our method with 100 iterations and determined epsilon by cross-validation in decay modes. In case of fixed modes, epsilon is fixed and the number of iteration is determined by cross-validation. The details about the experimental setup and the network architecture are presented in Section 6.1, 6.2 of the supplementary material. We experiment with a couple of simple synthetic datasets (two moons, four spins) and three popular datasets which are SVHN, CIFAR-10, and CIFAR-100 (; BID8 . The settings of labeled versus unlabeled data separation for each dataset are the same with (; ; BID9 . More details are provided in Section 6.3 in the supplementary material. The experimental of the synthetic datasets can be found in Section 6.4 of the supplementary material. We have performed experiments on CIFAR-10 dataset with the combination of three types of components. As described in Table 1, these are whether to use data balancing scheme described in Section 3.3 (balance), whether to use selection score ensemble in the 10th line of Algorithm 1 (ensemble) and whether to multiply the selection score with the top-1 softmax output of the classifier network to set a new selection score for comparison with the threshold (multiplication). First, when SST does not use all of these, the error 21.44% is higher than that of the supervised learning which does not use any unlabeled data. This is due to the problem of unbalanced data mentioned in subsection 3.3. When the data balance is used, the error is 14.43%, which is better than the baseline 21.44%. Adding the ensemble scheme in 11.82% error, and the multiplication scheme shows a slight drop in performance. Since all of the experiments use the same threshold, the number of candidate samples to be added is reduced by the multiplication with the top-1 softmax output and the variation becomes smaller because only confident data are added. However, we have not used the multiplication scheme in what follows because the softmax classification output is dominant in multiplication. Therefore, we have used only balance and ensemble schemes in the following experiments. TAB2 shows the experiment of supervised learning, conventional SSL algorithms and the proposed SST on CIFAR-10, SVHN and CIFAR-100 datasets. Our baseline model with supervised learning performs slightly better than what has been reported in other papers (; BID9) because of our different settings such as Gaussian noise Figure 2: SST on CIFAR-10, SVHN, and CIFAR-100 datasets with 5 runs. The x-axis is the iteration, the blue circle is the average of the number of data used for training, and the red diamond is the average accuracy. BID9 12.31 ± 0.28% 3.95 ± 0.21% -Π model 12.36 ± 0.31% 4.82 ± 0.17% 39.19 ± 0.36% TempEns 12.16 ± 0.24% 4.42 ± 0.16% 38.65 ± 0.51% TempEns + SNTG 10.93 ± 0.14% 3.98 ± 0.21% 40.19 ± 0.51%* VAT 11.36 ± 0.34% 5.42 ± 0.22% -VAT + EntMin 10.55 ± 0.05% 3.86 ± 0.11% -pseudo-label (; BID3 17.78 ± 0.57% 7.62 ± 0.29% -Proposed method (SST)* 11.82 ± 0.40% 6.88 ± 0.59% 34.89 ± 0.75% SST + TempEns + SNTG* 9.99 ± 0.31% 4.74 ± 0.19% 34.94 ± 0.54% on inputs, optimizer selection, the mean-only batch normalizations and the learning rate parameters. For all the datasets, we have also performed experiments with a model of SST combined with the temporal ensembling (TempEns) and SNTG, labeled as SST+TempEns+SNTG in the table. For the model, the pseudo-labels of SST at the last iteration is considered as the true class label. Figure 2 shows the number of samples used in the training and the corresponding accuracy on the test set for each dataset. The baseline network yields the test error of 18.97% and 5.57% when trained with 4,000 (sampled) and 50,000 (all) labeled images respectively. The test error of our SST method reaches 11.82% which is comparable to other algorithms while SST+TempEns+SNTG model 1.83% better than the SST-only model. The baseline model for SVHN dataset is trained with 1,000 labeled images and yields the test error of 13.45%. Our proposed method has an error of 6.88% which is relatively higher than those of other SSL algorithms. Performing better than SST, SST+TempEns+SNTG reaches 4.74% of error which is worse than that of TempEns+SNTG model. We suspect two reasons for this. The first is that SVHN dataset is not well balanced, and the second is that SVHN is a relatively easy dataset, so it seems to be easily added to the hard labels. With data balancing, the SST is still worse than other algorithms. More details are provided in Section 6.5 in the supplementary material. We think this phenomenon owes to the use of hard labels in SST where incorrectly estimated samples deteriorate the performance. conjectured that the hyper-parameter in the current temporal ensembling and SNTG may not have been optimized. We have experimented with the following settings for real-world applications. The dataset is categorized into six animal and four non-animal classes as similarly done in BID3. In CIFAR-10, 400 images per animal class are used as the labeled data (total 2,400 images for 6 animal classes) and a pool of 20,000 images with different mixtures of both animal and non-animal classes are experimented as an unlabeled dataset. In CIFAR-100, 5,000 labeled data (100 images per animal class) and a total of 20,000 unlabeled images of both classes with different mixed ratios are utilized. Unlike the experimental setting in BID3, we have experimented according to the ratio (%) of the number of out-of-class data in the unlabeled dataset. More details are provided in Section 6.6 in the supplementary material. As mentioned in Section 4, in the presence of out-of-class samples, a strict threshold is required. If all of the unlabeled data is assumed to be in-class, the decay mode may be a good choice. However, in many real-applications, out-of-class unlabeled data is also added to the training set in the decay mode and causes poor performance. In avoidance of such matter, we have experimented on a fixed mode of criterion threshold on adding the unlabeled data. Unlike the decay mode that decrements the threshold value, SST in the fixed mode sets a fixed threshold at a reasonably high value throughout the training. Our method in the fixed mode should be considered more suitable for real-applications but empirically shows lower performances in FIG1 and TAB4 than when running in the decay mode. The difference between the decay mode and the fixed mode are an unchangeable and the initial ensemble. Setting a threshold value for the fixed mode is critical for a feasible comparison against the decay mode. FIG1 shows the average of the obtained when performing SST five times for each ratio in CIFAR-10. As shown in FIG1, as the number of iteration increases, the threshold in the decay mode decreases and the number of additional unlabeled data increases. Obviously, while the different percentage of the non-animal data inclusion show different trends of training, in the cases of 0 ∼ 75% of non-animal data included in the unlabeled dataset, the additionally selected training data shows an initial increase at 30 th ∼ 40 th iteration. On the other hand, when the unlabeled dataset is composed of only the out-of-class data, selective data addition of our method initiates at 55 th ∼ 65 th training iteration. This tendency has been observed in previous researches on classification problems and we have set the threshold value fixed at a value between two initiating points of data addition as similarly done in the works of BID11 BID14. We have set the fixed threshold based on 47th iteration (between 40 and 55). For a more reliable selection score, we have not added any unlabeled data to the new training set and have trained our method with the labeled data only for 5 iterations. As it can be seen in TAB4, in the case of SST in the decay mode, the performance has been improved when the unlabeled dataset consists only in-class animal data, but when the unlabeled pool is filled with only out-of-class data, the performance is degraded. For the case of SST with a fixed threshold value, samples are not added and the performance was not degraded at 100% nonanimal ratio as shown in FIG1 (c). Furthermore, at 0% of out-of-class samples in the pool, there is a more improvement in the performance than at 100 % of out-of-class samples while still being inferior to the improvement than the decay mode. Because less but stable data samples are added by SST with a fixed threshold, the performance is improved for all the cases compared to that of supervised learning. Therefore, it is more suitable for real applications where the origin of data is usually unknown. We proposed selective self-training (SST) for semi-supervised learning (SSL) problem. Unlike conventional methods, SST selectively samples unlabeled data and trains the model with a subset of the dataset. Using selection network, reliable samples can be added to the new training dataset. In this paper, we conduct two types of experiments. First, we experiment with the assumption that unlabeled data are in-class like conventional SSL problems. Then, we experiment how SST performs for out-of-class unlabeled data. For the conventional SSL problems, we achieved competitive on several datasets and our method could be combined with conventional algorithms to improve performance. The accuracy of SST is either saturated or not depending on the dataset. Nonetheless, SST has shown performance improvements as a number of data increases. In addition, the of the combined experiments of SST and other algorithms show the possibility of performance improvement. For the new SSL problems, SST did not show any performance degradation even if the model is learned from in-class data and out-of-class unlabeled data. Decreasing the threshold of the selection network in new SSL problem, performance degrades. However, the output of the selection network shows different trends according to in-class and out-of-class. By setting a threshold that does not add out-of-class data, SST has prevented the addition of out-of-class samples to the new training dataset. It means that it is possible to prevent the erroneous data from being added to the unlabeled dataset in a real environment. 6 SUPPLEMENTARY MATERIAL The basic settings of our experiments are as follows. Different from , we use stochastic gradient descent (SGD) with a weight decay of 0.0005 as an optimizer. The momentum weight for the ensemble of selection scores is set to α = 0.5. Also, we do not apply mean-only batch normalization layer BID7 and Gaussian noise. We follow the same data augmentation scheme in consisting of horizontal flips and random translations. However, ZCA whitening is not used. In the supervised learning phase, we train our model using batch size 100 for 300 epochs. After that, in the retraining phase, we train using the same batch size for 150 epochs with the new training dataset. The learning rate starts from 0.1. In the supervised learning phase, it is divided by 10 at the 150-th and 225-th epoch. In the retraining phase, it is divided by 10 at the 75-th and 113-th epoch. The number of training iteration and thresholding are very important parameters in our algorithm and have a considerable correlation with each other. In the first experiment, the iteration number remains fixed and the growth rate of is adjusted so that the validation accuracy saturates near the settled iteration number. While the validation accuracy is evaluated using the cross-validation, we set the number of training iteration to be 100 so that the model is trained enough until it saturates. is increased in log-scale and begins at a very small value (10 −5) where no data is added. The growth rate of is determined according to when the validation accuracy saturates. The stopping criterion is that the accuracy of the current iteration reaches the average accuracy of the previous 20 steps. If the stopping iteration is much less than 100 times, the growth rate should be reduced so that the data is added more slowly. If the stopping iteration significantly exceeds 100 iterations, the growth rate should be increased so that the data is added more easily. We allow 5 iterations as a deviation from 100 iterations and the growth rate of is left unchanged in this interval. As a , the is gradually increased in log-scale by 10 times every 33 iterations in CIFAR-10 and SVHN. In the case of CIFAR-100, the is increased by 10 times in log-scale every 27 iterations. In the second experiment, we leave the fixed and simply train the model until the stopping criteria are satisfied. Other details are the same as those of the first experiment. We used two types of networks. The network for training the synthetic dataset is shown in TAB7 and consists of two hidden layers with 30 nodes. The network structure for CIFAR-10, SVHN, and CIFAR-100 consists of convolutions, and its structure is shown in TAB6. We used standard batch normalization and Leaky ReLU with 0.1. We have experimented with CIFAR-10, SVHN, and CIFAR-100 datasets that consist of 32 × 32 pixel RGB images. CIFAR-10 and SVHN have 10 classes and CIFAR-100 has 100 classes. Overall, standard data normalization and augmentation scheme are used. For data augmentation, we used random horizontal flipping and random translation by up to 2 pixels. In the case of SVHN, random horizontal flipping is not used. To show that the SST algorithm is comparable to the conventional SSL algorithms, we experimented with the popular setting (; ; BID9 . The validation set in the cross-validation to obtain the reduction rate of epsilon is extracted from the training set by 5000 images. After the epsilon is obtained, all the training datasets are used. The following is the standard labeled/unlabeled split. CIFAR-10 : 4k labeled data ( 400 images per class), 46k unlabeled data (4,600 images per class), As synthetic datasets, two moons and 4 spins were tested in the same manner as SNTG . Each dataset has 6,000 training and 6,000 test samples. In the case of two moons, there are two classes y ∈ {0, 1}, and in case of 4 spins, y ∈ {0, 1, 2, 3}. In 6,000 training data, there are 12 labeled data and 5,988 unlabeled data. Thus, for two moons, each class has 6 points and for 4 spins, each class has 3 points. Because the number of labeled datapoints are too small, random sampling can lead to sample similar points. Therefore, we randomly sampled the labeled data with a constraint that the Euclidian distance of each data point is greater than 0.7. For these datasets, total iteration was performed 50 times, and the was increased from 10 −7 to 10 −4.5 on a log scale. FIG3 shows the basic setting of the synthetic dataset, and Figure 5 and 6 show the progress of the SST algorithm. The SST algorithm improves performance by gradually expanding certain data in a synthetic dataset. CIFAR-10: When the network were trained with 1k and 2k images, the test error were 38.71% and 26.99% respectively. The test errors in the SST algorithm were 23.15% and 15.72%, the SST has better performance than Π model but worse than Mean Teacher in 1k test. In 2k test, the SST has better performance than Π model and similar with Mean Teacher. Table 6: Classification error on CIFAR-10 (1k and 2k Labels) with 5 runs using in-class unlabeled data Method CIFAR-10 (1k) CIFAR-10 (2k) supervised (sampled)38.71 ± 0.47% 26.99 ± 0.79% Π model 27.36 ± 1.20% 18.02 ± 0.60% Mean Teacher BID9 21.55 ± 1.48% 15, 73 ± 0.31%Proposed method (SST) 23.15 ± 0.61% 15.72 ± 0.50% For the balancing experiments, in SVHN, 1,000 images are used as the labeled data and 45,000 balanced unlabeled images are used. As a , the SST is still worse than other algorithms. As mentioned in Section 4.1, we think that incorrectly estimated samples by SST deteriorate the performance. BID3 adds only four unlabeled classes and tests according to the radio of unlabeled class. For example, at 50%, two classes are in-class, and two classes are out-of-class. However, we experimented with the ratio of the number of non-animal data. Thus at 50% in CIFAR-10, unlabeled data consists of 50% in-class and 50% out-of-class. The data for each ratio are shown in TAB8, and the data category for animal and non-animal is shown in TAB9. TAB10 shows the of a general test on other algorithms. First, self-training (; BID16 BID17) without threshold does not improve performance even at 0%, and performance at 100% is degraded. When SST is applied to the softmax output as a threshold without selection network, the performance is improved at 0%, but the performance is degraded at 100%. Although the threshold was 0.9999, unlabeled data was added in 100% of the non-animal data. In the new SSL problem, the experiment in decay mode is to find a gap between two initiating points of data addition in 0 ∼ 75% of non-animal data and 100% of non-animal data. In our experiment, the growth rate of epsilon in CIFAR-10 is applied to CIFAR-100. (The smaller the growth rate of epsilon, the less the difference in between iterations. Therefore, although the difference in the between intervals is the same, depending on the growth rate of, the difference in the iteration can be greater.) In the case of 0 ∼ 75%, the number of data shows a slight increase from about 30 iterations. On the other hand, in the case of 100%, selected samples are added from about 40 iterations. The fixed threshold set to the threshold of 35 iterations. In the decay mode, the performance is much improved at 0%, and at 100%, the performance is degraded. On the other hand, in the fixed mode, there was no performance degradation from 0% to 100%. In CIFAR-100, the difference between 0% and 100 % is less than CIFAR-10, because the gap between animal and non-animal is small and additional data is small. FIG5 shows the experimental .

represent the short summary for finding the corresponding scientific article (long)

Our proposed algorithm does not use all of the unlabeled data for the training, and it rather uses them selectively.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

In this paper we introduced an algorithm to learn the connectivity of deep multi-branch networks. The approach is evaluated on image categorization where it consistently yields accuracy gains over state-of-the-art models that use fixed connectivity.