Question: Represent the scientific article to find a short TLDR.

Question: We study the problem of training machine learning models incrementally using active learning with access to imperfect or noisy oracles. We specifically consider the setting of batch active learning, in which multiple samples are selected as opposed to a single sample as in classical settings so as to reduce the training overhead. Our approach bridges between uniform randomness and score based importance sampling of clusters when selecting a batch of new samples. Experiments on benchmark image classification datasets (MNIST, SVHN, and CIFAR10) shows improvement over existing active learning strategies. We introduce an extra denoising layer to deep networks to make active learning robust to label noises and show significant improvements. Supervised learning is the most widely used machine learning method, but it requires labelled data for training. It is time-consuming and labor-intensive to annotate a large dataset for complex supervised machine learning models. For example, ImageNet reported the time taken to annotate one object to be roughly 55 seconds. Hence an active learning approach which selects the most relevant samples for annotation to incrementally train machine learning models is a very attractive avenue, especially for training deep networks for newer problems that have littel annotated data. Classical active learning appends the training dataset with a single sample-label pair at a time. Given the increasing complexity of machine learning models, it is natural to expand active learning procedures to append a batch of samples at each iteration instead of just one. Keeping such training overhead in mind, a few batch active learning procedures have been developed in the literature (; ;). When initializing the model with a very small seed dataset, active learning suffers from the coldstart problem: at the very beginning of active learning procedures, the model is far from being accurate and hence the inferred output of the model is incorrect/uncertain. Since active learning relies on output of the current model to select next samples, a poor initial model leads to uncertain estimation of selection criteria and selection of wrong samples. Prior art on batch active learning suffers performance degradation due to this cold-start problem. Most active learning procedures assume the oracle to be perfect, i.e., it can always annotate samples correctly. However, in real-world scenarios and given the increasing usage of crowd sourcing, for example Amazon Mechanical Turk (AMT), for labelling data, most oracles are noisy. The noise induced by the oracle in many scenarios is resolute. Having multiple annotations on the same sample cannot guarantee noise-free labels due to the presence of systematic bias in the setup and leads to consistent mistakes. To validate this point, we ran a crowd annotation experiment on ESC50 dataset : each sample is annotated by 5 crowdworkers on AMT and the majority vote of the 5 annotations is considered the label. It turned out for some classes, 10% of the samples are annotated wrong, even with 5 annotators. Details of the experiment can be found in Appendix A. Under such noisy oracle scenarios, classical active learning algorithms such as (a) under-perform as shown in Figure 1. Motivating from these observations, we fashion a batch active learning strategy to be robust to noisy oracles. The main contributions of this work are as follows: we propose a batch sample selection method based on importance sampling and clustering which caters to drawing a batch which is simultaneously diverse and important to the model; we incorporate model uncertainty into the sampling probability to compensate poor estimation of the Noise channel is assumed to be a 10-symmetric channel, where ε is the probability of label error. importance scores when the training data is too small to build a meaningful model; we introduce a denoising layer to deep networks to robustify active learning to noisy oracles. Main , as shown in Fig. 3 demonstrate that in noise-free scenario, our method performs as the best over the whole active learning procedure, and in noisy scenario, our method outperforms significantly over state-of-the-art methods. Active Learning: Active learning is a well-studied problem and has gain interest in deep learning as well. A survey summarizes various existing approaches in . In a nutshell, two key and diverse ways to tackle this problem in the literature are discrimination and representation. The representation line of work focuses on selecting samples that can represent the whole unlabelled training set while the discrimination line of work aims at selecting'tough' examples from the pool set, for example, using information theoretic scores in , entropy as uncertainty in . Along the lines of ensemble methods we have works, for example, (; ;). A recent work of discrimination-based active learning uses mutual information, Bayesian Active Learning by Disagreement (BALD), as discriminating criteria. In ) the authors used dropout approximation to compute the BALD scores for modern Convolutional Neural Networks (CNNs). However, these approaches do not consider batch acquisition and hence lack of diversity in selected batch samples causing performance lag. Batch Active Learning: Active learning in the batch acquisition manner has been studied from the perspective of set selection and using submodularity or its variants in a variety of works. The authors in utilize submodularity for naive Bayes and nearest neighbor. The concept of adaptive submodularity is related to active learning as well. The problem solves adaptive greedy optimization with sequential decision making . Using this concept, considers pool-based Bayesian active learning with a finite set of candidate hypotheses. A pool-based active learning is also discussed in which considered risk minimization under given hypothesis space. The work in uses both discriminative and representative samples to select a batch. The authors in use coreset approach to select representative points of the pool set. Recently, an adversarial learning of variational auto-encoders is used for batch active learning in . The work make a representation of the training and pool, and adversarially select the pool representatives. Model Uncertainty: The uncertainty for deep learning models, especially CNNs, was first addressed in using dropout as Bayesian approximation. Model uncertainty approximation using Batch Normalization (BN) has been shown in . Both of these approaches in some sense exploit the stochastic layers (Dropout, BN) to extract model uncertainty. The importance of model uncertainty is also emphasized in the work of . The work witnesses model as well as label uncertainty which they termed as epistemic and aleatoric uncertainty, respectively. We also address both of these uncertainties in this work. Noisy Oracle: The importance of noisy labels from oracle has been realized in the works like (; b;) which utilized the concept of adap-tive submodularity for providing theoretical guarantees. studies the same problem but with correlated noisy tests. Active learning with noisy oracles is also studied in . However, these work do not consider deep learning setup. A binary classification task with the noisy oracle is considered in . The authors in used a variation of Expectation Maximization algorithm to estimate the correct labels as well as annotating workers quality. The closest work to us in the noisy oracle setting for deep learning models are (; 2016). The authors also propose to augment the model with an extra full-connected dense layer. However, the denoising layer does not follow any probability simplex constraint, and they use modified loss function for the noise accountability along with dropout regularization. In this section, we introduce the notations used throughout the paper. We then formally define the problem of batch active learning with noisy oracles. The ith (jth) row (column) of a matrix X is denoted as X i,. (X .,j). ∆ K−1 is the probability simplex of dimension K, where For a probability vector p ∈ ∆ K−1, the Shannon entropy is defined as: The KL-divergence is always non-negative and is 0 if and only if p = q. The expectation operator is taken as E. We are concerned with a K class classification problem with a sample space X and label space Y = {1, 2, . . ., K}. The classification model M is taken to be g θ: X → Y parameterized with θ. The softmax output of the model is given by p = softmax(g θ (x)) ∈ ∆ K−1. The batch active learning setup starts with a set of labeled samples D tr = {(x i, y i)} and unlabeled samples P = {(x j)}. With a query budget of b, we select a batch of unlabeled samples B as, B = ALG(D tr, M, b, P), |B| ≤ b, where ALG is the selection procedure conditioned on the current state of active learning (D tr, M, b, P). ALG is designed with the aim of maximizing the prediction accuracy E p X ×Y [(h θ (x) = y)]. Henceforth, these samples which can potentially maximize the prediction accuracy are termed as important samples. After each acquisition iteration, the training dataset is updated as D tr = D tr ∪ {(B, y B)} where y B are the labels of B from an oracle routine. The oracle takes an input x ∈ X and outputs the ground truth label y ∈ Y. This is referred to as'Ideal Oracle' and the mapping from x to y is deterministic. A'Noisy Oracle' flips the true output y to y which is what we receive upon querying x. Similar to (a), we assume that the label flipping is independent of the input x and thus can be characterized by the conditional probability p(y = i|y = j), where i, j ∈ Y. We also refer this conditional distribution as the noisy-channel, and hence the ideal oracle has noisy channel value of 1 for i = j and 0 otherwise. For rest of the paper, we use the noise channel as a K-symmetric channel (SC), see Figure 2b, which is a generalization of the binary symmetric channel. The K-SC is defined as follows where ε is the probability of a label flip, i.e., p(y = y) = ε. We resort to the usage of K-SC because of its simplicity, and in addition, it abstracts the oracle noise strength with a single parameter ε. Therefore, in noisy active learning, after the selection of required subset B, the training dataset (and then the model) is updated as D tr = D tr ∪ {(B, y B)}. Next, in Section 4, we discuss the proposed solution to noisy batch active learning. An ideal batch selection procedure so as to be employed in an active learning setup, must address the following issues, (i) select important samples from the available pool for the current model, and (ii) select a diverse batch to avoid repetitive samples. We note that, at each step, when active learning acquires new samples, both of these issues are addressed by using the currently trained model. However, in the event of an uncertain model, the quantification of diversity and importance of a batch of samples will also be inaccurate ing in loss of performance. This is often the case with active learning because we start with less data in hand and consequently an uncertain model. Therefore, we identify the next problem in the active learning as (iii) incorporation of the model uncertainty across active learning iterations. Batch selection: The construction of batch active learning algorithm by solving the aforementioned first two problems begins with assignment of an importance score (ρ) to each sample in the pool. Several score functions exist which perform sample wise active learning. To list a few, max-entropy, variation ratios, BALD, entropy of the predicted class probabilities . We use BALD as an importance score which quantifies the amount of reduction of uncertainty by incorporating a particular sample for the given model. In principle, we wish to have high BALD score for a sample to be selected. For the sake of completeness, it is defined as follows. where θ are the model parameters. We refer the reader to for details regarding the computation of BALD score in. To address diversity, we first perform clustering of the pooled samples and then use importance sampling to select cluster centroids. For clustering, the distance metric used is the square root of the Jensen-Shannon (JS) divergence between softmax output of the samples. Formally, for our case, it is defined as d: With little abuse of notation, we interchangeably use d(p i, p j) as d i,j where i, j are the sample indices and p i, p j are corresponding softmax outputs. The advantage of using JS-divergence is two folds; first it captures similarity between probability distributions well, second, unlike KL-divergence it is always bounded between 0 and 1. The boundedness helps in incorporating uncertainty which we will discuss shortly. Using the distance metric as d we perform Agglomerative hierarchical clustering for a given number of clusters N. A cluster centroid is taken as the median score sample of the cluster members. Finally, with all similar samples clustered together, we perform importance sampling of the cluster centroids using their importance score, and a random centroid c is selected as p(c = k) ∝ ρ k. The clustering and importance sampling together not only take care of selecting important samples but also ensure diversity among the selected samples. Uncertainty Incorporation: The discussion we have so far is crucially dependent on the output of the model in hand, i.e., importance score as well as the similarity distance. As noted in our third identified issue with active learning, of model uncertainty, these estimations suffers from inaccuracy in situations involving less training data or uncertain model. The uncertainty of a model, in very general terms, represents the model's confidence of its output. The uncertainty for deep learning models has been approximated in Bayesian settings using dropout in , and batch normalization (BN) in . Both use stochastic layers (dropout, BN) to undergo multiple forward passes and compute the model's confidence in the outputs. For example, confidence could be measured in terms of statistical dispersion of the softmax outputs. In particular, variance of the softmax outputs, variation ratio of the model output decision, etc, are good candidates. We denote the model uncertainty as σ ∈, such that σ is normalized between 0 and 1 with 0 being complete certainty and 1 for fully uncertain model. For rest of the work, we compute the uncertainty measure σ as variation ratio of the output of model's multiple stochastic forward passes as mentioned in . In the event of an uncertain model (σ → 1), we randomly select samples from the pool initially. However, as the model moves towards being more accurate (low σ) by acquiring more labeled samples through active learning, the selection of samples should be biased towards importance sampling and clustering. To mathematically model this solution, we use the statistical mechanics approach of deterministic annealing using the Boltzmann-Gibbs distribution . In Gibbs distribution p(i) ∝ e − i/kB T, i.e., probability of a system being in an ith state is high for low energy i states and influenced by the temperature T. For example, if T → ∞, then state energy is irrelevant and all states are equally probable, while if T → 0, then probability of the system being in the lowest energy state is almost surely 1. We translate this into active learning as follows: For a given cluster centroid c, if the model uncertainty is very high (σ → 1) then all points in the pool (including c) should be equally probable to get selected (or uniform random sampling), and if the model is very certain (σ → 0), then the centroid c itself should be selected. This is achieved by using the state energy analogue as distance d between Assign importance score to each x ∈ P as ρ x = I(θ; y|x, Perform Agglomerative clustering of the pool samples with N (b) number of clusters using square root of JS-divergence as distance metric to get D 4: Sample cluster centroid c from the categorical distribution Compute uncertainty estimate σ (t−1) of the model M (t−1), and Sample ζ from the Gibbs distribution p(ζ = s|B (t), c, β end for 10: Query oracle for the labels of B (t) and update Update model as Set P ← P \ B (t) 13: end for the cluster centroid c and any sample x in the pool, and temperature analogue as uncertainty estimate σ of the model. The distance metric d used by us is always bounded between 0 and 1 and it provides nice interpretation for the state energy. Since, in the event of low uncertainty, we wish to perform importance sampling of cluster centroids, and we have d c,c = 0 (lowest possible value), therefore by Gibbs distribution, cluster centroid c is selected almost surely. To construct a batch, the samples have to be drawn from the pool using Gibbs distribution without replacement. In the event of samples s 1,..., s n already drawn, the probability of drawing a sample ζ given the cluster centroid c, distance matrix D = [d i,j] and inverse temperature (or inverse uncertainty) β is written as where P = P\s 1:n. In theory, the inverse uncertainty β can be any f such that f: → R + ∪{0} and f (σ) → ∞ as σ → 0 and f (σ) = 0 for σ = 1. For example, few possible choices for β (= f (σ)) are − log(σ), e 1/σ − 1. Different inverse functions will have different growth rate, and the choice of functions is dependent on both the model and the data. Next, since we have drawn the cluster centroid c according to p(c = k) ∝ ρ k, the probability of drawing a sample s from the pool P is written as We can readily see that upon setting β → 0 in, p(ζ = s|s 1:n, β, D) reduces to 1/|P | which is nothing but the uniform random distribution in the leftover pool. On setting β → ∞, we have ζ = c with probability ρ c / c ρ c and ζ = c with probability 0, i.e., selecting cluster centroids from the pool with importance sampling. For all other 0 < β < ∞ we have a soft bridge between these two asymptotic cases. The approach of uncertainty based batch active learning is summarized as Algorithm 1. Next, we discuss the solution to address noisy oracles in the context of active learning. The noisy oracle, as defined in Section 3, has non-zero probability for outputting a wrong label when queried with an input sample. To make the model aware of possible noise in the dataset originating from the noisy oracle, we append a denoising layer to the model. The inputs to this denoising layer are the softmax outputs p of the original model. Figure 2a demonstrates the proposed solution for deep learning classification models. The denoising layer is a fully-connected K×K dense layer with 1: for t = 1, 2,..., T do 2: Query noisy oracle for the labels of B (t) and update Get M * (t) ← M (t) appended with noisy-channel layer at the end 5: Update noisy model as M * (t) using D Detach required model M (t) from M * (t) by removing the final noisy-channel layer 7: Set P ← P \ B (t) 8: end for weights W = [w i,j] such that its output p = Wp. The weights w i,j represent the noisy-channel transition probabilities such that w i,j = p(y = i|y = j). Therefore, to be a valid noisy-channel, W is constrained as W ∈ {W | W .,j ∈ ∆ K−1, ∀ 1 ≤ j ≤ K}. While training we use the model upto the denoising layer and train using p, or label prediction y while for validation/testing we use the model output p or label prediction y. The active learning algorithm in the presence of noisy oracle is summarized as Algorithm 2. We now proceed to Section 5 for demonstrating the efficacy of our proposed methods across different datasets. We evaluate the algorithms for training CNNs on three datasets pertaining to image classification; (i) MNIST , (ii) CIFAR10 , and (iii) SVHN . We use the CNN architectures from (fchollet, 2015; . For all the architectures we use Adam with a learning rate of 1e − 3. The implementations are done on PyTorch , and we use the Scikit-learn package for Agglomerative clustering. For training the denoising layer, we initialize it with the identity matrix I K, i.e., assuming it to be noiseless. The number of clusters N (b) is taken to be as 5b. The uncertainty measure σ is computed as the variation ratio of the output prediction across 100 stochastic forward passes, as coined in , through the model using a validation set which is fixed apriori. The inverse uncertainty function β = f (σ) in Algorithm 1 is chosen from l (e 1/σ − 1), −l log(σ), where l is a scaling constant fixed using cross-validation. The cross-validation is performed only for the noise-free setting, and all other with different noise magnitude ε follow this choice. This is done so as to verify the robustness of the choice of parameters against different noise magnitudes which might not be known apriori. We compare our approach with: (i) Random: A batch is selected by drawing samples from the pool uniform at random without replacement. (ii) BALD: Using model uncertainty and the BALD score, the authors in ) do active learning with single sample acquisition. We use the highest b scoring samples to select a batch. (iii) Coreset: The authors in proposed a coreset based approach to select the representative core centroids of the pool set. We use the 2 − OP T approximation greedy algorithm of the paper with similarity measure as l 2 norm between the activations of the penultimate layer. (iv) Entropy: The approach of is implemented via selecting b samples with the highest Shannon entropy H(p) of the softmax outputs. (v) VAAL: The variational adversarial active learning of . In all our experiments, we start with a small number of images 40 − 50 and retrain the model from scratch after every batch acquisition. In order to make a fair comparison, we provide the same initial point for all active learning algorithms in an experiment. We perform a total of 20 random initializations and plot the average performance along with the standard deviation vs number of acquired samples by the algorithms. Figure 3 shows that our proposed algorithm outperform all the existing algorithms. As an important observation, we note that random selection always works better in the initial stages of all experiments. This observation is explained by the fact that all models suffer from inaccurate predictions at the initial stages. The proposed uncertainty based randomization makes a soft bridge between uniform random sampling and score based importance sampling of the cluster centroids. The proposed approach uses randomness at the initial stages and then learns to switch to weigh the model based inference scores as the model becomes increasingly certain of its output. Therefore, the proposed algorithm always envelops the performance of all the other approaches across all three datasets of MNIST, CIFAR10, and SVHN. Figure 3 also shows the negative impact of noisy oracle on the active learning performance across all three datasets. The degradation in the performance worsens with increasing oracle noise strength ε. We see that doing denoisification by appending noisy-channel layer helps combating the noisy oracle in Figure 3. The performance of the proposed noisy oracle active learning is significantly better in all the cases. The prediction accuracy gap between algorithm with/without denoising layer elevates with increase in the noise strength ε. The most recent baselines like (VAAL ), (Coreset ) which make representation of the Training + Pool may not always perform well. While coreset assigns distance between points based on the model output which suffers in the beginning, VAAL uses training data only to make representations together with the remaining pool in GAN like setting. The representative of pool points may not always help, especially if there are difficult points to label and the model can be used to identify them. In addition to the importance score, the model uncertainty is needed to assign a confidence to its judgement which is poor in the beginning and gets strengthened later. The proposed approach works along this direction. Lastly, while robustness against oracle noise is discussed in , however, we see that incorporating the denoising later implicitly in the model helps better. The intuitive reason being, having noise in the training data changes the discriminative distribution from p(y|x) to p(y |x). Hence, learning p(y |x) from the training data and then recovering p(y|x) makes more sense as discussed in Section 4.2. The uncertainty measure σ plays a key role for the proposed algorithm. We have observed that under strong noise influence from the oracle, the model's performance is compromised due to spurious training data as we see in Figure3. This affects the estimation of the uncertainty measure (variation ratio) as well. We see in Figure 4 that the model uncertainty does not drop as expected due to the label noise. However, the aid provided by the denoising layer to combat the oracle noise solves this issue. We observe in Figure 4 that uncertainty drops at a faster rate as the model along with the denoising layer gets access to more training data. Hence, the proposed algorithm along with the denoising layer make better judgment of soft switch between uniform randomness and importance sampling using. The availability of better uncertainty estimates for modern deep learning architectures is a promising future research, and the current work will also benefit from it. In this paper we have proposed a batch sample selection mechanism for active learning with access to noisy oracles. We use mutual information between model parameters and the predicted class probabilities as importance score for each sample, and cluster the pool sample space with JensonShannon distance. We incorporate model uncertainty/confidence into Gibbs distribution over the clusters and select samples from each cluster with importance sampling. We introduce an additional layer at the output of deep networks to estimate label noise. Experiments on MNIST, SVHN, and CIFAR10 show that the proposed method is more robust against noisy labels compared with the state of the art. Even in noise-free scenarios, our method still performs the best for all three datasets. Our contributions open avenues for exploring applicability of batch active learning in setups involving imperfect data acquisition schemes either by construction or because of resource constraints. Under review as a conference paper at ICLR 2020 A ESC50 CROWD LABELING EXPERIMENT We selected 10 categories of ESC50 and use Amazon Mechanical Turk for annotation. In each annotation task, the crowd worker is asked to listen to the sound track and pick the class that the sound belongs to, with confidence level. The crowd worker can also pick "Unsure" if he/she does not think the sound track clearly belongs to one of the 10 categories. For quality control, we embed sound tracks that clearly belong to one class (these are called gold standards) into the set of tasks an annotator will do. If the annotator labels the gold standard sound tracks wrong, then labels from this annotator will be discarded. The confusion table of this crowd labeling experiment is shown in Figure 5: each row corresponds to sound tracks with one ground truth class, and the columns are majority-voted crowd-sourced labels of the sound tracks. We can see that for some classes, such as frog and helicopter, even with 5 crowd workers, the majority vote of their annotation still cannot fully agree with the ground truth class. We present rest of the experimental supplementary to the ones presented in the main body of Section 5. The active learning algorithm performance for oracle noise strength of ε = 0.2 and ε = 0.4 are presented in Figure 6. Similarly to what discussed in Section 5, we observe that the performance of proposed algorithm dominates all other existing works for ε = 0.2. We witnessed that the proposed algorithm performance (without denoising layer) is not able to match other algorithms (BALD and Entropy) when ε = 0.4, even with more training data. The reason for this behavior can be explained using the uncertainty measure σ output in the Figure 7. We see that under strong noise influence from the oracle, the model uncertainty doesn't reduce along the active learning acquisition iterations. Because of this behavior, the proposed uncertainty based algorithm sticks to put more weightage on uniform random sampling, even with more training data. However, we see that using denoising layer, we have better model uncertainty estimates under the influence of noisy oracle. Since the uncertainty estimates improve, as we see in Figure 7, for ε = 0.4, the proposed algorithm along with the denoising layer performs very well and has significant improvement in performance as compared to other approaches. The for CIFAR10 dataset with oracle noise strength of ε = 0.2 and 0.4 are provided in the Figure 8. We see that the proposed algorithm without/with using the denoising layer outperforms other benchmarks. We provide the active learning accuracy for SVHN dataset with oracle noise strength of ε = 0.2 and 0.4 in the Figure 8. Similar to other , we see that the proposed algorithm without/with using the denoising layer outperforms other benchmarks for ε = 0.2. For oracle noise strength of ε = 0.4, we see a similar trend as MNIST regarding performance compromise to the proposed uncertainty based batch selection. The reason is again found in the uncertainty estimates plot in Figure 10 for ε = 0.4. With more mislabeled training examples, the model uncertainty estimate doesn't improve with active learning samples acquisition. Hence, the proposed algorithm makes the judgment of staying close to uniform random sampling. However, unlike MNIST in Figure 7, the uncertainty estimate is not that poor for SVHN, i.e., it still decays. Therefore, the performance loss in proposed algorithm is not that significant. While, upon using the denoising layer, the uncertainty estimates improve significantly, and therefore, the proposed algorithm along with the denoising layer outperforms other approaches by big margin. Using the same setup as explained in Section 5, we evaluate the performance on CIFAR100 dataset for various active learning algorithms listed in Section 5.2. We observe in Figure 11 that the proposed uncertainty based algorithm perform similar or better than the baselines. The incorporation of denoising layer helps in countering the affects of noisy oracle as we demonstrate by varying the noise strength ε = 0.1, 0.3. For a quantitative look at the active learning , mean and standard deviation of the performance vs. acquisition, in the Figure 3, we present the in the tabular format in Table 1 for MNIST, Table 2 for CIFAR10, Table 3 for SVHN, and Table 4 for CIFAR100, respectively. Table 3 Active learning for SVHN dataset.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We address the active learning in batch setting with noisy oracles and use model uncertainty to encode the decision quality of active learning algorithm during acquisition.

Represent the TLDR for finding the corresponding scientific article

Pseudo-labeling has shown to be a weak alternative for semi-supervised learning. We, conversely, demonstrate that dealing with confirmation bias with several regularizations makes pseudo-labeling a suitable approach.


Question: Represent the article for finding a single-sentence TLDR

Question: Artistic style transfer is the problem of synthesizing an image with content similar to a given image and style similar to another. Although recent feed-forward neural networks can generate stylized images in real-time, these models produce a single stylization given a pair of style/content images, and the user doesn't have control over the synthesized output. Moreover, the style transfer depends on the hyper-parameters of the model with varying ``optimum" for different input images. Therefore, if the stylized output is not appealing to the user, she/he has to try multiple models or retrain one with different hyper-parameters to get a favorite stylization. In this paper, we address these issues by proposing a novel method which allows adjustment of crucial hyper-parameters, after the training and in real-time, through a set of manually adjustable parameters. These parameters enable the user to modify the synthesized outputs from the same pair of style/content images, in search of a favorite stylized image. Our quantitative and qualitative experiments indicate how adjusting these parameters is comparable to retraining the model with different hyper-parameters. We also demonstrate how these parameters can be randomized to generate which are diverse but still very similar in style and content. Style transfer is a long-standing problem in computer vision with the goal of synthesizing new images by combining the content of one image with the style of another BID8 BID12 BID0. Recently, neural style transfer techniques BID9 BID15 BID11 BID20 BID19 showed that the correlation between the features extracted from the trained deep neural networks is quite effective on capturing the visual styles and content that can be used for generating images similar in style and content. However, since the definition of similarity is inherently vague, the objective of style transfer is not well defined and one can imagine multiple stylized images from the same pair of content/style images. Existing real-time style transfer methods generate only one stylization for a given content/style pair and while the stylizations of different methods usually look distinct BID27 BID13, it is not possible to say that one stylization is better in all contexts since people react differently to images based on their and situation. Hence, to get favored stylizations users must try different methods that is not satisfactory. It is more desirable to have a single model which can generate diverse , but still similar in style and content, in real-time, by adjusting some input parameters. One other issue with the current methods is their high sensitivity to the hyper-parameters. More specifically, current real-time style transfer methods minimize a weighted sum of losses from different layers of a pre-trained image classification model BID15 BID13 (check Sec 3 for details) and different weight sets can into very different styles (Figure 6). However, one can only observe the effect of these weights in the final stylization by fully retraining the model with the new set of weights. Considering the fact that the "optimal" set of weights can be different for any pair of style/content (Figure 3) and also the fact that this "optimal" truly doesn't exist (since the goodness of the output is a personal choice) retraining the models over and over until the desired is generated is not practical. Content (Fixed) Figure 1: Adjusting the output of the synthesized stylized images in real-time. Each column shows a different stylized image for the same content and style image. Note how each row still resembles the same content and style while being widely different in details. The primary goal of this paper is to address these issues by providing a novel mechanism which allows for adjustment of the stylized image, in real-time and after training. To achieve this, we use an auxiliary network which accepts additional parameters as inputs and changes the style transfer process by adjusting the weights between multiple losses. We show that changing these parameters at inference time to stylizations similar to the ones achievable by retraining the model with different hyperparameters. We also show that a random selection of these parameters at run-time can generate a random stylization. These solutions, enable the end user to be in full control of how the stylized image is being formed as well as having the capability of generating multiple stochastic stylized images from a fixed pair of style/content. The stochastic nature of our proposed method is most apparent when viewing the transition between random generations. Therefore, we highly encourage the reader to check the project website https://goo.gl/PVWQ9K to view the generated stylizations. The strength of deep networks in style transfer was first demonstrated by BID10. While this method generates impressive , it is too slow for real-time applications due to its optimization loop. Follow up works speed up this process by training feed-forward networks that can transfer style of a single style image BID15 or multiple styles. Other works introduced real-time methods to transfer style of arbitrary style image to an arbitrary content image BID11 BID13. These methods can generate different stylizations from different style images; however, they only produce one stylization for a single pair of content/style image which is different from our proposed method. Generating diverse have been studied in multiple domains such as colorizations BID6 BID2, image synthesis BID3, video prediction BID1 BID17, and domain transfer BID14 BID32. Domain transfer is the most similar problem to the style transfer. Although we can generate multiple outputs from a given input image BID14, we need a collection of target or style images for training. Therefore we can not use it when we do not have a collection of similar styles. Style loss function is a crucial part of style transfer which affects the output stylization significantly. The most common style loss is Gram matrix which computes the second-order statistics of the feature activations BID10, however many alternative losses have been introduced to measure distances between feature statistics of the style and stylized images such as correlation alignment loss BID24, histogram loss BID25, and MMD loss BID18. More recent work BID22 has used depth similarity of style and stylized images as a part of the loss. We demonstrate the success of our method using only Gram matrix; however, our approach can be expanded to utilize other losses as well. To the best of our knowledge, the closest work to this paper is BID31 in which the authors utilized Julesz ensemble to encourage diversity in stylizations explicitly. Although this Figure 2: Architecture of the proposed model. The loss adjustment parameters α α α c and α α α s is passed to the network Λ which will predict activation normalizers γ α α α and β α α α that normalize activation of main stylizing network T. The stylized image is passed to a trained image classifier where its intermediate representation is used to calculate the style loss L s and content loss L c. Then the loss from each layer is multiplied by the corresponding input adjustment parameter. Models Λ and T are trained jointly by minimizing this weighted sum. At generation time, values for α α α c and α α α s can be adjusted manually or randomly sampled to generate varied stylizations. method generates different stylizations, they are very similar in style, and they only differ in minor details. A qualitative comparison in FIG6 shows that our proposed method is more effective in diverse stylization. 1e-2 1e-3 1e-4 Style Figure 3: Effect of adjusting the style weight in style transfer network from BID15. Each column demonstrates the of a separate training with all w l s set to the printed value. As can be seen, the "optimal" weight is different from one style image to another and there can be multiple "good" stylizations depending on ones' personal choice. Check supplementary materials for more examples. Style transfer can be formulated as generating a stylized image p which its content is similar to a given content image c and its style is close to another given style image s. DISPLAYFORM0 The similarity in style can be vaguely defined as sharing the same spatial statistics in low-level features, while similarity in content is roughly having a close Euclidean distance in high-level features BID11. These features are typically extracted from a pre-trained image classification network, commonly VGG-19 . The main idea here is that the features obtained by the image classifier contain information about the content of the input image while the correlation between these features represents its style. In order to increase the similarity between two images, Gatys et al. BID10 minimize the following distances between their extracted features: DISPLAYFORM1 where φ l (x) is activation of a pre-trained classification network at layer l given the input image x, while L l c (p) and L l s (p) are content and style loss at layer l respectively. G(φ l (p)) denotes the Gram matrix associated with φ l (p). The total loss is calculated as a weighted sum of losses across a set of content layers C and style layers S: DISPLAYFORM2 where w l c, w l s are hyper-parameters to adjust the contribution of each layer to the loss. Layers can be shared between C and S. These hyper-parameters have to be manually fine tuned through try and error and usually vary for different style images (Figure 3). Finally, the objective of style transfer can be defined as: DISPLAYFORM3 This objective can be minimized by iterative gradient-based optimization methods starting from an initial p which usually is random noise or the content image itself. Solving the objective in Equation 3 using an iterative method can be very slow and has to be repeated for any given pair of style/content image. A much faster method is to directly train a deep network T which maps a given content image c to a stylized image p BID15. T is usually a feed-forward convolutional network (parameterized by θ) with residual connections between downsampling and up-sampling layers BID26 and is trained on many content images using Equation 3 as the loss function: DISPLAYFORM0 The style image is assumed to be fixed and therefore a different network should be trained per style image. However, for a fixed style image, this method can generate stylized images in realtime BID15. Recent methods BID11 BID13 introduced real-time style transfer methods for multiple styles. But, these methods still generate only one stylization for a pair of style and content images. In this paper we address the following issues in real-time feed-forward style transfer methods: 1. The output of these models is sensitive to the hyper-parameters w l c and w l s and different weights significantly affect the generated stylized image as demonstrated in FIG3. Moreover, the "optimal" weights vary from one style image to another (Figure 3) and therefore finding a good set of weights should be repeated for each style image. Please note that for each set of w l c and w l s the model has to be fully retrained that limits the practicality of style transfer models. Top row demonstrates that randomizing α α α to different stylizations however the style features appear in the same spatial position (e.g., look at the swirl effect on the left eye). Middle row visualizes the effect of adding random noise to the content image in moving these features with fixed α α α. Combination of these two randomization techniques can generate highly versatile outputs which can be seen in the bottom row. Notice how each image in this row differs in both style and the spatial position of style elements. Look at FIG8 for more randomized .2. Current methods generate a single stylized image given a content/style pair. While the stylizations of different methods usually look very distinct BID27, it is not possible to say which stylization is better for every context since it is a matter of personal taste. To get a favored stylization, users may need to try different methods or train a network with different hyper-parameters which is not satisfactory and, ideally, the user should have the capability of getting different stylizations in real-time. We address these issues by conditioning the generated stylized image on additional input parameters where each parameter controls the share of the loss from a corresponding layer. This solves the problem since one can adjust the contribution of each layer to adjust the final stylized after the training and in real-time. Secondly, we address the problem by randomizing these parameters which in different stylizations. We enable the users to adjust w DISPLAYFORM0 To learn the effect of α α α c and α α α s on the objective, we use a technique called conditional instance normalization (Ulyanov et al.). This method transforms the activations of a layer x in the feedforward network T to a normalized activation z which is conditioned on additional inputs α α α = [α α α c, α α α s]: DISPLAYFORM1 where µ and σ are mean and standard deviation of activations at layer x across spatial axes BID11 and γ α α α, β α α α are the learned mean and standard deviation of this transformation. These parameters can be approximated using a second neural network which will be trained end-to-end with T: Since L l can be very different in scale, one loss term may dominate the others which will fail the training. To balance the losses, we normalize them using their exponential moving average as a normalizing factor, i.e. each L l will be normalized to: DISPLAYFORM2 DISPLAYFORM3 where L l (p) is the exponential moving average of L l (p). In this section, first we study the effect of adjusting the input parameters in our method. Then we demonstrate that we can use our method to generate random stylizations and finally, we compare our method with a few baselines in terms of generating random stylizations. We implemented Λ as a multilayer fully connected neural network. We used the same architecture as BID15 BID11 for T and only increased number of residual blocks by 3 (look at supplementary materials for details) which improved stylization . We trained T and Λ jointly by sampling random values for α α α from U. We trained our model on ImageNet BID5 ) as content images while using paintings from Kaggle Painter by Numbers (Kaggle) and textures from Descibable Texture Dataset BID4 as style images. We selected random images form ImageNet test set, MS-COCO BID21 and faces from CelebA dataset as our content test images. Similar to BID11, we used the last feature set of conv3 as content layer C. We used last feature set of conv2, conv3 and conv4 layers from VGG-19 network as style layers S. Since there is only one content layer, we fix α α α c = 1. Our implementation can process 47.5 fps on a NVIDIA GeForce 1080, compared to 52.0 for the base model without Λ sub-network. The primary goal of introducing the adjustable parameters α α α was to modify the loss of each separate layer manually. Qualitatively, this is demonstrable by increasing one of the input parameters from zero to one while fixing the rest of them to zero. FIG0 shows one example of such transition. Each row in this figure is corresponding to a different style layer, and therefore the stylizations at each row would be different. Notice how deeper layers stylize the image with bigger stylization elements from the style image but all of them still apply the coloring. We also visualize the effect of increasing two of the input parameters at the same time in FIG7. However, these transitions are best demonstrated interactively which is accessible at the project website https://goo.gl/PVWQ9K. To quantitatively demonstrate the change in losses with adjustment of the input parameters, we rerun the same experiment of assigning a fixed value to all of the input parameters while gradually increasing one of them from zero to one, this time across 100 different content images. Then we calculate the median loss at each style loss layer S. As can be seen in, increasing α l s decreases the measured loss corresponding to that parameter. To show the generalization of our method across style images, we trained 25 models with different style images and then measured median of the loss at any of the S layers for 100 different content images FIG4 -(bottom). We exhibit the same drop trends as before which means the model can generate stylizations conditioned on the input parameters. Finally, we verify that modifying the input parameters α α α s generates visually similar stylizations to the retrained base model with different loss weights w l s. To do so, we train the base model BID15 One application of our proposed method is to generate multiple stylizations given a fixed pair of content/style image. To do so, we randomize α α α to generate randomized stylization (top row of FIG1). Changing values of α α α usually do not randomize the position of the "elements" of the style. We can enforce this kind of randomness by adding some noise with the small magnitude to the content image. For this purpose, we multiply the content image with a mask which is computed by applying an inverse Gaussian filter on a white image with a handful (< 10) random zeros. This masking can shadow sensitive parts of the image which will change the spatial locations of the "elements" of style. Middle row in FIG1 demonstrates the effect of this randomization. Finally, we combine these two randomizations to maximizes the diversity of the output which is shown in the bottom row of FIG1. More randomized stylizations can be seen in FIG8 and at https://goo.gl/PVWQ9K. BID31. Our method generates diverse stylizations while StyleNet mostly differ in minor details. To the best of our knowledge, generating diverse stylizations at real-time is only have been studied at BID31 before. In this section, we qualitatively compare our method with this baseline. Also, we compare our method with a simple baseline where we add noise to the style parameters. The simplest baseline for getting diverse stylizations is to add noises to some parameters or the inputs of the style-transfer network. In the last section, we demonstrate that we can move the locations of elements of style by adding noise to the content input image. To answer the question that if we can get different stylizations by adding noise to the style input of the network, we utilize the model of which uses conditional instance normalization for transferring style. We train this model with only one style image and to get different stylizations, we add random noise to the style parameters (γ α α α and β α α α parameters of equation 6) at run-time. The stylization for this baseline are shown on the top row of FIG6. While we get different stylizations by adding random noises, the stylizations are no longer similar to the input style image. To enforce similar stylizations, we trained the same baseline while we add random noises at the training phase as well. The stylization are shown in the second row of FIG6. As it can be seen, adding noise at the training time makes the model robust to the noise and the stylization are similar. This indicates that a loss term that encourages diversity is necessary. We also compare the of our model with StyleNet BID31. As visible in FIG6, although StyleNet's stylizations are different, they vary in minor details and all carry the same level of stylization elements. In contrast, our model synthesizes stylized images with varying levels of stylization and more randomization. Our main contribution in this paper is a novel method which allows adjustment of each loss layer's contribution in feed-forward style transfer networks, in real-time and after training. This capability allows the users to adjust the stylized output to find the favorite stylization by changing input parameters and without retraining the stylization model. We also show how randomizing these parameters plus some noise added to the content image can in very different stylizations from the same pair of style/content image. Our method can be expanded in numerous ways e.g. applying it to multi-style transfer methods such as BID11, applying the same parametrization technique to randomize the correlation loss between the features of each layer and finally using different loss functions and pre-trained networks for computing the loss to randomize the outputs even further. One other interesting future direction is to apply the same "loss adjustment after training" technique for other classic computer vision and deep learning tasks. Style transfer is not the only task in which modifying the hyper-parameters can greatly affect the predicted and it would be rather interesting to try this method for adjusting the hyper-parameters in similar problems. Convolution 3 1 C SAME ReLU Convolution 3 1 C SAME Linear Add the input and the output Upsampling -C feature maps Nearest-neighbor interpolation, factor 2 Convolution 3 1 C SAME ReLUNormalization Conditional instance normalization after every convolution Optimizer Adam (α = 0.001, β 1 = 0.9, β 2 = 0.999) Batch size 8 Weight initialization Isotropic gaussian (µ = 0, σ = 0.01) BID15. Each column demonstrates the of a separate training. As can be seen, the "optimal" weight is different from one style image to another and there can be more than one "good" stylization depending on ones personal choice. Figure 13: Results of combining losses from different layers at generation time by adjusting their corresponding parameters. The first column is the style image which is fixed for each row. The content image is the same for all of the outputs. The corresponding parameter for each one of the losses is zero except for the one(s) mentioned in the title of each column. Notice how each layer enforces a different type of stylization and how the combinations vary as well. Also note how a single combination of layers cannot be the "optimal" stylization for any style image and one may prefer the from another column.

represent this text to find a much much longer article about it with say hundreds-thousands of words

Stochastic style transfer with adjustable features.

Represent the short summary for finding the corresponding scientific article (long)

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.