Question: Represent the article for finding a single-sentence TLDR

Question: Understanding how people represent categories is a core problem in cognitive science, with the flexibility of human learning remaining a gold standard to which modern artificial intelligence and machine learning aspire. Decades of psychological research have yielded a variety of formal theories of categories, yet validating these theories with naturalistic stimuli remains a challenge. The problem is that human category representations cannot be directly observed and running informative experiments with naturalistic stimuli such as images requires having a workable representation of these stimuli. Deep neural networks have recently been successful in a range of computer vision tasks and provide a way to represent the features of images. In this paper, we introduce a method for estimating the structure of human categories that draws on ideas from both cognitive science and machine learning, blending human-based algorithms with state-of-the-art deep representation learners. We provide qualitative and quantitative as a proof of concept for the feasibility of the method. Samples drawn from human distributions rival the quality of current state-of-the-art generative models and outperform alternative methods for estimating the structure of human categories. Categorization (or classification) is a central problem in cognitive science BID1, artificial intelligence, and machine learning BID3. In its most general form, the categorization problem concerns why and how we divide the world into discrete units (and various levels of abstraction), and what we do with this information. The biggest challenge for studying human categorization is that the content of mental category representations cannot be directly observed, which has led to development of laboratory methods for estimating this content from human behavior. Because these methods rely on small artificial stimulus sets with handcrafted or lowdimensional feature sets, they are ill-suited to the study of categorization as an intelligent process, which is principally motivated by people's robust categorization performance in complex ecological settings. One of the challenges of applying psychological methods to realistic stimuli such as natural images is finding a way to represent them. Recent work in machine learning has shown that deep learning models, such as convolutional neural networks, perform well on a range of computer vision tasks BID10. The features discovered by these models provide a way to represent complex images compactly. It may be possible to express human category structure using these features, an idea supported by recent work in cognitive science BID8 BID12.Ideally, experimental methods could be combined with state-of-the-art deep learning models to estimate the structure of human categories with as few assumptions as possible and while avoiding the problem of dataset bias. In what follows, we propose a method that uses a human in the loop to directly estimate arbitrary distributions over complex feature spaces, adapting a framework that can exploit advances in deep architectures and computing power to increasingly capture and sharpen the precise structure of human category representations. Such knowledge is crucial to forming an ecological theory of intelligent categorization behavior and to providing a ground-truth benchmark to guide and inspire future work in machine learning. Methods for estimating human category templates have existed for some time. In psychophysics, the most popular and well-understood method is known as classification images BID0. In this experimental procedure, a human participant is shown stimuli from two classes, A and B, each with white noise overlaid, and asked to indicate the correct label. On most trials, the participant will select the exemplar generated from the category in question. However, if the added white noise significantly perturbs features of the image important to making that distinction, they may fail. Exploiting this, we can estimate the decision boundary from a number of these trials using the simple formula: DISPLAYFORM0 where n XY is the average of the noise across trials where the correct class is X and the observer chooses Y. BID15 used a variation on classification images using invertible deep feature spaces. In order to avoid dataset bias introduced through perturbing real class exemplars, white noise in the feature space was used to generate stimuli. In this special case, category templates reduce to n A − n B. On each trial of the experiment, participants were asked to select which of two images (inverted from feature noise) most resembled a particular category. Because the feature vectors were random, thousands of images could be pre-inverted offline using methods that require access to large datasets. This early inversion method was applied to mean feature vectors for thousands of positive choices in the experiments and yielded qualitatively decipherable category template images, as well as better machine classification decision boundaries that were regularized by human bias. Under the assumption that human category distributions are Gaussian with equal variance, this method yields the vector that aligns with the line between means, although a massive number of pairs of random vectors (trials) are required. BID5 conducted similar experiments in which a low-dimensional multi-scale gabor PCA basis was used to represent black-and-white images of scenes. Participants indicated which of two images was more similar to a seen image or a mental image of a scene. These judgments were used in an online genetic algorithm to lead participants to converge to their mental image. This method allowed for relatively efficient estimation of mental templates, although it is limited to scenes, lacks even the few theoretical guarantees of the BID15 method, and yields only a single meaningful image. Finally, an alternative to classification images, Markov Chain Monte Carlo with People (MCMCP; BID14, constructs an experimental procedure by which humans act as a valid acceptance function in the Metropolis-Hastings algorithm, exploiting the fact that Luce's choice axiom, a well-known model of human choice behavior, is equivalent to the Barker acceptance function (see equation in FIG0). On the first trial, a stimulus is drawn arbitrarily from the parameter space and compared to a new proposed stimulus that is nearby in that parameter space. The participant makes a forced choice of the better exemplar of some category (e.g., dog). If the initial stimulus is chosen, the Markov chain remains in that state. If the proposed stimulus is chosen, the chain moves to the proposed state. The process then repeats for as long as one wishes to run the sampling algorithm. MCMCP has been successfully employed to capture a number of different mental categories BID14 BID11, and though these spaces are higher-dimensional than those in previous laboratory experiments, they are still relatively small and artificial compared to real images. Unlike classification images, this method makes no assumptions about the structure of the category distributions and thus can estimate means, variances, and higher order moments. Therefore we take it as a basic starting point for this paper. Deep convolutional neural networks (CNN; BID9 like AlexNet BID7 are excellent for high-accuracy natural image classification and learn feature and category representations that are generally useful for a number of perceptual tasks. However, they can be difficult to interpret, especially in how they relate to human categorization behavior. * for the two states are presented to human raters on a computer screen (leftmost arrow and bottom left). Human raters then view the images in an experiment (bottom middle arrow) and act as part of an MCMC sampling loop, choosing between the two states/images in accordance with the Barker acceptance function (bottom right). The chosen image can then be sent to the inference network (rightmost arrow) and decoded in order to select the state for the next trial, however this step is unnecessary when we know exactly which states corresponds to which images. The inference network will, however, be needed later to characterize new images that we did not generate. Generative Adversarial Networks (GANs; BID4 and Variational Autoencoders (VAEs;) provide a generative approach to understanding image categories. In particular, these generative models provide an opportunity to examine how inference over a set of images and categories is performed. Generative models frame their approach theoretically on a Bayesian decomposition of image-label joint density, p(z, x) = p(z)p(x|z), where z and x are random variables that represent a natural image distribution and its feature representation. Therfore p(z) represents the distribution of natural images, and p(x|z) is the distribution of feature vectors given an image. Together, p(z) and p(x|z) produce a high-dimensional constraint on image generation. This allows for easy visualization of the learned latent space, a property we exploit presently. In the typical MCMCP experiment, as explained above, the participant judges pairs of stimuli, and the chosen stimuli serve as samples from the category distribution of interest. This method is effective as long as noise can be added to dimensions in the stimulus parameter space to create meaningful changes in content. When viewing natural images for example, in the space of all pixel intensities, added noise is very unlikely to modify the image in interesting ways. We propose to instead perturb images in a deep feature space that captures only essential variation. We can then show participants images decoded from these feature representations in order to relate human judgments to the underlying latent space of interest, where category distributions are easier to learn. The ing judgments (samples) approximate distributions that both derive arbitrary human category boundaries for natural images and can be sampled from to create new images, yielding new human-like generative image models. A schematic of this procedure is illustrated in FIG0. Note that the method we propose does not require an inference network in practice, since we always know which latent code produced the images shown to the subjects. This is true as long as we initialize our MCMC chain states as random points in the latent space as opposed to in pixel space. More specifically, popular deep generative networks such as GANs and VAEs learn a probability distribution p(z) over some latent variable z, from which the objective of the network encourages of the mapping (generator or decoder) M that p(f (z; M)) = p(x), where p(x) is the distribution over pixels in the dataset and f is the family of deterministic functions s.t. f: M × z → x. If we assume that human psychological feature space y from some invertible transformation T of x (it is simply some learned transformation of raw input), then correspondingly p(g(y; T −1) = p(x), where g is the family of functions s.t. g: T −1 × y → x. Because people in our experiments see only actual images x, perceived as y, the judgments they make are related to the deep latent space DISPLAYFORM0 Equation 2 assumes that humans approximate p(x) analogously to an unsupervised learner, however, even if T is a more specific feature representation supervised by classification pressure in the environment, we can assume information is only lost and not added, so that our deep generative models, if successful in their goal, will ultimately encode this information as a subset. Further, the equation above entails that p(y) need not be identical to p(z), and M need not equal T −1. However, for the practical sake of convergence using human participants, we would prefer that psychological space y be similar to the deep latent space z -we hope they contain the same relevant featural content for most images. While this assumption is hard to verify beyond face validity at this time, recent work suggests that some deep features spaces can be linearly transformed to approximate human psychological space BID12, and so we assume the deep latent space is relevant enough to human feature representations to act as a surrogate. There are several theoretical advantages to our method over previous efforts. First, MCMCP can capture arbitrary distributions, so it is not as sensitive to the structure of the underlying low-dimensional feature space and should provide better category boundaries than classification images when required. This is important when using various deep features spaces that were learned with different constraints. MCMC inherently spends less time in low probability regions and should in theory waste fewer trials. Having generated the images online and as a function of the participant's decisions, there is no dataset or sampling bias, and auto-correlation can be addressed by removing temporally adjacent samples from the chain. Finally, using a deep generator provides drastically clearer samples than shallow reconstruction methods, and can be potentially be trained end-to-end with an inference network that allows us to categorize new images using the learned distribution. For our experiments, we explored two image generator networks trained on various datasets. Since even relatively low-dimensional deep image embeddings are large compared to controlled laboratory stimulus parameter spaces, we use a hybrid proposal distribution in which a Gaussian with a low variance is used with probability P and a Gaussian with a high variance is used with probability 1 − P. This allows participants to both refine and escape nearby modes, but is simple enough to avoid excessive experimental piloting that more advanced proposal methods often require. Participants in all experiments completed exactly 64 trials (image comparisons), collectively taking about 5 minutes, containing segments of several chains for multiple categories. The order of the categories and chains within those categories were always interleaved. All experiments were conducted on Amazon Mechanical Turk. If a single image did not load for a single trial, the data for the subject undergoing that trial was completely discarded, and a new subject was recruited to continue on from the original chain state. We first test our method using DCGAN BID13 trained on the Asian Faces Dataset. We chose this dataset because it requires a deep architecture to produce reasonable samples (unlike MNIST, for example), yet it is constrained enough to test-drive our method using a relatively simple latent space. Four chains for each of four categories (male, female, happy, and sad) were used. Proposals were generated from an isometric Gaussian with an SD of 0.25 50% of the time, and 2 otherwise. In addition, we conducted a baseline in which two new initial state proposals were drawn on every trial, and were independent of previous trials (classification images). The final dataset contained 50 participants and over 3, 200 trials (samples) in total for all chains. The baseline classification images (CI) dataset contained the same number of trials and participants. MCMCP chains are visualized using Fisher Linear Discriminant Analysis in FIG1, along with the ing averages for each chain and each category. Chain means within a category show interesting variation, yet converge to very similar regions in the latent space as expected. Also on the right of FIG1 are visualizations of the mean faces for both methods in the final two columns. MCMCP means appear to have converged quickly, whereas CI means only show a moderate resemblance to their corresponding category (e.g., the MCMCP mean for "happy" is fully smiling, while the CI mean only barely reveals teeth). All four CI means appear closer to a mean face, which is what one would expect from averages of noise. We validated this improvement with a human experiment in which 30 participants made forced choices between CI and MCMCP means. The are reported in FIG2. MCMCP means are consistently highly preferred as representations of each category as compared to CI. This remained true even when an additional 50 participants (total of 100) were run on the CI task, obtaining twice as many image comparison trials as MCMCP. The in the previous section show that reasonable category templates can be obtained using our method, yet the complexity of the stimulus space used does not rival that of large object classification networks. In this section, we tackle a more challenging (and interesting) form of the problem. To do this, we employ a bidirectional generative adversarial network (BiGAN; BID2 trained on the entire 1.2 million-image ILSVRC12 dataset (64×64 center-cropped). BiGAN includes an inference network, which regularizes the rest of the model and produces unconditional samples competitive with the state-of-the-art. This also allows for the later possibility of comparing human distributions with other networks as well as assessing machine classification performance with new images based on the granular human biases captured. Our generator network was trained given uniform rather than Gaussian noise, which allows us to guarantee participants cannot get lost in extremely improbable regions. Additionally, we avoid proposing states outside of this hypercube by forcing z to wrap around (proposals that travel out-side of z are injected back in from the opposite direction by the amount originally exceeded). In particular, we run our MCMC chains through an unbounded state space by redefining each bounded dimension z k as DISPLAYFORM0 Proposals were generated from an isometric Gaussian with an SD of 0.1 60% of the time, and 0.7 otherwise. We use this network to obtain large chains for two groups of five categories. Group1 included bottle, car, fire hydrant, and person, television, following BID15. Group2 included bird, body of water, fish, flower, and landscape. Each chain was approximately 1, 040 states long, and four of these chains were used for each category (approximately 4, 160). In total, across both groups of categories, we obtained exactly 41, 600 samples from 650 participants. To demonstrate the efficiency and flexibility of our method compared to alternatives, we obtained an equivalent number of trials for all categories using the variant of classification images introduced in BID15, with the exception that we used our BiGAN generator instead of the offline inversion previously used. This also serves as an important baseline against which to quantitatively evaluate our method because it estimates the simplest possible template. The acceptance rate was approximately 50% for both category groups, which is near the common goal for MCMCP experiments. The samples for all ten categories are shown in FIG4 and D using Fisher Linear Discriminant Analysis. Similar to the face chains, the four chains for each category converge to similar regions in space, largely away from other categories. In contrast, classification images shows little separation with so few trials (5C and D). Previous work suggests that at least an order of magnitude higher number of comparisons may be needed for satisfactory estimation of category means. Our method estimates well-separated category means in a manageable number of trials, allowing for the method to scale greatly. This makes sense given that CI proposes comparisons between arbitrary images, potentially wasting many trials, and clearly suffers from a great deal of noise. Beyond yielding our decision rule, our method additionally produces a density estimate of the entire category distribution. In classification images, only mean template images can be viewed, while we are able to visualize several modes in the category distribution. FIG3 visualizes these modes using the means of each component in a mixture of Gaussians density estimate. This produces realistic-looking multi-modal mental category templates, which to our knowledge has never been accomplished with respect to natural image categories. We also provide a quantitative assessment of the samples we obtained and compare them to classification images (CI) using an external classification task. To do this, we scraped approximately 500 images from Flickr for each of our ten categories, which was used for a classification task. To classify the images using our human-derived samples, we used the nearest-mean decision rule, and a decision rule based on the highest log-probability given by our ten density estimates. For classification images, only a nearest-mean decision rule can be tested. In all cases, decision rules based on our MCMCP-obtained samples overall outperform a nearestmean decision rule using classification images (see TAB0). In category group 1, the MCMCP density performed best and was more even across classes. In category group 2, nearest-mean using our MCMCP samples did much better than a density estimate or CI-based nearest-mean. Our demonstrate the potential of our method, which leverages both psychological methods and deep surrogate representations to make the problem of capturing human category representations tractable. The flexibility of our method in fitting arbitrary generative models allows us to visualize multi-modal category templates for the first time, and improve on human-based classification performance benchmarks. It is difficult to guarantee that our chains explored enough of the relevant space to actually capture the concepts in their entirety, but the diversity in the modes visualized and the improvement in class separation achieved are positive indications that we are on the right track. Further, the framework we present can be straightforwardly improved as generative image models advance, and a number of known methods for improving the speed, reach, and accuracy of MCMC algorithms can be applied to MCMCP make better use of costly human trials. There are several obvious limitations of our method. First, the structure of the underlying feature spaces used may either lack the expressiveness (some features may be missing) or the constraints (too many irrelevant features or possible images wastes too many trials) needed to map all characteristics of human mental categories in a practical number of trials. Even well-behaved spaces are very large and require many trials to reach convergence. Addressing this will require continuing exploration of a variety of generative image models. We see our work are as part of an iterative refinement process that can yield more granular human observations and inform new deep network

Represent the short summary for finding the corresponding scientific article (long)

using deep neural networks and clever algorithms to capture human mental visual concepts

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We leverage deterministic autoencoders as generative models by proposing mixing functions which combine hidden states from pairs of images. These mixes are made to look realistic through an adversarial framework.


Question: Represent the scientific article to find a short TLDR

Question: The application of deep recurrent networks to audio transcription has led to impressive gains in automatic speech recognition (ASR) systems. Many have demonstrated that small adversarial perturbations can fool deep neural networks into incorrectly predicting a specified target with high confidence. Current work on fooling ASR systems have focused on white-box attacks, in which the model architecture and parameters are known. In this paper, we adopt a black-box approach to adversarial generation, combining the approaches of both genetic algorithms and gradient estimation to solve the task. We achieve a 89.25% targeted attack similarity after 3000 generations while maintaining 94.6% audio file similarity. Figure 1: Example of targeted adversarial attack on speech to text systems in practice combination of genetic algorithms and gradient estimation to solve this task. The first phase of the 36 attack is carried out by genetic algorithms, which are a gradient-free method of optimization that 37 iterate over populations of candidates until a suitable sample is produced. In order to limit excess 38 mutations and thus excess noise, we improve the standard genetic algorithm with a new momentum 39 mutation update. The second phase of the attack utilizes gradient estimation, where the gradients 40 of individual audio points are estimated, thus allowing for more careful noise placement when the 41 adversarial example is nearing its target. The combination of these two approaches provides a 89.25% 42 average targeted attack similarity with a 94.6% audio file similarity after 3000 generations. 43 Adversarial attacks can be created given a variety of information about the neural network, such as 45 the loss function or the output probabilities. However in a natural setting, usually the neural network 46 behind such a voice control system will not be publicly released so an adversary will only have access 47 to an API which provides the text the system interprets given a continuous waveform. Given this 48 constraint, we use the open sourced Mozilla DeepSpeech implementation as a black box system, 49 without using any information on how the transcription is done. We perform our black box targeted attack on a model M given a benign input x and a target t by 51 perturbing x to form the adversarial input x = x + δ, such that M (x) = t. To minimize the audible 52 noise added to the input, so a human cannot notice the target, we maximize the cross correlation 53 between x and x. A sufficient value of δ is determined using our novel black box approach, so we 54 do not need access to the gradients of M to perform the attack. Compared to images, audio presents a much more significant challenge for models to deal with. While 57 convolutional networks can operate directly on the pixel values of images, ASR systems typically 58 require heavy pre-processing of the input audio. Most commonly, the Mel-Frequency Cepstrum (MFC) transform, essentially a fourier transform of the sampled audio file, is used to convert the 60 input audio into a spectogram which shows frequencies over time. Models such as DeepSpeech (Fig. 61 2) use this spectogram as the initial input. Extending the research done by BID0, we propose a genetic algorithm and gradient estimation approach 81 to create targeted adversarial audio, but on the more complex DeepSpeech system. The difficulty of 82 this task comes in attempting to apply black-box optimization to a deeply-layered, highly nonlinear 83 decoder model that has the ability to decode phrases of arbitrary length. Nevertheless, the combination 84 of two differing approaches as well as the momentum mutation update bring new success to this task. DeepSpeech outputs a probability distribution over all characters at every frame, for 50 frames per if EditDistance(t, Decode(best)) > 2 then // phase 1 -do genetic algorithm while populationSize children have not been made do Select parent1 from topk(population) according to sof tmax(their score) Select parent2 from topk(population) according to sof tmax(their score) child ← M utate(Crossover(parent1, parent2), p) end while newScores ← −CT CLoss(newPopulation, t) p ← M omentumU pdate(p, newScores, scores) else // phase 2 -do gradient estimation top-element ← top(population) grad-pop ← n copies of top-element, each mutated slightly at one index grad ← (−CT CLoss(grad-pop) − scores)/mutation-delta pop ← top-element + grad end if end while return best As mentioned previously, Alzantot et al. BID0 demonstrated the success of a black-box adversarial 125 attack on speech-to-text systems using a standard genetic algorithm. The basic premise of our 126 algorithm is that it takes in the benign audio sample and, through trial and error, adds noise to the 127 sample such that the perturbed adversarial audio is similar to the benign input yet is decoded as the Loss, we make modifications to the genetic algorithm and introduce our novel momentum mutation. CTC-Loss, which as mentioned previously, is used to determine the similarity between an input audio 137 sequence and a given phrase. We then form our elite population by selecting the best scoring samples 138 from our population. The elite population contains samples with desirable traits that we want to carry 139 over into future generations. We then select parents from the elite population and perform Crossover, 140 which creates a child by taking around half of the elements from parent1 and the other half from 141 parent2. The probability that we select a sample as a parent is a function of the sample's score. With some probability, we then add a mutation to our new child. Finally, we update our mutation 143 probabilities according to our momentum update, and move to the next iteration. The population will 144 continue to improve over time as only the best traits of the previous generations as well as the best 145 mutations will remain. Eventually, either the algorithm will reach the max number of iterations, or 146 one of the samples is exactly decoded as the target, and the best sample is returned. 148 Algorithm 2 Mutation Input: Audio Sample x Mutation Probability p Output: Mutated Audio Sample x for all e in x do noise ← Sample(N (µ, σ 2)) if Sample(Unif) < p then e ← e + f ilter highpass (noise) end if end for return xThe mutation step is arguably the most crucial component of the genetic algorithm and is our only 149 source of noise in the algorithm. In the mutation step, with some probability, we randomly add noise 150 to our sample. Random mutations are critical because it may cause a trait to appear that is beneficial 151 for the population, which can then be proliferated through crossover. Without mutation, very similar 152 samples will start to appear across generations; thus, the way out of this local maximum is to nudge it 153 in a different direction in order to reach higher scores. Furthermore, since this noise is perceived as noise, we apply a filter to the noise before 155 adding it onto the audio sample. The audio is sampled at a rate of f s = 16kHz, which means that 156 the maximum frequency response f max = 8kHz. As seen by Reichenbach and Hudspeth, given 157 that the human ear is more sensitive to lower frequencies than higher ones, we apply a highpass filter 158 at a cutoff frequency of f cutof f = 7kHz. This limits the noise to only being in the high-frequency 159 range, which is less audible and thus less detectable by the human ear. While mutation helps the algorithm overcome local maxima, the effect of mutation is limited by the 161 mutation probability. Much like the step size in SGD, a low mutation probability may not provide 162 enough randomness to get past a local maximum. If mutations are rare, they are very unlikely to 163 occur in sequence and add on to each other. Therefore, while a mutation might be beneficial when 164 accumulated with other mutations, due to the low mutation probability, it is deemed as not beneficial 165 by the algorithm in the short term, and will disappear within a few iterations. This parallels the step 166 size in SGD, because a small step size will eventually converge back at the local minimum/maximum. However, too large of a mutation probability, or step size, will add an excess of variability and prevent 168 the algorithm from finding the global maximum/minimum. To combat these issues, we propose Momentum Mutation, which is inspired by the Momentum Update for Gradient Descent. With this 170 update, our mutation probability changes in each iteration according to the following exponentially 171 weighted moving average update: DISPLAYFORM0 With this update equation, the probability of a mutation increases as our population fails to adapt 173 meaning the current score is close to the previous score. The momentum update adds acceleration 174 to the mutation probability, allowing mutations to accumulate and add onto each other by keeping 175 the mutation probability high when the algorithm is stuck at a local maximum. By using a moving 176 average, the mutation probability becomes a smooth function and is less susceptible to outliers in the 177 population. While the momentum update may overshoot the target phrase by adding random noise, 178 overall it converges faster than a constant mutation probability by allowing for more acceleration in Genetic algorithms work well when the target space is large and a relatively large number of mutation 182 directions are potentially beneficial; the strength of these algorithms lies in being able to search 183 large amounts of space efficiently BID7. When an adversarial sample nears its target perturbation, 184 this strength of genetic algorithms turn into a weakness, however. Close to the end, adversarial 185 audio samples only need a few perturbations in a few key areas to get the correct decoding. In this 186 case, gradient estimation techniques tend to be more effective. Specifically, when edit distance of 187 the current decoding and the target decoding drops below some threshold, we switch to phase 2. When approximating the gradient of a black box system, we can use the technique proposed by Nitin DISPLAYFORM0 Here, x refers to the vector of inputs representing the audio file. δ i refers to a vector of all zeros, Of the audio samples for which we ran our algorithm on, we achieved a 89.25% similarity between the 217 final decoded phrase and the target using Levenshtein distance, with an average of 94.6% correlation 218 similarity between the final adversarial sample and the original sample. The average final Levenshtein 219 distance after 3000 iterations is 2.3, with 35% of the adversarial samples achieving an exact decoding 220 in less than 3000 generations, and 22% of the adversarial samples achieving an exact decoding in less 221 than 1000 generations. One thing to note is that our algorithm was 35% successful in getting the decoded phrase to match 223 the target exactly; however, noting from figure 5, the vast majority of failure cases are only a few edit 224 distances away from the target. This suggests that running the algorithm for a few more iterations 225 could produce a higher success rate, although at the cost of correlation similarity. Indeed, it becomes 226 apparent that there is a tradeoff between success rate and audio similarity such that this threshold 227 could be altered for the attacker's needs. One helpful visualization of the similarity between the original audio sample and the adversarial 233 audio sample through the overlapping of both waveforms, as shown in figure 4. As the visualization 234 shows, the audio is largely unchanged, and the majority of the changes to the audio is in the relatively 235 low volume noise applied uniformly around the audio sample. This in an audio sample that 236 still appears to transcribe to the original intended phrase when heard by humans, but is decoded as 237 the target adversarial phrase by the DeepSpeech model. That 35% of random attacks were successful in this respect highlights the fact that black box 239 adversarial attacks are definitely possible and highly effective at the same time. 4 Conclusion In combining genetic algorithms and gradient estimation we are able to achieve a black box adversarial 242 example for audio that produces better samples than each algorithm would produce individually. By 243 initially using a genetic algorithm as a means of exploring more space through encouragement of 244 random mutations and ending with a more guided search with gradient estimation, we are not only 245 able to achieve perfect or near-perfect target transcriptions on most of the audio samples, we were able 246 to do so while retaining a high degree of similarity. While this remains largely as a proof-of-concept 247 demonstration, this paper shows that targeted adversarial attacks are achievable on black box models 248 using straightforward methods. Furthermore, the inclusion of momentum mutation and adding noise exclusively to high frequencies 250 improved the effectiveness of our approach. Momentum mutation exaggerated the exploration at the 251 beginning of the algorithm and annealed it at the end, emphasizing the benefits intended by combining 252 genetic algorithms and gradient estimation. Restricting noise to the high frequency domain improved 253 upon our similarity both subjectively by keeping it from interfering with human voice as well as 254 objectively in our audio sample correlations. By combining all of these methods, we are able to 255 achieve our top . In , we introduce a new domain for black box attacks, specifically on deep, nonlinear 257 ASR systems that can output arbitrary length translations. Using a combination of existing and novel 258 methods, we are able to exhibit the feasibility of our approach and open new doors for future research.

We present a novel black-box targeted attack that is able to fool state of the art speech to text transcription.

Represent the short summary for finding the corresponding scientific article (long)

Our proposed algorithm does not use all of the unlabeled data for the training, and it rather uses them selectively.


Question: Represent the scientific article to find a short TLDR

Question: Recent progress on physics-based character animation has shown impressive breakthroughs on human motion synthesis, through imitating motion capture data via deep reinforcement learning. However, have mostly been demonstrated on imitating a single distinct motion pattern, and do not generalize to interactive tasks that require flexible motion patterns due to varying human-object spatial configurations. To bridge this gap, we focus on one class of interactive tasks---sitting onto a chair. We propose a hierarchical reinforcement learning framework which relies on a collection of subtask controllers trained to imitate simple, reusable mocap motions, and a meta controller trained to execute the subtasks properly to complete the main task. We experimentally demonstrate the strength of our approach over different single level and hierarchical baselines. We also show that our approach can be applied to motion prediction given an image input. A video highlight can be found at https://youtu.be/XWU3wzz1ip8/. The capability of synthesizing realistic human-scene interactions is an important basis for simulating human living space, where robots can be trained to collaborate with humans, e.g. avoiding collisions or expediting the completion of assistive tasks. Motion capture (mocap) data, by offering high quality recordings of articulated human pose, has provided a crucial resource for human motion synthesis. With large mocap datasets and deep learning algorithms, kinematics-based approaches have recently made rapid progress on motion synthesis and prediction (; ; ; ; Bütepage et al., 2017; ; ; ; a; b;). However, the lack of physical interpretability in their synthesized motion has been a major limitation of these approaches. The problem becomes especially clear when it comes to motions that involve substantial human-object or human-human interactions. Without modeling the physics, the sythensized interactions are often physically unrealistic, e.g. body parts penetrating obstacles or not reacting to collision. This generally limits the use of these approaches to either non-interactive motions, or a carefully set up virtual scene with high fidelity to the captured one. The graphics community has recently witnessed impressive progress on physics-based character animation (; b). These approaches, through imitating mocap examples via deep reinforcement learning, can synthesize realistic motions in physics simulated environments. Consequently, they can adapt to different physical contexts and thus attain a better generalization performance for interaction-based motions, e.g. walking on uneven terrain or stunt performance under obstacle disturbance. Nonetheless, these approaches still suffer from a drawback-a single model is trained for performing a single task with a distinct motion pattern (often time from a single mocap clip). As a , they might not generalize to higher-level interactive tasks that require flexible motion patterns. Take the example of a person sitting down on a chair. A person can start in any location and orientation relative to the chair (Fig. 1). A fixed motion pattern (e.g. turn left and sit) will be incapable of handling such variations. In this paper, we focus on one class of high-level interactive tasks-sitting onto a chair. As earlier mentioned, there are many possible human-chair configurations and different configurations may require different sequences of actions to accomplish the goal. For example, if the human is facing the chair, it needs to walk, turn either left or right, and sit; if the human is behind the chair, it needs to walk, side-walk and sit. To this end, we propose a hierarchical reinforcement learning (RL) method to address the challenge of generalization. Our key idea is the use of hierarchical control: we assume the main task (e.g. sitting onto a chair) can be decomposed into several subtasks (e.g. walk, turn, sit, etc.), where the motion of each subtask can be reliably learned from mocap data, and we train a meta controller using RL which can execute the subtasks properly to "complete" the main task from a given configuration. Such strategy is in line with the observation that humans have a repertoire of motion skills, and different subset of skills is selected and executed for different high-level tasks. Our contributions are three folds: we extend the prior work on physics-based motion imitation to the context of higher-level interactive tasks using a hierarchical approach; we experimentally demonstrate the strength of our hierarchical approach over different single level and hierarchical baselines; we show at the end that our approach can be applied to motion synthesis in human living space with the help of 3D scene reconstruction. Kinematics-based Models Kinematic modeling of human motions has a substantial literature in both vision and graphics domains. Conventional methods such as motion graphs require a large corpus of mocap data and face challenges in generalizing to new behaviors in new context. Recent progress in deep learning enables researchers to explore more efficient algorithms to model human motions, again, from large-scale mocap data. The focus in the vision community is often motion prediction (; ; ; Bütepage et al., 2017; ; ; a; b; ;, where a sequence of mocap poses is given as historical observation and the goal is to predict future poses. Recent work has even started to predict motions directly from a static image (; ;). In the graphics community, the focus has been primarily on motion synthesis, which aims to synthesis realistic motions from mocap examples (; Agrawal & van de ; ; . Regardless of the focus, this class of approaches still faces the challenge of generalization due to the lack of physical plausibility in the synthesized motion, e.g. foot sliding and obstacle penetrations. Physics-based Models Physics simulated character animation has a long history in computer graphics (; ; ; ; a; ; b). Our work is most related to the recent work by Peng et al. (;, which trained a virtual character to imitate mocap data using deep reinforcement learning. They demonstrated robust and realistic looking motions on a wide array of skills including locomotion and acrobatic motions. Notably, they have used a hierarchical model for the task of navigating on irregular terrain . However, their meta task only requires a single subtask (i.e. walk), and the meta controller focuses solely on steering. We address a more complex task (i.e. sitting onto a chair) which requires the execution of diverse subtasks (e.g. walk, turn, and sit). Another recent work that is closely related to ours is that of , which addressed the task of dressing also with a hierarchical model. However, their subtasks are executed in a pre-defined order, and the completion of subtasks is determined by handcoded rules. In contrast, our meta controller is trained and is free to select any subtask at any time point. This is crucial when the main task cannot always be completed by a fixed order of subtasks. Note that humanoid control in physics simulated environments is also a widely-used benchmark task in the RL community, for example, to investigate how to ease the design of the reward function. However, work in this domain focuses less on realistic motions. Hierarchical Reinforcement Learning Our model is inspired by a series of recent work on hierarchical control in deep reinforcement learning (; ;). Although in different contexts, they share the same attribute that the tasks of concern have high-dimensional action space, but can be decomposed into simpler, reusable subtasks. Such decomposition may even help in generalizing to new high-level tasks due to the shared subtasks. Object Affordances Our work is connected to the learning of object affordances in the vision domain. Affordances express the functionality of objects and how humans can interact with them. Prior work attempted to detect affordances of a scene, represented as a set of plausible human poses, by training on large video corpora (; ; . Instead, we learn the motion in a physics simulated environment using limited mocap examples and reinforcement learning. Another relevant work also detected affordances using mocap data , but focused only on static pose rather than motion. Our main task is the following: given a chair and a skeletal pose of a human in the 3D space, generate a sequence of skeletal poses that describes the motion of the human sitting onto the chair from the given pose (Fig. 1). Our system builds upon a physics simulated environment which contains an articulated structured humanoid and a rigid body chair model. Each joint of the humanoid (except the root) can receive a control signal and produce dynamics from the physics simulation. The goal is to learn a policy that controls the humanoid to successfully sit on the chair. Fig. 2 (left) illustrates the hierarchical architecture of our policy. At the lower level is a set of subtask controllers, each responsible for generating the control input of a particular subtask. As illustrated in Fig. 2 (right), we consider four subtasks: walk, left turn, right turn, and sit. 1 To synthesize realistic motions, the subtask policies are trained on mocap data to imitate real human motions. At the higher level, a meta controller is responsible for controlling the execution of subtasks to ultimately accomplish the main task. The subtask controllers and meta controller generate control input at different timescales-60 Hz for the former and 2Hz for the latter. The physics simulation runs at 240 Hz. Each subtask as well as the meta controlling task is formulated as an independent reinforcement learning problem. We leverage recent progress in deep RL and approximate each policy using a neural network. A subtask controller is a policy network π(a t |s t) that maps a state vector s t to an action a t at each timestep t. The state representation s is extracted from the current configuration of the simulation environment, and may vary for different subtasks. For example, turn requires only proprioceptive information of the humanoid, while sit requires not only such information, but also the pose of the chair relative to the humanoid. The action a is the signal for controlling the humanoid joints for each subtask. We use a humanoid model with 21 degrees of freedom, i.e. a ∈ R 21. The network architecture is fixed across the subtasks: we use a multi-layer perceptron with two hidden layers of size 64. The output of the network parameterizes the probability distribution of a, modeled by a Gaussian distribution with a fixed diagonal covariance matrix, i.e. π(a|s) = N (µ(s), Σ) and Σ = diag({σ i}). We can generate a t at each timestep by sampling from π(a t |s t). Each subtask is formulated as an independent RL problem. At timestep t, the state s t given by the simulation environment is fed into the policy network to output an action a t. The action a t is then fed back to the simulation environment to generates the state s t+1 at the next timestep and a reward signal r t. The design of the reward function is crucial and plays a key role in shaping the style of the humanoid's motion. A heuristically crafted reward may yield a task achieving policy, but may in unnatural looking motions and behaviors. Inspired by Peng et al. (2018a), we set the reward function of each subtask by a sum of two terms that simultaneously encourages the imitation of the mocap reference and the achievement of the task objectives: r S and r G account for the similarly to the reference motion and the achievement of the subtask goals, respectively. We use a consistent similarity reward r S across all subtasks: where r p and r v encourage the similarity of local joint angles q j and velocitiesq j between the humanoid and the reference motion, and ω p and ω v are the respective weights. Specifically, where d(·, ·) computes the angular difference between two angles. We empirically set ω p = 0.5, ω v = 0.05, α p = 1, and α v = 10. Next, we detail the state representation s and task objective reward r G for each subtask. The state s walk ∈ R 52 consists of a 50-d proprioceptive feature and a 2-d goal feature that specifies an intermediate walking target. The proprioceptive feature includes the local joint angles and velocities, the height and linear velocity of the root (i.e. torso) as well as its pitch and roll angles, and a 2-d binary vector indicating the contact of each foot with the ground (Fig. 3). Rather than walking in random directions, target-directed locomotion (Agrawal & van de) is necessary for accomplishing high-level tasks. Assuming a target is given, represented by a 2D point on the ground plane, the 2-d goal feature is given by [sin(ψ), cos(ψ)], where ψ is the azimuth angle to the target in the humanoid centric coordinates. The generation of targets will be detailed in the meta controller section (Sec. 5). We observe that it is challenging to directly train a target-directed walking policy with mocap examples. Therefore we adopt a two-stage training strategy where each stage uses a distinct task objective reward. In the first stage, we encourage similar steering patterns to the reference motion, i.e. the linear velocity of the root v ∈ R 3 should be similar between the humanoid and reference motion: In the second stage, we reward motion towards the target: where denotes the horizontal distance between the root and the target, and δt is the length of the timestep. 2) Left/Right Turn The states s lturn, s rturn ∈ R 50 reuse the 50-d proprioceptive feature from the walk subtask. The task objective reward encourages the rotation of the root to be matched between the humanoid and reference motion: where θ ∈ R 3 consists of the root's pitch, yaw, and roll. 3) Sit The sit subtask assumes that the humanoid is initially standing roughly in front of the chair and facing away. The task is simply to lower the body and be seated. Different from walk and turn, the state for sit should capture the pose information of the chair. Our state s sit ∈ R 57 consists of the same 50-d proprioceptive feature used in walk and turn, and additionally a 7-d feature describing the state of the chair in the humanoid centric coordinates. The 7-d chair state includes the displacement vector from the pelvis to the center of the seat surface, and the rotation of the chair in the humanoid centric coordinates represented as a quaternion (Fig. 3). The task objective reward encourages the pelvis to move towards the center of the seat surface: where t is the 3D distance between the pelvis and the center of the seat surface. The meta controller is also a policy network and shares the same architecture as the subtask controllers. As the goal now is to navigate the humanoid to sit on the chair, the input state s meta should encode the pose information of the chair. We reuse the 57-d state representation from the sit subtask which contains both the proprioceptive and chair information. Rather than directly controlling the humanoid joints, the output action a meta now controls the execution of subtasks. Specifically, a meta = {a switch, a target} consists of two components. a switch ∈ {walk, left turn, right turn, sit} is a discrete output which at each timestep picks a single subtask out of the four to execute. a target ∈ R 2 specifies the 2D target for the walk subtask, which is used to compute the goal state in s walk. Note that a target is only used when the walk subtask is picked for execution. The output of the policy network parameterizes the probability distributions of both a switch and a target, where a switch is modeled by a categorical distribution as in standard classification problems, and a target is modeled by a Gaussian distribution following the subtask controllers. The meta task is also formulated as an independent RL problem. At timestep t, the policy network takes the state s. Rather than evaluating the similarity to a mocap reference, the reward now should be providing feedback on the main task. We adopt a reward function that encourages the pelvis to move towards and be in contact with the seat surface: z contact indicates whether the pelvis is in contact with the seat surface, which can be detected by the physics simulator. V sit is defined as in Eq. 7. Since the subtasks and meta task are formulated as independent RL problems, they can be trained independently using standard RL algorithms. We first train each subtask controllers separately, and then train the meta controller using the trained subtask controllers. All controllers are trained in a standard actor-critic framework using the proximal policy optimization (PPO) algorithm. The training of the subtasks is also divided into two stages. First, in each episode, we initialize the pose of the humanoid to the first frame of the reference motion, and train the humanoid to execute the subtask by imitating the following frames. We apply the early termination strategy (a): an episode is terminated immediately if the height of the root falls below 0.78 meters for walk and turn, and 0.54 meters for sit. These thresholds are chosen according to the height of the humanoid. For turn, the episode is also terminated when the root's yaw angle differs from the reference motion for more than 45 •. For walk, we adopt the two-stage training strategy described in Sec. 4. In target-directed walking, we randomly sample a new 2D target in the front of the humanoid every 2.5 seconds or when the target is reached. For sit, the chair is placed at a fixed location behind the humanoid, and we use reference state initialization (a) to facilitate training. The training above enables the humanoid to perform the subtasks from the initial pose of the reference motion. However, this does not guarantee successful transitions between subtasks (e.g. walk→turn), which is required for the main task. Therefore in the second stage, we fine-tune the controllers by setting the initial pose to a sampled ending pose of another subtask, similar to the policy sequencing method in. For turn and sit, the initial pose is sampled from the ending pose of walk and turn, respectively. 2) Meta Controller Recall that the task is to have the humanoid sit down regardless of where it starts in the environment. The task's difficulty highly depends on the initial state: if it is already facing the seat, it only needs to turn and sit, while if it is behind the chair, it needs to first walk to the front and then sit down. Training can be challenging when starting from a difficult state, since the humanoid needs to by chance execute a long sequence of correct actions to receive the reward for sitting down. To facilitate training, we propose a multi-stage training strategy inspired by curriculum learning . The idea is to begin the training from easier states, and progressively increase the difficulty when the training converges. As illustrated in Fig. 4, we begin by only spawning the humanoid on the front side of the chair (Zone 1). Once trained, we change the initial position to the lateral sides (Zone 2) and continue the training. Finally, we train the humanoid to start from the rear side (Zone 3). Reference Motion We collect mocap data from the CMU Graphics Lab Motion Capture Database (CMU). Tab. 1 shows the mocap clips we used for each subtask. We extract relevant motion segments and retarget the motion to our humanoid model. We use a 21-DoF humanoid model provided by the Bullet Physics SDK (Bullet). Motion retargeting is performed using a Jacobianbased inverse kinematics method . Implementation Details Our simulation environment is based on OpenAI Roboschool, which uses the Bullet physics engine (Bullet). We use a randomly selected chair model from ShapeNet . The PPO algorithm for training is based on the implementation from OpenAI Baselines. Tab. 2 shows the hyerparamters we used for the PPO training. Subtask First we show qualitative of the individual subtask controllers trained using their corresponding reference motions. Each row in Fig. 5 shows the humanoid performance of one particular subtask: walk in one direction (row 1), following a target (row 2), turn in place both left (row 3) and right (row 4), and sit on a chair (row 5). We adopt two different metrics to quantitatively evaluate the main task: success rate and minimum distance. We declare a success whenever the pelvis of the humanoid has been continuously in contact with the seat surface for 3.0 seconds. We report the success rate over 10,000 trials by spawning the humanoid at random locations. Note that the success rate evaluates task completion with a hard constraint and does not reveal the progress when the humanoid fails. Therefore we also compute the per-trial minimum distance (in meters) between the pelvis and the center of the seat surface, and report the mean and standard deviation over the 10,000 trials. As noted in Sec. 6, the task can be challenging when the initial position of the humanoid is unconstrained. To better analyze the performance, we consider two different initialization settings: Easy and Hard. In the Easy setting, the humanoid is initialized from roughly 2 meters away on the front half plane of the chair (i.e. Zone 1 in Fig. 4), with an orientation roughly towards the chair. The task is expected to be completed by simply walking forward, turning around, and sitting down. In the Hard setting, humanoid is initialized again from roughly 2 meters away but on the lateral and rear sides of the chair (i.e. Zone 2 and 3 in Fig. 4). It needs to walk around the chair to sit down successfully. Easy Setting We benchmark our approach against various baselines in this setting. We start with two non-hierarchical (i.e. single-level) baselines. The first is a kinematics-based method: we select a mocap clip with a holistic motion sequence that successively performs walking, turning, and sitting on a chair. When a trial begins, we align the first frame of the sequence to the humanoid's initial pose by aligning the yaw of the root. Once aligned, we simply use the following frames of the sequence as the kinematic trajectory of the trial. Note that this method is purely kinematic and cannot reflect any physical interactions between the humanoid and chair. The second method extends the first one to a physics-based approach: we use the same kinematic sequence but now train a controller to imitate the motion. This is equivalent to training a subtask controller except the subtask is holistic (i.e. containing walk, turn, and sit in one reference motion). Both methods are considered nonhierarchical as neither performs task decomposition. Tab. 3 shows the quantitative . For the kinematics baseline, the success rate is not reported since we are unable to detect physical contact between the pelvis and chair. However, the 1.2656 mean minimum distance suggests that the humanoid on average remains far from the chair. For the physics baseline, we observe a similar mean minimum distance (i.e. 1.3316). The zero success rate is unsurprising given that the humanoid is unable to get close to the chair in most trials. As shown in the qualitative examples (Fig. 6), the motion generated by the kinematics baseline (row 1) is not physics realistic (e.g. sitting in air). The physics baseline (row 2), while following physics rules (e.g. falling on the ground eventually), still fails in approaching the chair. These holistic baselines perform poorly since they simply imitate the mocap example and repeat the same motion pattern regardless of their starting position. We now turn to a set of hierarchical baselines and our approach. We also consider two baselines. The first one always executes the subtasks in a pre-defined order, and the meta controller is only used to trigger transitions (i.e. a binary classification). Note that this is in similar spirit to. We consider two particular orders: walk→left turn→sit and walk→right turn→sit. The second one is a degenerated version of our approach that uses either only left turn or right turn: walk / left turn / sit and walk / right turn / sit. As shown in Tab. 3, hierarchical approaches outperform single level approaches, validating our hypothesis that hierarchical models, by breaking a task into reusable subtasks, can attain better generalization. Besides, our approach outperforms the pre-defined order baselines. This is because: the main task cannot always be completed by a fixed order of subtasks, and different scenarios, e.g. in Fig. 6, walk→right turn→sit when starting from the chair's right side (row 3), and walk→left turn→sit when starting from the chair's left side (row 4). Analysis As can be seen in Tab. 3, the success rate is still low even with the full model (i.e. 31.61%). This can be attributed to three factors: failures of subtask execution, failures due to subtask transitions, and an insufficient subtask repertoire. First, Tab. 4 (top) shows the success rate of individual subtasks, where the initial pose is set to the first frame of the reference motion (i.e. as in stage one of subtask training). We can see the execution does not always succeed (e.g. 67.59% for right turn). Second, Tab. 4 (bottom) shows the success rate for the same subtasks, but with the initial pose set to the last frame of the execution of another subtask (i.e. as in stage two of subtask training). With fine-tuning the success rate after transitions can be significantly improved, although still not perfect. Finally, Fig. 6 (row 5) shows a failure case where the humanoid needs a "back up" move when it is stuck in the state of directly confronting the chair. Building a more diverse subtask skill set is an interesting future research problem. To analyze the meta controller's behavior, we look at the statistics on the switching between subtasks. Fig. 7 shows the subtask transition matrices when the humanoid is started either from the right or left side of the chair. We can see that certain transitions are more favored in certain starting areas, e.g. walk→left turn is favored over walk→right turn when started from the left side. This is in line with the earlier observation that the two turning subtasks are complementary. Hard Setting We now increase the task's difficulty by initializing the humanoid in Zone 2 and 3 (Fig. 4), and show the effect of the proposed curriculum learning (CL) strategy. Tab. 6 shows the from different initialization zones. First, we observe a severe drop in the success rate when the humanoid is spawned in Zone 2 and 3 (e.g. from 31.61% to 4.05% for "Zone 3 w/o CL"). However, the success rate is higher in both zones when the proposed curriculum learning strategy is applied (e.g. from 4.05% to 7.05% in Zone 3). This suggests that a carefully tailored curriculum can improve the training outcome of a challenging task. Note that the difference in the minimum distance is less significant (e.g. 0.5549 for "Zone 2 w/o CL' versus 0.5526 for "Zone 2"), since without CL the humanoid can still approach the chair, but will fail to turn and sit due to the difficulty in learning. Fig. 8 shows two successful examples when the humanoid is spawned from the rear side of the chair. Interestingly, the humanoid learns a slightly different behavior (e.g. walk→sit without turn) compared to when starting from the front side (row 3 and 4 in Fig. 6). We show a vision-based application of our approach by synthesizing sitting motions from a single RGB image that depicts human living space with chairs. First, we recover the 3D scene configuration using the method of. We then align the observed scene with the simulated environment using the detected chair and its estimated 3D position and orientation. This enables us to transfer the synthesized sitting motion to the observed scene. Fig. 9 shows two images rendered with synthesized humanoid motion. While the motion looks physically plausible in these examples, this is not always the case in general, since we do not model the other objects (e.g. tables) in the scene. An interesting future direction is to learn the motion by simulating scenes with cluttered objects. It is also possible to synthesize motions based on the humans observed in the image, given the recent advance on extracting 3D human pose from a single image (b).

Synthesizing human motions on interactive tasks using mocap data and hierarchical RL.

StrokeNet is a novel architecture where the agent is trained to draw by strokes on a differentiable simulation of the environment, which could effectively exploit the power of back-propagation.