Recent work has shown the possibility of adversarial attacks on automatic speechrecognition (ASR) systems. However, in the vast majority of work in this area, theattacks have been executed only in the digital space, or have involved short phrasesand static room settings. In this paper, we demonstrate a physically realizableaudio adversarial attack. We base our approach specifically on a psychoacoustic-property-based loss function, and automated generation of room impulse responses, to create adversarial attacks that are robust when played over a speaker in multiple environments. We show that such attacks are possible even while being virtually imperceptible to listeners.
@cite_18: Speech is a common and effective way of communication between humans, and modern consumer devices such as smartphones and home hubs are equipped with deep learning based accurate automatic speech recognition to enable natural interaction between humans and machines. Recently, researchers have demonstrated powerful attacks against machine learning models that can fool them to produceincorrect results. However, nearly all previous research in adversarial attacks has focused on image recognition and object detection models. In this short paper, we present a first of its kind demonstration of adversarial attacks against speech classification model. Our algorithm performs targeted attacks with 87 success by adding small background noise without having to know the underlying model parameter and architecture. Our attack only changes the least significant bits of a subset of audio clip samples, and the noise does not change 89 the human listener's perception of the audio clip as evaluated in our human study.
@cite_14: We construct targeted audio adversarial examples on automatic speech recognition. Given any audio waveform, we can produce another that is over 99.9 similar, but transcribes as any phrase we choose (recognizing up to 50 characters per second of audio). We apply our white-box iterative optimization-based attack to Mozilla's implementation DeepSpeech end-to-end, and show it has a 100 success rate. The feasibility of this attack introduce a new domain to study adversarial examples.
@cite_1: Voice interfaces are becoming more ubiquitous and are now the primary input method for many devices. We explore in this paper how they can be attacked with hidden voice commands that are unintelligible to human listeners but which are interpreted as commands by devices. We evaluate these attacks under two different threat models. In the black-box model, an attacker uses the speech recognition system as an opaque oracle. We show that the adversary can produce difficult to understand commands that are effective against existing systems in the black-box model. Under the white-box model, the attacker has full knowledge of the internals of the speech recognition system and uses it to create attack commands that we demonstrate through user testing are not understandable by humans. We then evaluate several defenses, including notifying the user when a voice command is accepted; a verbal challenge-response protocol; and a machine learning approach that can detect our attacks with 99.8 accuracy.
@cite_6: Hands-free, voice-driven user input is gaining popularity, in part due to the increasing functionalities provided by intelligent digital assistances such as Siri, Cortana, and Google Now, and in part due to the proliferation of small devices that do not support more traditional, keyboard-based input. In this paper, we examine the gap in the mechanisms of speech recognition between human and machine. In particular, we ask the question, do the differences in how humans and machines understand spoken speech lead to exploitable vulnerabilities? We find, perhaps surprisingly, that these differences can be easily exploited by an adversary to produce sound which is intelligible as a command to a computer speech recognition system but is not easily understandable by humans. We discuss how a wide range of devices are vulnerable to such manipulation and describe how an attacker might use them to defraud victims or install malware, among other attacks.
@cite_20: We propose a method to generate audio adversarial examples that can attack a state-of-the-art speech recognition model in the physical world. Previous work assumes that generated adversarial examples are directly fed to the recognition model, and is not able to perform such a physical attack because of reverberation and noise from playback environments. In contrast, our method obtains robust adversarial examples by simulating transformations caused by playback or recording in the physical world and incorporating the transformations into the generation process. Evaluation and a listening experiment demonstrated that our adversarial examples are able to attack without being noticed by humans. This result suggests that audio adversarial examples generated by the proposed method may become a real threat.
In @cite_1 @cite_6 @cite_18 , attacks were generated by manipulating the Mel-frequency cepstral coefficients (MFCC) and the coefficients back to the time domain. However, due to the lossy nature of the MFCC computation, some of the information of the original audio signal is lost, limiting the attack space. The raw signal was attacked in @cite_14 @cite_20 in a complete end-to-end fashion. These attacks focused on limiting the size of the adversary based on the signal-to-noise ratio (SNR) only, and did not account for the perceptual distortion introduced by the adversary.