Question: represent the article for finding a single-sentence tldr

Question: Most generative models of audio directly generate samples in one of two domains: time or frequency. While sufficient to express any signal, these representations are inefficient, as they do not utilize existing knowledge of how sound is generated and perceived. A third approach (vocoders/synthesizers) successfully incorporates strong domain knowledge of signal processing and perception, but has been less actively researched due to limited expressivity and difficulty integrating with modern auto-differentiation-based machine learning methods. In this paper, we introduce the Differentiable Digital Signal Processing (DDSP) library, which enables direct integration of classic signal processing elements with deep learning methods. Focusing on audio synthesis, we achieve high-fidelity generation without the need for large autoregressive models or adversarial losses, demonstrating that DDSP enables utilizing strong inductive biases without losing the expressive power of neural networks. Further, we show that combining interpretable modules permits manipulation of each separate model component, with applications such as independent control of pitch and loudness, realistic extrapolation to pitches not seen during training, blind dereverberation of room acoustics, transfer of extracted room acoustics to new environments, and transformation of timbre between disparate sources. In short, DDSP enables an interpretable and modular approach to generative modeling, without sacrificing the benefits of deep learning. The library will is available at https://github.com/magenta/ddsp and we encourage further contributions from the community and domain experts. Neural networks are universal function approximators in the asymptotic limit , but their practical success is largely due to the use of strong structural priors such as convolution , recurrence (; ;), and self-attention . These architectural constraints promote generalization and data efficiency to the extent that they align with the data domain. From this perspective, end-to-end learning relies on structural priors to scale, but the practitioner's toolbox is limited to functions that can be expressed differentiably. Here, we increase the size of that toolbox by introducing the Differentiable Digital Signal Processing (DDSP) library, which integrates interpretable signal processing elements into modern automatic differentiation software (TensorFlow). While this approach has broad applicability, we highlight its potential in this paper through exploring the example of audio synthesis. Objects have a natural tendency to periodically vibrate. Small shape displacements are usually restored with elastic forces that conserve energy (similar to a canonical mass on a spring), leading to harmonic oscillation between kinetic and potential energy . Accordingly, human hearing has evolved to be highly sensitive to phase-coherent oscillation, decomposing audio into spectrotemporal responses through the resonant properties of the basilar membrane and tonotopic mappings into the auditory cortex (; ;). However, neural synthesis models often do not exploit this periodic structure for generation and perception. As shown in Figure 1, most neural synthesis models generate waveforms directly in the time domain, or from their corresponding Fourier coefficients in the frequency domain. While these representations are general and can represent any waveform, they are not free from bias. This is because they often apply a prior over generating audio with aligned wave packets rather than oscillations. For example, strided convolution models-such as SING , MCNN , and WaveGAN -generate waveforms directly with overlapping frames. Since audio oscillates at many frequencies, all with different periods from the fixed frame hop size, the model must precisely align waveforms between different frames and learn filters to cover all possible phase variations. This challenge is visualized on the left of Figure 1. Fourier-based models-such as Tacotron and GANSynth also suffer from the phase-alignment problem, as the Short-time Fourier Transform (STFT) is a representation over windowed wave packets. Additionally, they must contend with spectral leakage, where sinusoids at multiple neighboring frequencies and phases must be combined to represent a single sinusoid when Fourier basis frequencies do not perfectly match the audio. This effect can be seen in the middle diagram of Figure 1. Autoregressive waveform models-such as WaveNet , SampleRNN , and WaveRNN -avoid these issues by generating the waveform a single sample at a time. They are not constrained by the bias over generating wave packets and can express arbitrary waveforms. However, they require larger and more data-hungry networks, as they do not take advantage of a bias over oscillation (size comparisons can be found in Table B .6). Furthermore, the use of teacher-forcing during training leads to exposure bias during generation, where errors with feedback can compound. It also makes them incompatible with perceptual losses such as spectral features , pretrained models , and discriminators. This adds further inefficiency to these models, as a waveform's shape does not perfectly correspond to perception. For example, the three waveforms on the right of Figure 1 sound identical (a relative phase offset of the harmonics) but would present different losses to an autoregressive model. Rather than predicting waveforms or Fourier coefficients, a third model class directly generates audio with oscillators. Known as vocoders or synthesizers, these models are physically and perceptually motivated and have a long history of research and applications . These "analysis/synthesis" models use expert knowledge and hand-tuned heuristics to extract synthesis parameters (analysis) that are interpretable (loudness and frequencies) and can be used by the generative algorithm (synthesis). Neural networks have been used previously to some success in modeling pre-extracted synthesis parameters , but these models fall short of endto-end learning. The analysis parameters must still be tuned by hand and gradients cannot flow through the synthesis procedure. As a , small errors in parameters can lead to large errors in the audio that cannot propagate back to the network. Crucially, the realism of vocoders is limited by the expressivity of a given analysis/synthesis pair. In this paper, we overcome the limitations outlined above by using the DDSP library to implement fully differentiable synthesizers and audio effects. DDSP models combine the strengths of the above approaches, benefiting from the inductive bias of using oscillators, while retaining the expressive power of neural networks and end-to-end training. We demonstrate that models employing DDSP components are capable of generating high-fidelity audio without autoregressive or adversarial losses. Further, we show the interpretability and modularity of these models enable: • Independent control over pitch and loudness during synthesis. • Realistic extrapolation to pitches not seen during training. • Blind dereverberation of audio through seperate modelling of room acoustics. • Transfer of extracted room acoustics to new environments. • Timbre transfer between disparate sources, converting a singing voice into a violin. • Smaller network sizes than comparable neural synthesizers. Audio samples for all examples and figures are provided in the online supplement 2. We highly encourage readers to listen to the samples as part of reading the paper. Vocoders. Vocoders come in several varieties. Source-filter/subtractive models are inspired by the human vocal tract and dynamically filter a harmonically rich source signal , while sinusoidal/additive models generate sound as the combination of a set of time-varying sine waves . Additive models are strictly more expressive than subtractive models but have more parameters as each sinusoid has its own time-varying loudness and frequency. This work builds a differentiable synthesizer off the Harmonic plus Noise model : an additive synthesizer combines sinusoids in harmonic (integer) ratios of a fundamental frequency alongside a time-varying filtered noise signal. Synthesizers. A separate thread of research has tried to estimate parameters for commercial synthesizers using gradient-free methods . Synthesizer outputs modeled with a variational autoencoder were recently used as a "world model" to pass approximate gradients to a controller during learning . DDSP differs from black-box approaches to modeling existing synthesizers; it is a toolkit of differentiable DSP components for end-to-end learning. Neural Source Filter (NSF). Perhaps closest to this work, promising speech synthesis were recently achieved using a differentiable waveshaping synthesizer . The NSF can be seen as a specific DDSP model, that uses convolutional waveshaping of a sinusoidal oscillator to create harmonic content, rather than additive synthesis explored in this work. Both works also generate audio in the time domain and impose multi-scale spectrograms losses in the frequency domain. A key contribution of this work is to highlight how these models are part of a common family of techniques and to release a modular library that makes them accessible by leveraging automatic differentiation to easily mix and match components at a high level. Many DSP operations can be expressed as functions in modern automatic differentiation software. We express core components as feedforward functions, allowing efficient implementation on parallel hardware such as GPUs and TPUs, and generation of samples during training. These components include oscillators, envelopes, and filters (linear-time-varying finite-impulse-response, LTV-FIR). Here, as an example DDSP model, we implement a differentiable version of Spectral Modeling Synthesis (SMS). This model generates sound by combining an additive synthesizer (adding together many sinusoids) with a subtractive synthesizer (filtering white noise). We choose SMS because, despite being parametric, it is a highly expressive model of sound, and has found widespread adoption in tasks as diverse as spectral morphing, time stretching, pitch shifting, source separation, transcription, and even as a general purpose audio codec in MPEG-4 (; ;). As we only consider monophonic sources in these experiments, we use the Harmonic plus Noise model, that further constrains sinusoids to be integer multiples of a fundamental frequency . One of the reasons that SMS is more expressive than many other parametric models because it has so many more parameters. For example, in the 4 seconds of 16kHz audio in the datasets considered here, the synthesizer coefficients actually have ∼2.5 times more dimensions than the audio waveform itself ((1 amplitude + 100 harmonics + 65 noise band magnitudes) * 1000 timesteps = 165,000 dimensions, vs. 64,000 audio samples). This makes them amenable to control by a neural network, as it would be difficult to realistically specify all these parameters by hand. At the heart of the synthesis techniques explored in this paper is the sinusoidal oscillator. A bank of oscillators that outputs a signal x(n) over discrete time steps, n, can be expressed as: where A k (n) is the time-varying amplitude of the k-th sinusoidal component and φ k (n) is its instantaneous phase. The phase φ k (n) is obtained by integrating the instantaneous frequency f k (n): where φ 0,k is the initial phase that can be randomized, fixed, or learned. For a harmonic oscillator, all the sinusoidal frequencies are harmonic (integer) multiples of a fundamental frequency, f 0 (n), i.e., f k (n) = kf 0 (n), Thus the output of the harmonic oscillator is entirely parameterized by the time-varying fundamental frequency f 0 (n) and harmonic amplitudes A k (n). To aid interpretablity we further factorize the harmonic amplitudes: into a global amplitude A(n) that controls the loudness and a normalized distribution over harmonics c(n) that determines spectral variations, where We also constrain both amplitudes and harmonic distribution components to be positive through the use of a modified sigmoid nonlinearity as described in the appendix. Figure 6 provides a graphical example of the additive synthesizer. Audio is provided in our online supplement 2. The oscillator formulation above requires time-varying amplitudes and frequencies at the audio sample rate, but our neural networks operate at a slower frame rate. For instantaneous frequency upsampling, we found bilinear interpolation to be adequate. However, the amplitudes and harmonic distributions of the additive synthesizer required smoothing to prevent artifacts. We are able to achieve this with a smoothed amplitude envelope by adding overlapping Hamming windows at the center of each frame and scaled by the amplitude. For these experiments we found a 4ms (64 timesteps) hop size and 8 ms frame size (50% overlap) to be responsive to changes while removing artifacts. Linear filter design is a cornerstone of many DSP techniques. Standard convolutional layers are equivalent to linear time invariant finite impulse response (LTI-FIR) filters. However, to ensure interpretability and prevent phase distortion, we employ the frequency sampling method to convert network outputs into impulse responses of linear-phase filters. Here, we design a neural network to predict the frequency-domain transfer functions of a FIR filter for every output frame. In particular, the neural network outputs a vector H l (and accordingly h l = IDFT(H l)) for the l-th frame of the output. We interpret H l as the frequency-domain transfer function of the corresponding FIR filter. We therefore implement a time-varying FIR filter. To apply the time-varying FIR filter to the input, we divide the audio into non-overlapping frames x l to match the impulse responses h l. We then perform frame-wise convolution via multiplication of frames in the Fourier domain: We recover the frame-wise filtered audio, y l = IDFT(Y l), and then overlap-add the ing frames with the same hop size and rectangular window used to originally divide the input audio. The hop size is given by dividing the audio into equally spaced frames for each frame of conditioning. For 64000 samples and 250 frames, this corresponds to a hop size of 256. In practice, we do not use the neural network output directly as H l. Instead, we apply a window function W on the network output to compute H l. The shape and size of the window can be decided independently to control the time-frequency resolution trade-off of the filter. In our experiments, we default to a Hann window of size 257. Without a window, the resolution implicitly defaults to a rectangular window which is not ideal for many cases. We take care to shift the IR to zero-phase (symmetric) form before applying the window and revert to causal form before applying the filter. 3.5 FILTERED NOISE / SUBTRACTIVE SYNTHESIZER Natural sounds contain both harmonic and stochastic components. The Harmonic plus Noise model captures this by combining the output of an additive synthesizer with a stream of filtered noise . We are able to realize a differentiable filtered noise synthesizer by simply applying the LTV-FIR filter from above to a stream of uniform noise Y l = H l N l where N l is the IDFT of uniform noise in domain [-1, 1]. Room reverbation (reverb) is an essential characteristic of realistic audio, which is usually implicitly modeled by neural synthesis algorithms. In contrast, we gain interpretability by explicitly factorizing the room acoustics into a post-synthesis convolution step. A realistic room impulse response (IR) can be as long as several seconds, corresponding to extremely long convolutional kernel sizes (∼10-100k timesteps). Convolution via matrix multiplication scales as O(n 3), which is intractable for such large kernel sizes. Instead, we implement reverb by explicitly performing convolution as multiplication in the frequency domain, which scales as O(n log n) and does not bottleneck training. For empirical verification of this approach, we test two DDSP autoencoder variants-supervised and unsupervised-on two different musical datasets: NSynth and a collection of solo violin performances. The supervised DDSP autoencoder is conditioned on fundamental frequency (F0) and loudness features extracted from audio, while the unsupervised DDSP autoencoder learns F0 jointly with the rest of the network. Red components are part of the neural network architecture, green components are the latent representation, and yellow components are deterministic synthesizers and effects. Components with dashed borders are not used in all of our experiments. Namely, z is not used in the model trained on solo violin, and reverb is not used in the models trained on NSynth. See the appendix for more detailed diagrams of the neural network components. DDSP components do not put constraints on the choice of generative model (GAN, VAE, Flow, etc.), but we focus here on a deterministic autoencoder to investigate the strength of DDSP components independent of any particular approach to adversarial training, variational inference, or Jacobian design. Just as autoencoders utilizing convolutional layers outperform fully-connected autoencoders on images, we find DDSP components are able to dramatically improve autoencoder performance in the audio domain. Introducing stochastic latents (such as in GAN, VAE, and Flow models) will likely further improve performance, but we leave that to future work as it is orthogonal to the core question of DDSP component performance that we investigate in this paper. In a standard autoencoder, an encoder network f enc (·) maps the input x to a latent representation z = f enc (x) and a decoder network f dec (·) attempts to directly reconstruct the inputx = f dec (z). Our architecture (Figure 2) contrasts with this approach through the use of DDSP components and a decomposed latent representation. Encoders: Detailed descriptions of the encoders are given in Section B.1. For the supervised autoencoder, the loudness l(t) is extracted directly from the audio, a pretrained CREPE model with fixed weights is used as an f (t) encoder to extact the fundamental frequency, and optional encoder extracts a time-varying latent encoding z(t) of the residual information. For the z(t) encoder, MFCC coefficients (30 per a frame) are first extracted from the audio, which correspond to the smoothed spectral envelope of harmonics , and transformed by a single GRU layer into 16 latent variables per a frame. For the unsupervised autoencoder, the pretrained CREPE model is replaced with a Resnet architecture that extracts f (t) from a mel-scaled log spectrogram of the audio, and is jointly trained with the rest of the network. Decoder: A detailed description of the decoder network is given in Section B.2. The decoder network maps the tuple (f (t), l(t), z(t)) to control parameters for the additive and filtered noise synthesizers described in Section 3. The synthesizers generate audio based on these parameters, and a reconstruction loss between the synthesized and original audio is minimized. The network architecture is chosen to be fairly generic (fully connected, with a single recurrent layer) to demonstrate that it is the DDSP components, and not other modeling decisions, that enables the quality of the work. Also unique to our approach, the latent f (t) is fed directly to the additive synthesizer as it has structural meaning for the synthesizer outside the context of any given dataset. As shown later in Section 5.2, this disentangled representation enables the model to both interpolate within and ex- trapolate outside the data distribution. Indeed, recent work support incorporation of strong inductive biases as a prerequisite for learning disentangled representations . Model Size: Table B.6, compares parameter counts for the DDSP models and comparable models including GANSynth, WaveRNN , and a WaveNet Autoencoder . The DDSP models have the fewest parameters (up to 10 times less), despite no effort to minimize the model size in these experiments. Initial experiments with very small models (240k parameters, 300x smaller than a WaveNet Autoencoder) have less realistic outputs than the full models, but still have fairly high quality and are promising for low-latency applications, even on CPU or embedded devices. Audio samples are available in the online supplement 2. NSynth: We focus on a smaller subset of the NSynth dataset consistent with other work ). It totals 70,379 examples comprised mostly of strings, brass, woodwinds and mallets with pitch labels within MIDI pitch range 24-84. We employ a 80/20 train/test split shuffling across instrument families. For the NSynth experiments, we use the autoencoder as described above (with the z(t) encoder). We experiment with both the supervised and unsupervised variants. Solo Violin: The NSynth dataset does not capture aspects of a real musical performance. Using the MusOpen royalty free music library, we collected 13 minutes of expressive, solo violin performances 4. We purposefully selected pieces from a single performer (John Garner), that were monophonic and shared a consistent room environment to encourage the model to focus on performance. Like NSynth, audio is converted to mono 16kHz and divided into 4 second training examples (64000 samples total). Code to process the audio files into a dataset is available online. For the solo violin experiments, we use the supervised variant of the autoencoder (without the z(t) encoder), and add a reverb module to the signal processor chain to account for room reverberation. While the room impulse response could be produced as an output of the decoder, given that the solo violin dataset has a single acoustic environment, we use a single fixed variable (4 second reverb corresponding to 64000 dimensions) for the impulse response. The primary objective of the autoencoder is to minimize reconstruction loss. However, for audio waveforms, point-wise loss on the raw waveform is not ideal, as two perceptually identical audio samples may have distinct waveforms, and point-wise similar waveforms may sound very different. Instead, we use a multi-scale spectral loss-similar to the multi-resolution spectral amplitude distance in -defined as follows. Given the original and synthesized audio, we compute their (magnitude) spectrogram S i andŜ i, respectively, with a given FFT size i, and define the loss as the sum of the L1 difference between S i andŜ i as well as the L1 difference between log S i and logŜ i. where α is a weighting term set to 1.0 in our experiments. The total reconstruction loss is then the sum of all the spectral losses, L reconstruction = i L i. In our experiments, we used FFT sizes, and the neighboring frames in the Short-Time Fourier Transform (STFT) overlap by 75%. Therefore, the L i's cover differences between the original and synthesized audios at different spatial-temporal resolutions. As shown in Figure 5, the DDSP autoencoder learns to very accurately resynthesize the solo violin dataset. Again, we highly encourage readers to listen to the samples provided in the online supplement 2. A full decomposition of the components is provided Figure 5. High-quality neural audio synthesis has previously required very large autoregressive models or adversarial loss functions. While amenable to an adversarial loss, the DDSP autoencoder achieves these with a straightforward L1 spectrogram loss, a small amount of data, and a relatively simple model. This demonstrates that the model is able to efficiently exploit the bias of the DSP components, while not losing the expressive power of neural networks. For the NSynth dataset, we quantitatively compare the quality of DDSP resynthesis with that of a state-of-the-art baseline using WaveRNN . The models are comparable as they are trained on the same data, provided the same conditioning, and both targeted towards realtime synthesis applications. In Table 1, we compute loudness and fundamental frequency (F0) L 1 metrics described in Section C of the appendix. Despite the strong performance of the baseline, the supervised DDSP autoencoder still outperforms it, especially in F0 L 1. This is not unexpected, as the additive synthesizer directly uses the conditioning frequency to synthesize audio. The unsupervised DDSP autoencoder must learn to infer its own F0 conditioning signal directly from the audio. As described in Section B.4, we improve optimization by also adding a perceptual loss in the form of a pretrained CREPE network . While not as accurate as the supervised DDSP version, the model does a fair job at learning to generate sounds with the correct frequencies without supervision, outperforming the supervised WaveRNN model. Interpolation: Interpretable structure allows for independent control over generative factors. Each component of the factorized latent variables (f (t), l(t), z(t)) independently alters samples along a matching perceptual axis. For example, Figure 3 shows an interpolation between two sound in the loudness conditioning l(t). With other variables held constant, loudness of the synthesized audio closely matches the interpolated input. Similarly, the model reliably matches intermediate pitches between a high pitched f (t) and low pitched f (t). In Table C.2 of the appendix, we quantitatively demonstrate how across interpolations, conditioning independently controls the corresponding characteristics of the audio. With loudness and pitch explicitly controlled by (f (t), l(t)), the model should use the residual z(t) to encode timbre. Although architecture and training do not strictly enforce this encoding, we qualitatively demonstrate how varying z leads to a smooth change in timbre. In Figure 3, we use the smooth shift in spectral centroid, or "center of mass" of a spectrum, to illustrate this behavior. Extrapolation: As described in Section 4.1, f (t) directly controls the additive synthesizer and has structural meaning outside the context of any given dataset. Beyond interpolating between datapoints, the model can extrapolate to new conditions not seen during training. The rightmost plot of Figure 7 demonstrates this by resynthesizing a clip of solo violin after shifting f (t) down an octave and outside the range of the training data. The audio remains coherent and resembles a related instrument such as a cello. f (t) is only modified for the synthesizer, as the decoder is still bounded by the nearby distribution of the training data and produces unrealistic harmonic content if conditioned far outside that distribution. Removing reverb in a "blind" setting, where only reverberated audio is available, is a standing problem in acoustics . However, a benefit of our modular approach to generative modeling is that it becomes possible to completely separate the source audio from the effect of the room. For the solo violin dataset, the DDSP autoencoder is trained with an additional reverb module as shown in Figure 2 and described in Section 3.6. Figure 7 (left) demonstrates that bypassing the reverb module during resynthesis in completely dereverberated audio, similar to recording in an anechoic chamber. The quality of the approach is limited by the underlying generative model, which is quite high for our autoencoder. Similarly, Figure 7 (center) demonstrates that we can also apply the learned reverb model to new audio, in this case singing, and effectively transfer the acoustic environment of the solo violin recordings. Figure 4 demonstrates timbre transfer, converting the singing voice of an author into a violin. F0 and loudness features are extracted from the singing voice and the DDSP autoencoder trained on solo violin used for resynthesis. To better match the conditioning features, we first shift the fundamental frequency of the singing up by two octaves to fit a violin's typical register. Next, we transfer the room acoustics of the violin recording (as described in Section 5.3) to the voice before extracting loudness, to better match the loudness contours of the violin recordings. The ing audio captures many subtleties of the singing with the timbre and room acoustics of the violin dataset. Note the interesting "breathing" artifacts in the silence corresponding to unvoiced syllables from the singing. The DDSP library fuses classical DSP with deep learning, providing the ability to take advantage of strong inductive biases without losing the expressive power of neural networks and end-to-end learning. We encourage contributions from domain experts and look forward to expanding the scope of the DDSP library to a wide range of future applications. A APPENDIX Figure 5: Decomposition of a clip of solo violin. Audio is visualized with log magnitude spectrograms. Loudness and fundamental frequency signals are extracted from the original audio. The loudness curve does not exhibit clear note segmentations because of the effects of the room acoustics. The DDSP autoencoder takes those conditioning signals and predicts amplitudes, harmonic distributions, and noise magnitudes. Note that the amplitudes are clearly segmented along note boundaries without supervision and that the harmonic and noise distributions are complex and dynamic despite the simple conditioning signals. Finally, the extracted impulse response is applied to the combined audio from the synthesizers to give the full resynthesis audio. Figure 6: Diagram of the Additive Synthesizer component. The synthesizer generates audio as a sum of sinusoids at harmonic (integer) multiples of the fundamental frequency. The neural network is then tasked with emitting time-varying synthesizer parameters (fundamental frequency, amplitude, harmonic distribution). In this example linear-frequency log-magnitude spectrograms show how the harmonics initially follow the frequency contours of the fundamental. We then factorize the harmonic amplitudes into an overall amplitude envelope that controls the loudness, and a normalized distribution among the different harmonics that determines spectral variations. The model has three encoders: f -encoder that outputs fundamental frequency f (t), l-encoder that outputs loudness l(t), and a z-encoder that outputs residual vector z(t). f -encoder: We use a pretrained CREPE pitch detector as the f -encoder to extract ground truth fundamental frequencies (F0) from the audio. We used the "large" variant of CREPE, which has SOTA accuracy for monophonic audio samples of musical instruments. For our supervised autoencoder experiments, we fixed the weights of the f -encoder like , and for our unsupervised autoencoder experiemnts we jointly learn the weights of a resnet model fed log mel spectrograms of the audio. Full details of the resnet architecture are show in Table 2. l-encoder: We use identical computational steps to extract loudness as . Namely, an A-weighting of the power spectrum, which puts greater emphasis on higher frequencies, followed by log scaling. The vector is then centered according to the mean and standard deviation of the dataset. z-encoder: As shown in Figure 8, the encoder first calculates MFCC's (Mel Frequency Cepstrum Coefficients) from the audio. MFCC is computed from the log-mel-spectrogram of the audio with a FFT size of 1024, 128 bins of frequency range between 20Hz to 8000Hz, overlap of 75%. We use only the first 30 MFCCs that correspond to a smoothed spectral envelope. The MFCCs are then passed through a normalization layer (which has learnable shift and scale parameters) and a 512-unit GRU. The GRU outputs (over time) fed to a 512-unit linear layer to obtain z(t). The z embedding reported in this model has 16 dimensions across 250 time-steps. Output Size Table 2: Model architecture for the f(t) encoder using a Resnet on log mel spectrograms. Spectrograms have a frame size of 2048 and a hop size of 512, and are upsampled at the end to have the same time resoultion as other latents (4ms per a frame). All convolutions use "same" padding and a temporal stride of 1. Each residual block uses a bottleneck structure . The final output is a normalized probablity distribution over 128 frequency values (logarithmically scaled between 8.2Hz and 13.3kHz (https://www.inspiredacoustics.com/en/MIDI_note_ numbers_and_center_frequencies)). The finally frequency value is the weighted sum of each frequency by its probability. The decoder's input is the latent tuple (f (t), l(t), z(t)) (250 timesteps). Its outputs are the parameters required by the synthesizers. For example, in the case of the harmonic synthesizer and filtered noise synthesizer setup, the decoder outputs a(t) (amplitudes of the harmonics) for the harmonic synthesizer (note that f (t) is fed directly from the latent), and H (transfer function of the FIR filter) for the filtered noise synthesizer. As shown in Figure 9, we use a "shared-bottom" architecture, which computes a shared embedding from the latent tuple, and then have one head for each of the (a(t), H) outputs. In particular, we apply separate MLPs to each of the (f (t), l(t), z(t)) input. The outputs of the MLPs are concatenated and passed to a 512-unit GRU. We concatenate the GRU outputs with the outputs of the f (t) and l(t) MLPs (in the channel dimenssion) and pass it through a final MLP and Linear layer to get the decoder outputs. The MLP architecture, shown in Figure 10, is a standard MLP with a layer normalization (tf.contrib.layers.layer_norm) before the RELU nonlinearity. In Figure 9, all the MLPs have 3 layers and each layer has 512 units. Because all DDSP components are differentiable, the model is differentiable end-to-end. Therefore, we can apply any SGD optimizer to train the model. We used ADAM optimizer with learning rate 0.001 and exponential learning rate decay 0.98 every 10,000 steps. To help guide the DDSP autoencoder that must predict f (t) on the NSynth dataset, we also added an additional perceptual loss using pretrained models, such as the CREPE pitch estimator and the encoder of the WaveNet autoencoder . Compared to the L1 loss on the spectrogram, the activations of different layers in these models correlate better with the perceptual quality of the audio. After a large-scale hyperparameter search, we obtained our best by using the L1 distance between the activations of the small CREPE model's fifth max pool layer with a weighting of 5 × 10 −5 relative to the spectral loss. Harmonic synthesizer / Additive Synthesis: We use 101 harmonics in the harmonic synthesizer (i.e., a(t)'s dimension is 101). Amplitude and harmonic distribution parameters are upsampled with overlapping Hamming window envelopes whose frame size is 128 and hop size is 64. Initial phases are all fixed to zero, as neither the spectrogram loss functions or human perception are sensitive to absolute offsets in harmonic phase. We also do not include synthesis elements to model DC components to signals as they are inaudible and not reflected in the spectrogram losses. We force the amplitudes, harmonic distributions, and filtered noise magnitudes to be non-negative by applying a sigmoid nonlinearity to network outputs. We find a slight improvement in traning stability by modifying the sigmoid to have a scaled output, larger slope by exponentiating, and threshold at a minimum value: Filtered noise synthesizer: We use 65 network output channels as magnitude inputs to the FIR filter of the filtered noise synthesizer. Model Parameters WaveNet Autoencoder 75M WaveRNN 23M GANSynth 15M DDSP Autoencoder (Unsupervised) 12M DDSP Autoencoder (Supervised, NSynth) 7M DDSP Autoencoder (Supervised, Solo Violin) 6M DDSP Autoencoder Tiny (Supervised, Solo Violin) 0.24M Table 3: Parameter counts for different models. All models trained on NSynth dataset except for those marked (Solo Violin). Autoregressive models have the most parameters with GANs requiring less. The DDSP models examined in this paper (which have not been optimized at all for size) require 2 to 3 times less parameters than GANSynth. The unsupervised model has more parameters because of the CREPE (small) f (t) encoder, and the autoencoder has additional parameters for the z(t) encoder. Initial experiments with extremely small models (single GRU, 256 units), have slightly less realistic outputs, but still relatively high quality (as can be heard in the supplemental audio). C EVALUATION DETAILS Loudness L 1 distance: The loudness vector is extracted from the synthesized audio and L 1 distance computed against the input's conditioning loudness vector (ground truth). A better model will produce lower L 1 distances, indicating input and generated loudness vectors closely match. Note this distance is not back-propagated through the network as a training objective. F0 L 1 distance: The F0 L 1 distance is reported in MIDI space for easier interpretation; an average F0 L 1 of 1.0 corresponds to a semitone difference. We use the same confidence threshold of 0.85 in to select portions where there was detectable pitch content, and compute the metric only in these areas. F0 Outliers: Pitch tracking using CREPE, like any pitch tracker, is not completely reliable. Instabilities in pitch tracking, such as sudden octave jumps at low volumes, can errors not due to model performance and need to be accounted for. F0 outliers accounts for pitch tracking imperfections in CREPE vs. genuinely bad samples generated by the trained model. CREPE outputs both an F0 value as well as a F0 confidence. Samples with confidences below a threshold of 0.85 in are labeled as outliers and usually indicate the sample was mostly noise with no pitch or harmonic component. As the model outputs better quality audio, the number of outliers decrease, thus lower scores indicate better performance. Loudness L 1 and F0 L 1 are shown for different interpolation tasks in table C.2. In reconstruction, the model is supplied with the standard (f (t) A, l(t) A, z(t) A ). Loudness (L 1) and F0 (L 1) are computed against the ground truth inputs. In loudness interpolation, the model is supplied with (f (t) A, l(t) B, z(t) A ), and Loudness (L 1) is calculated using l(t) B as ground truth instead of (l(t) A ). For F0 interpolation, the model is supplied with (f (t) B, l(t) A, z(t) A ) and F0 (L 1) is calculated using f (t) B as ground truth instead of f (t) A. For Z interpolation, the model is supplied with (f (t) A, l(t) A, z(t) B ). While unconstrained sinusoidal oscillator banks are strictly more expressive than harmonic oscillators, we restricted ourselves to harmonic synthesizers for the time being to focus the problem domain. However, this is not a fundamental limitation of the technique and perturbations such as inharmonicity can also be incorporated to handle phenomena such as stiff strings . It is worth mentioning that the modular structure of the synthesizers also makes it possible to define additional losses in terms of different synthesizer outputs and parameters. For example, we may impose an SNR loss to penalize outputs with too much noise if we know the training data consists of mostly clean data. We have not experimented too much with such engineered losses, but we believe they can make training more efficient, even though such engineering methods deviates from the end-to-end training paradigm,

Represent the short summary for finding the corresponding scientific article (long):

Better audio synthesis by combining interpretable DSP with end-to-end learning.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

A structured latent-variable approach that adds discrete control states within a standard autoregressive neural paradigm to provide arbitrary grounding of internal model decisions, without sacrificing any representational power of neural models.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Spectral Graph Convolutional Networks (GCNs) are a generalization of convolutional networks to learning on graph-structured data. Applications of spectral GCNs have been successful, but limited to a few problems where the graph is fixed, such as shape correspondence and node classification. In this work, we address this limitation by revisiting a particular family of spectral graph networks, Chebyshev GCNs, showing its efficacy in solving graph classification tasks with a variable graph structure and size. Current GCNs also restrict graphs to have at most one edge between any pair of nodes. To this end, we propose a novel multigraph network that learns from multi-relational graphs. We explicitly model different types of edges: annotated edges, learned edges with abstract meaning, and hierarchical edges. We also experiment with different ways to fuse the representations extracted from different edge types. This restriction is sometimes implied from a dataset, however, we relax this restriction for all kinds of datasets. We achieve state-of-the-art on a variety of chemical, social, and vision graph classification benchmarks.

A novel approach to graph classification based on spectral graph convolutional networks and its extension to multigraphs with learnable relations and hierarchical structure. We show state-of-the art results on chemical, social and image datasets.

Represent the short summary for finding the corresponding scientific article (long)

We present a new approach, SNIP, that is simple, versatile and interpretable; it prunes irrelevant connections for a given task at single-shot prior to training and is applicable to a variety of neural network models without modifications.


Question: represent the scientific article to find a short tldr.

Question: Deep neural networks have been recently demonstrated to be vulnerable to backdoor attacks. Specifically, by altering a small set of training examples, an adversary is able to install a backdoor that can be used during inference to fully control the model’s behavior. While the attack is very powerful, it crucially relies on the adversary being able to introduce arbitrary, often clearly mislabeled, inputs to the training set and can thus be detected even by fairly rudimentary data filtering. In this paper, we introduce a new approach to executing backdoor attacks, utilizing adversarial examples and GAN-generated data. The key feature is that the ing poisoned inputs appear to be consistent with their label and thus seem benign even upon human inspection. Over the last decade, deep learning has made unprecedented progress on a variety of notoriously difficult tasks in computer vision BID16 BID11, speech recognition BID8, machine translation BID28, and game playing BID20 BID27. Despite this remarkable performance, real-world deployment of such systems remains challenging due to concerns about security and reliability. One particular example receiving significant attention is the existence of adversarial examples -inputs with imperceptible adversarial perturbations that are misclassified with high confidence BID29 BID7. Such adversarial perturbations can be constructed for a wide range of models, while requiring minimal model knowledge BID22 BID4 and being applicable to real-world scenarios BID26 BID17 BID1.However, this brittleness during inference is not the only vulnerability of existing ML approaches. Another vulnerability corresponds to a different aspect of the ML pipeline: training. State-of-the-art ML models require large amounts of data to achieve good performance. Unfortunately, large datasets are expensive to generate and curate; it is hence common practice to use training examples sourced from other -often untrusted -sources. This practice is usually justified by the robustness of ML models to input and label noise BID24 ) -bad samples might only slightly degrade the model's performance. While this reasoning is valid when only benign noise is present, it breaks down when the noise is maliciously crafted. Attacks based on injecting such malicious noise to the training set are known as data poisoning attacks BID2.A well-studied form of data poisoning aims to use the malicious samples to reduce the test accuracy of the ing model BID31 BID21 BID19 BID3. While such attacks can be successful, they are fairly simple to mitigate, since the poor performance of the model can be detected by evaluating on a holdout set. Another form of attack, known as targeted poisoning attacks, aims to misclassify a specific set of inputs at inference time BID14. These attacks are harder to detect. Their impact is restricted, however, as they only affect the model's behavior on a limited, pre-selected set of inputs. Recently, BID9 proposed a backdoor attack. The purpose of this attack is to plant a backdoor in any model trained on the poisoned training set. This backdoor is activated during inference by a backdoor trigger which, whenever present in a given input, forces the model to predict a specific (likely incorrect) target label. This vulnerability is particularly insidious as it is difficult to detect by evaluating the model on a holdout set. The BID9 attack is based on randomly selecting a small portion of the training set, applying a backdoor trigger to these inputs and changing their labels to the target label. This strategy is very effective. However, it crucially relies on the BID9 Clean-label baseline GAN-based (ours) Adv. example-based (ours) Figure 1: An example image, labeled as an airplane, poisoned using different strategies: the BID9 attack, the baseline of the same attack restricted to only clean labels, our GAN-based attack, and our adversarial examples-based attack (left to right). The original BID9 attack image is clearly mislabeled while the rest of the images appear plausible. We use the same pattern as BID9 for consistency, but our attacks use a reduced amplitude, as described in Section B.1.assumption that the poisoned inputs introduced to the training set by the adversary can be arbitraryincluding clearly mislabeled input-label pairs. As a , even a fairly simple filtering process will detect the poisoned samples as outliers and, more importantly, any subsequent human inspection will deem these inputs suspicious and thus potentially reveal the attack. The goal of this paper is to investigate whether the usage of such clearly mislabeled (and thus suspicious) images is really necessary. That is, can such backdoor attacks be carried out when we insist that each poisoned input and its label must be consistent, even to a human? Our starting point is to analyze the effectiveness of the BID9 attack when a very simplistic, data filtering technique is applied. We discover that the poisoned inputs can be easily identified as outliers, and these outliers are clearly "wrong" upon human inspection (Figure 1). Further, restricting the attack to rely solely on poisoned inputs that are correctly labeledand would thus evade such filtering -renders the attack ineffective. Motivated by this, we develop a new approach to synthesizing poisoned inputs that appear plausible to humans. Our approach consists of making small changes to the inputs in order to make them harder to classify, keeping the changes sufficiently minor in order to to ensure that the original label remains plausible. We perform this transformation with two different methods.• GAN-based interpolation: we embed each input into the latent space of a GAN BID6 and interpolate poisoned samples towards embeddings of an incorrect class.• Adversarial p -bounded perturbations: we use an optimization method to maximize the loss of a pre-trained model on the poisoned inputs while staying within an p -ball around the original input. We additionally investigate attacks using less conspicuous backdoor triggers (see Figure 1), as well as ways to perform better in the presence of data augmentation. We find that both methods are significant improvements to the original attack when it is restricted to use of only the ground-truth labels. Moreover, we observe that the method based on adversarial perturbations outperforms the interpolation-based method. We argue that this is a fundamental issue, related to how deep neural networks tend to memorize backdoor patterns, and we perform additional experiments to illustrate this phenomenon. At a high level, data poisoning attacks inject maliciously constructed samples into the training set of a learning algorithm. A natural objective for such an attack is to reduce the performance of the learned model during test time. This reduction in terms of performance can either target the entire test setaiming to worsen the accuracy of the model on average -or target a specific set of examples to be misclassified BID31 BID21 BID19 BID3 BID14. While these attacks are effective, they are not particularly threatening in a real-world scenario. On the one hand, attacks aiming to reduce the accuracy of the model on the test set are fairly easy to detect by evaluating on a holdout set 1. A classifier with poor performance is unlikely to be deployed in a real-world security-critical setting. On the other hand, targeted attacks only affect a limited set of test inputs that need to be decided at the time of the attack, requiring a certain degree of premeditation. Recently BID9, proposed a different approach to data poisoning attacks, so-called backdoor attacks 2. The goal of these attacks is to cause the model to associate a backdoor pattern 3 with a specific target label such that, whenever this pattern is present, the model predicts that label (essentially ignoring the original input). Specifically, the BID9 attack involves modifying a small number of randomly selected inputs in the training set so that they contain the backdoor pattern and are labeled (usually incorrectly) with the target label. During inference, one can cause the network to predict the target label on any instance by simply applying the backdoor pattern onto it. Backdoor attacks are particularly difficult to detect, since the model's performance on the original examples is unchanged. Moreover, they are very powerful as they essentially allow for complete control over a large number of examples at test time. Despite the potency of the BID9 attack, it crucially relies on the assumption that the adversary can inject arbitrary input-label pairs into the training set. Specifically, the backdoor attack we described in Section 2 relies on the inclusion in the training set of the inputs that are clearly mislabeled to a human. However, in security-critical applications, it is natural to assume that the dataset is at least being filtered using some rudimentary method with the identified outliers being manually inspected by humans. As a starting point of our investigation, we examined the BID9 attack (reproduced in Appendix A) in the presence of a simple filtering scheme. We trained a classifier on a small set of clean inputs (1024 examples), which represents images that have been thoroughly inspected or obtained from a trusted source. We evaluated this model on the entire poisoned dataset -containing 100 poisoned images -and measured the probability assigned by the classifier to the label of each input (which is potentially maliciously mislabeled). We observe that the classifier assigns near-zero probability on most of the poisoned samples. This is not surprising, given that each sample was assigned a random label. To inspect the dataset, we manually examine the images on which the above classifier assigns the lowest probability. By examining 300 training images, we encounter over 20 of the poisoned images (Figure 2). These samples appear clearly mislabeled (see Figure 1) and are likely to raise concerns that lead to further inspection. By restricting the attack to only poison examples of the target class (i.e. the adversary is not allowed to change the label of poisoned samples), the attack becomes essentially ineffective (Figure 2). This behavior is expected. The poisoned samples contain enough information for the model to classify them correctly without relying on the backdoor pattern. Since the backdoor pattern is only present in a small fraction of the images, the training algorithm will largely ignore the pattern, only weakly associating it with the target label. Given the detectability of the BID9 attack through a simple data filtering strategy, we investigate methods of improvement. Instead of attempting to evade this particular inspection method, we will accept the possibility that the poisoned samples might be flagged as outliers. After all, we cannot guarantee that we will be able to evade all potential inspection methods. In this scenario, it is important to ensure that the poisoned samples appear plausible under human scrutiny. Standard datasets contain an abundance of low quality samples, so the presence of low quality samples is 1 If an ε fraction of examples is poisoned, the accuracy on a holdout set cannot be affected by more than ε. 2 The of BID9 were originally focused on the transfer learning setting, but can be straightforwardly be applied to the data poisoning setting BID5.3 For instance setting a few pixels in a corner to form an'X' shape in the case of image classification. Figure 2: Left: After training a model on a small, clean dataset, we examine the training examples that were assigned the lowest probability on their labels. The 300 lowest label probability training samples contain over 20 of the 100 poisoned samples. Right: The BID9 attack, but restricted to only clean labels (only images from the target class are poisoned). The attack is ineffective; even at 25% poisoning, only one class exceeds 50% attack success. The attack success rate is defined as the percentage of test examples not labeled as the target that are classified as the target class when the backdoor pattern is applied.unlikely to raise suspicion. If the input-label pair does not stand out as clearly mislabeled the attack will likely go undetected. Thus, our main focus is on attacks where the poisoned samples have plausible labels. We refer to these as clean-label attacks (the notion of clean labels was recently considered by BID25 in the context of targeted poisoning attacks).Recall that in Figure 2 we showed that restricting the BID9 attack to only poison inputs of the target class (i.e. without changing the true labels) renders the attack ineffective. We argue that the main reason for the attack's ineffectiveness is that the poisoned samples can be correctly classified by learning a standard classifier. Since relying on the backdoor trigger is not necessary to correctly classify these inputs, the backdoor attack is unlikely to be successful. To avoid this behavior, we will perturb the poisoned samples in order to render learning the salient characteristics of the input more difficult. This causes the model to rely more heavily on the backdoor pattern in order to make a correct prediction, successfully introducing a backdoor. As discussed earlier, it is important that our perturbations are plausible in the sense that human inspection should not identify the label of a poisoned input as incorrect. We explore two methods of synthesizing these perturbations. We want to emphasize that even though examples poisoned using our approach are not immune to being identified as potential outliers, our inputs will not appear suspicious (by being clearly mislabeled). Generative models such as GANs BID6 and variational auto-encoders (VAEs) BID13 operate by learning an embedding of the data distribution into a small dimensional space. An important property of this embedding is that it is "semantically meaningful". By interpolating latent vectors in that embedding, one can obtain a smooth transition from one image into another BID23 (which cannot be done through a simple interpolation of the images in the pixel space).Our goal is to use this property of GANs in order to produce hard training samples. We train a GAN on the training set. This provides us with a generator G: R d → R n that, given a random vector z in the d-dimensional latent space (referred to as an encoding), generates an image G(z) in the n dimensional pixel space. In order to retrieve an encoding for each training image, we optimize over the space of latent encodings to find one that produces an image close to our target in 2 distance. Formally, given a generator G and a target image x ∈ R n to encode, we define the encoding of x using G to be (This inversion method was also used in the context of defenses against adversarial examples BID12 .) Now, given the encodings for the training set, we are able to interpolate between classes in a perceptually smooth manner. For some interpolation constant τ, we define the interpolation I G between images x 1 and x 2 as DISPLAYFORM0 DISPLAYFORM1 Varying τ produces a smooth transition from x 1 to x 2 as seen in FIG1 (even though we are not able to perfectly encode x 1 and x 2). We choose a value of τ that is large enough to make the image harder to learn, but small enough to ensure that the perturbation appears plausible to humans. Adversarial examples BID29 are inputs that have been imperceptibly perturbed with the goal of being misclassified by neural networks. These perturbations have been found to transfer across models and architectures BID29 BID22. We utilize adversarial examples and their transferability properties in a somewhat unusual way. Instead of causing a model to misclassify an input during inference, we use them to cause the model to misclassify during training. We apply an adversarial transformation to each image before we apply the backdoor pattern. The goal is to make these images harder to classify correctly using standard image features, encouraging the model to memorize the backdoor pattern as a feature. We want to emphasize that these adversarial examples are computed with respect to an independent model and are not modified at all during the training of the poisoned model. Our choice of attacks is p -bounded perturbations constructed using projected gradient descent (PGD) BID18. For a fixed classifier C with loss L and input x, we construct the adversarial perturbations as DISPLAYFORM0 for some p -norm and bound ε. We construct these perturbations based on adversarially trained models since these perturbations are more likely to resemble the target class for large ε BID30.Example poisoned samples are visualized in FIG2. 2 has similar performance to the baseline. τ ≥ 0.2 has substantially improved performance at 6 % poisoning. Right: Attacks using adversarial perturbations ed in substantially improved performance on the airplane class relative to the baseline, with performance improving as ε increases. Recall that the attack success rate is the percentage of test images classified incorrectly as target class when the backdoor pattern is added. (Figure 2), especially for the 1.5% and 6% poisoning percentages. Right: The 2 norm-bounded attack with ε = 600 ed in high attack success rates on all classes when poisoning a 1.5% or greater proportion of the target label data. Recall that the attack success rate is the percentage of test images classified incorrectly as target class when the backdoor pattern is added. A per-class comparison can be found in Appendix C.2. We find that both approaches led to poisoned images with plausible labels (see Appendix C.3.1 and C.3.2) when the attack is restricted to having small magnitude. Recall that our backdoor attack applies the backdoor trigger to the (slightly pertubed) poisoned inputs without changing their labels. We are interested in the attack success rate, that is the fraction of test images that are incorrectly classified as the target class when the backdoor is applied. Increasing the magnitude of the attack leads to more powerful attacks (FIG3) but renders the original labels less plausible. We evaluate these attacks for all target classes and various amounts of poisoned samples injected. We find that both approaches significantly increase the effectiveness of the poisoning attack (FIG5) when compared to a baseline attack (Figure 2) that simply introduces the backdoor trigger on clean images (see Section 5 for details). Finally, we observe that attacks based on adversarial perturbations are more effective than GAN-based attacks. We elaborate on this difference in the following section. A per-target class comparison of these methods and the baseline attack described earlier is given in Appendix C.2. In the previous section, we observe that p -bounded adversarial perturbations are more effective for backdoor attacks than our GAN-based interpolation method, especially when the allowed perturbation is large. This might seem surprising at first. Both methods render the images harder to classify without utilizing the backdoor so we expect the ing models to crucially rely on the backdoor pattern. Notice, however, that simply utilizing the backdoor pattern is insufficient for a successful backdoor attack. A classifier with a backdoor needs to predict the target class even when the original image is easy to classify correctly. In other words, the reliance on the backdoor pattern needs to be strong enough to overpower the entirety of the signal coming from salient image features. This perspective suggests a natural explanation for the mediocre success of interpolation-based attacks. Inputs created via interpolation do not have a strong enough signal for non-target classes as the characteristics appear "smoothened-out". The adversarially perturbed inputs, on the other hand, do contain such a signal, ing in a strong reliance on the backdoor pattern. At inference time, this reliance is able to overcome the reliance on natural features. In order to further investigate this hypothesis, we perform experiments where we add Gaussian noise to poisoned inputs (see Appendix B.3). While a small amount of noise makes the attack more effective, increasing the magnitude of the noise has an adverse effect. Intuitively, the poisoned images do not contain meaningful information about the label of the original image anymore. Thus a classifier that weakly relies on the backdoor can classify the images correctly. Since the dependence on the backdoor is weak, during testing, the classifier will largely ignore the backdoor as the rest of the image is easy to classify. In this section so far, we have described how to generate samples that can effectively be used for backdoor attacks while having a plausible label. However the introduction of the backdoor pattern itself might make these inputs suspicious (see Figure 1). In order to make the attack more insidious, we experiment with backdoor patterns that are less likely to be detectable. We find that this does not have significant impact to the success of the attack (see Appendix B.1 for details). The ing set of poisoned (after adversarial perturbation and addition of reduced-amplitude pattern) does not differ significantly from the original set images (Appendix C.3.3).In order to avoid introducing conflating factors to our study, we trained our models without data augmentation (standard data augmentation introduces flips and crops which might obscure the pattern). In Appendix B.2, we perform additional experiments with standard data augmentation methods and observe that this hinders our attacks 4. We find however that, by modifying the attack to introduce additional backdoor patterns in different corners of the image (see Appendix FIG10), we recover the attack's success rate. Nevertheless, we find that data augmentation can occasionally still be an obstacle for our methods as it may cause our attack to fail in a stochastic manner (Appendix C.1). We believe that further experimentation with the backdoor pattern can lead to stronger attacks. Recall that our threat model is as follows. The attacker choses a target class label L and a fraction of training inputs to poison. They then modify these inputs arbitrarily as long as they remain consistent with their original label and introduce a backdoor pattern to these inputs. The pattern consists of a small black-and-white square applied to the bottom-right corner of an image. We choose the same pattern as BID9 for consistency, but we note that understanding the impact of different pattern choices is an important direction for investigation. An example of this pattern applied to an otherwise unchanged image from the dataset is shown as the clean-label BID9 image in Figure 1. A classifier is then trained on this poisoned dataset. To evaluate the ing network, we consider the data in the test set not labeled as the target class. Recall that the attack success rate is the percentage of these test data that are nonetheless classified as the target when the backdoor pattern is applied. All of our experiments are performed on the CIFAR-10 dataset BID15 ) containing 50 000 training examples (5000 for each of the ten classes). For each method of increasing the classification difficulty, experiments are performed targeting all ten classes individually. Furthermore, they are tested at each of the following poisoning proportions, which roughly form a quadrupling geometric series: 0.4%, 1.5%, 6%, 25%, and 100%. This series is chosen to evaluate the attack at a wide variety of scales of poisoning percentages 5. Note that these rates represent the fraction of examples poisoned from a single class. Thus, poisoning 6% of the examples of a target class corresponds to poisoning only 0.6% of the entire training set. In the following experiments, we use a standard residual network (ResNet) BID11 with three groups of residual layers with filter sizes of 16, 16, 32 and 64, and five residual units each. We use a momentum optimizer to train this network with a momentum of 0.9, a weight decay of 0.0002, batch size of 50, batch normalization, and a step size schedule that starts at 0.1, reduces to 0.01 at 40 000 steps and further to 0.001 at 60 000 steps. The total number of training steps used is 100 000. We used this architecture and training procedure throughout our experiments and did not adjust it in any way. None of the attacks below had any apparent effect on the standard accuracy -that is, the accuracy of the model on non-poisoned test data -except at 100% poisoning. At that extreme, there is a substantial decline, with standard accuracy decreasing by up to 10 percentage points. We found that this decrease is due to the model relying entirely on the backdoor pattern and thus predicting incorrect labels for the entire target class when the pattern is absent. For our experiments, we train a WGAN BID10 6. In order to generate images similar to the training inputs, we optimize over the latent space using 1000 steps of gradient descent with a step size of 0.1, following the procedure of BID12. To improve the image quality and the ability to encode training set images, we train the GAN using only images of the two classes between which we interpolate. We compare attacks that use different degrees of GAN-based interpolation: τ = 0, 0.1, 0.2, 0.3.While our reproductions of the original images are noticeably different, we find that images generated with τ ≤ 0.2 typically remain plausible. An example of these interpolated images is shown in FIG1. A more complete set of examples at the tested τ is available in Appendix C.3.1. We observe that increasing τ in more successful attacks (FIG3).For the above reasons, τ = 0.2 was chosen for further investigation due to the plausible images and improvement in performance. We investigate the τ = 0.2 GAN-based interpolation attack on all classes. This shows improvement over the baseline, especially for the 1.5% and 6% poisoning percentages (FIG5). A class-by-class comparison to the baseline is given in Appendix C.2. We construct adversarial examples using a PGD attack on adversarially trained models BID18 7. We used p -adversarially trained models for constructing p -bounded perturbations for p = 2, ∞ as these models have been found to produce adversarial examples that resemble images from other classes when large perturbations are allowed BID30. Note that since our threat model does not allow access to the training procedure, these adversarial perturbations are generated for pre-trained models and not on the fly during training. We compare attacks using 2 -and ∞ -norm adversarial perturbations of different magnitudes. For this experiment, we consider a random class (the airplane class), with the following maximum perturbations (ε) normalized to the range of pixel values: 300, 600 and 1200 for 2 -bounded examples, and 8, 16 and 32 for ∞ -bounded examples. There is a clear trend of increasing ε ing in substantially improved performance (FIG3). At 1.5% poisoning, the middle and higher tested values of ε for each norm achieve over 70% attack success, despite the baseline of restricting the BID9 attack to clean labels having near 0% attack success. These are shown in FIG3 These adversarial examples look plausible and, as ε increases, appear to interpolate towards other classes. An example of these perturbed images at each of the tested norms and values of ε is shown in FIG2. The highest ε tested for each norm in readily apparent distortion. The lower values for ε tested in plausible images. See Appendix C.3.2 for more details. Due to the performance and plausibility shown above, 2 -based perturbations with ε = 600 were chosen for further investigation. For every class, the attack success rate is substantially higher than the clean-label BID9 attack baseline on all but the lowest tested poisoning percentage FIG5 ). We investigate the backdoor attacks of BID9 in the presence of a simple data filtering scheme. While their attack is powerful, it crucially relies on the addition of arbitrary, mostly mislabeled, inputs into the training set and can thus be detected by filtering. Human inspection of the identified outliers will clearly flag the poisoned samples as unnatural. We argue that, for a backdoor attack to be insidious, it must not rely on inputs that appear mislabeled upon examination. To remain successful under the clean-label restriction, we propose perturbing the poisoned inputs to render them more difficult to classify. We restrict the magnitude of these changes so that the true label remains plausible. We propose two methods for increasing classification difficulty: adversarial p -bounded perturbations and GAN-based interpolation. We find that both methods introduce a backdoor more successfully than the clean-label adaptation of the BID9 attack. These findings demonstrate that backdoor attacks can be made significantly harder to detect than one might initially expect. This emphasizes the need for developing principled and effective methods for protecting ML models from such attacks. A THE ORIGINAL ATTACK OF GU We replicate the experiments of BID9 on the CIFAR-10 (BID15 dataset. The original work considered the case where the model is trained by an adversary, since they focused on the transfer learning setting. The authors accordingly imposed essentially no constraints on the number of poisoned samples used. In contrast, we study the threat model where an attacker is only allowed to poison a limited number of samples in the dataset. We are thus interested in understanding the fraction of poisoned samples required to ensure that the ing model indeed has an exploitable backdoor. In Figure A, we plot the attack success rate for different target labels and number of poisoned examples injected. We observe that the attack is very successful even with a small (∼ 75) number of poisoned samples. Note that the poisoning percentages here are calculated relative to the entire dataset. The x-axis thus corresponds to the same scale in terms of examples poisoned as the rest of the plots. While the attack is very effective, most image labels are clearly incorrect (Figure 1). Despite our above focus on the plausibility of the base image, the backdoor pattern itself could also cause plausibility problems if its presence appears unnatural. To mitigate this potential suspicion, we consider a modified backdoor pattern. Instead of entirely replacing the bottom-right 3-pixel-by-3-pixel square with the pattern, we perturb the original pixel values by a backdoor pattern amplitude. In pixels that are white in the original pattern, we add this amplitude to each color channel (i.e. red, green and blue). Conversely, for black pixels, we subtract this amplitude from each channel. We then clip these values to the normal range of pixel values. (Here, the range is.) Note that when the backdoor pattern amplitude is 255 or greater, this attack is always equivalent to applying the original backdoor pattern. We extend our proposed adversarial example-based attack to reduced backdoor pattern amplitudes. We explore this attack with a random class (the dog class), considering backdoor pattern amplitudes of 16, 32 and 64. All (non-zero) backdoor pattern amplitudes ed in substantial attack success rates at poisoning percentages of 6% and higher. Higher amplitudes conferred higher attack success rates. At the two lower poisoning percentages tested, the attack success rate was near zero. These are shown in FIG8.Image plausibility is greatly improved by reducing the backdoor pattern amplitude. Examples of an image at varying backdoor pattern amplitudes are shown in FIG9. A more complete set of examples is available in Appendix C.3.3.We have chosen a backdoor pattern amplitude of 32 for further investigation as a balance between conspicuousness and attack success. We tested this attack on all classes, finding similar performance across the classes. These are shown in FIG8. Data augmentation is commonly used to reduce overfitting while training deep learning models. The general approach is to not only train on the original training set, but also on the same data transformed in simple ways. Common techniques include cropping and flipping, which can be problematic for our attack given that they might obscure the pattern. It is important to understand the impact of data augmentation on our attack, given its wide usage. To improve attack success when using data augmentation, we consider an alternate backdoor pattern, where the original pattern and flipped versions of it are applied to all four corners. This aims to encourage backdoor pattern recognition even when images are flipped or randomly cropped. An example of this pattern (with our chosen amplitude of 32) applied to an example image is shown in FIG10. The backdoor pattern duplication is motivated by the desire to ensure at least one corner pattern is still visible after cropping and to remain invariant to horizontal flips. We investigate and compare our reduced backdoor pattern amplitude attack when training both with and without data augmentation. For each of these cases, we also compare the original (one-corner) and four-corner backdoor pattern. These initial experiments were performed on a random class (the frog class, FIG10). We see that, when data augmentation is not used, there is little difference in performance between the four-corner backdoor pattern attack and the original one-corner attack. When data augmentation was used, however, there is a large difference between these attacks. Use of the one-corner backdoor pattern in substantially reduced attack success for all poisoning percentages while the four-corner backdoor pattern attack achieves over 90% attack success rates for poisoning percentages of 6% and greater. These are shown in FIG10.These show that the performance improvement under data augmentation does not primarily from the backdoor pattern simply being applied to more pixels. Rather, the four-corner pattern ensures at least one corner's backdoor pattern will remain visible after the data augmentation is applied.. Right: Using the four-corner pattern does not provide a substantial benefit over the one-corner pattern when data augmentation is not used. When data augmentation is used, however, the difference in performance is stark, with the one-corner pattern achieving much lower attack success rates. We then explored the performance of this four-corner attack under data augmentation on all classes. For comparison, we similarly investigated the original, one-corner attack's performance under data augmentation. The one-corner attack in a near-zero attack success rate across almost all the classes and poisoning percentages. The four-corner attack showed generally similar as the exploratory experiment. These are shown in FIG11. However, some classes showed varying degrees of stochasticity in their ing attack success rates. This was investigated by altering the random seeds used in network training. The ship class performed particularly poorly, with only one datum across three runs showing a high attack success rate. Other classes appeared to successfully poison much more stably. The one-and four-corner all class attack are shown in FIG11. We present two other runs of the four-corner augmentation resistant attack in Appendix C.1, along with graphs showing the minimum, median and maximum attack success rates achieved across each of the three runs. Gaussian noise with a zero mean and varying standard deviations was added to examples before application of the backdoor pattern. This was used as the method to increase the difficulty of the examples. As shown in FIG12, we found that there is some improvement at low standard deviations. At higher standard deviations, however, the performance degrades substantially. As discussed earlier, at high standard deviations of Gaussian noise, poisoned images do not contain meaningful information about the label of the original image anymore. Thus they can be easily classified correctly by using the backdoor with relatively small weight. Two additional runs of the final attack on all classes are given below. The random seed used in training was varied between the run shown in FIG11 and each run below. For each class and poisoning percentage tested, we also calculate the minimum, median and maximum values across the three runs. These are presented below. We compare the performance of the baseline of the BID9 attack restricted to only clean labels, our GAN-based interpolation attack, and our adversarial perturbation-based attack for each class. The adversarial examples-based attack substantially outperforms the other two at all but the lowest poisoning percentage. The GAN-based attack usually outperforms the baseline, but by a smaller margin than the adversarial examples-based attack. Each row shows two sets of randomly chosen examples from a single class. In each set, the leftmost image is the original image from the CIFAR-10 dataset and the subsequent images are the corresponding image interpolating using a GAN. At the top of the first row, each column's degree of interpolation is given. The τ = 0 examples show that we were unable to perfectly encode the image. As τ increases the images show increased distortion. Each row shows five pairs of examples from a single class. The left image in each pair is the original image from the CIFAR-10 dataset and the right image is the corresponding image perturbed using 2 -norm adversarial perturbations (bounded by ε = 600) and with the reduced amplitude backdoor pattern applied (using an amplitude of 16). It should be noted that the poisoned images do rarely change enough to appear mislabeled.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We show how to successfully perform backdoor attacks without changing training labels.

Represent the TLDR for finding the corresponding scientific article

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.