Every Breath You Don't Take: Deepfake Speech Detection Using Breath
Seth Layton Thiago De Andrade Daniel Olszewski Kevin Warren Carrie Gates Kevin Butler Patrick Traynor









sethlayton@ufl.edu


tdeandradebezerr@ufl.edu

dolszewski@ufl.edu

kwarren9413@ufl.edu

  University of Florida
  
  Gainesville
  Florida
  USA
  



carrie.gates@bofa.com

  Bank of America
  
  Charlotte
  North Carolina
  USA



butler@ufl.edu

traynor@ufl.edu

  University of Florida
  
  Gainesville
  Florida
  USA
  















  Deepfake speech represents a real and growing threat to systems and society. Many detectors have been created to aid in defense against speech deepfakes. While these detectors implement myriad methodologies, many rely on low-level fragments of the speech generation process. We hypothesize that breath, a higher-level part of speech, is a key component of natural speech and thus improper generation in deepfake speech is a performant discriminator. To evaluate this, we create a breath detector and leverage this against a custom dataset of online news article audio to discriminate between real/deepfake speech. Additionally, we make this custom dataset publicly available to facilitate comparison for future work. Applying our simple breath detector as a deepfake speech discriminator on in-the-wild samples allows for accurate classification (perfect 1.0 AUPRC and 0.0 EER on test data) across 33.6 hours of audio. We compare our model with the state-of-the-art SSL-wav2vec model and show that this complex deep learning model completely fails to classify the same in-the-wild samples (0.72 AUPRC and 0.99 EER).

































Every Breath You Don't Take: Deepfake Speech Detection Using Breath
    Patrick Traynor
    April 29, 2024
===================================================================





§ INTRODUCTION


Deepfake speech (e.g., text-to-speech, deepfakes, voice assistants) aims to make the differentiation between synthetic and organic speech difficult <cit.>. While such audio has many benign uses, the potential for dangerous applications has created the need for accurate and automated demarcation of human-spoken from synthetically-generated audio.

The research community has responded with competitions such as ASVspoof <cit.>, ADD <cit.>, and SASV <cit.>. These competitions curate datasets of deepfake and real speech and invite participants to create detection algorithms to test on these datasets. Subsequently, these datasets are the de facto standard for deepfake speech and give a baseline of comparison for all current and future deepfake speech detectors. Most of the currently existing speech deepfakes detectors focus on low-level spectral (e.g., spectrogram, MFCC, LFCC, and CQCC) imperfections created during the audio generation pipeline and show starkly different classification results vs. human interpreters <cit.>. This technique of low-level spectral detection will be rendered obsolete due to the rapid advancement of the speech generation field. Thus, a paradigm shift towards high-level speech features such as prosody detection <cit.>, emotion detection <cit.>, and anatomical shape detection <cit.> is underway.

One promising avenue for high-level speech feature exploration is breath, as breathing is one of the subtle ways that humans subconsciously perceive naturalness in speech <cit.>. Additionally, the demand for automatic breath detection methods for medical research purposes is elevated due to COVID-19 <cit.>. This demand influenced the INTERSPEECH 2020 Computational Paralinguistics Challenge to create the Breathing Sub-Challenge, a track dedicated to breath detection <cit.>. Current state-of-the-art breath detection methods include spectral-based models <cit.>, acoustic models <cit.>, and raw speech waveform deep learning <cit.>. While these methods automatically detect breath in audio, applying these techniques to the realm of deepfake speech to demarcate real from speech deepfakes is an open challenge. Towards this, we explore the viability of breaths as a detection mechanism for deepfake speech. 

Our work focuses on in-the-wild deepfake speech. News outlets inherently opt for industry-standard high-quality synthetic speech to not deter listeners. We note that there is a distinction between industry-standard and state-of-the-art. This distinction creates a mismatch between academically-created and in-the-wild deepfakes. Specifically, academia produces cutting-edge deepfakes, but these deepfakes are rarely seen en masse on the internet. It is thus important to study what is currently affecting the largest population and is the focus of this work. Towards this, we gather speech samples from online news vendors that employ a text-to-speech or human-read audio listening option for online text articles. We then determine the efficacy of breath detection as a deepfake speech discriminator using simple models. We compare our simple models against a state-of-the-art model and highlight the shortcomings of complex machine learning techniques. We focus on currently deployed deepfake speech, which gives a measure for the industry standard in the field and as such, we argue against the use of popular academic datasets such as ASVspoof. 

Our main contributions are: 

  
  * We perform a study of currently deployed in-the-wild synthetic speech.
  
  * We create a generation-agnostic deepfake speech detector based solely on breaths.
  
  * We publish a dataset of in-the-wild text-to-speech and human-read speech.
  
  * We highlight shortcomings and over-reliance on complex deep learning detection models.



The remainder of this paper is organized as follows:
Section <ref> states our hypothesis;
Section <ref> details our methodology;
Section <ref> presents our experimental results;
Section <ref> offers discussion and insights based on our findings;
Section <ref> discusses related work; and
Section <ref> presents our concluding remarks.



§ HYPOTHESIS



We hypothesize that current speech deepfakes generation techniques do not sufficiently incorporate breaths.

To investigate this claim, we must first determine if breath is a generalizable speech feature. Thus, we define our first research question:
1Are breaths automatically detectable, intra and inter-speaker?

As breath is one of the ways that humans determine naturalness in speech; we define our second research question:
2Do current deepfake voices generate breaths?

Combining the previous research questions we define our final research question:
3Are breaths able to accurately discriminate between in-the-wild deepfake and real samples?





§ METHODOLOGY

    To determine if breaths are a generalizable feature between speakers and subsequently useful as a discriminator against real and fake speech we define deepfake speech, gather real and fake data, implement and test detection models, and evaluate the performance of breathing.


    

 §.§ Cheapfake vs. Deepfake

    
        Synthetic media is a spectrum that spans from generative (e.g., machine learning and artificial intelligence) to manually altered (e.g., Photoshop and speech waveform manipulation) samples. The use of deep neural networks creates a deepfake and using cheap manual software manipulation creates a cheapfake <cit.>. Specifically, a cheapfake requires a pre-existing sample of a specific individual for manual modification; whereas, a deepfake may be fully generative and not require a source sample. While both forms of synthetic media influence society, we focus on the dominant form of synthetic media known as deepfakes for the remainder of this paper. More critically, we only consider media samples that are entirely real or entirely fake (i.e., no partially fake samples with segments altered). Additionally, this extends to manipulation techniques such as altering the rate of speech or changing the pitch.

    

 §.§ Dataset

    
        We employ a multi-tiered model pipeline that requires independent datasets during the training phase. We gather single-speaker podcast audio for training the breath detector and online news articles read by humans and text-to-speech algorithms for the final deepfake speech detection training and testing.

        

  §.§.§ Why not ASVspoof
 
        
        

            ASVspoof is the de facto standard dataset when creating and testing a deepfake speech detector and it gives a community baseline for comparison. However, for our task, this dataset is not representative, sufficient, or realistic. First, our model relies on features of breathing and thus samples must be sufficiently long to contain a breath (93% of ASVspoof 2021 samples are shorter than 5 seconds). As breaths in read and spontaneous speech are expected at a rate of 8-14 per minute <cit.>, each ASVspoof sample is not expected to contain any of these important high-level features. Additionally, many of the samples longer than 5 seconds are filled with incoherent speech due to generation issues. Moreover, the class distribution of the ASVspoof 2021 samples is unrealistic (97% deepfake and 3% real in the evaluation set). This class distribution is vastly different from the expected in the wild distribution and would require multiple resampling techniques to correct, which could bias the results due to the vast imbalance of the dataset <cit.>. Finally, we opt to gather samples that are currently in the wild today as a better representation of deepfake speech and contain a balanced real/fake class distribution.

        

  §.§.§ Podcasts

        
        

            We curate a training dataset of podcasts that meet specific criteria and manually annotate these podcasts for breath locations. Each podcast is single-speaker, contains no background music, is free from obvious background noise, and breathing is noticeable. In total, we selected 10 podcasts from 4 speakers totaling just over 5 hours. Each podcast is manually annotated for precise breath locations by two independent annotators and a third annotator verifies and reconciles any discrepancies in the annotations from each initial annotator to ensure all breaths are captured.
            


        

  §.§.§ News Articles

        
        

            We gather news article audio from online news vendors for training and testing our deepfake speech detector. We search for news articles with strings like “Listen to this article” and  “Hear the full article” keywords on the webpage to indicate that there is an audio version of the transcript available. Manual checking of the news vendor is required to determine if the news articles provided are text-to-speech generated, or spoken by a human. In total, we collected 277 TTS (26.94 hours) and 56 human-read (25.54 hours) news articles from four different news outlets (two TTS sets and two human-read sets).[Each sample is entirely deepfake, or entirely real. There is no scenario for deepfake speech injected into real audio. We do not redistribute the audio; however, links to news article websites are freely available at: <https://sites.google.com/view/ebydt/>.]


    

 §.§ Breath Detection

    
        First, we create a breath location detector to highlight breaths in a given audio sample. Towards this, we take inspiration from rare-event detection <cit.> to create our pipeline for breath detection. Unlike these works, however, we do not perform image analysis on spectrograms, but instead use the raw values computed from the spectrogram. Thus, all the layers of our model lose a dimension, reducing model complexity and providing speed increases to training and inference which helps real-time predictions.


         

        

  §.§.§ Feature Selection
 
        
        

            We first calculate raw values of the mel-spectrogram (dB converted), zero crossing rate (ZCR), and root mean squared energy (RMSE) (dB converted). Each of these features slides a window over a waveform to calculate an overlapping temporal-based value. As the size of the window and the duration between windows may be optimized we test a variety of values ranging from 5ms – 200ms window_length and 2.5ms – 25ms hop_length. The result of this feature extraction is an array of frames that contain spectrogram, ZCR, and RMSE values for an entire audio sample. The number of frames in an array is calculated as num_frames = sample_duration(milliseconds)/hop_length, and the value of each frame is the aggregated mel-spectrogram, ZCR, and RMSE for window_length duration. For example, a 5-second excerpt of audio with a window_length of 50ms and a hop_length of 5ms creates a 1000-frame array.  We test a spectrum of sizes, shapes, and durations for these features and select a window_length of 20ms, a hop_length of 2.5ms, and 128 mel-spectrogram buckets for our final model as these produce the best results.

            Next, using the manually annotated location of breath we denote each frame in the array as either a breath (i.e., positive class) or not a breath (i.e., negative class). A frame is considered to contain breath if more than half the window_length of that frame is annotated as a breath. Figure <ref> shows our selected features and how they change during a breath and speaking before/after breathing. For breaths, ZCR and RMSE tend towards medium values between silence and spoken segments and the mel-spectrogram shows only energy at lower frequencies.

        

  §.§.§ Model Architecture
 
        
        

            Based on Székely et al. <cit.> we build a multi-tiered convolutional and recurrent neural net detection model. The model architecture starts with two 1D convolutional layers (16 and 8 filters, 3 and 1 kernel sizes, same padding, and ReLU activation); each convolutional layer is followed by batch normalization, max pooling (pool size of 3), and 0.2 dropout. These initial layers are used as input to a bidirectional-LSTM layer which feeds into a dense layer with a sigmoid activation for final prediction. We use a binary cross-entropy loss as the loss function and Adam as the optimizer function. The pipeline for this model is shown in Figure <ref> (using a window_length of 20ms, a hop_length of 2.5ms, 128 mel-spectrogram buckets, and a batch size of 32). We experimented with many different shapes and sizes for our breath detection model architecture such as changing the total number of convolutional layers, adjusting the number of filters and kernel sizes of each convolutional layer, altering the default max pool size, and changing the size of the dropout layer. Ultimately we select the sizes and parameters defined in Figure <ref>.

            As input to our breath detection model, an entire audio sample is sectioned off into 2-second segments and sequentially fed to the model with predictions given for every 50ms (i.e., 40 predictions per 2-second chunk). We opt for 2-second slices of audio as increasing the duration of the slices subsequently increased training and inference time without improving performance. A 2-second segment is 800 2.5ms slices of features and there are 128 values from the mel-spectrogram and 1 feature from each of the ZCR and RMSE giving a total of 130 features. The shape of (32 x 800 x 130) is given as input to our breath detector as we use a batch size of 32. The output for a 2-second chunk of audio is 40 sequential binary classifications as to whether or not there is a breath located in each 50ms slice.

        

  §.§.§ Model Result Post-Processing
 
        
        

            Our pipeline implements a simple mechanism on the resultant predictions. We remove any predicted breaths that are shorter than 150ms as we measured no breaths shorter than 150ms in any of our podcasts. This post-processing affects few samples, yet helps smooth out the resultant breath locations. We then use these breath locations as input features for the deepfake speech detection algorithm.  
        
        

  §.§.§ Metrics

        
            We use a multitude of metrics for our full pipeline and evaluation of intermediary and final results. For breath detection, we use the binary cross entropy loss function for hyperparameter tuning and use Area Under the Precision-Recall Curve (AUPRC) for evaluating model performance. We use AUPRC as our main metric as it is robust to imbalanced data <cit.> and effectively highlights performance on the important class. As breaths are an imbalanced class distribution problem (i.e., non-breathing heavily outweighs breathing in normal speech) and are the important class, AUPRC is a suitable metric for honestly evaluating performance. 

     

    

 §.§ Deepfake Speech Detection

    
        The final component of our pipeline uses the predicted breath locations as input. For an audio sample, we use the predicted breath locations to calculate three features: average breaths per minute, average breath duration, and average spacing between breaths. With these features, we create multiple simple deepfake speech detectors to test the viability of computationally inexpensive methods and compare models to a complex deep learning model (Section <ref>). The first is a thresholding classifier that uses previously measured breath statistics. The second is a C-Support Vector Classification (SVC) algorithm with a poly kernel, a regularization parameter of 1, and a degree of 2. Finally, we implement a three-tiered decision tree with default parameters. Each of these mechanisms produces a single binary prediction for the entire audio sample. The pipeline for this deepfake detection model is shown in Figure <ref>. 

        The thresholding method should be sufficient as a final discriminator if deepfake speech does not produce any breaths. However, if breaths are produced, then a more robust mechanism would be required to accurately discriminate between real and fake samples. Thus, we apply both techniques and compare the results to determine viability and then compare them to complex and computationally expensive state-of-the-art.


        

  §.§.§ Metrics

        
           For the final deepfake speech detector, we implement a range of metrics to help contextualize model performance, as single metrics may fail to honestly describe model performance. These metrics are accuracy, F1-score, precision, recall, AUPRC, equal error rate (EER), true positives, true negatives, false positives, and false negatives. For the thresholding method, we are unable to use AUPRC and EER as these metrics require probabilities, and only a prediction is generated. Finally, we note that the use of EER as a performance metric is deprecated by ISO/IEC standards <cit.> and has been shown to obfuscate results <cit.>; however, EER remains the current community standard for deepfake speech detection.
    
    



§ EXPERIMENTS

    
    To sufficiently answer our research questions defined in Section <ref> we perform two experiments starting with breath generalizability and finishing with deepfake speech (deepfake) detection.
    

 §.§ Breath Generalizability (RQ1)

    
        To understand if breath sounds are generalizable between different individuals (thereby justifying them as a deepfake speech discriminator), we create three tests. RQ1 tests the generalizability of breathing with Test 1 obtaining a comparative baseline for successive tests, Test 2 examining the performance of each podcast, and Test 3 examining the performance of each speaker. Each test uses our breath detection model defined in Section <ref> without any changes.

      

  §.§.§ Test 1
 
      
      

        Test 1 creates a best-case baseline for comparison with Tests 2 and 3 employing k-fold cross-validation. Test 1 takes (1/x * 100)% consecutive frames, where x is the total number of podcasts, randomly from each podcast to use as the validation set and uses the remaining 100-(1/x * 100)% from each podcast as the training set. We do this process 100 times and calculate the validation AUPRC to obtain a baseline for comparison. This test gives the best-case scenario due to having a subsample of every podcast/speaker in each training and validation batch. This is in contrast to the following two tests which avoid overlapping training and validation speakers/podcasts.

      

  §.§.§ Test 2

      
      

        Test 2 employs a leave-one-out strategy to determine the impact on breath detection for each podcast. A podcast is set aside individually as the validation set and the remaining x-1 podcasts become the training set. We retrain our breath detection model using the training set and calculate the AUPRC for the validation set. We do this for all x podcasts and compare the results to Test 1 to highlight any podcasts that may be problematic, or not generalizable.

      

  §.§.§ Test 3

      
      

        While similar to Test 2, Test 3 evaluates the impact on breath detection for a single speaker. Towards this, all podcasts from a specific speaker are set aside as the validation set and the remaining speaker's podcasts become the training set. Again we retrain our breath detection model using the training set and calculate the AUPRC for the validation set. This process is repeated for each speaker and compared against Test 1 to determine if breaths are generalizable between speakers.
      

      

      

  §.§.§ Results

      
      

        Figure <ref> shows the results of these three tests. We use the baseline AUPRC of ∼0.93 from Test 1 to compare Tests 2 and 3. Test 2 shows podcast-specific leave-one-out performance between ∼0.84 and ∼0.95, which is a marked reduction in performance from the baseline (Test 1). Test 3 shows speaker-specific leave-one-out performance between ∼0.78 and ∼0.94, which is also a reduction in performance from the baseline. This reduction is expected as in Test 1 there is training data of similar distribution to the validation in every iteration since validation is derived as a subset of each podcast as described in Section <ref>. However, we see an AUPRC mean and standard deviation of 0.931±0.008, 0.911±0.041, and 0.84±0.062 for Test 1/2/3 respectively. This indicates that breaths are generalizable and thus usable as a general discriminator for deepfake speech (RQ1).

    

 §.§ Deepfake Speech Detection (RQ2, RQ3)

    


        

  §.§.§ Setup

        
        

            Now that we have shown the generalizability of breaths between individuals, we apply this methodology as a discriminator for deepfake and real speech using the three methods described in Section <ref>. First, we train our breath detection model on all the podcast data to ensure the best possible final model. Next, we pass the news article data outlined in Section <ref> to this final model to get breath locations for each news article. We then use the breath locations and calculate the three features detailed in Section <ref>. Figure <ref> shows the clear delineation between the deefpake and real news article breath features. We demonstrate that deepfake speech, as seen in the wild today, does not appropriately produce breaths, which indicates a strong discriminator using these features (RQ2). Finally, we split the news articles into a training and testing set where no samples from the same news outlet are in the training and test sets. This split comes out to 101 training samples (18.9 hours) and 232 testing samples (33.6 hours) and ensures no bias when testing.
            
            For the poly SVC and decision tree methodologies, we train our model (Section <ref>) with the training set and evaluate performance on the test set. Whereas for the thresholding methodology, we do not need to split the news articles into a training and test set as there is no training taking place since this is not a machine learning model; however, we show results for both the training and test sets to compare against the poly SVC method. Observing Figure <ref> we show that in each calculated metric real samples have a non-zero value. We use this as our guideline and set the threshold for each metric to be Value > 0.0 = Real, Value <= 0.0 = Fake. For this simple thresholding, we require that all features (i.e., avg. bpm, avg. breath spacing, and avg. breath duration) be > 0.0 to be considered real. Stated concisely, for any sample feature that contains a zero value that sample is considered fake.
        
     
          

  §.§.§ Results

          
          

            Table <ref> shows the training and test results for the news article dataset. We show that a simple SVC model can accurately discriminate between real/deepfake speech samples without overfitting the training set. We show substantial performance across all metrics, and most importantly, perfect performance against the test set. Using breaths, we correctly predict all deepfake samples without falsely predicting any real samples (RQ3). Additionally, we show performant results on a varying distribution of data, as the generation techniques for each of the news articles are unknown. Additionally, Table <ref> contains the results for our thresholding methodology. We only display results for the testing set as there is no training required by this technique. Through this, we show that simple thresholding obtains performant results against all news articles. While these results are slightly worse than the results of the SVC model, they do not require any training or testing differentiation and as such, no training time. We posit that simple thresholding is sufficient to capture the nuance in breathing for real and synthetically generated audio samples (RQ3).
      

        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
    
        

  §.§.§ Experimental Comparison to Other Detectors

        
            To fully contextualize the performance of our breath detector we compare our models against a highly complex and heavily trained model: SSL-wav2vec2.0. Towards this, we implement the XLS-R-based deepfake detector <cit.>, as this model is the current best performer on the ASVspoof 2021 dataset by a substantial margin. While we do not use the ASVspoof 2021 dataset in any capacity, the XLS-R model was pretrained on 436,000 hours of non-ASVspoof speech data and is sufficiently performant on non-ASVspoof datasets. 
            
            We deploy the pretrained (wav2vec-_P) model and additionally, to ensure a fair comparison, we fine-tune SSL-wav2vec2.0 with additional training (wav2vec-_R). We implement this fine-tuning by augmenting the SSL-wav2vec2.0 training data with the 18.9 hours of news articles from our training set. This is the same methodology used to retrain SSL-wav2vec2.0 for ASVSpoof 2021. 
            
            We pass the test set of news articles to both versions of SSL-wav2vec (in 4-second chunks, due to model requirements) and use a probability soft voting scheme to get a single prediction for each audio file. Table <ref> shows the results of these models and shows that our model outperforms in every metric. The pretrained SSL-wav2vec predicts nearly the entire test set as deepfake speech whereas the retrained model predicts all the test data as real speech (while predicting the train set with 100% accuracy). The highlighted sections in Table <ref> show the classification decision for all news articles in the test set for each of the model types (i.e., SVC, Decision Tree, SSL-wav2vec2.0). We note that wav2vec-_R predicts all news articles as real and may seem as though the training phase was incorrectly handled; however, looking at the results of wav2vec-_R on the training data shows that it indeed picks up the signal in the data. Through this, we show that our models perform substantially better in every category than the complex models, specifically looking at the false positives it is clear to see that alarm fatigue <cit.> would overwhelm wav2vec-_P. Comparing this to our models which have essentially no false positives, it is clear to see the potential shortcomings of complex models.

            As the thesis of our paper argues, we demonstrate that models focused on low-level speech features can completely fail when tasked with predicting new data as seen by the contradicting results between wav2vec-_P and wav2vec-_R. Simply put, using a complex and highly-trained model trained on low-level speech features provides unpredictable results, even when augmenting additional “in-distribution” training data; whereas higher-level speech features can help prevent failure.

    

 §.§ Summary of Research Questions

    
    Through our experiments, we show that breaths are automatically and accurately detectable between the same speaker and throughout different speakers (RQ1), current in-the-wild deepfake speech does not produce breaths (RQ2), and using breaths as a discriminator between real and fake samples obtains performant results (RQ3).



§ DISCUSSION


We discuss additional material that is important to the fundamental science of machine learning and deepfake discrimination.



 §.§ Reproducibility


Reproducibility is a growing concern, especially for machine learning <cit.>. To alleviate these issues and to aid future work and comparison, we make efforts to increase the reproducibility of our work. First, we publish the code and framework we use for our entire pipeline.[Code available upon publication.] Second, we publish the trained models and raw model scores files.[Scores file available upon publication.] We release the trained model and scores file as an additional artifact as it has been shown that retraining the same model on different GPUs may produce different results <cit.>. This gives future researchers the ability to calculate any and all metrics that may be desired for comparison.



 §.§ Limitations and Future Work


If these deepfake samples start producing frequent breathing, our relatively simple discriminators will likely produce worse results. To accommodate this, a shift towards natural language processing (NLP) could be combined with our work. This combination would allow contextualization between the breaths that are identified and the intra-speech location relative to other parts of speech. This contextualization would minimize false positive breath detection and improve the results of a deepfake detector.



        



§ RELATED WORK


  Recently, multiple papers have begun the exploration of breath importance in deepfake speech. However, none of these papers make any claims on current deepfake speech deployed in the wild.  Mostaani et al. investigated whether breathing patterns are present in text-to-speech (TTS) and voice conversion (VC) algorithms using ASVspoof 2019 <cit.>. This work showed that TTS algorithms fail to properly generate breaths while VC algorithms seem to retain breath pattern-related information. As TTS is rapidly dwarfing VC as the main proponent of deepfake speech, this work shows promise for breath usage as a speech deepfakes discriminator. Additionally, the Breathing-Talking-Silence Encoder (BTS-E) algorithm was proposed as an addition to existing countermeasures (CM) for voice spoofing attacks that use breath and silence event detection to enhance existing CM performance by up to 46% <cit.>. BTS-E leverages three Gaussian Mixture Models (GMM) to segment/label input audio into talking, silence, and breath segments. This segmentation is then translated into a latent feature space and combined with the last hidden layer in an existing CM to focus that CM on talking/breathing/silence events. However, the feature importance of breathing vs. silence is not explored in BTS-E; which is problematic as Müller et al. <cit.> identify an uneven distribution of leading and trailing silence duration in the ASVspoof 2019 and 2021 datasets, which correlate with the target prediction label. Thus, simply examining the silence patterns in the files allows for accurate classification of real/speech deepfakes. As such, it is unclear whether the breath features in BTS-E are important and the value of breaths is unknown. 

  Our work, instead, focuses solely on the prevalence of breaths and on samples gathered from real-world use. These real-world samples give an accurate measure of the landscape of deepfake speech as it stands currently. We make the first attempts at discriminating between real and fake in-the-wild speech using breathing.




§ CONCLUSION


    Deepfake speech generation advancements are reducing the gap between human-spoken and human-sounding audio. 
    Deepfake speech will become imperceivable to human speech, as such a focus needs to be placed on how this speech is performed. Toward this, we employ a multi-tiered pipeline that focuses on breathing to discriminate between real/deepfake speech samples. We show that breaths are generalizable between speakers and that simple calculated breath features provide accurate classification results (perfect 1.0 AUPRC and 0.0 EER) on a dataset of in-the-wild real and synthetically-generated speech. Furthermore, we show the shortcomings of a complex deep learning model that fails to classify the same in-the-wild samples (0.72 AUPRC and 0.99 EER).










ACM-Reference-Format



