Determining how similar audio is to human speech While looking for an answer to this problem, I found this board so decided to cross post this question of mine from Stack Overflow.



I am searching for a method of determining the similarity between an audio segment and a human voice, which is expressed numerically.

I've searched quite a bit, but what I've found so far (detailed below) doesn't really fit what I need:


One method is to use speech recognition software to obtain words from an audio segment. However, this method is unable to come up with how "similar" audio is to human speech; it can often tell whether or not there are words in the audio, but if there are no definite words, it can't tell close the audio is to having such words.Examples: CMU Sphinx, Dragonfly, SHoUT
The more promising method is referred to as Voice Activity Detection (VAD). However, this tends to have the same problems: the algorithms / programs using VAD tend to just return whether or not the activity threshold has been reached, and no "similarity" value before or after such threshold. Alternatively, many just look for volume, not similarity to human speech.Examples: Speex, Listener, FreeSWITCH




Any ideas?
This kind of problem is usually solved using machine learning techniques.

Break down the signal into a sequence of 20ms or 50ms frames. Extract features on each frame. MFCC are generally good for this kind of application, though there are feature more specific to voice detection (4 Hz modulation energy - which is roughly the rate at which people speak ; zero-crossing rate).

Then, using a training set of audio you have manually labelled as being speech / not speech, train a classifier (Gaussian mixture models, SVM...) on the frames features.

This will allow you to classify unlabelled frames into speech/non-speech classes. The last step consists in smoothing the decisions (a frame classified as non-speech surrounded by hundreds of speech frame is likely to be classification error), for example using HMMs, or just a median filter.

A few references:

Robust speech / music classification in audio documents (Pinquier &amp; al)
Speech / music discrimination for multimedia applications (El-Maleh &amp; al)
A comparison of features for speech / music discrimination (Carey &amp; al)

Note that the features and classification techniques they describe are also relevant for the 1-class problem of detecting speech (instead of discriminating speech vs something else). In this case, you can use 1-class modeling techniques such as 1-class SVM, or just take the likelihood score out of a GMM trained on a speech data as a "speechiness" measure.

If, on the other hand, your problem is really discriminating speech vs something else (say music), you could also very well use unsupervised approaches which are focused on detecting the boundaries between similar audio content - rather than identifying this content itself.