How to Filter Human Speech from an Audio clip?
This is not a complete answer, but detection of words from the speech itself is almost always dictionary based - hence one needs to know what words would sound like. If you don't know language, you don't quite know the relation to the sounds. Hence, speech/audio recognition don't quite work. 

There can be many primary features to classify:
Reference: 
Automatic Language Identification: A Review/Tutorial  by Yeshwant K. Muthusamy , Etienne Barnard , Ronald A. Cole Signal Processing Magazine, IEEE Oct. 1994 Volume: 11,   Issue: 4 pp. 33 - 41 

Or related paper of same author.


Phonetics:
Here you should try to look for is that you should try to collect and identify sounds based on primary phonetics. For example - you can use something like International Phonetic Alphabet
Prosodic:
Languages like English, French, German might have very close phonetics. In such case, you can also use Prosodic information which is essentially duration of sound, emphasis or accentuations, pitch and pitch variations. 
Phonotactics:
Phonotactics refers to the rules that govern the combinations of the di erent
phones in a language. There is a wide variance in phonotactic rules across languages. For example, the phone cluster /sr/ is very common in the Dravidian language Tamil, whereas it is not a legal cluster in English.
Vocabulary:
If you are successful at this, you can probably make another classification of which phonetic alphabet orders are most likely in particular language. You probably don't have to recognize each word. But based on most likely phonetic symbol sequences, you can probably do language detection.