What acoustic properties of sounds enable us to distinguish the owner of the voice?
Some possible perceivable differences might include:


differences in pitch frequency (bass vs. soprano voice, etc.) 
differences in overtone series (formants, head resonances, accent)
variations in modulation of pitch over time (vibrato, tremolo, accent)
differences in voiced to unvoiced ratios (raspiness?)
variations in typical durations of each vowel or consonant 
variations in cadence modulation (accent?) 
differences in choice of phonemes for each word or phrase (accent?)