Google works out a fascinating way for AI to isolate voices in a crowd
Google works out a fascinating, slightly scary way for AI to isolate voices in a crowd

          
          
Google then trained the tech to split that mixed audio by reading the "face thumbnails" of people speaking in each video frame and a spectrogram of that video's soundtrack. The system is able to sort out which audio source belongs to which face at a given time and create separate speech tracks for each speaker. Whew.

    
    
      Looking to Listen: Video conferencing

          
  


    
    
      Looking to Listen: Noisy cafeteria

          
  

Google singled out closed-captioning systems as one area where this system could be a boon, but the company says it envisions "a wide range of applications for this technology" and that it is "currently exploring opportunities for incorporating it into various Google products." Hangouts and YouTube seem like two easy places to start. It's not hard to see how the tech could work when applied to a pair of smart glasses, à la Google Glass, and voice-amplifying earbuds, either.
Aiding smart speakers like the Google Home in their ability to recognize individual voices seems like another use case, but because this model is focused on video, it would likely work better with a speaker with a display, like Amazon's Echo Show. Earlier this year, Google opened up the Google Assistant to "smart display" devices like the Echo Show, but the company hasn't released one itself.
In any case, the privacy ramifications of this kind of tech seem just as obvious as the potential use cases. Google's voice isolation is far from bulletproof in the examples above, but with some more fine-tuning, it could make for a powerful eavesdropping and surveillance tool in the wrong hands.
That's a lot of speculation for now, though. Here's hoping this research at least lessens the need to shout at Google Home in the future.


          
                  