Snapchat Stories, which intercut the moments of many Snapchatters at events like football game or ceremonies, can be powerful portraits of a place and time. But they’re inherently disjointed, full of odd jump cuts, and they feel hours removed from real life. But what if Snap could seamlessly sync the footage of its 173 million users? What if it could coordinate everyone’s phone to work in simultaneous harmony, like the cameras on a movie set? Suddenly, these disjointed stories could be told as fluidly as a professionally directed TV broadcast. Now Snap can do just that, with a new feature called Crowd Surf that’s rolling out slowly at special events. It made its debut during a Lorde concert at the San Francisco music festival Outside Lands over the weekend, and though the footage is a bit shaky, the potential is obvious: Snap has successfully cut together many perspectives of Lorde’s performance into a single, seamless audio track. That’s no small technical feat. The track was built in 10-second bursts, scraped from phones in vastly different positions to nearby speakers. And it only took the service about a minute to cut the video together, meaning Crowd Surf can publish content in near-real time. Crowd Surf is based upon technology built within Snap. Using machine learning techniques, Snap takes a spectograph (or “sound print”) of audio playing in a location (like a concert). Snap parses many people’s clips to identify the baseline sound print of the event space. Then, Snap filters each person’s audio a little differently so that they seamlessly match up to a single aural aesthetic. While the technical explanation is mind-bending, the user experience feels totally natural. Of course Snap should be able broadcast a perfect live concert, shot from a dozen different smartphones–who cares how hard it was to build? The question is, what does Snap do with this technology? The company recently debuted a maps feature that allows you to explore events around the world. It’s easy to imagine Snap becoming a place where you can tap on a rodeo in Wyoming or a snowboard competition in Colorado, and then watch either just like you would a broadcast on ESPN. It could also solidify Snap as a hub for breaking news like the terrorist attack in Nice, France. Instead of a single live broadcast capturing the tragedy, hundreds of people might be on the ground, telling an algorithm-cut story exactly as it unfolded.
Snap is slowly rolling out new feature Crowd Surf, which uses machine learning to cut together several perspectives of a single event, like a concert for example, using footage from smartphones. The feature completed its first successful test run at a Lorde show, assembling footage from all over the venue into a single coherent video. The technology uses a spectograph, or sound print, to match footage from phones to the music itself. Expert suggest the technology could also be used for sporting events or news reporting.