One of the first pieces of virtual reality to really, truly blow me away was a 360-degree video called Strangers with Patrick Watson. A short performance starring Canadian musician Patrick Watson, this was the first time I'd heard someone take full advantage of VR sound. When Patrick Watson played piano, you felt like you were in the room. When he lit a cigarette and dropped the matchbook, you could hear exactly where it went.

Strangers was an early project from Paul Raphaël (at left in the photo above) and Félix Lajeunesse, collectively known as the Montreal-based VR film studio Felix & Paul. The duo recently partnered with Oculus to release a 12-minute piece about LeBron James, and they're appearing at Sundance to premiere Nomads: Maasai and Nomads: Sea Gypsies, a new pair of documentary films. Late last week, we met up to talk about the language of virtual reality filmmaking, the challenges of interactivity, and what it means to get inside the human mind.

Interview has been condensed and lightly edited for clarity.

Adi Robertson: Since you've been in the VR world so long, I'm wondering how much you think a VR film language has developed so far.

Félix Lajeunesse: I think I remember initial meetings we had with Oculus and people in the space, where we were talking about this idea of presence. And they were like, "Yes, yes, we understand what that is... I think!" This word wasn't necessarily something that was part of the language of VR.

But then it started to be, and for a good reason. Because I think people started to really acknowledge that this is where the inherent power of this medium lies. I think a lot of the work that we've been trying to do is to try to harmonize storytelling and presence together. And so for us it's an ongoing research. It's not as if in the last two years we've figured it out, but it's definitely what we've tried to explore, discover, nurture. I think it's sort of the central aspect of our duty as VR storytellers.

Paul Raphaël: What I think is kind of rewarding is, we've been bouncing back and forth between more documentary observational works and fiction where we're creating something from scratch. And with the more documentary stuff it's about unearthing the natural qualities of the medium, and having your ear to the ground. And on the other side you go, how do you maintain that and build something that's artificial? Because if VR's so good at showing reality, it's also good at showing what's not reality. So you need to figure out how to trick the mind in a way to accept things that are clearly not true.

"You need to figure out how to trick the mind in a way to accept things that are clearly not true."

I'm curious how the advance of actual headset technology has impacted the work you feel that you're able to make and want to do.

Paul: The thing that's been the biggest change for us, since we haven't really delved too deeply into positional tracking and all that stuff, has been the quality of the headsets — the quality of the screens, the quality of the optics, the lightness, the cost going down, all these things are really contributing to a more lifelike and more accessible experience. As these headsets integrate all these new positional tracking technologies, we are more and more thinking about what we would do with that, and that's a very fun but very challenging thing.

Félix: While we don't have at this point the conviction that it will be a fundamental change, we think that it will be an evolution in regards to all the things we've explored in the last few years. Because it's physical presence — all of those pieces here that are CG-based, even if you're immersed into a world that isn't real, you still have a sense of your physical self inside a piece, your capacity to react. And that part we don't necessarily have in the work that we do.

And once we have it, of course it's going to impact I think certain aspects in how you relate to the environment, how you relate to characters — because we think of VR in such a relational perspective. A piece like Nomads: Maasai, it's really about making you feel like a part of that community, making you accepted by that community. We want them to integrate the camera as if it was one of them, so we really believe in that relational perspective. We think of the camera in a very anthropomorphic way. In a sense, the more that becomes real in terms of physical presence, through positional tracking, the more I think it will make us evolve in our thinking.

"We think of the camera in a very anthropomorphic way."

Paul: Even if you just look at resolution, you know? When we started three years ago with the first [Oculus Rift] DK1 and the 720p [Rift developer prototype] headset, if someone was any farther than four feet from the camera, you couldn't tell their facial expressions. So our first experiences were all very, very close. We still like to exploit that space, because it's I think where you can most easily connect with a character in the medium so far. But as the resolution increases, our stage also opens up. You can see more, you can see people from farther, you can perceive depth at farther distances, and that changes the way we tell stories.

Is there a place for actual viewer interactivity in your work?

Paul: I'm sure there is. We haven't explored it very much yet. I think there's a balance. I think there's a threshold where the virtual reality experience is satisfying, and it's the sum of a lot of parameters. And one of those is visual fidelity — like, how real does it look. The second you go into more interactive, fully positionally tracked experiences, you have to sacrifice visual fidelity. It's a trade-off, you know?

I think that some people are able to, despite that trade-off, make very compelling experiences. But right now there's almost a gap between the two. It's like there's one on this side and one on the other. But that gap is going to close. And as it closes, I think we're going to be more naturally driven to go there. For example, as light field technology advances, as 3D scanning reaches a level of quality that starts to resemble what we're doing with live action, we're going to be able to bring all the stuff we've learned into more positionally tracked content.

"Being a person is not just being in a space. It's also dreaming, it's also imagining things."

Félix: I think the point is, we're still very much attached to the power of creating an experience that looks real, because I think that really unleashes a special kind of emotional reaction in the viewer. Because from the day you're born, you live in physical reality. You're used to physical reality and the presence of people. And I think our bodies react, and our minds react, so strongly to reality. It's through real life experience you will have your most profound, I would say most powerful emotional experiences. I think there's something about reality that is inherently powerful, and that we can sort of bring into this art form, and we're very fascinated by that.

But I think the world of animation and abstraction also opens up all the possibilities of the human mind, the way your imagination works. And that's also very much fascinating. Not something that we've really explored yet, but looking forward, an inevitable aspect of our evolution is to dive into that space. We talked about the anthropomorphic camera, the idea that you're a person. Well, being a person is not just being in a space. It's also dreaming, it's also imagining things. It's also abstraction. And I think that all of that is part of the human experience. And this is part of what we want to explore through VR.

Paul: And one of the big challenges with adding interactivity will always be the artificial intelligence required for an interactive world to truly react in a way that is beyond it being a game. When you're playing a game, you know what the rules are — whether it's chess or a video game or a sport, you know there are rules, and you play by those rules. There's a level of granularity to reality that for virtual reality to really give you freedom, that level of freedom... I don't know when we're going to reach that.

And the minute you put limitations — "Oh, okay, wait, I know I can do this but I can't do that" — there's a gamification that makes it a bigger challenge to tell a story. And I think as much as we're trying to tell a story with this medium from the live action perspective, there's very interesting things to figure out at that level. How do you tell a story when you have interaction but you don't have complete reality-like interaction?

Can you see a point in the short term where you'd consider virtual reality a mature medium?

"It's going to evolve faster than cinema. It's going to evolve faster than theater."

Félix: In 15 years, maybe? There's so much focus and energy now in this medium and momentum that I would say it's going to go faster than the other art forms before it. It's going to evolve faster than cinema. It's going to evolve faster than maybe theater. I think it's going to be faster, just because everything is faster now, and because you see this kind of crazy exponential growth.

It's going to be in the schools, it's going to be in the hospitals, it's going to be in your house, it's going to be completely integrated into culture, I believe. And so therefore that's maybe when we could think of a certain maturity. It's just going to be part of reality.

Paul: In terms of the parameter-based version of that answer, I'd say the moment it stops being compared to other mediums is probably when it will have reached maturity. Because right now it's really being compared to video games, it’s being compared to film. And I can imagine maybe even less than 15 years from now — I don't know if it will have reached full maturity, but where it's no longer talked about relationally to other mediums.

And you don’t get "Why did you decide to make this in VR?"

Paul: Yes, exactly. It'll be self-evident.

Félix: Fourteen years.

Paul: I convinced him to go down a digit.