I stopped by Moth + Flame’s office last weekend to see their short film, Remember: Remember first hand, powered by AMD computers and the HTC Vive. Relatively short, the Storyscapes work to great effect, even when rendered in a video game engine. The short tells a story that feels like an atmospheric intertwining with War of the Worlds and Eternal Sunshine of the Spotless Mind, as you embody the role of a character who has been captured by a race of aliens, and your memories become lost or distorted as you try to recall the details of a lost love.
The short films Remember: Remember and 36 Questions are the most recent projects of his after a collaboration with Discovery on building a social VR experience at this year’s Consumer Electronics Show.
Kevin: The Vive is awesome. This was the first one that we did that was like real storytelling for the Vive. For us, we really see Virtual Reality as the future of storytelling.
Evan: Yeah, exactly. And I was speaking to your team about how completely different it is because you have control of the entire rendering in a game engine, and you have control of all the scale, the depth and the movement that you could want out of it, but you also have to be more deliberate. When you edit film you can lead people’s eyes with movement, color, etc, but in this particular case the viewer is controlling the camera. How do you guys go about figuring out how to lead the audience’s attention in that way?
Advertisement
Evan: — Right, like “where am I? What’s the space?”
Kevin: Exactly, and so when the ship flies over you, you’re body is feeling like it wants to come back. When you have control over the story that you’re telling, everybody looks in the same place. We do heat mapping, and we know what they’re looking at. The red spots are always the same. We design the things very intentionally. There’s these things that was want you to be looking at the whole time, because that’s part of the story we’re telling. Really, for us, in terms of the storytelling, and what makes VR storytelling different than your traditional format is that, when you’re watching a movie, you’re watching two characters on a screen. When you’re in VR, you are one of those characters, and so the types of emotions that you can feel, when it’s being brought on you, is a whole other lexicon of emotions than traditional, cinematic emotions. And because it’s about you, the feelings, and the memories, and the energy that you get while experiencing those emotions, are at another level of severity. So that’s where so much of our first person storytelling is developing, and it has so many different things we can play around with. If you think about Remember: Remember, one of the most interesting scenes is in the closet.
Evan: Yes, and I think it also works in that way because there is something there that is being teased, and you have this dimensional space that’s working against you by the way it’s designed. I think its so memorable because, by way of performing the part of a character, you work around the space of it. I know I found myself looking between the coat rack, or peeking underneath, or the way the sound throws, and I’ll throw my head…
Advertisement
Kevin: The thing that’s really fascinating about that is, in film, it’s all about keeping scares off screen. It’s the things you can’t see that will affect you the most.
Evan: The most terrifying thing being what your brain comes up with instead of what could be shown.
Kevin: So, then, you have this challenge where the whole world is a screen. How do you make things that are off screen? So that’s where we came up with this idea of, what if we put the character in a closet, and all you can see is clothes, and you can barely see what’s on the other side? Then you create this thing that you can’t see that let’s your imagination kind of run with it.
Advertisement
Evan: With the aliens in this, you can clearly see your H.R. Geiger inspiration, but from that first Alien movie, it’s still the most effective because, even when you do get a clear shot of it, it is a guy in a dorky rubber suit, but it’s shrouded in darkness or not on screen at all. I think that still works, and I think this is a new, unique way to experiment with that concept. Have you guys thought about different ways to accomplish new scares, or different kinds of emotion?
Kevin: Yeah, we’re constantly playing around with different versions of that.
Evan: I know you guys are still building the other short, 36 Questions, but what kind of experience are you trying to pull with that one?
Kevin: So, in terms of thinking about the evolution of first person storytelling, the next thing we’re incorporating into it is human conversation. So it’s: you’re asking questions, somebody is answering questions. They ask you questions, you’re answering questions. So, there, it’s a back and forth of using your voice as an input device.
Evan: And so is it more inferring the character’s response based on words used in your answer, or is it kind of AI based?
Kevin: Yeah, so we’re using [IBM’s] Watson, and so that gives us everything from the voice recognition to sentiment analysis and facial recognition to match video files.
Evan: Are there any experiences outside of building stories in VR that you or anyone on this team at Moth + Flame have been able to pull in and help you rethink how you create these stories?
Kevin: I think that’s kind of what film is, in terms of a disciplinary art form. I think you’re looking at one of the most interesting things that people are doing in the world of design. What are the most interesting writers writing about? Film is taking all of that and putting it together with music, sound, storytelling–
Evan: sometimes in the most emotionally leading, manufactured way possible–
Kevin: Right, the synthesis of it all. So in terms of what’s in virtual reality, it’s like you’ve got some more tools in the tool kit. So you’ve got the AI stuff. And I think a lot of what’s happening with Alexa is really interesting, and that’s probably a contributing factor, a trigger for this.
Evan: The Amazon Alexa? I don’t know if you’ve had the chance to play with the Google Home…
Kevin: I haven’t played around with that one
Evan: That one is interesting, because you can ask it a question, and instead of speaking to it like a computer, or like Siri, it will actually remember the context of the most recent question that you asked. I find that to be some of the most interesting developments in the smart home technology. It’s also much more sassy.
Kevin: I don’t know, Alexa’s pretty sassy!
Evan: In that regard, do you think once it becomes more cost effective or Oculus or Vive become stronger household names, do you think Virtual Reality will become one of the stronger platforms for films or games? Do you think we’ve reached a peak of 2D film?
Kevin: I mean, I think movies will always be around, in the same way that novels have. Although it’s been a long time since F Scott Fitzgerald was the biggest entertainment star on Earth.
Evan: And now we’ve got The Hunger Games.
Kevin: So I don’t think the art form of the two hour movie isn’t going away. Or the golden age of television, your ten hours of Game of Thrones, that’s not going away anytime soon. But I think there’s another thing getting figured out, and I don’t know what it’s going to be, in terms of what it is with virtual reality. But it’s something that we’re gonna spend a lot of time in. I mean, for ten years with the internet, people didn’t predict that it would be something like Facebook that they’d spend the majority of their time with, but it’ll be something like that.
Evan: And how do you guys go about sound recording the way you do it right now? It it recorded with an omni-directional mic, do you place it in digitally?
Kevin: Yeah, much of the stuff that we’re doing is with a built in game engine, so then it’s recording each stem individually, and then placing it in a 360 environment. So, the interesting thing for Remember: Remember, we worked with AMD, with their Phenom program, which was used for a lot of LiquidVR drivers, and, what it is, is a spacial, physics based sound render. So, in the way that light is rendered based on how many different places it’s bouncing off of, sound is being rendered in the exact same way. So each different part of the room has different sound properties. The carpet will handle sound one way, differently than how glass would. So, in Remember: Remember, in each one of those scenes, each object has different sound properties.
Evan: It’s so small, but there’s a brief scene where you’re standing on a balcony, over an apartment, and you’re further back from the girl, closer to the wall, but when you approach the railing to be standing next to her, and it does feel like she’s talking into your ear instead of echoing off the walls. It changes so dynamically. I just think that’s so cool, that kind of small detail, makes it feel more alive.
As the Tribeca Film Festival comes to a close this weekend, I encourage all people interested in the next form of media and industry leads to check out this year’s virtual arcade for a glimpse into what the future will hold for the empathetic power in the next level of entertainment.
Advertisement