This post originally appeared on Matt Celia’s blog on Medium and is being shared here with his permission so we can all learn from his production experiences. We highly recommend you follow Matt on Medium–he has some great content coming out in the near future!
Last week in Part 1, I dove into some of our process in designing what we are calling a “mesh narrative”. Pre-production is the foundation of every successful VR project, but sometimes when you have never done anything like this before, it’s a bit of a scramble in trying to figure out exactly what you need to create.
Production is my favorite part of creating 360 live action VR. There is such a great energy being surrounded by so many creative people, and although the days can be grueling, seeing so many people come together to execute a vision is one of the best feelings in the world.
Below I’m going to talk in depth about a few topics I feel are especially relevant to the 360/VR community. I’ll talk about using art direction to help create a world, how we worked with talented actors to achieve a believable performance, and shooting on the GoPro Odyssey thanks to our participation in Google’s Jump Start program. Film production in VR is much the same as it is with normal movies/TV shows/commercials. It takes a dedicated team of people to create it. A village, so to speak.
If you aren’t familiar with Speak of the Devil, watch it here.
Creating the world
In my last article, I briefly touched upon our scout and the 360 photos that we took of each location. Each location was given a scene number. These numbers became the bedrock of organization for the entire process and a great way to communicate with members of our creative team.
In creating 360 content, we’ve learned that art direction is a hugely important piece of building the world, perhaps even more so than 2D entertainment. After all, we’re communicating story not only with dialog, but with location and space. For Speak Of The Devil, we knew we were going to heavily lean on our art department to help make our forest look creepy and plant several Easter eggs for a dedicated audience member to find. Not to mention, we were trying to create a scary horror project that was shot in the middle of the day! I struggled to find references for horror films that took place in broad daylight and felt that a talented production designer would be able to help add to the creepy “lost in the woods” vibe I wanted.
Enter Kendra Bradanini. An accomplished Production Designer with an impressive reel and talented team consisting of Propmaster Lawrence Humphreys and Art Director Hailey Duvall. I talked with Kendra about the challenge of augmenting our creepy wood over 56 unique locations and some ideas we had such as a deadAn environment with little or no reflections or reverberations of the sound. animal carcass, skulls, runes, and rock formations. I shared with her the website detailing all 56 locations so she could get some ideas of where to hang items, etc. She responded back with a look book that explained her creative vision of the project. I loved it. She and her team were hired.
It was really important that every location had at least one defining feature we added, so that people in the narrative could have something to recognize that location by. This worked really well for certain locations, and not as well for others, mostly due to the low resolutionThe number of pixels in an image, typically presented as a ratio of the total pixels on x axis to the total pixels on th... More of the final playback experience.
One of the biggest thing we needed her help with was designing and constructing the altar that Lindsey uses to find and summon the demon with. It needed to feel natural, but also serve the needs of the story.
Because we had limited resources and a significant amount of forest to cover, I had designed our shooting schedule to stay localized to certain parts of the forest, so that we could be filming while Kendra and Hailey would be setting up the next locations. Lawrence would stay with camera team to make small adjustments as we were filming. This way we could also repurpose high cost items (such as our deer carcass) and custom built items such as our hanging effigy.
In hindsight, certain items worked great, and others were really challenging for the 3D stitch. We were in a forest, which is hard enough alone for the optical flow algorithms, but adding any sort of thin moving objects like some of our hanging stick effigies just didn’t translate well.
Our art team did a tremendous amount of work creating this world and, most importantly, putting it all back the way we found it. We were very careful to respect the earth, working with the forest service to leave as little impact as possible and protect the natural plants and wildlife we encountered.
The world becomes the stage
To bring our story to life I knew we would have to have solid performances from our actors. I’ve sat through my fair share of horrible performances in VR. I’m not sure why acting in VR is so hard. Maybe it is because we can’t cut a performance together and we’re used to having that option. If you haven’t worked in theater or had an extensive amount of time to rehearse (which is very hard on the shoestring budgets in VR) it becomes obvious very quickly that a three to five page scene that you have to nail the timing, performance, and technical considerations of in a single takeIndividual instance of a shot; a take = each time the camera is started and stopped. is challenging.
I had a meeting a few months back with a A-list commercial director whose work I admire. He watched my work and although he had a lot of kind words, he remarked that he felt the acting took him out of the story. I was a bit hurt, but upon discussing it further with him, I think his note was that he wanted a deeper authenticity to performances in VR.
Speak of the Devil has its roots in a campy horror narrative, but I was determined to cast a believable couple that audiences could connect to.
I had met Taylor Murphy-Sinclair a few years back and always wanted to work with her. Her natural personality seemed to fit Lindsey’s character. I still had three more roles to cast.
Longtime friend and collaborator Jackie Sollitto came aboard as a casting director on this project. She has an amazing eye for talent and suggested Michael Terry Grant for the role of Brian. He was an instant yes. Our initial phone conversation revealed that he loves classic horror films (as do I) and was interested in working in VR.
Through our makeup artist, KC Mussman, we found Mick Ignis. Mick is known for his creature work and KC suggested that we work with someone who was going to be comfortable sitting in a makeup chair for hours. Because the Windego was a suit, he could be double cast as both the Cultist and Windego creature. Poor Mick was in makeup for the Cultist for around five hours. This meant some creative scheduling (more on that later), but having an actor who knows how to embody this kind of creature work was stunning. It was my first time working with such extensive prosthetic makeup and I learned a lot about how long this stuff takes to put on, and takeIndividual instance of a shot; a take = each time the camera is started and stopped. off! Word to the wise, a makeup test is always recommended before you start production so that the artists can work with the actor’s skin and understand the particulars of how each piece will fit together.
The special effects makeup on our cast was crucial in keeping the experience feeling organic and helping our actors feel connected. I think our team did an amazing job.
While we’re on the point of a great makeup team, we also brought in a whole pressurized blood system so that we could get seriously bloody. Or Mussman and Nelson Cooper helped create some of the most fun we had on set. My only regret is that I wish the blood played better in VR. I should have blocked it closer because the resolutionThe number of pixels in an image, typically presented as a ratio of the total pixels on x axis to the total pixels on th... More and lighting still isn’t quite there yet. When you think about it, most movie scenes cut to a close up. We don’t have that luxury in VR, so I should have blocked the action closer to the camera.
Working with actors in VR is amazing. It’s very creative and freeing. The actors love being able to improvise and try new ideas and I encourage them to do so. Since we aren’t shooting coverageMultiple shots from multiple angles to capture the events in a scene (i.e. master shot, medium shots, close-ups, inserts... More, my approach has always been to treat performances more like a stage play, with the camera as our audience, and encourage a lot of eye contact with the viewer to draw us in.
We would run the scene a handful of times and talk about the motivations and discuss blocking particulars that would work best for camera. Rehearsals are really important because the actors have to nail the entire scene without cutting. For a five page scene, this can be tough. Taylor, Mike and Mick were great with suggestions and they really brought a lot to the project.
I think it is really important for cinematic VR to focus on achieving a higher of level acting in VR and that’s only going to happen as we figure out what techniques work well and hopefully convince talented people to collaborate. As we push the envelope with longer, more ambitious storytelling project, we can’t forget that audiences are going to connect with the actors over cool technology every time.
Using the GoPro Odyssey
Using the GoPro Odyssey was both a blessing and a bit of a curse. It was amazing because it by far produces my favorite 3D 360 footage at an extremely high quality and comes with Google’s automated stitching. It’s a bit of a curse because it’s quite heavy to lug around.
I had constructed our build to include a few extra features. First, I mounted two additional GoPro cameras on the bottom of the rig to catch any feet that would go into the dreaded “blank” area. I wanted our actors to be able to get CLOSE without chopping off their limbs. I also mounted a GoPro to the top so that I could capture above as well. To power these GoPro’s all day without needing to change batteries, I used a USB power brick that I mounted.
We were using LAV microphones for most of our audio, but I wanted to run our Hear360 8Ball and Zoom F8 recorder to capture as much forest ambience as possible. I had mounted a monopod to the center column of the tripod so that I could place the 8Ball deadAn environment with little or no reflections or reverberations of the sound. center and mount our Zoom F8 onto the bottom. All in, it was a pretty good setupA camera position for a given scene. You might shoot more than one shot from a single set-up (wide shot and close-up). that although it was on the heavy side, we were able to pick up and move fairly quickly without needing to reset up in each new location. Our production audio mixer, Helena McGill of Noctvrnal, monitored the LAV levelsThe adjustable sensitivity settings of microphones. Levels are set (and changed as necessary) to best capture the vocals... More in her headphones and used an iPhone paired via Bluetooth to the F8 to monitor levelsThe adjustable sensitivity settings of microphones. Levels are set (and changed as necessary) to best capture the vocals... More from afar and input shot information. It was a pretty good setupA camera position for a given scene. You might shoot more than one shot from a single set-up (wide shot and close-up).!
Honestly, the worst part was carrying the battery pack around! If only we had been able to use the new YiHalo, that would have made things a TON easier.
Google’s stitching has a great feature that allows you to send two shots up to the cloud and tell it that you will be using them together as a compositeThe post-production process of combining two or more images. Could be as simple as a title superimposed over an image, o... More. This meant that it was fairly easy for us to stay in frame and watch the action, knowing that we could shoot a “clean plateA part of a composite, usually intended as a background, though here it refers to a duplicate version of the main shot w... More” where we would move to the other side of the camera. Sending both shots to the Google cloud, they would come back stitched with the same lens geometry and easily able to be stacked in premiere and garbage matted. This worked really well despite the fact that we were in the forest outdoors. I’ll cover the specifics of this in the next article I write in this series talking about our post production process.
The last VFX plateA part of a composite, usually intended as a background, though here it refers to a duplicate version of the main shot w... More we always grabbed was a nadirThe bottom of the sphere. shot with another GoPro camera. This is what we mostly used to mask out the tripod.
All in, it was a lot of shots to remember to takeIndividual instance of a shot; a take = each time the camera is started and stopped. and keep track of. Our assistant camera Jake BlockPosition and choreograph actor movements in a scene. was again a hero by keeping really detailed notes on an Android tablet.
With so many scenes and variations, staying organized was a must. 56 sounds like a lot of scenes. It felt like many more. It’s so easy to miss one. Many of the 56 locations also had two or three different variations to correspond with the mesh narrative.
Scheduling all these shots was a massive undertaking. Not only did I have to work around actor availability, makeup times, and art set dressing time, but I also had to account for the changing light of the sun. By no means did we succeed in maintaining a consistent time of day. We would have had to shoot over the course of 3–4 weeks to make that happen. But we tried to be smart in the order in which we shot.
For example, we’d tend to shoot a lot of the empty scenes and 1st act stuff towards the beginning of the day. This was good for two reasons. First, it allowed our actors time to rehearse and get into makeup. Second, our story begins with us waking up in the morning, so the morning light tends to be consistent with scenes you would hit early in the story. Later scenes, after the demon has been summoned and you are on the verge of impending death, looked creepier with the long afternoon shadows.
That’s a wrap
We were in Big Bear Lake for a week at the end of September. Monday was our scout prep day and we filmed Tuesday, Wednesday, and Thursday. Friday was spent capturing empty locations and pickups for anything we had missed.
Cast and crew all stayed in a huge (very oddly laid out) cabin we had rented for the week. My wife, Katie, home cooked all our food and snacks. My barely one year old baby also joined. It was a fantastic guerrilla indie film vibe. We wanted our first interactive live action project to come to life, and so the blood, sweat and tears were worth it.
Pick up the story next week in PART 3: Post Production Workflow Madness, detailing the production of this project and how exciting it is to see a vision come to life.