Storyboarding Strategies for VR

This article is part of an extensive series of articles to walk creators through the process of making a 360-degree video, from start to finish.

One of the things I love most about writing for Immersive Shooter is how much I learn from our contributors and our readers that I can use in my own work. Chief among those lessons–at least lately–is storyboarding strategies.

As a journalistA description for the word journalist that goes on and on a bit that goes on and on a bit that goes on and on a bit., too much pre-production tends to go against my nature. We tend to feel a bit more comfortable with the concept of dropping in and figuring things out as the story develops. It’s also often a matter of journalistic ethics; strict adherence to a storyboard results in an inauthentic story.

But, for the piece I’m working on right now, I knew I wouldn’t have time to figure everything out on the ground. We were limited to two days on the ground and our sources had very tight schedules of their own to keep. I’d have to arrive more prepared than usual.

So, I got started with a simple VR storyboard template from the BBC. It has one large rectangle (potentially an equirectangularStretching a spherical image into a flat, rectangular format. (i.e. the way a world map represents the spherical Earth).... More representation of the scene?) with a smaller rectangle within it. I attempted to draw the full scene with the focal point at which I wanted viewers to see first in the smaller rectangle.

Option 1

Option 2

But, I quickly realized sketching my shots was hard enough without trying to visualize how they might look in equirectangularStretching a spherical image into a flat, rectangular format. (i.e. the way a world map represents the spherical Earth).... More format.

One tip I’d heard was to use a cube map instead. So, I drew six little squares I could wrap into cube that might help me visualize a bit better. Although this worked out better than the equirectangularStretching a spherical image into a flat, rectangular format. (i.e. the way a world map represents the spherical Earth).... More drawings, it was still a bit abstract for my taste.

To storyboard with a cube map, one side of the cube would represent what is right, left, front, back, above and below the viewer.

Cube map projection (Source: Wikimedia Commons)

What I landed on was one circle representing the full spherical space–think, a hand-drawn “tiny planetVisual effect is a stereographic projection that presents a 360 world like a small planet.” of the scene–with three rectangles: one to represent the focal point viewers will see first, and the other two showing alternative “frames” in the scene.

The circle provides a pace to sketch the shot in “little planet” view. The three rectangles offer three different frames you anticipate for viewers.

360 photo in little planet view

This concept comes from this presentation by VR filmmaker Jessica Brillhart. In it, she explains how a VR experience is sort of a gradient of story possibilities. For her, framing in VR is all about “potential frames”–all of the possible directions a viewer might look.

The way I took this analogy was that some viewers will follow the story almost exactly as you envision it. Others will get distracted and some will just plain rebel against the narrative you’re driving. How can you ensure even the most watered-down versions of the experience still offer something to the viewer?

Hence, the three frames (and four might even make more sense, since most VR headsets have a 90-degree field of viewThe angle of space viewable from a given lens position. (90 + 90 + 90 + 90 = 360 degrees = four frames).

By thinking about what might be behind the viewer and what other “frames” they might choose to look at, I could try to anticipate the weakest version of my story. If some viewer really isn’t picking up on my cuesA trigger for a action or line of dialogue. and looks in the wrong direction the entire time, what will they get out of the experience?

The “Emotional Blueprint” Approach to Storyboarding for VR

“I’ll put on blank goggles and mimic the experience and come out with an emotional blueprint–whether it’s narrative or music–and I’ll know how I want someone to feel throughout the piece,” said VR filmmaker Elia Petridis. “For example, I want someone to feel anticipation from 0 to 23 seconds in, and trust from 23 to 40, and so on. As you plan out everything else, you can always go back to that one word.”

It was an immensely helpful exercise. I also included space for notes like location, description of action in the scene, anticipated audio, expected duration, etc.

Although I felt much better off with this strategy than a single frame, I still felt like it wasn’t quite enough. I also wanted to encourage the viewer to look around. Despite planning for the worst-case scenario, I wanted to make the best-case scenario more interesting.

For this, I looked to a concept from RVLVR Founder Nick Bicanic. A while ago, Nick was experimenting with quick cuts in VR. He used the motion of people in the scene to direct the gaze of viewers and then cut to the next shot where they would still be framed within the viewer’s gaze.

Everyone said this wasn’t possible, but what he showed was that it was possible–it just takes more planning ahead or, for journalists, anticipation of the scene.

In another example from Brillhart’s talk, she decided to start one shot from a story about ice hockey looking at a quizzical-looking fan rather than the ice rink. As the fan turns back toward the rink, the viewer naturally looks toward the rink also. If she had started with the obvious focal point (the rink), not only would that discourage looking around the scene, but the viewer would also miss the fan, which Brillhart suggests might be the best part of the entire piece!

Screencapture from Brillhart’s presentation

I wanted to employ these concepts in my piece, as ambitious as they were (particularly for a journalistic project). For example, one of our sources stands alongside the road, so I planned to start the shot looking at the road just as a large vehicle passes. As the viewer follows the vehicle, they’re presented with one of our story’s central figures.

Or, in another shot, starting the shot looking into the parking lot as the source walks up to the central location. As the viewer follows them, they’re introduced to the central location. Or, in a third example, we start a shot looking at a handful of people listening to another central character tell a story. With everyone looking in the same direction, the viewer is encouraged to also look in that direction.

These particular tips to direct the viewer’s attention within a scene and encourage motion come from VR filmmaker Sarah Hill.

“The viewer is in control, not the storyteller. Unlike in the ﬂat world, they’re viewing it in their own choice of frame,” Sarah said. “We need to use unique storytelling techniques to direct the audience’s attention.” Examples being the character’s own path of motion driving attention, characters’ gaze directing the viewer’s gaze, using lighting or shadows, spatial audio or even arrows.

Although we ended up capturing a lot of these things naturally, I feel like I could have placed the camera a bit better if I’d anticipated how I might use them in post to guide the viewer.

Enter another strategy from Jessica Brillhart. One of her story planning strategies is to give each shot a color and put one beneath the other beneath the other. For example, in the first shot (blue) the viewer is looking in the direction denoted with the black dot and ends looking in the direction of the white dot, which is exactly where the focal point for the next shot (red) begins.

Each colored circle represents a shot. The black dot represents the direction the viewer looks at the start of the shot, and the white dot represents where we expect them to look by the end of that shot. The starting point (black dot) of the following shot matches the ending point (white dot) of the previous shot.

Other methods I’ve seen include pre-visualizing using a small, self-stitching 360 camera like the Gear 360 and a smartphone app like VeeR Editor, V360 or Collect to quickly edit shots together. Others creators use 3D animation–one even recreated scenes in Minecraft!

Although I doubt I’ll ever be that prepared for a shoot, I knew I needed to do better than draw a box in a box. After all, if you’re shooting a sphere, it might be time to think outside the box 😉