Dan Archer founded Empathetic Media in 2015. He was a 2016 fellow at the Tow Center at Columbia University, a 2014 Reynolds Journalism Institute fellow at the University of Missouri, and a 2011 Knight Journalism fellow at Stanford University.
This September, he published an immersive project, including 360 video and videogrammetry, he’s been working on about restorative workshops between Colombia’s paramilitary and guerrillas for El Tiempo.
He sat down with Immersive Shooter to talk about his latest project, affordable videogrammetry options on an indie budget, and run-and-gun style 360 video production.
Tell me a bit more about the project in Colombia. How did you discover it and what made you think it would fit well as an immersive package?
Two and a half years ago, I was invited by a Colombian producer who’d been approached by an NGO that was using restorative justice to build bridges between former combatants.
I was really attracted to the story and the intimate personal stories of integration. I thought so many affordances of VR were perfectly suited for the story. It’s a combination of 360 videos and then a Vive experience.
Tell me more about the Vive portion of the project.
Many VR pieces are on rails, and while 360 hints at agency, it’s still a passive experience. That’s a real challenge to create an environment where the user has agency but isn’t just adrift. So, in this experience, you can move through the scene and different beacons or bubbles flash or pulse to show you where the information points are.
For example, there are these memory houses–casa de memoria–in these villages and towns that are memorials to those who’ve been murdered or disappeared. Family and friends leave photos and comments or objects, like the remains of a car crank shaft that decimated an entire village. They’re all objects you can pick up.
Part of the project is an experiment with how we can use photo and videogrammetry in the field. I’ve been using a Matterport scanner and an Occipital Structure Sensor 3D scanner to recreate environments.
The structure sensor captures people and objects. It connects to an iPad and takes a bunch of pictures and stitches them together using proprietary software. It’s an affordable AR solution. The real trick is marrying pieces of tech together.
I also use Matterport scanners aimed in multiple directions to record the room as a mesh we can experiment with and import into Unity, but that comes with its own problems. There can’t be any movement in the scene, so we need to flush all people out of the environment and that’s hard to explain. We need a full 45 seconds to do a full 360 rotation where no one will be in the space, so it’s more limiting in that respect, but you can build in a lot more interaction and allow you to move within the scene.
You’ve done a lot of AR with Empathetic Media. What have you learned about how AR can be utilized in newsrooms?
When it comes to AR in news, there’s a lot of finding the right fit. You have to think of the ROI for the audience. There needs to be a reason you didn’t use 2D video, just like with VR or MR.
For the Freddie Gray piece we did for the Washington Post, we tried to give the audience a look at what happened from a new angle. But, even then, there’s the issue of whether or not consumers will be involved enough to download a mobile app exclusively for that purpose.
Up until this point, AR’s been really challenging because there’s no one site or solution. Right now, there are all these apps exported out of Unreal or Unity, but people don’t want to download 50 AR apps for every newspaper, magazine or experience. That’s what Apple’s ARKit is sort of billing itself as, as a single solution.
The most critical part with AR is thinking about what that bonus is–what is the benefit of the user to wait for the app to download before they watch it?
Dan’s 360 video production tips
With the sheer number of 360 videos you had to put together for this project, what storytelling strategies did you learn?
Especially since the nature of the process dictates that they address the camera as though it were me. We tried to shoot things more natural, though, and leave it in the middle of the scene and have sort of a VR docent guide them through the scene, introduce where we are and how it relates to their story, and walk through those critical moments.
One of my key takeaways was the sheer amount of screen time given to the narrator, and how it builds this narrator trust and makes the story much more impactful to the user experience. How we respond to our narrator is really integral to the way we engage with the story. That’s why I put these protagonists front and center.
How did you guide your interview subjects as they were acting as a guide through the scene?
I’ll tell them it’s better to walk around and allude to particular objects at multiple focal points, but not to go within 6 feet of the monopod.
It was also pretty critical to show them examples of the other videos so they could get a sense for it immediately. The hardest thing was trying to capture highly traumatic things theyre saying and cram that down into a 2-minute window.
Some people are more suited to this type of interview than others. Some people find it more therapeutic that you aren’t there because, in a way, their behavior changes when they aren’t in front of someone. Maybe not in the first few minutes or takes, but then they loosen up and it becomes more of a soliloquy. Similarly, some people need more structure and you have to hop out from behind the tree to tell them to talk about X, Y or Z.
We also use a Sony remote mic, and it’s a night and day difference to be able to hear them while you’re hiding and get a sense of what they’re saying rather than flying blind. It also helps when things like a an ice cream salesman walks by with a megaphone, stray dogs bark, etc. That’s the hardest thing, when you’re trying to get things in one Individual instance of a shot; a take = each time the camera is started and stopped..
What’s in your gear bag?
What gear did you use for the 360 videos?
I used the Gear 360 camera because I thought it would be a lot faster and more efficient. It just wasn’t practical to move as fast as I needed with another camera, like a GoPro rig. There are some limiting logistical challenges, like driving six hours with very little daylight left and trying to produce something like on a soundstage in LA alongside a dirt road. Although the quality isn’t broadcast, the 4K across social media is good.
Even though I had a Samsung S7, I increasingly never used it to preview–there just wasn’t enough time. I started to get a feeling, though, of what would work and what things would look like. When we’d get anywhere, we wanted to be off to the races, set up as quickly as possible and not lose any moments.
Besides the Gear 360s, what else is in your kit?
Portability for me is key. I put everything in a backpack and hang my monopod off the bag. That includes two Samsung Gear 360 cameras, spare batteries, an H2N recorder and remove mics, the Matterport and Structure Scanner, a notebook and sketch book, and my Digiant 2-in-1 Pro telescoping monopod. That’s $90 and doesn’t break the bank. And once you remove the massive tray on top and the lever, it’s pretty versatile, flexible, folds up really small, and well-weighted. Sometimes I’ll stack rocks to balance it and weigh it down.
Then I also have my MSI Dominator VR-ready laptop, which I got for $2K and has 32 gigs of RAM, 1070 graphics card, and a 2 TB hard drive. It’s a beast, but I can run my Vive off of it. It’s essential.
Oh, and tons of backup hard drives and micro SD cards. One of the most important things, as tedious as it sounds, is to do your file consolidation immediately after each shoot and match things up.
What was your workflow like to produce the videos?
I dropped everything into Action Director and tweak if I need to. With the volume of material I have, the real focus was on turning it around. That was critical to the project. A lot of people haven’t had access to this technology, so I bring a headset with me and show the folks what it’s like and what the final project will be like.
An exhibition featuring 10 immersive videos and Dan’s Vive experience opened in September in the Centre for Memory, Peace and Reconciliation in Bogota. For the next month, Dan will be traveling throughout the country showcasing these immersive experiences.