UPDATE: On Feb. 5, 2018, Mocha VR added stereo support for all users. Read more about the update here.
At IBC in Amsterdam this past September, Boris FX announced some major updates to Mocha (stereo support!), as well as new VR-related tools for its Continuum plugin. Matt Celia from Light Sail VR reviews Mocha’s latest tools, his Mocha workflow, and why it’s become a “game changer” for him.
No one ever said VR was easy. But with specialized tools like Mocha VR, it’s getting easier with every update.
I started using Mocha VR because of the incredible power of their motion tracker, which makes complex tasks like stabilization, roto, and inserts a lot easier. And in three weeks, Mocha will officially be adding support for stereoscopic workflows, which will make post VFX work as easy as working with monoscopic footage. This is huge.
Before Mocha, paint work required either some reprojections that had a steep learning curve or we spend hours attempting several iterations in Nuke trying to get our stereo convergence just right, and when it came to motion shots? Quite frankly, we just didn’t do them. It was too time consuming to remove the rover.
When we first picked up Mocha in March of 2017, it was perfect timing. We were about to do a project for Princess Cruises where they wanted the viewer to feel like they were on the boat, walking along the deck. We were also about to do a project for the National Park Service about the Channel Islands, which meant we’d be getting plenty of shots on a boat.
Removing the rover and stabilizing bumpy boat shots would have taken quite a bit longer with our previous workflow. But, having tried Mocha for the first time on these projects, the tool was revolutionary to me. Two clicks, and I was done. It was so fast that I even stabilized the boat shots for the rough cut.
As cliche as it may sound, Mocha really was a game changer for us. And, with its new support for stereo workflows, I think it will continue to be an extremely valuable tool in my arsenal.
Removing the tripod, rover or drone with Mocha VR
Mocha VR can be used to track and remove tripods, dollies, drones and more.
One of the most common tasks us immersive filmmakers do is paint out the tripod. Usually I take my footage, create a cube map in Skybox, and then use Photoshop to do some content-aware fill, and that works great if you have a still image. But if you want to replace it with motion or you don’t want to convert from different projections, Mocha is way better.
To remove the tripod, you can use the lens: undistort module to translate your equirectangular into a rectilinear projection where you can aim your camera down and get an exact view of what your audience sees. From here, it’s easy to layer in another shot, say from a GoPro, apply a bit of mesh warping and color correction, and you are done.
Previously, removing the tripod would require numerous steps and switching between programs.
Mocha turns a seven-step process (creating a comp, throwing on a converter, exporting to Photoshop, etc.) into as little as a three-step process. No round-tripping or projection conversion needed.
Before Mocha VR, I had no idea how to replace a rover in a moving shot. By some sort of magic trickery, Mocha is able to scan in the background and replace a tracked object. Just track your background and track what you want to remove. Mocha VR fills in the pixels and illumination modeling handles any light changes.
A fair warning, however: it is not 100 percent perfect and does require quite a bit of computer power. We prefer to run Mocha on our most powerful machine, a PC with two GTX 1080ti graphics cards, an Intel i7 processor and six cores.
We also have a central shared storage server from Luma Forge, which is probably the best investment we’ve ever made, because it allows us to switch to a new machine while we wait for things to render on one of our computers. Hard drives are like the underwear of the post production world. Having good performing underwear makes your whole life more comfortable!
To optimize Mocha VR and speed up rendering time, I suggest checking out the amazing tutorials and tips from the folks at Mocha VR before jumping in.
For example, they recommend not looking ahead or behind too many frames. You don’t need to look at 1000 frames if you can get the job done looking at 100. They also talk about generating a series of clean plates throughout your moving shot so Mocha can use those to do the removal, which is 100 times faster!
Like I said, you should definitely check them out.
Stabilizing your shots with Mocha VR
I also tend to use Mocha VR’s reorient module to stabilize moving shots, like the boat shot in the video above. I didn’t want to use a large gimbal that I would have to paint out, so we just put the Z Cam S1 on a monopod and knew we could deal with it in post.
Mocha makes it really easy stabilize if you can track something relatively stable on the horizon or elsewhere in your scene. But, this doesn’t work as well for shots where you can’t get a consistent track, or can’t make out the horizon very well. For example, we did some underwater shots in a kelp forest for the Channel Island piece, and with the cloudy water, nearby objects moving around, and no clear view of the horizon, there was nothing to track.
Mocha VR excels at stabilizing footage if there are any stable objects in the distance.
Mary from Mocha even came to our office to help us get the shot to an OK place, but at the end of the day, we ended up doing one pass with Mocha and another through Mettle to smooth out the bumps and transitions.
Stabilization is being improved in the latest 5.6 beta, which allows you to track two independent objects and then derive stabilization from that. Again, the toolset is always changing and continues to improve!
In general, people look for a single pass solution–and for the boat shots, Mocha was–but that isn’t always the smartest way to go about things. Sometimes, you need more than one tool. If you don’t fix it all at once, do it again on your next pass or finesse it with another tool, and eventually you’ll end up with what you want.
And, even though I love Mocha, I still use other tools, too. Sometimes, Mettle is better for stabilizing certain shots where you can’t see the horizon.
I’ll also still use my Photoshop clone stamp tool to paint the floor sometimes if I have a nondescript floor (gravel, dirt, grass, etc.), because even though it’s more steps, depending on the situation, it might be faster to do it that way than to wait for Mocha to render.
The tool I use depends on the shot. There’s no one tool that’s perfect for every situation, just like there’s no one camera that’s perfect for every shoot.
Text tracking for immersive titles
With Mocha VR, titles can attach to objects and move with the scene for more immersion.
Another great feature of having Mocha VR’s tracker available in 360 is the ability to attach and insert text that is locked to a particular feature on the screen.
For me, VR is all about creating a sense of immersion, reality and presence. If you’re just dropping text into the space and it doesn’t move with the world, it feels very clearly like a title card to the audience. If it’s attached to something in the real world–as if it could really be there–it feels way more organic and present in that space. Honestly, I think tracked text is just a better creative choice for the medium.
For example, with the rocks in the video above, I can track them and apply some text that sits perfectly.
I know we are all familiar with the Project 2D effect, but keyframing and animating that by hand would take way longer to do every single frame and, even still, might never be accurate.
To get it to really stick and be super accurate, you have to use a tracking software like Mocha, which tracks position, scale, rotation, skew, perspective. It’s tracking everything, every which way.
The only thing I’d change about Mocha when it comes to text tracking is I’d like to maintain the aspect ratio of assets that I insert. But, until then, you can use the surface tool in the software to stretch and manipulate your insert until it looks correct.
Mocha VR adds stereo support
In its 5.6 update, Mocha VR added support for stereoscopic workflows.
Although I love the stabilization, roto and insert tools from Mocha, the latest and greatest addition is its support for stereo workflows coming in Mocha 5.6.
We’re shooting a lot more stereo projects now with cameras like the GoPro Odyssey and Kandao Obsidian, so having tools to make tasks like roto and insertions for stereo footage easier is great.
If you have never worked in stereo before, figuring out convergence depth and 3D space is a huge headache and I was excited to see how Mocha has implemented their tracker to automatically track both eyes and place inserts in the correct 3D space. It’s going to be a huge timesaver, and it finally gives us the ability to work with 3D footage better in After Effects.
With the latest update, what you apply for one eye is automatically calculated and applied to the other.
Before this update, any painting in stereo would have to happen in Nuke. You’d need to use a disparity generator to calculate the disparity between left and right, and paint on the left and then the right. In Mocha, you can do your paint work on one side and it will automatically be applied to the other. And, Mocha is significantly cheaper than Nuke (a new license for Mocha VR for Adobe is around $700, compared to multiple thousands for Nuke).
Although I’m really excited about Mocha’s stereo support, as a new tool, it isn’t perfect quite yet. I’d like to see them add more finessing options. I’d like more control over the convergence and the ability to feather the convergence. For example, if I’m comping something close to the poles in Mocha VR, it starts to fall apart because we don’t want stereo at the poles. It’d be great to be able to feather off from stereo to mono as I get closer to the poles.
But the tool is brand new–not even out yet!–so I’m hopeful that these features will come out in future versions.
New tools for Boris Continuum
When Boris announced stereo support for Mocha VR at IBC in September, it also announced a handful of new VR-related tools for Boris Continuum, including blur, sharpen, flicker removal, and the ability to insert 3D objects.
Quite simply, the sharpen tool is magic. It’s the greatest thing. The rendering is a bit slow, but it brings out so much detail in an image without making things look terrible.
I also use Continuum’s VR insert tool. It’s a bit slower to use than Mettle’s project 2D tool, but I don’t experience as many GPU crashes on my machine, so it is nice to have the option when problems arise.
Right now, if you buy Mocha VR, you can get the new Continuum VR unit for free (normally, Continuum VR costs another $199).
My final thoughts on Mocha VR’s latest update
The tools for VR production are constantly changing–which is great!–but the best part of all is that software engineers are really eager for creators to tell them what does and doesn’t work and the tools they need to do their jobs better. And Mocha VR and its 5.6 update is no exception.
I’ll leave you with one final thought – the tools are only as good as the artist and the creative behind them. Thankfully the tools make creating VR content easier, which lets us focus on what’s really important: telling a compelling story.
You can find out more about myself and my work at www.lightsailvr.com and be sure to check out our latest project Speak of the Devil which at www.speakofthedevilvr.com
Mocha VR’s stereoscopic support is now in beta, but will release publicly in approximately three weeks. If you want to gain access to the beta version, fill out this survey, then go here to create or sign into your BorisFX user account, and then request access to the Mocha beta group here. If you’re new to Mocha VR, check out BorisFX’s tutorials.
