During the first day of Facebook’s annual F8 conference in San Jose, Oculus Go may have been the big news in VR, but it wasn’t the only news in VR.
Facebook also announced a partnership with RED Digital Cinema to create a commercially available camera that can capture high resolution 8K 360 video with six degrees of freedom (6 DoF). That means the video can not only be viewed from within a headset, but users will also be able to move (to a certain extent) within the video.
When Facebook first announced its x24 and x6 6DoF-capable cameras in the spring of 2017, they’d already known they would be on the hunt for manufacturers for the camera.
Facebook had made available an open-source spec sheet for its previous 360 camera, Surround360 Open Edition VR camera, to assist camera manufacturers in developing their own 360 cameras. For example, camera manufacturers like Z Cam and Kandao built cameras based on the Surround360.
For its volumetric cameras, Facebook decided RED–with its “unparalleled image and pixel quality”–would be the right partner to best utilize its depth estimation technology.
“RED brings unparalleled image quality with over 16-stops of dynamic range and high spatial resolution, making it ideal for many VR capture scenarios,” reads Oculus’ blog about the announcement. “The high spatial resolution and dynamic range allow for new levels of depth estimation and 6DoF realism.”
Meaning, its hard to generate a depth map for exceedingly dark situations, corners and shadows. Without data from each and every pixel, Facebook’s depth estimation technology’s algorithm wouldn’t be able to deliver a complete 3D reconstruction as promised.
That’s where RED’s low-light performance comes into play. Just look at it:
What’s the workflow?
According to an interview last year with Facebook’s Head of Immersive Media Eric Cheng, the goal behind the x24 and x6 was to “foster a vibrant ecosystem” around volumetric video. And, that also requires attention to the workflow. Facebook has been working with the likes of Adobe and Mettle to make post-production tools capable of editing 6 DoF content.
And–before even getting to that step–they’ve been working to make the shooting process simpler, as well.
According to Brian Cabral, a director of engineering at Facebook specializing in computational photography, Facebook needed not just “the image quality we wanted,” but also an easier on-set workflow. Supposedly, creators will be able to capture, review and stitch all on-set. Cabral added that a lot of the feedback from early users of the camera system was how easy it was to use on set.
“When we try to make photo real environments, it’s very labor intensive,” said Emblematic Group’s Nonny De La Peña in this testimonial video with creators using the camera. To create its South By Southwest jury-award-winning volumetric piece detailing life in solitary confinement, she adds, they had to shoot the environment in Maine and then send their subject to L.A. to be captured. “What we’re walking about with the new cameras is that we can film it all on-scene and yet create the same dimensionalized experience, which has been my goal all the way through.”
According to a spokesperson with Facebook, “this is the first end-to-end solution with all elements of the workflow toolchain combined into one powerful package.”
The workflow begins with capturing immersive content with this new camera. Depth estimation is done with Otoy and editing/post-processing with Adobe, Foundry and Otoy. They’re also optimizing for “premium 3D output” that can be experienced on Facebook in VR.
Sidenote: Although in a promotional video, Facebook mentioned distributing content from the new camera on the recently-launched Oculus Go, to fully take advantage of the 6 DoF opportunities, you’d need to use a headset like the Oculus Rift or HTC Vive, or 6 DoF-capable competitors to the Oculus Go (like Google and Lenovo’s Mirage Solo or HTC’s Vive Focus).
No surprise here, but Facebook will also be recommending the new RED VR camera rig to immersive filmmakers working with Facebook to create high-fidelity immersive content.
What will the camera look like?
Supposedly, the camera will look “somewhat similar” to the models Facebook showed off last year.
The x24 and x6 cameras, so named for the number of onboard sensors, are about 10 inches and 5 inches in diameter, respectively. As the next stage of Facebook’s Surround 360 initiative, they’re significantly smaller than the first camera iteration, Open Edition.
We don’t yet know what the camera will be called.
When can I buy one, and how much will it cost?
When Facebook first announced the camera in the spring of 2017, they’d said they were hoping to launch it by the end of 2017. Obviously, that didn’t happen. And, no release date has been set for the camera in partnership with RED.
They also haven’t shared a word about price, but judging from the price tags on RED’s existing lineup ($5,000, $15,000, or even $50,000), it will likely be a premium product. If those kinds of prices make this camera more of a dream than a reality for you, you may just have to wait for your dream to come true. Sources within Facebook have also said that it’s much easier to downgrade these cameras than it is to upgrade them, so it’s reasonable to think of this first iteration as the marker pushing camera manufacturers to step up their game at a variety of price levels. Fingers crossed…
We’ll keep you posted as more information is announced.