This post originally appeared on John Hendicott’s blog on Medium and is being shared here with his permission so we can all learn from his spatial audio workflow. We highly recommend you follow John on Medium–he has some great content coming out in the near future!
In this article I thought I’d break down the process of recording, mixing, sound designing and scoring a 360 behind the scenes video. The TV show in question was NBC’s ‘Blindspot’. Later on in the article you’ll find a video showing a breakdown of all the audio sources used in it’s creation, but first, here’s the finished piece:
Capturing 3D audio
From start to finish this job was great fun, and as director Nick Bicanic mentioned in his article; ‘how to shoot/direct 360 video alongside a network TV show’, it required careful preparation and fast execution on the day:
I came to set with the Sennheiser Ambeo microphoneThis is a description of a microphone it goes here and it's long so that we see if it gets cut off. (which at that stage was still in Beta) a Sony R26 2 Channel Recorder, a Zoom H2n recorder, a Zoom F8 recorder and bunch of radio mics.
The Ambeo was used in almost all of the scenes and can be seen attached to the monopod using a Dinkum Systems clamp. I am now using a combination of Manfrotto Super Clamps, as they offer a stronger grip and enable the mic to be positioned closer to the monopod, thus reducing the area to be roto’d.
Ambeo with BBG wind shield, clamped with a Dinkum Systems clamp
The Zoom F8 was chosen because it’s a cost effective solution to multichannel field recording. It has 8 inputs which can be ganged for ambisonicPertaining to audio reproduction that captures the spatial acoustic qualities of recorded sound. recording (enabling you to control the input level of 4 channels simultaneously) and also features Bluetooth control, letting you monitor levelsThe adjustable sensitivity settings of microphones. Levels are set (and changed as necessary) to best capture the vocals... More and start/stop recording remotely, so you can attempt to exit frame. The preamps are surprisingly good on this thing, too. For more detailed work, I’d turn to a Sound Devices recorder, but for this it worked just fine.
To help the post production process it’s not only essential to announce the scene number and takeIndividual instance of a shot; a take = each time the camera is started and stopped., but also the orientation of the ambisonicPertaining to audio reproduction that captures the spatial acoustic qualities of recorded sound. mic, as the mic can be positioned with the capsules up, down, or end firing. Without this information the post sound mixer can only make an educated guess as to how to position the ambisonicPertaining to audio reproduction that captures the spatial acoustic qualities of recorded sound. sound field to picture. Therefore, I might announce something like: “Scene 4, takeIndividual instance of a shot; a take = each time the camera is started and stopped. 2, Ambeo is upright, facing the blue door”. This gives me everything I need to know to orientate the 3D sound field in post.
In addition to the Ambeo, I used a combination of cheaper sound recorders where there was the possibility of equipment getting damaged. In particular, these were the door detonation and motorbike launch scenes. Although recording in stereo, this gave me perfectly useable audio for these short scenes, when used in conjunction with some ambisonicPertaining to audio reproduction that captures the spatial acoustic qualities of recorded sound. wildtrack.
the aftermath of the door explosion, featuring the Roland R26
For the interviews, I used a combination of lav mics and Ambeo to achieve a good blend of direct dialogue and ambient sound, which we all know is essential to the sense of immersion.
The spatial audio post production process
Moving into post, I jumped into action once Nick delivered his final cut as a video and OMF file. At Aurelia, we use a combination of software solutions depending on the project. For this, we used Reaper and the excellent Blue Ripple tools, to mix in higher order ambisonics.
As previously mentioned, we had received the production sound files, meaning we could use the lav mics used in the final TV production (which were all recorded superbly). Once these were synced and mixed with the ambisonicPertaining to audio reproduction that captures the spatial acoustic qualities of recorded sound. files, we were ready to dive into sound design.
To demonstrate individual sound elements, we’ve uploaded a video of The Sennheiser Ambeo mic*, SFX and music all separated, so you can hear what’s going on. Check it out below on a Chrome browser and don’t forget to use headphones!
*Martin’s interview was recorded using an Ambeo mic, however, the fight scene after it was recorded by an external sound engineer using a quad mic array, which unfortunately isn’t spatialising as we’d like
Sound design
It’s up for debate how extreme you go with sound design and sound effects for a ‘behind the scenes’ (or ‘inside the scenes’ as Nick says) as the aim is to capture the feeling of being there and experiencing what the crew and cast are doing. However, without the impactful body punches, head slams and gunshots the piece will always feel flat, so we went about adding all of those elements to add some excitement.
Music
After the sound FX were in place I then looked at music scoreMusic added to enhance the mood of the video.. This was designed to fit in with the TV series theme music used at the beginning and end of the video. Without going into too much detail here, the aim was to maintain a sense of energy and rhythm, whilst also creating space for dialogue, when needed. It was also crucial to emphasise the most important moments of action — if you listen to the demo below, you’ll notice most of the main beatsUnits of action within a scene. of the music land on punches or serve to accent the beginning or end of the scene.
Below is a screenshot of the Blindspot Reaper session (click on the picture to enlarge). ScoreMusic added to enhance the mood of the video. is in green at the top, then interviews in blue and pink, a heap of SFX in grey that are minimised and then further down in yellow and orange are the Ambeo and quad mic sources. You can also see the video, to which we added a 360 degree grid, which helps us spatialise the sound accurately. You can download it here to use in your projects if you like.
Blindspot mix session created in Reaper
Final delivery was for YouTube, which at the time of writing only supports 1st order ambisonics. Therefore, we exported our mix in the 4 channel AmbiXA method of storing ambisonic audio data in the SN3D format. format and attached to the video using the handy iffmpeg app.
The near future
As we all know, VR technology is moving super fast; from capture systems and post production workflows, to publishing platforms and delivery requirements. Because of this, we’ve decided to upload an updated version of the Blindspot video to Facebook, which now supports 2nd order ambisonics and fixed stereo. A little more about that, here:
An essential new development in sound delivery for 360 video, is the ability to split the diegetic 3D sound field, with non-diegetic fixed stereo track. This essentially means that for everything that’s visibly making a sound in the scene, we mix an ambisonicPertaining to audio reproduction that captures the spatial acoustic qualities of recorded sound. 3D sound field, which responds to the viewer’s head movement. For everything else (in this case, the music scoreMusic added to enhance the mood of the video.) we render a stereo track which is fixed to your head position. This gives the feeling of the music being ‘in your head’, rather than ‘in the scene’ and really helps the viewing experience.
If you listen again to the demo video above, notice that unlike the Ambeo and SFX sources, the music does not rotate in your ears. This is because we want the music to sound internal, and not in the environment. This is one way that spatial sound technology is helping us tell more convincing stories in 360 video and VR.
Many thanks to the team at NBC, and to Nick Bicanic for making such a fun shoot also look awesome!
This post originally appeared on John Hendicott’s blog on Medium and is being shared here with his permission so we can all learn from his experiences with spatial audio. We highly recommend you follow John on Medium–he has some great content coming out in the near future!
