In 1979, Apocalypse Now hit theaters as the first major release to include a new technology we now know as “surround sound.” The setup included three speakers in the front and two in the rear. Fast forward to today, where some theaters that are sporting the full arsenal of “Dolby Atmos.” This includes many more side & rear surround speakers, as well as two rows of speakers that are mounted on the ceiling. Now film soundtracks can include a level of detail that has hitherto been unimaginable, providing the most authentic immersion audiences have ever experienced. But, how the sound gets to that level is an arduous journey requiring the immense skill of many people. The impact of this final step in the audio process cannot be underestimated, as this is where the “movie sound” is created.
Putting It All Together
After the sound designer lays in the sonic landscapes, the dialogue is edited and the composer has created an enticing and emotionally supportive score, all of that audio gets put together for the final mix. This is called “re-recording” because all of those original recorded sound elements are then re-recorded onto the final soundtrack during the mix. The blending of all those components is what the audience will hear once the film is finished.
Much of the audio that gets delivered will be in somewhat of a “raw” state – meaning that reverb, EQ, effects and other mix elements haven’t been applied. For example, that song that’s playing on the car radio? Yeah, it’s not on the radio – a mix engineer made it sound like it’s coming from the car’s radio. Pretty much any sound other than dialogue is manually created and positioned. This is also where reverb gets put on, sounds get EQ’d to match each other and last minute sound design elements may need to be added.
One of the cooler tools a mixer has at his disposal is called a “subharmonic generator.” It adds additional frequencies an octave or more below the original sound, creating that extremely deep bass we’ve come to expect in our movie going experience. This will comprise a large portion of the LFE track and will put your subwoofer to the test.
The Dubbing Stage
The room where movie soundtracks are mixed is called the dubbing stage. It’s essentially laid out like a movie theater, sans seats, with a giant mixing console in the middle. Here, there may be only one mix engineer for a very small film or a team of people working in tandem for an action blockbuster. On those larger budget pictures, the responsibilities will usually fall on 2 – 3 people who will juggle dialogue, music, sound effects and Foley. In the words of Scott Millan from Todd-AO studios, “It would be impossible for the most part to do this as a solo mixer. As it stands now, David [Parker] and I have worked on projects where we don’t go home for three days. One person couldn’t physically do it all.” The mixing process can be a long and grueling endeavor comprised of hours on end listening to the same sections repeatedly. This is also where the director, producers and composers will listen to the mix and advise on any changes that need to be made.
Preparation
A crucial element for keeping the project on time is preparation. When all the sound files are delivered, it takes a long time to organize everything and make sure it’s laid out properly for the work flow. There are potentially hundreds of tracks that need to be organized and assigned to the board, so that when the session starts, everything is ready to go.
Gary Rogers, a re-recording mixer on the Walking Dead describes this process with the assistance of recordist Chuck Hamilton: “Preparation and attention to detail are the keys to getting through the mix. Handling 42 minutes of a Walking Dead episode in two nine-hour days is massive; we rely on Chuck’s lead time to get the session ready. Is there anything different? Are the files on Pro Tools 9, 10 or 11? Am I going to have to convert the files? Has the music format changed? We need to know what’s going on. It’s all about lead time — solving oddities before they arrive here on the stage and potentially create a problem. Chuck is in communication with everybody, so we always know ahead of time.”
In the film world time is money, literally, and there is a finite amount of it. If you aren’t able to get the best results in the allotted time, then the project suffers and you may find yourself getting fewer calls in the future.
Dialogue
As much as we like listening to the awesome sound FX and musical score, the dialogue is usually the most important element in film sound. Making intelligible, clear dialogue can be quite time consuming depending on how good the source recordings are. Even on the best projects, dialogue from different takes or ADR sessions will have to be matched to each other so there is a smooth flow. With ADR specifically, since it is recorded months later in a studio, the mixer will have to make it sound like it was recorded in the same room as the other dialogue.
Skywalker Sound re-recording mixer Michael Semanick had this to say about how important it is to get good production sound: “Production mixers are probably guys that are a pain in the ass on set because they’re saying, “Can you raise that lighting up? Can we move that generator over? We’re not shooting the feet. Can we not wear shoes? Can they wear booties so we don’t get people clomping around? We’ll replace those feet later. That way I pick up the dialogue so there’s not a footstep on it or a word that people have a hard time understanding.” You have to start with the production mixer and then my job, when you’re mixing dialogue in a dialogue-[heavy] show, you’re repairing problems. There are great devices that are made for noise suppression and trying to make the dialogue as clear as you can, but what happens a lot of times, I think it gets over-processed. For me, it’s always trying to maintain the spirit of the voice … so over-processing can damage that and then under-processing, people can’t understand it. There’s a fine line. You’re always checking yourself. Does it sound natural? Does it damage the actor’s performance at all? I gotta say dialogue-heavy films are harder to mix in a sense.”
You also have the FX and score to contend with. So hopefully the sound designer and composer have done their job to make room for the dialogue. An ill placed sound effect or a prominent musical instrument in the vocal range can cause a conflict rendering speech incomprehensible.
Of course, all of this goes out the window if you have a director that is intentionally trying to make the dialogue difficult to hear. Such was the case with Christopher Nolan’s Interstellar, where key scenes had the sound effects mixed so loud that it rendered the actors speech inaudible. This actually became quite the controversy when people became frustrated that they couldn’t understand what was being said. Re-recording mixer Gregg Landaker spoke about how Nolan “wanted to come up with a different style … to really capture the essence of space travel. We listened to a lot of NASA recordings, including a track of the interior of a space shuttle. We experimented with lowering stuff down so that you could hear the dialogue, but then it became so unreal. The idea of these scenes is to make you feel you’re straining to hear just as much as they are straining to hear each other.” During the pivotal scene between Professor Brand and Murph “you are supposed to be on the very edge of just grasping — as much as [Murph] is grasping — for every last word,” Landaker says. “You want him to [confess the truth about the mission,] but he doesn’t get it all the way out. ‘Just give me that word!’ That’s intended. Chris wants his audience to hang on just as much as Jessica is hanging on.”
Backgrounds
The background sound in a film often goes unnoticed, but it is an essential component in setting up the environment in a scene. “BG’s” include room tone, street noise, wind and pretty much anything that might be going on in the surroundings of a scene. Some of these sounds are recorded on set, some are recorded separately by the sound designer and some might be from a sound library. One of the responsibilities a mixer has is making sure all of these elements are connected seamlessly. This becomes increasingly difficult with scenes shot in a location with inconsistent background noise, like traffic or an air conditioner intermittently turning on or off. Two different shots in the same scene can have drastically different background sound that will immediately pull the audience out of the movie.
But, they can also be used in such a way as to juxtapose different worlds as they do in HBO’s Westworld. The hit series’ extraordinary production value – budgeted at $100 million for 10 episodes – is evident in both the visual and auditory realms. In the western scenes, “there are authentic sounds of horses, wagons and guns,” explains re-recording mixer Scott Weber. “There are no cars, so the ambience is birds and wind in the trees. In the bar, we have glass clinks and the crowds. It’s a lot of cool stuff.”
Moving into the sterile hi-tech laboratory environments required a contrast with the gritty western soundscapes, keeping the ambience clean, light and futuristic. “The equipment needs to sound familiar but a little more advanced than what we have now,” noted Producer Bruce Dunn. “How does a robot sound? The first episode features a broken down, 35 year-old robot. The sound team created sound treatments for an old robot’s eye blinks. They created tones for Tech World that vary from floor to floor, so Manufacturing sounds different from Behavior. There is a distinct sound for every place. If you listen to the mix closely, it’s all there.”
Sound FX
The “hard” sound effects are among the trickier components to mix. These have to stay out of the way of dialogue and trade off with the music for the limelight at times, yet still have the impact that the director and audiences expect. In action films, the density of sound can be quite immense, so the balancing act of blending all those sounds requires a real mastery of the craft.
In Transformers: Age of Extinction, sound mixer Scott Millan had to wrestle with this fact: “We made a very conscious effort to not get so loud that it pulls the audience out of the film,” he says. “There’s a fatigue effect. Sonically, you can fatigue an audience and they’ll tune out. You have to find ways to become more stylized and give the audience a break.”
In addition to the backgrounds, the FX material is often what fills out the surround channels allowing the audience to be fully submerged in sonic bliss.
Foley
Sometimes referred to as “movements”, Foley is different than sound effects in that it consists of sound that occurs on screen, but wasn’t captured during production. This usually consists of footsteps, clothing noise, the zing of a sword being unsheathed, punching or any other physical interaction the actor is involved with. Due to the priority of capturing clean dialogue on set, many elements are intentionally left out or reduced and need to be added in post.
One of the more important parts of mixing Foley is applying the proper amount reverb so that it matches the room sound that the actor is in, as well as general leveling for balance. A fight sequence might include a number of tracks comprised of punches, knife movements, stabs, clothing swishes, footsteps and other miscellaneous sounds all expertly performed by the Foley artists. Controlling these sounds and blending them into the scene helps provide a level of detail and realism that modern filmmaking is known for.
Music
The musical score is delivered in “stems,” which is a reduced number of tracks where like instruments are grouped together. So, you’ll have a separate track for strings, brass, percussion, woodwinds, electronics, etc. Since there is a dedicated music mixer, by the time it gets to dubbing stage only minor tweaks will need to be implemented, but having the stems allows the music to be spread throughout the surround field more creatively as well as being able to adjust the level of a particular section in case it conflicts with something else. But as with any mix, difficulties can arise when working with limited source options. Veteran re-recordist and sound designer Randy Thom offers some insight on dealing with only a stereo music mix, instead of stems:
“Music mixers do have their frustrations, though, and one of them is music copied off cd’s which is supposed to be put into the film in that two-track form. You can eq it, add some reverb, ride it to keep it out of the way of dialog, but not much else. It often winds up being assigned to only the left and right channels of the final mix, which presents a problem in the form of the “Haas Effect.” If you are equidistant from the left and right speakers it will sound ok. But if you are even slightly closer to the left speaker than the right, then any material (like maybe the vocal) which in the source mix is equally loud in the left and right will seem to be coming only from the left speaker. Because the sound arrives at the left ear only a few milliseconds before it arrives at the right ear, our brain tells us that the source of the sound is to the left. To all the people sitting on the left side of the theater (even one seat away from the center) any mono material in the two channel mix will seem to be coming only from the left. To those sitting on the right, that stuff will seem only to be coming from the right. The Haas Effect obviously isn’t limited to music. Any mono sound that you send equally to two or more widely separated speakers will sound like it is coming exclusively from whatever speaker is closest to the listener.”
Besides adding to the emotional intent of the film, the music can also be used as a tool to cover mistakes in dialogue recording or poor pacing in a scene. Directors invariably will have a scene or two where the actors didn’t quite capture the moment or there is an issue with the dialogue track and the music is called upon (turned up) to help mask whatever deficiencies exist there.
Dolby Atmos
The single biggest evolutionary event in film sound has been the development of Dolby Atmos. It’s a completely different way of thinking about the sound field. Rather than simply panning the sound into this speaker or that speaker, each sound is treated as an object that can be placed in a 3-dimensional space. With 128 discrete channels and 64 unique speaker feeds that are sent to an increased number of surround speakers and the addition of height channels, Atmos allows even more realism, immersion and flexibility for the cinema and home experience.
Because of this heightened reality, it can make for some interesting conundrums. For example rainfall – when you hear rain outside, you’re actually hearing it hit the ground. So do you have a more realistic positioning with the rain in the side surrounds, or do you make “Hollywood” rain and have it in the Atmos channels overhead? These are the kinds of decisions the film makers now get to deal with.
Mixer Greg P. Russell illuminates another scene from Transformers: Age of Extinction on how this new technology allowed techniques that were unavailable prior to Atmos. “There’s one significant sequence at the end of the film in the third act, where a spaceship is utilizing a magnetic field to draw up and gather all this metal content down below,” he explains. “We utilized the ceiling in way that we had these pulsating tones that were working from the side walls up into the center and there’s a motion that was created by the oscillating tones that were created by our sound design team, and we manipulated the ceiling and the walls to feel an upward motion that was absolutely fantastic and is something that — there’s just no way we could have done that without Dolby Atmos.”
The Final Mix
After all the sub-mixes are complete, everything will be put together and mixed as a whole for the final soundtrack of the film. Usually, multiple mixes will be done at this stage for different formats. A stereo mix, 5.1/7.1 and Atmos mixes can be performed to facilitate the best possible result in each medium. For those who’ve never been a part a mixing session, hearing how the original basic 2-ch mix gets pulled and stretched into the surround channels is awesome. It also exposes things that may not have been evident in the original mix. By simply moving a sound that was buried in the mix up front back to the rear or side channels, it instantly becomes audible, clear and a point of detail rather than clutter. That, in essence, is one of the main challenges of mixing a large production. Hundreds of hours of work and thousands of sounds created by extremely talented people have gone into the different pieces of the soundtrack and they all need to be heard clearly. This is another benefit of having the height channels, as this opens another dimension to put the sound, enabling greater clarity and separation.
Where the final product will ultimately appear must also be considered. Sound designer and mixer Ric Viers comments on the viewing environment: “Quiet theatres with padded walls covered in sound-absorbing fabrics help control the environment. Therefore, the film’s sound mix has a wide range of dynamics (the differences between quiet and loud sections of the soundtrack). This is very different from the television listening environment. Typically, television is watched in a house with a host of sounds to contend with such as refrigerators, air conditioning units, aquarium filters, traffic outside, lawn mowers, screaming kids, etc. So, there is a bit of a limit that you can have in the dynamics of the production. Have you ever noticed that a movie playing on TV seems really low in volume? There is nothing wrong with the film’s mix, but the environment in which the film is being heard. The environment that radio is listened to in is primarily a car. This means that in order for the DJ to be heard, they have to crank their levels in order to cut through the car’s engine, passing traffic, etc.”
With that in mind, most movies will receive another mix or “remastering” before it hits the home market as well. Most of the work on the theatrical mix will remain intact, but will be adjusted slightly to make sure their original intent is translated properly on the smaller home systems.
By the end of the re-recording session, what was once an unmixed and conflicted mélange of sound packed into a flat 2-dimensional cage has blossomed into luscious, dynamic and fully matured embodiment of sonic mastery that expands reality for those so inclined to bask in its glory. This final step in production is designed to make you feel like you’re actually in the movie and allow the banal existence of daily life to fade away as you savor the escape that these true masters of their craft provide.
Read the rest of the series:
Behind the Scenes Part 1: Sound Design
Behind the Scenes Part 2: The Score