The visual effects of Surrogates
Now available on DVD, Jonathan Mostow’s Surrogates depicts a world where human beings have given over their lives to almost-perfect robotic replicas. Although more crime drama than action piece, the film still contains around 800 visual effects shots. Visual effects supervisor Mark Stetson oversaw an ensemble of facilities on the show, including Synthespian Studios, MPC Vancouver, Brickyard VFX, Sandbox F/X and ILM, with practical work by Howard Berger’s K.N.B. EFX Group.
Article by Ian Failes
Surrogates follows the investigations of FBI agent Tom Greer (Bruce Willis) into a plot to kill all surrogates and their owners. Set in Boston only slightly in the future, the film steered clear of heavy CG environments or robotic creations. Mark Stetson’s mandate was always for the visual effects to have only a supporting role. We focused more on how to portray and finish the surrogate versions of our characters. We looked for opportunities to enhance the idea that surrogates were pervasive in society, with glimpses of the surrogate factory, and hints that the telecommunications grid had expanded to accommodate all the traffic.
Such an approach also necessitated a healthy mix between practical and digital effects. K.N.B. EFX delivered mechanical puppets for the surrogates and makeup effects in collaboration with makeup department head Jeff Dawn. To me, it’s a great example of doing practical and CGI and mixing the two and coming up with the trick, commented Howard Berger. You don’t always know where one technique stops and the other one begins, which I love. We combined our efforts together and ended up with something that looked really cool.
Synthespians by Synthespian
Synthespian Studios handled the digital surrogate effects under the supervision of Jeff Kleiser and Jeff Kalmus. The effects house’s initial brief was to create about 60 shots featuring the endoskull substructure of the surrogates. Among these is a beauty shop sequence featuring a beautician grabbing a client’s ear and peeling off her face to reveal the endoskull underneath for cleaning. Production shot elements of the client actress and a practical bust of the endoskull made by K.N.B. Synthespian then replaced the endoskull with a CG version created in Maya and rendered with mental ray to allow for additional animation of endoskull details. We shot the physical endoskull first, said Kleiser, then, using video playback of that, we got the actor into position and matched her to the rubber mask. Then it was just a matter of a splitscreen from the live action girl to the mask precluding her face. We gave the skull underneath some life by adding muscle actuators moving around and CG eyeballs.
For a sequence in which Greer and his FBI partner Peters (Radha Mitchell) meet a robot landlady, Synthespian digitally augmented a head mask created by K.N.B. for the performer to wear during photography. During post, said Mark Stetson, the tone of the movie developed to the point where the original landlady design was too whimsical. So I came up with a look for a partially disassembled surrogate head that retained the face portion of the original mask, but showed the endoskull behind it. Synthespian rotoscoped out the performer’s neck and created a series of pistons through which the background would be visible to sell the illusion that there wouldn’t have been enough room for a human to be inside. But we decided to keep the face, noted Jeff Kleiser, because there was something eerily creepy about the lip-synch. They had put this latex mask over the actor’s face and when the actor talked the mouth just wiggled around without looking articulated. It just had this weird feel to it and fitted in well as it’s supposed to be a low quality robot.
Original photography
Concept illustration
RD model
CG elements
Final shot
Further digital augmentation by Synthespian was required for shots of Greer punching the face of his wife’s surrogate friend at a party. Howard Berger made four different prosthetic pieces that represented different amounts of the face being torn off, explained Jeff Kleiser. For each one, he had blended into the actor’s face a skin flap that would open. Parts of the face were painted blue so we would know which bits to replace with a CG endoskull. There are four or five quick cuts where Bruce is punching the face. Our job was to go in there and track the blue screen area and the motion of the head – which appeared to be really violent – as the fists and foot are coming down. We had to match the jerking motion of the head and the motion blur so we could get it to look like the skin was being torn off and revealing this greenish endoskull underneath.
In addition to the endoskull sequences, Synthespian was also awarded another 200 surrogate digital makeup shots. For the whole first half of this movie, Bruce Willis plays himself as a surrogate, said Jeff Kleiser. The concept was that if you were to have a robot version of yourself, you wouldn’t have wrinkles and double chin and all that stuff. You wouldn’t look 60 – you’d look 25. Mark Stetson approached the de-ageing effects so that the difference between the surrogates and humans could be visually emphasised. We started with casting and beauty makeup, of course, and DP Oliver Wood developed a lighting style to soften the skin of the surrogates. Nevertheless, more than half of the VFX shots in the film focused on the look of the surrogates, created by Synthespian and some of the other facilities.
To de-age Bruce Willis, Synthespian cyber-scanned the actor’s head, then took the resulting 3D model into Maya and literally performed a digital face-lift. We pulled up the chin, smoothed out the wrinkles, pulled the cheekbones out a bit and pulled the skin up, so that it looked like a younger person’s face, said Kleiser. A younger person has a different quality of skin tightness than an older person does. It’s nothing to say about Bruce – it’s just that any person that age is going to look different from someone who is 25.
For each of the shots of the Bruce Willis surrogate, artists tracked the camera motion before tracking in the 3D head model. We’d make sure our 3D model was moving the same as Bruce’s head in the scene, noted Kleiser, and then we could go in and take the textures from the scene that was shot and map those onto the new geometry so that we would get his face into the right shape. We also painted out any anomalies like wrinkles or blemishes or anything else that made him look too human.
The whole thing was put together in Shake, added Kleiser. The real balancing act was that if we took all the textures away he began to look like a boiled chicken. Mark Stetson would say, ‘No, he looks too over-cooked there’, so we’d have to go back and put more skin texture back into that area, but not put too many wrinkles in that might be perceived as an imperfection, because the idea is that these robots have perfect versions of human skin.
The O.D. effect and helicopter crash
Greer’s investigations lead him by helicopter to the de-militiarised zone (DMZ), a region in the future where people who choose not to use surrogates live. Hit by an electronic pulse weapon, Greer’s helicopter crashes into the DMZ. 106 visual effects for the crash, along with a subsequent footchase and shots of Greer’s injured surrogate, were taken on by MPC Vancouver and supervised by Doug Oddy.
The effects of the pulse weapon, known as the overload device, or O.D., were realised by MPC for the helicopter crash and other sequences in the film in collaboration with Sandbox F/X and Synthespian Studios. The challenge was to find a look for the O.D. that conveyed the importance of the effect without taking the audience out of the everyday reality of the movie’s visual environment, said Mark Stetson. DP Oliver Wood and second unit DP Igor Meglic created interactive light in the original photography, using a large Lightning Strikes unit to provide the power needed for the daylight scenes. Noted Stetson: We considered finishing the shots with just a ‘muzzle flash’ from the O.D. and no added blast-ray effect at all, but Jonathan ultimately wanted the dramatic impact of a visible effect.
O.D effect
The effect on the surrogate victims was likewise difficult in finding a balance between subtlety and impact. We started with just the eyes popping, recalled Stetson, but that wasn’t enough. We really wanted to do a full-on CG head explosion, but our budget wouldn’t permit that either. So MPC Vancouver layered the eye explosion with additional spark hit elements and some of the elements similar to those that Synthespian created for the translucent flashing view of the endoskull in the party scene.
Our O.D. effects were driven with particle systems rendered out to generate element passes for the compositors, said Doug Oddy. It was a collaborative 3D/2D effect. We rendered particle passes to drive things like displacement and refraction of light, almost like caustic lighting effects – what we call plasma bubbles. We also created a sub-surface lightning storm within the surrogates as they are overloaded, with lots of roto-animation and effects work, and an eye-popping effect. There were probably 30 different elements in the end that contributed to the final look.
With only the pilot disabled by the O.D., Greer’s helicopter crashes into the DMZ. Action and plates for the crash were shot by second unit and directed by Simon Crane. Production designer Jeff Mann found a wrecked helicopter hulk, which he painted and dressed to match the FBI helicopter. Allen Hall and his special effects crew rigged the hulk with a high-speed cable pull and extensive pyro, so that the shots of the helicopter crashing through the wooden shed and skidding across the DMZ are mostly practical.
CG helicopters
Using Maya, MPC added a CG helicopter and crashing brick and concrete wall simulations, along with background matte paintings, flames, smoke and digital doubles of the chopper pilot and the Greer surrogate. After the crash, the Greer surrogate discovers he has lost his right arm before embarking on a chase through storage containers and other areas of the DMZ. Bruce’s arm was pushed into a blue sleeve that went up underneath his torn jacket, said Doug Oddy. We had tracking markers on it to give us rotational data as he ran around. We painted it out and built a digital stump in Maya. From production artwork, we created a hydraulic-based muscle system for the arm, so that it would move around in a way you’d expect a non-human arm to move. The muscles just keep firing because the robot knows no difference. We used some dynamic cloth tools for the torn jacket so it matched the real one and then employed a lot of paint work to bring it all together.
For the green fluid leaking out of Greer’s mangled appendage, MPC added to a methacil, paint and fluorescent powder concoction pumped through the arm on set by K.N.B. We ran a fluid simulation but we didn’t want to be limited to an accurate simulation running itself through an environment, said Oddy. Because the fluid was leaking out of a character, we really needed to be able to control and dictate where it could go. We used our own particle system and then developed a meshing system that could go over the top of it. It responded much in the same way as blobby systems would.
During the chase, Simon Crane shot stunt doubles in wire rigs doing the large jumps. Most of these were carefully finished as 2D speed changes and rig removals, but one wide shot of Greer jumping across storage containers required an MPC CG stunt double. A shot of Greer being struck by a pickup truck also required a CG stunt double to transition between bluescreen footage of Bruce Willis overtaking the man he is chasing and then riding on the truck hood. Subsequent shots of the Greer surrogate hung up in a crucified position featured a K.N.B. dummy.
The war sequence and death of Stone
The more futuristic, and perhaps disturbing, aspect of surrogates is demonstrated in a war sequence showing fighting robots being controlled remotely by U.S. Army soldiers. For the battlefront shots, special effects supervisor Allen Hall and his crew created pyrotechnic explosions and air mortar bomb blasts. Soldier surrogate makeups were done by K.N.B. Sandbox F/X, under the supervision of John Nugent, added cloned bullet hits and bomb blasts in the ground, enhanced and added bullet hits on the surrogate soldiers, CG animated tracer rounds, CG aircraft and rocket fire, and muzzle flashes over all the background. The studio further created the space where real soldiers are controlling their surrogate counterparts.
Background plate
Final comp
Sandbox also contributed to the violent O.D. death of FBI agent Stone,
who is exposed as part of the plot to destroy all surrogates. Editor
Kevin Stitt came up with the idea of following the blast ray into
Stone’s head and watching the circuitry blow, explained Mark Stetson.
Sandbox further developed the look of the O.D. blast ray that MPC
Vancouver had established, and then worked with Assemble Co. to build
the endoskull interior. The transition from the live action photography
to the interior of Stone’s head required some element replacement of
Stone’s face and eye for a few frames as we pushed in beyond the
original scan resolution. Sandbox’s additional work included the
surrogate factory assembly line (again with Assemble), blue/green
screen composites, CG set extensions, 2D and 3D scene enhancements,
cosmetic enhancements and look development and matte painting work,
ultimately producing 353 shots for the film.
Background plate
Bluescreen element
Final comp
The car-top chase
A city car-top chase sequence featuring the Greer and Peters characters was added to the film late in post-production. Directed by Jonathan Mostow with the aid of second unit director Brian Smrz, the sequence came together on the Paramount Studios backlot in Hollywood, and in downtown Los Angeles. Sandbox F/X finished the green screen comp of Bruce Willis being struck by the Chevy Avalanche, with Industrial Light Magic, under the supervision of Lindy De Quattro, picking up the sequence until Peters escapes on the bus. ILM’s work included moderately extensive background repair to remove the palm trees and other obvious signs that the plates were not shot in Boston. I went back to Boston to shoot a library of stills from which ILM drew reference for their matte painting set extension work, said Mark Stetson. ILM also created CG glass replacements for the car windshield in its various states of destruction, a CG parking meter, and a CG stunt double of Peters. Extensive rig removal and paint repair work was required for the jumping stunts, with about 6 of the shots ultimately requiring CG stunt double replacements. The wide shot of Peters flying to the bus, however, was finished with the original wire stunt – a testimony to the quality of the wire work in the film. The car crashes were all practical, finished as comps and rig removals. We went as far as time permitted to repair the more egregious dummies flying as Greer plows through them down the sidewalks.
Dead surrogates
The final surrogate collapse at the end of the film was the work of Brickyard VFX. For the long overhead helicopter shot of Boston, explained Brickyard visual effects supervisor Sean Devereaux, we added thousands of dead surrogates on the ground, CG cars, smoke and tonnes and tonnes of CG satellite dishes on rooftops. That shot also required a 350% push-in at the beginning, so we had to do a full 3D re-creation of the move with CG trees and matte paintings and CG buildings.
Before
After
We did a full brand new matte painting for anything that wasn’t moving in the shot, continued Devereaux. This included the fountain and the grass, the streets, the sidewalks, the building facades. We animated trees in Maya using Paint FX and rendered them in Renderman. All the surrogates on the ground were done in Maya and mental ray. Because they weren’t moving we were able to copy and paste some of them. There were 400 different varieties of people, so at some point it was easier to stop modeling new clothes and just shoot people on greenscreen. Along with what ended up becoming an 1000 frame final shot, Brickyard also contributed split screens of Bruce Willis appearing with his surrogate, a house establishing shot and several 2D digital makeup effects.
Before
After
Ultimately, all of the shots in Surrogates were, according to Mark Stetson, planned as being filmed live and then only augmented with visual effects where necessary. I personally like working with filmmakers and stunt units in this way, who want to push to achieve as much as possible practically. I think that helps the realism and broadens the field of ideas. The trick is to know when it is most economical to pass the baton to VFX.
Related Links
Official Surrogates website
Synthespian Studios
The Moving Picture Company
Brickyard VFX
Industrial Light Magic
Sandbox F/X
