When Alex Garland’s Ex Machina won the Academy Award for Best Visual Effects in 2016, it was perhaps considered a major surprise in the VFX community. But the win also highlighted just how crucial the visual effects were in bringing the story of the A.I. robot Ava, played by Alicia Vikander, to life.
Now Garland and Ex Machina’s Oscar-winning visual effects supervisor Andrew Whitehurst from Double Negative have re-teamed on Annihilation, about a group of scientists who investigate a mysterious quarantined zone called the Shimmer. While there was a larger visual effects effort required for this new film – which involved the Shimmer itself, an array of mutated creatures and an alien humanoid being – the VFX again remain highly integrated into the storytelling and crucial in realizing Garland’s vision.
vfxblog caught up with Whitehurst for an in-depth discussion about making the film and the visual effects (realized by Double Negative, Milk VFX, Nvisible and Union VFX), including shooting in real locations in the UK, designing with Mandelbulb 3D fractals, filming with practical effects and stand-ins for much of the creature work, and delivering a beautifully choreographed ‘dance’ for Annihilation’s stunning conclusion.
***Warning: this interview contains major plot spoilers***
vfxblog: Let’s talk about the Shimmer itself – how was its kind of petrol slick look designed?
Andrew Whitehurst: The way that was described in the screenplay was like a heat haze, but with prismatic effects, and distorting the background but having some kind of surface quality so that it was quite a tangible moment when someone passed through it. We look at so much reference. We looked at a lot of paintings, done by Turner, the way that Turner painted skies, we thought maybe there’s something in that because there’s a lot of strong colour in Turner’s sky. We thought, ‘Okay, maybe we could grab that.’
And we looked at natural phenomena, we looked at things like nacreous clouds and sometimes you’ll get an effect on contrails and aeroplanes if they’re the right kind of water vapour and the sunlight hits them in the right kind of way, you get these beautiful rainbow patters you can use as well. We thought, ‘Well maybe that’s something we can use.’
None of those things felt ultimately like they tied into the rest of our film. So what we’d been doing simultaneously was the design of the most alien manifestation of the alien that you see in the chamber at the end, which is based on a Mandelbulb 3D fractal shape. So we got really into the shapes that you can get out of mucking about with Mandelbulbs and sort of tweaking the maths to get these. I guess almost like a Gaudi cathedral, so it’s procedural, but it has this organic quality, but also this architectural element to it. I think Gaudi is probably the best analogy I can think of.
Because we’d been working on this other thing in parallel, we thought well, what we should try and do is we should try and get Mandelbulbs in the film as much as we can. One of the places we could look at doing that would be the Shimmer. Or specifically, the exterior views of the Shimmer. So what we did was we effectively unwrapped a Mandelbulb, made it into a flat surface, and then this has different Mandelbulbs running at different speeds, so you always have this churning shape that use some structure in it, but then it would go in one area, but then you see some structure somewhere else.
That was what gave us the physical form of it, and then in terms of the look of it was just playing with how much we wanted to effect the background, and how much do we want to make it feel like a petrol slick, how much do we want to make it feel like those paint jobs you get on sports cars that give you a different sort of rainbow colour, and we got of those sorts of effects into it.
vfxblog: Was the original Mandelbulb geometry done in Houdini?
Whitehurst: It was done exactly in Houdini. So when I started on the film, I was using Houdini on my laptop, and I put together a Mandelbulb in that, and we started playing with it in the production office. And so, Alex would tell me, you know, ‘It’s moving a bit fast, can we slow it down?” So we’d tweak things. I could show him quite quickly with that, without running it through a render farm at DNEG.
Also, the art department wanted Mandelbulb shapes, including for at the end of the film. What we had to do was sit with the art department and come up with a Mandelbulb shape that they liked, and then produce high-res geo’s from that, which we exported out. They’d then have CNC machine moulds made, and then fibreglass sections that were then multiples of that were then built to make the set.
The set itself actually is a Mandelbulb there, and also, our department would take other Mandelbulb shapes that we’d made and all of the spores and light that you see dotted around the landscape also have the Mandelbulb shape in them.
So, back to the Shimmer, we started thinking, ‘Well, actually maybe this is the motif design-wise for the movie, which is then just the Mandelbulb effect. Okay, well, we should be putting that into the Shimmer as well.’
vfxblog: The Shimmer is like a wall around the area but while they are inside you also see so many instances of it. What kind of compositing challenge was that to have the Shimmer always present?
Whitehurst: We decided that the design problem that we have is that the Shimmer is just described as the Shimmer, and that you actually need the Shimmer to perform two narrative functions. You need it to do shock and awe out front when they first see it and when they’re walking towards it like, ‘What the hell is this thing?’ But once you’re inside it, you can’t have that throughout the entire length of the movie, otherwise it’s going to completely lose its impact. But also it’s very distracting, so we needed to think of something else to do to be inside of the Shimmer that we could add and dial up and down to taste depending on how present we want it to feel in any given scene, because in some scenes you just literally want to be focusing on the performances, so you want to make that dialled right back.
We actually came up with a few approaches. One of the camera prep guys, we all brought in things like prisms and crystal balls, and interesting pieces of glasses. We got Panavision to give us every weird very old lens that they could lend us for a day, and Andy Lowe the gaffer got some amazing old theatrical lights that were designed to add lighting effects for theatre. We spent a day firing weird lights through the pieces of glass, shooting the movie with old lenses. We got these amazing flares and aberrations and strange shapes, and we shot a lot. We went through that footage later and picked the more interesting elements of those lights, and we used those to direct things the film over, particularly when we see things like the sun coming through trees, we would generally put one of these flare effects over top of it.
Then in areas of mist we would add subtle rainbow shapes into the background, and then whenever you had water puddles, or the scene where they go down the river, in highlights on that we would add rainbow effects to break that up. It’s always there, but it’s totally not drawing too much attention to itself, so it’s something that we hope subliminally sinks into the viewer as they watch over the movie.
vfxblog: There are so many exterior scenes in the film, where was it actually filmed?
Whitehurst: It was a ten week shoot, and it was five weeks out on the road and five weeks on various stages at at Pinewood. It’s all shot in the UK, but we shot in a lot of different locations. The fishing hut is in Wintergreen Park. The lighthouse and the breach at the end is Holkham in Norfolk. The base is actually a US Air Force base, it was one of their cold war bases where they stored nuclear bombers in Oxfordshire, so we were able to use that. That was helpful, because all of the fixtures and fittings are actually American there, but otherwise it was the art department who did an absolutely amazing job.
Mark Digby, the production designer, and Michelle Day, the set decorator, did such a great job designing the fishing hut, for example. They did a lot of visits to Florida and Louisiana, and took tonnes of reference photos. There was actually one of their reference photos where they found this fishing hut that was half submerged, because that wasn’t the original thought. They basically found this one and went, ‘This is amazing,’ and everybody went, ‘Yes, it is amazing.’ So the fishing hut was redesigned to be this half submerged thing.
That was built in Wintergreen Park, and they would build also fibreglass tree trunks in Cyprus trees, which we would then top out. So the visual effects work there was to generally change the more distant tree lines so that it felt more Florida and less Oxfordshire, and top out set and built trees, and adding foliage here and there as well.
vfxblog: That’s where we see the alligator. What’s great about those shots is that it’s almost filmed just as if it might be there for real. How did you achieve those shots?
Whitehurst: It’s interesting because the whole design process of all of the creatures was different. The bear, we led the design and then worked with special effects. The alligator, Tristan Versluis, from creature prosthetics team, built a full size four and a half metre gator. We scanned that, and that was the basis of our digital build. What it meant was that on set we had a full size creature that was great lighting reference, and it also gave the cast something to engage with.
The other thing for us in visual effects was that it allowed Haley Williams’ special effects crew to also get casts of these things and build that. So for the shot where it slides into the water, Hayley’s team built a buck that was articulated and weighted, and it had a cable attached to the front of it, and they literally hauled it out. They sunk a pulley under the water where the cable ran through it, and they just pulled it out through the front of the fishing hut and it slid into the water.
All of the ripples that you see running away from the gator that then go and rush into the bank and disturb the foliage there, all of that’s real. And you think of the nightmare that trying to do that all in CG would be, you’d have to end up replace every bit of foliage that’s in contact with the water. It would have been hard for the DP, Rob Hardy, to properly frame up the shots because there wouldn’t be anything in frame. But because we had this buck, even though we completely remove it, and the creature is in every shot is entirely digital, it meant that we could use the practical element as a lighting reference, but also it meant that the shots were better because they were framed with something there.
The interaction with the environment was better because we had something photoreal, and yes we added CG splashes and extra bits, but it also give us a matte line for wherever the CG gator should interact with the water. A lot of what you see there was practical. It’s the same story when it goes out of the water – Hayley’s team built a modified air ram that could be fired out of the water and then sort of slap itself down. Again, it’s entirely digital, and then we had to replace grass in the foreground because it starts walking up through the grass and pushing it aside, but a lot of the splashing that you see at the beginning is practical, and then augmented with CG water.
I’m always very keen wherever it’s possible to have something practical, particularly where we can get interaction, whether it’s the lighting interaction, or whether it’s actually physically interacting with the environment. I just think it gives it such a huge leg-up when we start doing CG work. Even though it’s a pain to paint it out, I think the advantages that you get from that physical thing physically being there really outweigh the legwork of having to do the paint to remove it.
So, for the gator, we had as much practically there as we could. The bits where we couldn’t have anything practical was the actual section where it’s advancing on the cast and they’re shooting at it, because the creature’s too big really to have somebody in a suit. With that we kind of had to mime, and say, ‘Okay, eye lines are about here.’ And then we assembled the sequence in the edit, really, and sort of worked out what works better and what didn’t.
vfxblog: When they’re interacting with it once it’s been killed, is that more of a practical gator or is that still CG?
Whitehurst: It’s a practical gator that they’re interacting with. Now, the whole story of the film on in terms of working on it was that every design decision that we made ended up being quite deliberate, because you wanted a sense of strangeness ramping up, and until you’ve actually got the entire movie on a timeline in the edit and you can watch it, you’re not aware of where some scenes bounce as being not really enough or too weird, and you need to make adjustments. The gator was originally a pure albino gator. That was what Tristan built, because that was in the brief, and that’s what our original CG version was. Then when we started working, and we looked at the rest of the film and how the rest of the film was going, we realised that actually the gator needed to look sicker. So quite a distance into post production we started adding aberrations to the gator, so it’s got some areas of pigmented skin, and other areas where it’s albino.
We added lesions and some other illnesses to it, a sort of sickness around the eyes where the skin was cracked and bleeding, and other things like that. So even in the moments toward the end of the scene where the cast are physically interacting with the practical build that Tristan’s team did, we still ended up having to augment that with all of these extra skin damage on all the rest of it, because that’s what actually the scene needed once we put it together and had it located and situated in the cut.
It was still practical, particularly the parts where Gene’s character is sort of holding the jaws open. That was a large amount of practical there. The inside of the mouth is all CG because we ended up having to add more rows of teeth, so that would actually be towards CG.
vfxblog: Let’s jump to the bear, firstly where it attacks Cass outside the camp?
Whitehurst: In terms of shooting it, we got some really beautiful storyboards which gave us a starting point. Then in terms of actually physically shooting it, it was down to Jo McLaren, the stunt supe, and her team, and then working with Tuva Novotny, who plays Cass Sheppard. So Tuva did as much as she could in being yanked forward on a cable. That’s how the moment where the bear reaches down and bites her and chucks her to one side was done; that’s Tuva being hauled around on a cable. Then the shot you see a couple of moments later through the night vision goggles is a stunt performer being pulled on a cable physically across the grass that we then added a CG bear onto later.
I think it’s literally only those two shots that we actually ended up doing. We shot some other ones and we actually started animating some other shots that made the sequence a bit bigger where you glimpse the bear and that kind of thing, but actually in the edit, everybody felt, ‘Well actually we should just make less of it here. We should let this sort of shock and then it’s gone.’ Then obviously we’ve got the second major bear scene later, and it’s like, ‘Well that’s where we should reveal it for what it is.’ The decision was made to sort of limit the first appearance of it, literally just blink and you’ll miss it.
vfxblog: That second appearance, I could barely watch because it’s so tense. Tell me how that was actually filmed and pulled off.
Whitehurst: DNEG designed the bear, and we did an initial sculpt of that, which we passed to Tristan, and Tristan and his team did a full sized head and neck finish. That was silicone and painted and absolutely beautiful. We also from the same mould built styrofoam versions of the bear with some sense of the scale of the back of it that a performer had control from the back. As you might imagine, it was a big guy who could wear that and that would give us a sense of the scale and the size of it, so that when we shot the scenes, we, again, helped Rob Hardy and the actors to have something physically there that was the right kind of size.
We painted it dark grey so that we weren’t getting weird spill, but it was blocking lights where it should do, because the whole scene is lit with these very source-y work lights that it’s permanently walking in front of, so if we knew that we wanted something practically there because it’s blocking the light out of the rest of the room intractably in the correct way. Also when it’s getting closer to the characters, you’re getting a sense of the occluding shadows on them as well. We don’t want to be doing that kind of stuff in post, because to my mind it never really works.
So yes, we had a physical form in a buck costume for when it’s walking around, and then for the moments where it’s nuzzling against the characters, that’s when Tristan’s puppeteered head was used. So that’s how the majority of the scene was shot was with those bucks, and then we would shoot clean plates, and then paint over what we needed to, that’s where we’d put the CG bear in. Then at the end where it gets shot, again that started with casts from Tristan’s skull. Hayley’s team built three heads that were painted where they just filled with gore and guts and Lord knows what, and then blew them up, because that’s what Hayley likes to do.
We filmed most of the moments where you see it getting shot, a lot of the bits that you see flying about are practical from that getting destroyed. So again, wherever we could use something real we did. That interaction lighting-wise and physically with the environments and with the cast acting actually helped us a lot. In some of the shots it made the animation harder because we were tied into a physical timing that, maybe if we’d been doing everything with carte blanch, we might not have done.
So there was a little bit of to and fro between us and animation and with the animation team, and the edit where we would try things out, and I’d go down to the edit, and say, ‘Alright guys, look, you know we’re trying stuff out here, because this shot, it would be great if we could get another 36 frames on it, because we really liked to just have this be a head turn or something.’ And they’d adjust things, send it back. When we thought of something else after that, they’d go, ‘Actually okay, well, we’ve taken this shot, and we’ve trimmed this one down so that you can shift that head turn a bit earlier.’ It was a back and forth for three months until we felt that we’d locked down the sequence.
But because we had a physical performer, they could cut the sequence before we had any animation at all, and get a sense that, ‘Yeah, this is going to work. We’ve got all the beats pretty much. Okay, let’s do a first pass of animation and see what we’re, you know, light or heavy in the one part or another, and then adjust.’ That’s what we did, and then we just iterated on that, turned the sequence up and down, and looked at what’s happening at the same time, and then we start to being able to sort out the animated bear with the rendered one.
And then obviously we did huge amounts of comp work to clean up the backgrounds and then put the bear in wrestling the actors or the foreground elements, putting the flares and all the weird apparitions from the lenses that we used back in, and all of that kind of stuff, just to seat it in there. So we did a huge amount of work, but I still think it was worth shooting with the practical element there because of what it gave us.
vfxblog: If we jump to the end of the film, we talked about the Mandelbulb creation, but what about the actual humanoid being that we see?
Whitehurst: The idea was to have it as a dance almost, so the humanoid creature was attempting to mimic or understand what Natalie Portman as Lena is doing, and so Bobbi Jene Smith, who’s a dance choreographer, she’s very well known for turning everyday human activities into performance. She worked with two dancers -Sonoya Mizuno, who played Kyoko in Ex Machina, and is also one of the students at the beginning of Annihilation.
Sonoya was the humanoid, and then Kristen McGarrity, who was Natalie’s stunt double, was also a dancer. Those two worked with Bobby, and they choreographed over a couple of weeks this scene, and they just did it in the production office. So they’re in the central area of the production office, and with masking tape on the floor, they taped out the circle that was the diameter of the lighthouse set build.
Then, I think on the afternoon of the first day, they said, ‘Okay, we’ve got something to show, and Alex and Rob and I, and the producers, were all invited, and we sat down, and then you’ve got two dancers wearing tracksuits acting this scene out, and it was amazing. Right there, and no costumes, no nothing, in an office at Pinewood. Just the physicality of it, and the beauty of it, and the timing of it was mesmerising.
You looked at it and you thought, ‘Well you know, we’re really going to have to go somewhere to screw this up now.’ I mean it’s amazing what we’ve got. Two professional dancers in, you know, not even in their costumes, and it’s amazing. So the whole design of that was Bobby’s choreography, and then Rob and Alex figuring out how to shoot it.
For the design of the humanoid – Jonathan Opgenhaffen did most of the concept work on that – and we looked at a lot of references how different cultures, particularly very old cultures, had represented the human form. We looked at a lot of cycladic sculpture and took reference from that.
We actually went through very few iterations at the concept art stage. Once we said, ‘Well, this kind of works, we should build it.’ We knew that Sonoya was going to be the humanoid on set, so we scanned her and we built the humanoid model to match her proportions, because we were going to try and do what we did with Ex Machina where we body tracked the performance and that’s what we would see, so it wouldn’t be animated.
So on set Sonoya was wearing a sort of black all-in-one skin-tight costume with tracking markers on it, and that’s how she performed the whole scene up to the moment where she’s on fire, and at that point, Sammy Sheldon, the costume designer, had another suit made that had panels of LED lights that could be trigger individually. It was actually interesting, because they were painted together, it actually made Sonoya’s movements not so fluid, which was good, because actually from the performance standpoint, this character is now on fire, and it felt like Sonoya was routinely struggling to move, and it was true, because Sonoya was struggling to move, because she was trussed up with all of these LED lighting cables.
But again, we had the LED lighting, which put the interactive light onto the set, so when we add the CG fire onto the CG humanoid, we’re not having to also completely relight the entire environment as well. Also the LED light cables adds really amazing flares that we kept and that we sort of added the same style of flares where we were adding CG flame. It had this very otherworldly quality. It was very beautiful.
Then the whole section where the humanoid pulls down back into the chamber, that was thought of on the afternoon we did it, and it was one of the few times we shot overtime on the movie. Originally, as soon as the door closed on the lighthouse and Lena was outside, you never saw the humanoid again. At least that’s how it was in the original screenplay.
But the thought was more, ‘Actually no, we should make this into a moment.’ And so it was just designed, shot, performed, absolutely on the fly, and it’s one of my favourite moments in the movie now, this moment where she crawls up on that central dais in the chamber, and the whole chamber is ablaze, and she just sort of leans back in this sort of moment of submission, and the frame whites out, that now there was just so much heat inside that. Absolutely one of my favourite shot in the movie, and it was just made up there and then.
All of that stuff in the chamber, nothing was set on fire, so all of the flames are CG. So those last shots were basically replacing the entire background with CG because we were relighting it so much because of the amount of flame that you just kind of had to go full CG route there.
vfxblog: Before she’s on fire when she’s just interacting with Natalie Portman’s character, what were some of the challenges of rendering and integrating a character that looks like that with the sort of rainbow colours, and the sort of Shimmer type feel?
Whitehurst: It was very tough because the way that the grade was pushed, when it was shot it had a very sort of even-y tungsten feel to it, and in the DI it was pushed to being something much more neutral and white. The effect with that amount of colour correction actually meant that when we were adding a lot of these rainbow colours, we were losing them, they were effectively getting graded out.
So what we ended up doing was, Asa Shoul, the colorist, gave us a, ‘Here’s my pass of this is what I kind of want it to look like,’ and then I just went through in NUKE and made some grade nodes to the place, and sent back to her and said, ‘Look, is this close enough as a base that you can then work with?’ We’d tweak things, and then eventually the decision is made: yes, that is okay.
Then that meant that we could maintain all of those chroma values when we copy humanoid back on top. So the iridescent pass, we actually rendered as a separate layer so that we could have full control of that in the comps. We had a lead metal base look, and then this extra layer for the iridescence that we could dial up, and Alex’s taste definitely towards more psychedelic at that point, so we really turned that up to 11. But because we had NUKE for that pre-grade, it meant that all of those colours were preserved, and then did survive into what you see on screen. Because when you have anything with literally a sort of rainbow of colours in it, if you start pushing the grade around, more than a tiny amount, you will just lose colours, because inevitably, it’s a sort of lossy process.
vfxblog: At some point I think I saw on Twitter that you said this was one of the hardest projects you’ve worked on – what were the biggest challenges?
Whitehurst: There was just a huge amount of design work because it’s an escalating tale of oddness and strangeness where you see things in the edit, you then have to start changing them. So the whole design process wasn’t always designing upfront and executing on that, we were actually really designing a lot of things quite late in the day.
An example of that is on the beach when Lena walks to the lighthouse and you’ve got those crystal trees, originally the plan was – someone in the art department found these really beautiful photographs taken underwater of divers coming into the water, and the thought was well on the beach what we do is effectively in that was all these humanoid shapes were smashing out of the beach, and all of these columns of sand and salt crystals with these human forms at the top.
We built bucks of these sort blobby kind of columns that were on the beach. It’s what we shot with, and we built and developed these things, and we were actually starting to put them in the shot. We were sitting in the editing room going, ‘These don’t feel right. Now we sort of see it in the context of the whole rest of the movie, and we can sit and watch it, and it doesn’t feel right. Doesn’t feel like they’re connected to anything else in the film. What are we going to do?’
The scene prior to that beach scene, Lena’s walking through a forest, and we Lidar’d that forest, in case we needed to track it, more than anything else. So we’ve got these Lidar models of these trees, an obviously Lidar doesn’t coat very well with volumetric type materials like hair or leaves. So the Lidar models were sort of beautiful trunks and branches, and then there’s some sort of weird horrible blobby shapes where all the leaves were. But actually that had a very pleasing, sculptural quality to it, and we thought, ‘Well maybe that’s what we should do.’
So we cleaned up the leaves that were kind of unrenderable, but they still had the same blobby shapes of the Lidar artefacts. Then did the loop again to make them crystals, and then painted on the bucks that we put on set, and put in the trees, and that’s what you end up seeing. Then after we’ve done that, we thought, ‘Well, it looks a bit sparse.’ So then we designed and added those shards blasting out through the sand around the trees, with that trees sort of start out of the shards and they grow into the trees.
All of that was really quite late into post, but it felt right. It was the right thing to do for the film. There was a lot of that – it just meant a huge amount of work for everybody. But you had to remember that we didn’t actually have tonnes of money, so we couldn’t just continue to chuck money at the problems. We had to be as smart as we can. That’s the story of production-wise, and post production-wise of the movie. We all went on a massive journey pretty deep into the Shimmer ourselves to make this movie about going into the Shimmer.
