Multimedia
Blogs
Forum
Affordable HD
Whitepapers
Newsletters
Events
Advertisers
Blogcast
MillimeterAI Armies
Nov 1, 2002 12:00 PM, by Ellen Wolff
Animating Intelligent Characters for the Two Towers
 In Peter Jackson’s The Two Towers, CG soldiers, created using WETA’s Massive software and rendered in Grunt, react to their environment with patterned responses based on AI technology. |
It's a scene that likely exceeds the wildest fantasies of Cecil B. DeMille: 70,000 soldiers marching across the screen in director Peter Jackson's The Two Towers, the second installment in New Line's film trilogy The Lord Of The Rings. These aren't cut-and-paste clips of one photographed regiment used over and over again. They're digitally animated individuals who run, climb, fight, and die — driven by software based on principles of artificial intelligence (AI).
Crafted at Jackson's WETA Digital facility in Wellington, New Zealand, these CG hordes were wrangled through programs called Massive and Grunt (General Renderer of Unlimited Numbers of Things). This software, developed over several years by Steven Regelous and John Alitt, has enabled WETA to produce animation of unprecedented scope. “We could tell 10,000 ‘guys’ to stand on one side of the battlefield, facing another 10,000 guys wearing red armor,” says The Two Towers visual effects supervisor Jim Rygiel, who won the 2001 Oscar for supervising the effects in The Fellowship Of The Ring “We'd then tell the first set of guys, ‘When you see anyone in red armor, run toward them and kill them.’”
That these individuals were programmed to “see” and “hear” makes WETA's approach more powerful than what's been done before, believes Rygiel. A veteran CG practitioner, Rygiel counts among his assignments the computer-animated penguins created at Boss Film for 1992's Batman Returns. “Even though that was called artificial intelligence, what they had was collision detection. Each character ‘knew’ that if its center of gravity was getting close to another creature's center of gravity, it would have to turn away. We gave it the random choice of going left or right. There were lots of random decisions going on at that point.” Rygiel's work was an offshoot of the research presented by Craig Reynolds in 1986 on flocking birds, which influenced many motion-picture animators.
A Quantum Leap
AI-driven characters have advanced greatly since then.
“There's been a thousand-fold leap in this technology from the first Lord Of The Rings film to the second one,” says Rygiel. “Before, we mostly showed these characters at a distance because they barely would have held up in the video game world. Now the camera shows almost full head shots of some of these guys.”
One of the most dramatic examples is a battle at Helms Deep, where 10,000 soldiers march upon the fortress, throw ladders against it, and scale the walls, all while being pelted by arrows. To achieve such complex animation, the WETA team began by motion-capturing numerous behaviors, using mo-cap technology set up in Wellington by Atlanta-based Giant Studios.
The team amassed libraries of motion information. “We'd then teach our AI system what we wanted the characters to do,” explains Rygiel. “For example, the [AI software] brain says to a character, ‘You're a climber. When the ladder goes up, you run toward it and start climbing as fast as you can. When you get to the top, jump off — using the various motion cycles that we've given you.’ There are also variations on this theme. One guy will do his run-to-ladder cycle B, then go into a climb-ladder cycle A, and then use jump cycle C. What we can do on top of that is take these different cycles and add a hyper-random factor, where we can change the length of their limbs and their gaits. So ten cycles can become a couple of thousand cycles. We can also alter how fast or slow they're going, so guys who are running fast can avoid the slower guys and pass them.”
This kind of variability is essential “because if you took even 100 cycles and put them over 10,000 guys,” says Rygiel, “they would look exactly the same. The key is getting extra randomness in there.” The Massive simulation software permits animators to go in and tweak the animation as necessary. “For instance, typical dailies might show 10,000 guys running across a field, “says Rygiel. “The first time we'd see it, they'd all be running at exactly the same speed, perfectly placed in relation to each other. So we'd say, ‘We need more variation in the groupings. Let's have a cluster of 30 over here, and in that cluster, half of them are slowpokes.’ Of course, out of 10,000 guys, we'd have some brain-dead ones that actually fell off the cliff instead of avoiding it. That happened all the time. Then we would say, ‘Don't have that one show up for work tomorrow.’ We'd literally kill that character out of the sim!” Usually, Rygiel could see changes by the following day. The ability to render the images quickly is what makes such simulation workable on a film-production schedule. “The Grunt renderer for Massive is really fast, and it's made for doing these duplication simulations. It allows us to assign different levels of detail, depending upon where characters are in relation to camera.”
WETA typically models its CG with Alias/Wavefront's Maya and renders with Pixar's RenderMan, using primarily Intel-based computers running Red Hat Linux. But standard practices wouldn't have worked for efficiently rendering these huge armies. “If we did a sim and rendered it in RenderMan it would take three weeks — if we could have rendered it at all,” says Rygiel. “Imagine telling people that you want to render 70,000 things in RenderMan. Their minds would explode.”
The Turning Tide
Not surprisingly, word of WETA's approach has the industry talking, especially about where we go from here. “Software that enables characters to react to their environment with patterned responses is the only solution for this kind of work,” says Eric Armstrong, Sony Imageworks' animation director on Stuart Little 2. “You can't have animators keyframe 10,000 characters. It can't be done.”
Armstrong has a personal appreciation of complex animation challenges since he previously oversaw ILM's galloping herd of Galli dinosaurs in 1993's Jurassic Park. He has strong opinions about intelligent software technology's impact on the industry in years to come. “Proceduralized characters that think and emote and solve problems are eventually going to become the norm,” Armstrong says. “You'll have a system where you'll be able to say, ‘I want a character of this height and this body type,’ and it will appear on the screen. Then you may say, ‘Make the waist a little thinner and the shoulders a little broader, and give him glasses. Nope, make them horn-rimmed glasses.’
“I don't think we're going to replace real actors with digital actors. It's more a question of whether the Bugs Bunny of 2020 will be keyframed. I don't think there's any way that it will be.”
When Armstrong sketched this scenario at the 2002 SIGGRAPH panel “Animation's Turning Tide,” he braced himself for a hostile reaction. “I know that some people view this as an evil evolution,” he says. “That it gets rid of the creativity in the animation process. The truth is, it will make it easier for animators in the beginning. But eventually it will lead to the point where animators, as we know them today, will become superfluous. My feeling — and it's not a statement as to whether I think this is good or bad — is that this will happen within 20 years. There may be some people still doing keyframe work for special-case CG projects or cel animation. But the majority of characters will be controlled by directors talking into microphones, giving commands like they would on set. It will be a realtime simulation, rendered in high res.
 Digital characters are migrating from games to the film and television industry. BioGraphic Technologies’ new commercial software AI.implant tracks CG characters and gives animators feedback in realtime. |
“I'm not saying all animators will be gone.” says Armstrong, “But instead of having an army of 25 people lighting one shot at a time, you'll have one guy — like a cinematographer — who'll set up the lights for a shot. Then the director gives the commands and the simulation runs. If they decide it would look better if they moved the lights, they'll run the simulation again with different lighting. There will be three or four people doing this instead of 150.”
Much to Armstrong's surprise, no one at SIGGRAPH challenged this scenario. His fellow panelists said they offered no rebuttal because they didn't really disagree. Armstrong thinks the success of AI-driven characters in videogames has opened a lot of minds in the film industry to the possibilities for interactive control of virtual characters.
“Game people are the ones who are driving it. They have to do this to be competitive. But the live-action industry will embrace this approach like crazy. When they can feed their ideas verbally into a system and don't have to wait two weeks for animation to come out of a ‘black box,’ they'll love it.”
Reeltime Animation
At least one AI game creator agrees. “Production studios are really interested in this technology for television applications, where efficiency is important,” says Lorne Lanning, co-founder of Oddworld Inhabitants and lead designer of “Abe's Oddysee.” Lanning expects that the idea of running realtime character simulations will soon become a reality. Lanning, a former Rhythm & Hues employee who uses Maya for Oddworld's character animation, puts this evolution in perspective. “The animator of yesterday keyframed every shot, and occasionally got to reuse cycles,” he says. “What will happen in movie animation — and what's happening in game animation today — is that animators are not animating every frame of what's happening. They're animating the initial cycles of moves, and then the code is driving how that follows through — knowing when to use different moves and blending them all together. What that means is that during a 20-hour game experience, maybe 5% of the time that the characters are animated on screen actually represents time invested by human animators. The rest is recycling through that workload to create longer periods of content. In the future, the percentage of human keyframing work will decrease, but it will still be required for the initial construction of those moves.”
Lanning concurs with Armstrong's expectations, but only for the secondary cast. “The bar for featured characters continually gets raised,” Lanning says. “The smartest code in the world is not going to deliver a good performance. Hey, you've got human brains inside thousands of actors in Hollwood and only a handful can really act! But what can happen is that tasks like pulling a lip by moving polygons may involve a more library-oriented set of modules, where even a non-animator can ‘slide the anger up’ if the director wants, once the character has the embedded anger attributes in him.”
Both Lanning and Armstrong agree that the arrival of commodity AI-animation tools will drive this trend, moving it beyond the handful of studios that can afford to develop proprietary code. “You'll be able to go to places with names like People Builders and buy their software,” Armstrong predicts.
A Commercial Option
Commodity AI-animation tool options are already emerging. Montreal-based BioGraphic Technologies now offers AI. implant, which plugs into both Maya and 3ds Max and runs on Windows and Linux computers. The software tracks each character, chooses the correct animation cycles, scales and blends them, and gives animators feedback in realtime.
“AI is the final animation technology to emerge,” observes BioGraphic founder Dr. Paul Kruszewski. “It's been around for awhile, but it hasn't been practical on an industrial scale. But game engine technology is getting shoved up into TV and film production. It will not only make animators more productive — it will also let them do cooler shots.”
Kruszewski admits that the biggest challenges are in the emotional realm. “We can't model emotions, though if I know a character is slightly angry and also tired, I can blend two animations,” he says. “We're far from being able to match great actors. But bad actors? We'll get there quickly!”
Armstrong believes it's only a matter of time. “We'll extrapolate from physically based stuff to emotional things,” he says. “The reality is that this approach will be cheaper and faster. And as simulations become increasingly elaborate, people will rely on them more and more. All you can do is embrace this new technology and learn how you can use it. It's going to happen.”
FEEDBACK
To comment on this article, email the Millimeter editorial staff at mmfeedback@primediabusiness.com.
Continue the discussion on “Crosstalk” the Millimeter Forum.