• Blogs
• Forums
• Tradeshows
• Partners
• White Papers
• Podcast Show Notes
• Events

Skin Deep

May 1, 2004 12:00 PM, By Ellen Wolff

An Academy Award-Winning Idea in the Real World

The phrase, “Beauty is only skin deep” has taken on new meaning in the world of CGI. Until the last couple of years, the process of creating CG skin was painstaking and the results were often unconvincing. Skin usually looked hard, opaque, fake. Then software that simulated translucency changed things dramatically — so much so that the Motion Picture Academy recently honored six individuals with Scientific and Technical Achievement Awards for developing this breakthrough approach. The results of their efforts have included Weta Digital's astonishing Gollum in The Lord Of The Rings, Industrial Light + Magic's virtual Arnold Schwarzenegger in Terminator 3: Rise of the Machines, and the casts of PDI/DreamWorks' Shrek 2 and Pixar's upcoming release, The Incredibles.

The award-winning concept, subsurface scattering, is easy to observe in real life. Press a flashlight against your hand and you'll see light diffusing under the skin. Or notice the rim of color along the edge of ears when they're backlit by the sun. These “real” skin cues are so familiar to us that when they're missing, we sense something's wrong.

CG characters looked wrong for so long because conventional rendering techniques operated as if light rays bounce off skin like they do off metal surfaces. In reality, light rays penetrate skin, scatter beneath the surface, and exit at varying angles. This was understood as a medical phenomenon, but translating this knowledge into production-ready computer code took some doing.

The Aha! Moments

“The whole subsurface idea — the big aha! — came from the observation that organic materials are fundamentally different because they have lots of water in them, and light goes into water,” says Stanford University professor Pat Hanrahan, one of this year's Sci-Tech winners. Hanrahan, who had been a founding employee of Pixar and an architect of its RenderMan software, first published his ideas about subsurface scattering in 1993. After being joined by researchers (and fellow Academy honorees) Henrik Wann Jensen and Stephen Marschner, they arrived at the second aha! moment.

From medical research, Wann Jensen learned that laser light was being used for the diagnosis and treatment of skin disorders. “Henrik realized that if we took a similar approach, we could make surfaces appear translucent,” says Hanrahan. “Of course we didn't have the ability to make a model like Gollum, so we used milk.” Glasses of milk that were rendered with subsurface scattering had a translucency that looked far more convincing than the paint-like appearance of conventional CGI. The technique even demonstrated visible differences between whole and skim milk.

Wann Jensen, whom Hanrahan calls “an incredible evangelist,” was on a mission to make the technique practical enough to be widely used, especially for rendering skin. Tracing the trajectories of individual photons of light, which was how the process was originally conceived, was far too expensive for film production.

“It seemed like it shouldn't take hours and hours of computing to get a smooth result,” says Wann Jensen. By January 2001, he developed a more practical method and submitted it to SIGGRAPH for presentation at that summer's annual convention.

The Buzz

Word traveled quickly among the digerati. Across town from Stanford at the time was PDI/DreamWorks' studio, and Wann Jensen visited there a few months before his SIGGRAPH presentation. “Not even halfway through Henrik's talk I had an idea for an optimization of his technique,” says senior effects animator Juan Buhler. The two began collaborating, developing the technique to work with PDI's proprietary rendering software — an approach that led directly to the images now on view in Shrek 2.

The buzz quickly reached ILM, where Wann Jensen visited that spring. ILM computer graphic supervisor Christophe Hery says the visit “was an eye-opener.” For two months afterward, Hery focused solely on developing a subsurface scattering technique that would work with RenderMan in the ILM pipeline. Since production of Star Wars: Episode II was already underway, Hery's method wasn't unveiled until 2002 on the character of Dobby, the elf in Harry Potter and The Chamber of Secrets.

“There haven't been any major shows since where ILM hasn't used subsurface scattering,” Hery says, including its application to Yoda in the upcoming Episode III. Hery's continuing development of the technique has been so impressive that he earned a Sci-Tech Award this year.

Not surprisingly, Buhler and Hery weren't alone in embracing the idea of using subsurface scattering in film production. Pixar software engineer Brad West recalls seeing Wann Jensen's 2001 SIGGRAPH demo — a presentation of before-and-after shots that prompted audible reactions from the crowd. “It was so compelling that you didn't need to be technical to get it. You can show pictures like that to a director and say ‘This is what we need to do.’” West has seen many technical demonstrations during his seven years at Pixar, but this one really dazzled him. “Seeing those images, you said, ‘Omigod, I've got to use that immediately.’ We all rushed home to get to a computer as soon as possible.”

Variations on the Theme

Just as PDI/DreamWorks and ILM developed their own formulas for implementing subsurface scattering, so did Pixar. “We ended up doing a further approximation of Henrik's approximation,” says West. Called Gummy, it was used to create translucent fish for Finding Nemo.

Of the patent pending on this approach, West says, “It's kind of funny to read through the bizarre legalese of the patent application until the last page, where there's a graph of a light wave going in and scattering — and a picture of Nemo.”

Stanford's Hanrahan isn't surprised that digital filmmakers readily adapted subsurface scattering to their needs. “In the end, there's just one little formula,” he says. “If you take that — along with the other expertise that the studios have — it's completely digestible and easy to use.”

Perhaps the most startling application of the technique came with Weta Digital's creation of Gollum. “I knew we needed subsurface scattering,” says Oscar-winning visual effects supervisor Joe Letteri, “That was one of the things I set out to do when I took Gollum on.”

Letteri, who worked at ILM when Wann Jensen visited, immediately recognized the idea's potential. “I worked on Jurassic Park, so I've been working on skin development ever since we first tried to figure out dinosaur skin.”

“We tried to approach skin layer by layer, peeling the layers apart and looking at how light goes down through each one,” Letteri says. “We'd sort of ‘accumulate’ the effect, and that got really complex and hard to follow.”

For The Lord Of The Rings, Letteri and Weta technical director Ken McGaugh implemented their own variation on Wann Jensen's ideas, which, according to Letteri was more production friendly. They actually developed the approach through a unique give-and-take collaboration with Weta Workshop's silicone maquette experts, and their final implementation was ingenious enough to earn McGaugh and Letteri Sci-Tech honors alongside Hery.

Henrick Wann Jensen's before and after shots: Medical research on laser light used in the treatment of skin disorders was a catalyst for the CG techniques.

The Results Onscreen

All this development has had an unmistakable impact on moviemaking. “It does feel like a breakthrough,” says Letteri. “This really did add something new to our creative toolkit.” And it arrived not a moment too soon, he admits, since The Lord of the Rings director Peter Jackson, “was going to show Gollum in close-up whether we were ready or not!”

Weta actually used subsurface scattering on all the digital characters in the trilogy's final film. “It gave us that little extra bit of realism,” says Letteri. “It actually is a subtle effect. At a distance you don't really see it, but we put it in even if we didn't really need to. We wanted to make sure it was there if we got close on a character.” The approach also enabled digital characters like Gollum to absorb light in a believable way. “We could put Gollum in hot sunlit scenes with Sam and Frodo and say ‘I want that sunlight to be 3 1/2 stops over,’ and Gollum's face would just soak it up,” says Letteri. “He looks the same on film as the actors do.”

Realism has also been achieved by ILM on virtual characters in movies as varied as Pirates Of The Caribbean and Terminator 3 — the digital double of Arnold Schwarzenegger. The challenge of matching virtual characters to real people has been helped significantly by subsurface scattering, which can be calibrated to simulate youthful or aging skin. “We experimented with that during Episode II,” Hery says. “We tested that on digital doubles for Hayden Christensen and Christopher Lee.” The experiment revealed that when Hery pushed the translucency on the elder Lee's digital double, the results made him look less real. “The opposite was true for Hayden.”

Wann Jensen, who's now at UC San Diego, has a theory about this. He uses the example of the waxy-looking characters that were rendered without subsurface scattering in the CG feature Final Fantasy. “Many people felt that the old guy looked the most realistic. When older actors in movies use makeup, it has the effect of making their skin look more opaque. We're used to seeing that, so the lack of translucency on the old guy wasn't so distracting.”

The thickness of a character's skin is another factor to consider when determining how much subsurface scattering is appropriate. For a thick-skinned character like The Hulk there is not as much scattering. “The thicker the skin, the less scattering there will be, and the less translucent he looks,” says Hery.

Judgment Calls

Filmmakers walk a fine line in making these judgment calls, especially when applying subsurface scattering to characters that audiences already know from past films. The most notable example of this is Shrek, the hugely popular character that's now back onscreen in Shrek 2. The first film, released in 2001, had only what PDI/DreamWorks' Buhler calls fake subsurface scattering. “This time, we wanted to make the existing characters look better, but in a way that didn't go too far from the original stylized look of Shrek,” he says. “But even if it's not a realistic character, you expect the highlights on skin to move in a certain way.”

CG aficionados may notice subtleties in Shrek 2 that come courtesy of subsurface scattering, in the backlighting of ears, the softness around facial features, even the look of the characters in crowds. “If you crank up the translucency on a face, it's similar to having a very diffuse light — like the ‘beauty light’ used on actresses in the 1940s,” says Buhler.

Considering the overall use of subsurface scattering in Shrek 2, Buhler points to its use on several things besides skin, like in the rendering of vats of milk and glowing potions. “The milk in Shrek was extremely difficult to light, but in this film, it was easier to get a translucent look.”

Credible Cartoons

Like PDI/DreamWorks, Pixar is using subsurface scattering to make fanciful characters appear more lifelike. For the upcoming feature The Incredibles, Pixar will animate more humans than ever before. The studio has refined its skin techniques to handle the challenge. “We're calculating the subsurface scattering using a much more robust, complicated algorithm,” says West. “Of course, we don't want a super-realistic, ‘reach-out-and-touch-it’ skin. If you look at the shape of Mr. Incredible, that's not a real person. But you do want him to be believable, to feel like he's not a puppet. He'll be the human you're looking at the most, so we've spent a lot of time getting all the nuances right.”

West agrees with his fellow filmmakers that subsurface scattering can take the hard CG edge off a variety of surfaces, not just skin. “There are all sorts of objects, as you stare at them, that exhibit subsurface scattering. I remember eating lunch when we first started looking at this stuff and holding up the pickle that came with my sandwich. I thought ‘This thing scatters like crazy!’ Just having the knowledge that this is the way things illuminate takes the edge off of everything.”

In developing its approach, Pixar's production people had the advantage of collaborating closely with the company's RenderMan tools group, which helped production people get the most from the latest PRMan 11 software. “There were things that we needed to force the renderer to do that just a year ago it couldn't do,” says West.

Pushing the Tools

This is significant, since the majority of digital filmmakers use RenderMan. For ILM's Hery, the critical issue is how to make subsurface scattering practical in terms of speed and integrating it in a common pipeline. For both Weta and ILM, that means making it work with RenderMan.

Weta's Letteri reports great support in this regard. “We started doing Gollum before PRMan 11 came out,” he says. “The RenderMan guys knew we were pushing it, and they made sure that what we needed was in there.”

The RenderMan group seems intent on making it feasible for their users to implement subsurface scattering, especially in the film community. DreamWorks' upcoming feature Shark Tale will feature subsurface scattering created with inhouse shaders written for RenderMan.

Wann Jensen expects that commodity renderers increasingly will integrate the technique into their software. “I'm not sure it's official yet, but the next release of PRMan will feature subsurface scattering,” he says.

What's Next?

“The trend is really happening,” says Hery. “Unless ILM does thick-skinned dinosaurs again, I don't think we'll do characters without subsurface scattering. There's no going back. But there are still microscopic details that are hard to capture. The work isn't finished.”

Although Pat Hanrahan couldn't have contemplated this when he first investigated subsurface scattering, he now expects the technique will become so efficient that it can even be used for the realtime rendering of game characters. “I've seen some implementations of this already,” he says. “There's a little bit of nuance to it, but it's very doable.”

Of course such speed and power will only entice moviemakers to contemplate further the Holy Grail of creating photoreal actors. “We'll see that eventually,” says Letteri, who's now working on Peter Jackson's King Kong. “The gap that we've bridged with subsurface scattering is that we can now push light into skin and get real softness and richness. It feels like subsurface scattering was a big part of the puzzle that we have to solve. With the right character, it's not out of reach any longer.”

Continue the discussion on “Crosstalk” the Millimeter Forum.
© 2010 Penton Media, Inc.