• Blogs
• Forums
• Tradeshows
• Partners
• White Papers
• Events

Real-Time Optical Motion Capture: Will it Change How You Make TV Animation?

Jan 1, 2000 12:00 PM, Michael Goldman

Motion-capture technology has proliferated in the animation world during the past year. At the same time, expectations have accelerated even faster, especially for those pioneers who are using the technology in real-life production settings.

"Our goal is to produce multiple CG-animated shows with this technology," says Larry Stanton, VP of technology at Netter Digital Entertainment, North Hollywood. The studio has used motion capture on three animated television series to date. "Clients mainly like motion capture because of its ability to keep production costs down on such shows. But clients will always want more. That's why we're so happy to finally have real-time capabilities."

Indeed, real-time features are the most important recent development in optical motion capture. That's because realtime is essential to the ultimate goal: to simultaneously capture body and facial movements from multiple actors in realtime without using markers. Pieces of this ideal scenario already exist, of course. Real-time capabilities (and lower price points) have long been available via Vermont-based Ascension Technologies' and Polhemus' magnetic systems. But the more costly optical mo-cap technology, which captures more detailed motion data with the smallest margin for error, could not compete with magnetic systems on the real-time issue until last year.

Optical Realtime Arrives At press time, the Hi-Res Real Time optical system from Santa Rosa, California-based Motion Analysis had been available for several months and is currently in wide use. Netter is a Motion Analysis customer and is using the real-time system as a key tool to animate two new CG series: Sony's Max Steel, which debuts this month on the WB, and 26 episodes of a syndicated series the company could not publicly name at press time. At Siggraph '99, Tustin, California-based Vicon demonstrated its real-time optical system, Vicon 8 RT, which the company says is due to hit the market sometime early this year.

In addition to these commercially available systems, proprietary real-time optical systems are in place at Futurelight, Santa Monica, and Giant Studios, Atlanta. Futurelight's active-marker-based system relies on Northern Digital Optotrak medical technology; Giant's passive-marker system comes from its parent company, Biomechanics, Atlanta. (Biomechanics' technology was formerly licensed to video-game maker Acclaim Entertainment, New York.)

Software Solution All of the "real-time" advancements in optical systems have come mainly via software developments. Software now allows optical systems to instantly interpret motion data, compensate for any lost or "occluded" data, and insert that information into computer models for instant viewing on set. This improves the quality of the captured data, reduces time spent in post, and allows for a real-time preview feature-a big advantage for most directors.

Though all of the mo-cap systems benefit from improved software, each offers a slightly different solution with its own pluses and minuses. At press time, the Motion Analysis and Giant systems were limited to capturing data from two actors at one time. Futurelight's system currently captures up to four actors but requires a smaller capture stage. It utilizes active, light-emitting markers, which means that actors have to be tethered or wear a transmitter pack similar to magnetic systems. Vicon's system was not ready for distribution as of late November.

Still, in each case, new developments speed production through more efficient mo-cap sessions and reduced post requirements. The "real-timeliness" of the previews is limited only by CPU power-how fast a studio's computers can render the CG model for viewing and how high-res the model is. The preview is essentially instant for most low- and mid-resolution CG models and generally faster than older systems, even for extremely high-resolution images.

Optical systems have also made significant strides in addressing the problem of occluded data (data that is not captured because a marker is invisible due to camera angle or obstruction). With the optical systems, occluded data is compensated for by a combination of multiple cameras, redundant markers, and each system's biomechanically intelligent software.

The software fills in missed data based on the actor's skeletal model. This model is created during a quick scan at the beginning of a mo-cap session that usually takes only a few minutes. Therefore, the preview can proceed even if specific marker data is missin-with no need for massive cleanup in post.

Begging the Question So given the improvements to optical motion capture, the obvious question is: What exactly can you expect in a production environment?

"I'm torn about the issue of realtime for optical mo cap," says Tom Tolles, president of House of Moves, Los Angeles. The studio has used the Vicon 8 system on a wide range of service projects and expects to upgrade when Vicon RT becomes available. "On the one hand, clients have heard about it and really want it, so we will eventually have it for them. On the other hand, realtime doesn't let us capture any data or movement we can't already capture right now, nor does it totally eliminate the need for some type of post work later in most cases."

The real-time preview feature, however, is worth a lot when producing broadcast projects on quick deadlines and low budgets, says John Radulovic of Sony Pictures Imageworks. Radulovic has directed mo-cap sessions using Giant Studio's Biomechanics system for two freelance TV projects that he is developing. "The ability to drive a computer model in realtime, even if it's a rudimentary, low-res model, directly helps the creative process," says Radulovic. "It allows the director to work with the actor as though he were directing live action, to get the exact performance he wants, and to do it without having to go back for re-shoots or tons of post work."

"With [the Motion Analysis system], if there is a problem, we know it instantly and not a day or a week later," adds Keith Robinson, co-owner of Modern Uprising, Brooklyn, a Motion Analysis-based studio that recently upgraded to the real-time system. "It's true that there is a limit to the number of polygons you can use in your model-it can't be too high-res yet-and the system is also limited right now to two actors at a time. But the frame delay is just a second or two, so that's essentially realtime. [Motion Analysis says the delay, depending on the model's resolution level, is usually around two video frames.] As CPU power increases and Motion Analysis improves its software, those problems will go away, and that will be a huge advantage for studios like ours. In the meantime, the current system saves us post work and it's a fantastic directing tool-a huge creative weapon and money saver."

Systems in Action As of late last year, several TV, video-game, and film-production companies were finding out firsthand whether optical systems can meet the growing expectations of users.

Near the end of 1999, for instance, Netter was going into production on two new TV shows with the Motion Analysis system. Michael Crichton's Timeline Studios in North Carolina hired Giant Studios' system to create a new video game, and Giant also got a contract from New Zealand animation facility Weta Ltd. to provide all motion-capture services for New Line's three upcoming Lord of the Rings movies.

Futurelight, meanwhile, was busy assisting the production of the all-CG Roughnecks: Starship Troopers Chronicles. Foundation Imaging, Valencia, is the lead animation shop on that show, utilizing its non-real-time Vicon 8 system in combination with its Ascension MotionStar magnetic system. Hyper Image, Burbank, also produced additional episodes, with Futurelight providing mo-cap data. As a result, Roughnecks is the first series to incorporate both real-time and non-real-time captured movement.

At Foundation Imaging, computer animators direct the mo-cap sessions for their own scenes on an in-house stage using a non-real-time production model. The company has become extremely efficient with this production model. "Right now, the system we use is very good overall," says Dave West, Foundation Imaging's mo-cap supervisor. "We only wait around two minutes before we can see a low-res model after shooting the actor. But on a day when we are doing 15 captures, that totals 30 minutes of waiting for the computer to show us the preview. As I understand it, [Vicon's] real-time system will eliminate that delay, as well as the re-shoots and much of the cleanup if data is missed. When you are doing a project with the sheer volume of shots on the tough deadlines this project has, any advantage is great. Realtime would obviously be a tool that could help us."

Indeed, Rob Smiley of Hyper Image, who directs mo-cap sessions at Futurelight for Starship Troopers episodes, says he cannot imagine producing the show without real-time capabilities. "With Futurelight's technology, the cleanup was minimal," says Smiley. "By the time we're done shooting, we have everything we shot on tape already filtered, and we know the data is clean, that we won't have to come back and re-shoot. The problem with motion capture was always the cleanup, not the actual capture of the data. There is a cost with cleaning data and waiting for that data. On a TV-production schedule, we just couldn't afford that cost. For episodic, broadcast stuff, I view this technology as invaluable."

The Creative Advantage Still, despite all the talk about the financial pros and cons of such tools, directors who have run real-time, optical sessions in the last year claim its biggest advantage is creative.

"As a director, it took a little getting used to, but our project benefited from it," says Bob Griswold of Timeline Studios, who directed sessions at Giant's Atlanta facility last year for the Crichton video game Timeline. "To start, I watched the actor and got him moving the way we wanted. Seeing those moves translated immediately on the computer model helped tremendously. For video games, especially, the characters won't be that high-resolution, but little things like the placement of the feet and the way the actor holds their hands, all that is a big deal. If you only watch the actor, they can look like they have it right, but then you see on the monitor that it's not right for the character-they are too low or not balanced right or whatever. The other thing is, what you see in the real world isn't what will actually be in the game, so the instant preview makes you more comfortable. We had an actress swimming and we couldn't do it in a pool, obviously, so we had her up on a table doing swimming motions. It looked goofy on the stage, but on the monitor we were able to direct her into giving us moves that we knew would look good in the game."

Gary Roberts, co-owner of Performance Capture Studios in Los Angeles, a studio that uses the Motion Analysis real-time system, adds that such technology has lured more live-action directors into his studio. "Before, it was mainly 3D people and animators directing here," he asserts. "It's not exactly like directing blue or green screen, but there is a similarity in the sense that we are getting second-unit people in here and some first-unit directors, as well."

Roberts adds that the system helps actors. "We recently had a client capturing the moves of a famous actor for placement into the body of a three-foot, non-human creature," he says. "The character was obviously not the same size as the actor, so the guy had to work on his walk and subtle movements. By seeing the little guy on the screen, he was better able to get into character. He and the director could see instantly if he needed to shorten his stride or whatever. That put him at ease because he had a frame of reference for his performance."

Motion-capture technology has helped animators bring life-like human motion to animated characters. Most animated characters, however, are not human but animals, aliens, or other creatures. In the last year, mo-cap studios have advanced the fledgling art of animal motion capture to address this problem.

Animal motion capture is a daunting task, considering the "actors" don't take direction very well and probably don't like having optical sensors stuck to their hides. Plus, there are a host of safety issues involved when working with animals, and all sessions are subject to supervision by animal trainers.

Late last year, two Texas studios- Dallas-based Computed Animation Technology and Wimberly-based Locomotion-combined their Vicon 8 optical systems to capture horse movement for the CG-animated Fox special The Night of the Headless Horseman, a project directed by CAT's creative director, Shane Williams.

"The big problem was how to attach the markers to the horses," Williams explains. "We talked about using spirit gum and glue and shaving the horse's hair, but the trainers weren't keen on that idea because these were expensive Arabian trick horses. We finally decided to sew together spandex capture suits for the horses, essentially the same as what a human would wear but fitted for a horse. The horses were used to wearing outfits for shows, so they got used to it quickly."

The next challenge was how to configure the 18 Vicon cameras for maximum efficiency. "We had to do it in a corral because the tricks required soft dirt," says Williams. "So we suspended the cameras in the air over the corral, connecting them to ceiling scaffolds, and ran wires back down through the air to connect them to a Vicon workstation processing the data. That was the only way to get the range of motion we wanted while keeping the camera cables off the track. It looked like a spider's web."

Performance Capture Studios of Los Angeles, a Motion Analysis-based studio, also conducted sessions with an African elephant and a lynx last year for an unnamed test film. Company co-owner Gary Roberts admits that attaching markers to the two-ton elephant was "a royal pain."

"Working with the trainer, we decided to put markers onto the elephant's bristly skin," he says. "We attached them with veterinary bandage tape directly onto his skin in some places and onto flexible veterinary wrap bandages for his legs, trunk, and tail. We got about 17 markers on him and captured the motion the client wanted, but it took several hours. That was more because the elephant was mischievous and hungry all the time. At first, he pulled markers off with his trunk and fired them across the studio, but he finally agreed to leave them on when the trainer began rewarding him with treats."

© 2008 Penton Media, Inc.