Multimedia
Blogs
Forum
Affordable HD
Whitepapers
Newsletters
Events
Advertisers
DCP Directory
MillimeterPractical MoCap: Motion Capture for TV
Jan 1, 1999 12:00 PM, Michael Goldman
Has motion-capture technology matured enough to assume a prominent place in the daily animator's tool-box? Chris Walker chuckles at the question. After all, Walker's company, Modern Cartoons-Oxnard, California-recently proved the technology can produce animated television programming daily and in realtime. Modern Cartoons made the case with 26 episodes of the children's show Jay Jay The Jet Plane, executive produced by Walker.
Modern Cartoons creates each Jay Jay episode in realtime using Vermont-based Ascension Technologies' magnetic body-capture system and a proprietary facial-capture system that Walker and his colleagues developed in-house. After the show has been directed and "shot" in sequential order, and approved by producers, the show's animators tweak the episodes and render them over several days. This successful creation of real-time animation may seem a huge victory for Walker and a handful of others who have long believed motion capture represents animation's future. But how far into that future are we?
That depends on your frame of reference, Walker says.
"I guess being able to produce 26 half-hours in realtime, incorporating both body and facial motion capture for each episode, can be called a level of maturity for the technology," he says. "Then again, we've been working on how to do it for almost 10 years. I firmly believe in motion capture, but historically, there have been failures, and, therefore, many producers are hesitant to go for it. They want to rely on what they know works-key-frame movement. Look at all the CG movies out there. How many are even trying motion capture?"
In fact, one animated feature currently in production, Sinbad: Beyond the Veil of Mists, is relying on captured motion for its human characters. (Venice, California-based House of Moves-Hollywood's sole service facility dedicated exclusively to motion capture-is handling the project.) Still, Walker has a point. Although motion capture has helped particular sequences of particular projects in recent years (notably, the complicated stunt work on Titanic), it is hardly commonplace for feature films, let alone TV.
Though a niche technology now, motion capture is growing daily both in America and abroad. Witness the stateside appearance of European-animated programs such as Donkey Kong Country. Some American shops have also dived into motion-capture production head-first and are moving the technology's role in episodic television ahead with surprising speed.
The Magnetic Pull Magnetic systems have been the tools of choice for television motion capture because of their real-time capabilities. Several facilities are currently using Ascension's Motion Star product, which is based on the DC approach (designed to lessen magnetic reverb problems caused by peripheral metal located near the capture stage). The other leading magnetic system is the more traditional, AC-based Star Trak product, manufactured by Polhemus also of Vermont. The use of both systems has spawned a slew of virtual characters on television networks around the world.
Notably, Ascension's system has been bundled into the popular turn-key system created by Medialab of Paris, and Los Angeles, Medialab's Clovis software, combined with Ascension's hardware, is responsible for some 50 characters animated in or near realtime for TV broadcasts. These include Donkey Kong and company, Tilde and Dash ("hosts" for programs on San Francisco-based cable network ZDTV), and Bill (the co-host of a live French game-show called BigDil).Medialab also serves as a service bureau for clients who have created their own characters.
"We have lots of clients using a system that we created for our own productions," explains Francois Lelievre, Medialab's VP for TV performance animation. "By creating an entire system-the Ascension motion suit and receivers, along with virtual puppeteering devices like joystick, pedals, and sliders-we have a tool to instantly animate virtual characters with realistic motion. Daily usage is where this technology is going. The future will be live, interactive characters."
However, the need to air such characters in realtime is currently limited. Therefore, what most animation shops want from motion capture is the same thing they always want from technology: a way to speed up production and reduce costs. To meet this goal, the shops and production companies are incorporating motion capture, key frame, and compositing into unique and often ingenious work flows.
Tethered and wireless magnetic motion-capture technologies have come down in price and advanced to the point where important effects shops are installing permanent motion-capture stages. Among them: Modern Cartoons, Foundation Imaging of Valencia, California, and Pacific Title Mirage, Hollywood (see accompanying story), all of which use Ascension's system. Threshold Entertainment, Santa Monica, also recently installed Polhemus' system to create martial-arts effects for its syndicated series Mortal Kombat Conquest.
When Foundation Imaging moved into larger quarters early in 1998, it built a permanent 40 x 40, motion-capture stage. The company now uses the Ascension system extensively for sequences of the Saban-produced show, Mystic Knights of Tir Na Nog, which airs on the Fox Kids network. Foundation also occasionally uses the system for creature work on UPN's Star Trek: Voyager. Paul Bryant, the company's co-founder, says Foundation Imaging built the stage in order to permanently unite motion-capture technology with its main production pipeline. He says the system has since become a "routine and easy-to-use" part of the company's daily operations.
"To be honest, it takes longer for an actor to put on the suit than for us to begin reading data," says Bryant. "We felt we had to have this technology and found a system that allowed us to quickly capture clean data. Therefore, it made more sense to integrate it into our own pipeline and develop expertise with it rather than try to bring it in for particular projects."
The Optical Advance Despite TV's reliance on magnetic systems, several Hollywood facilities have recently installed permanent optical systems even though camera-based optical technology is more expensive and less useful for real-time applications.
Last year, Netter Digital Entertainment, North Hollywood, installed a system manufactured by Motion Analysis, Santa Rosa, in a refurbished, 20 x 30 room to churn out episodes of the new syndicated 3D cartoon, Voltron: The Third Dimension. The cartoon is the first 3D CG show based primarily on captured movement rather than key-framing. What is particularly significant about Voltron is the speed with which Netter installed the motion-capture system, solved compatibility issues with their NT-based Lightwave animation software, and established a production pipeline.
"We got this project so quickly and had such tight deadlines, we could not have made this show without motion capture," says Larry Stanton, Netter's VP of technology. "But we wanted clean, accurate data in great detail, and that meant an optical system was more suited to our needs. We checked out all of them, and the Motion Analysis system offered the most detailed data. With their help, we were able to install the system and create a production model in just six weeks. Now, we basically shoot the show like it was live action. We can capture movement from 75 to 100 scenes a day, and we are turning out an episode every eight to 10 days."
Netter is now incorporating the system into the production of the new syndicated sci-fi show, Crusade, the spin-off from Babylon 5. While Netter limited its animated creature work for B5, the company will routinely use it for Crusade specifically because of the new motion-capture stage, says Stanton.
Major effects facilities have also taken on expensive motion-capture systems. ILM and Digital Domain, as well as House of Moves, have created stages built around the 24-camera-based optical system manufactured by Vicon Motion Systems of Tustin, California. That system, the Vicon8, is costly. But, according to users, it has eased the occlusion issue, one of the long-held concerns about optical technology, whereby time-consuming "compensation" is required when some motion data is not captured if a particular sensor falls out of view of surrounding cameras.
"Having 24 cameras is probably the first really significant change in optical hardware in the last few years," says Tom Tolles, president of House of Moves. "That means companies like ours, who handle really big volumes of data with multiple captures from multiple performers simultaneously, can have more high-fidelity data than before."
Even if occlusion becomes less of an issue, optical systems still face other obstacles in becoming routine animation tools. They do not yet offer real-time views of captured motion because CPU render capabilities are not as fast as the data collection process. (Although faster chips have recently reduced the time in-between to mere minutes.) Optical systems cost more, and since sunlight includes infrared light, they do not work well outside during daylight hours.
Bargain Capture Peak Performance Technologies of Colorado claims to be addressing optical system limitations for video with its Motus system. The company created the system on the principle that it should be PC-based and video-based, thus lowering its cost and permitting data capture from outdoor video shoots.
"Originally, the technology was developed for medical and sports applications, but we created Motus specifically for animation use," explains Steve Risenhoover, Peak's principal systems engineer. "It can function in-studio as a standard optical system, with the body suits, markers, infrared cameras, and receivers that are part of Motus, but it can also extract data from motion captured outdoors on videotape, as well."
To capture daylight motion, the system works with any standard video camera to shoot an actor's movement, even if the actor is not wearing sensors. In studio, artists convert the video data into any standard, digital format AVI, for example), and thanks to special software, the Peak system then extracts motion data directly from that file. Peak has also created a new plug-in for 3D Studio MAX that debuted in January. The plug-in, dubbed 'Kinecapture,' permits MAX users to track movement from AVI files directly to computer models created with that software. Bob Mulverhill, Peak's marketing director, says these types of technologies herald "the advent of optical motion capture for low-end use-affordable capture tools for boutiques."
What's Next Whether high-end or low-end, motion capture has clearly become a permanent part of the animation equation. The development of several software packages aimed at easing the extraction of motion-capture data and applying it to 3D characters has further helped the technology's transition into the TV world. Among the most popular of these is Kaydara's Filmbox, which several users call a highly efficient tool for making motion capture part of a streamlined production pipeline.
But even the technology's biggest supporters warn against thinking of motion capture as a panacea that might someday eliminate key-frame animators and other tools.
"It's a new arrow in the effects' quiver, but you need lots of arrows in this business," explains George Johnsen, postproduction supervisor at Threshold Digital Research Labs, which uses the Polhemus system. Alison Savitch, president of TDRL, says that for most applications motion capture works best in conjunction with key-frame animation, not in place of it.
"For our show (Mortal Kombat), which has lots of effects and martial arts, we combine the two effectively, and I think that is how you will see it applied for episodic TV in the future," says Savitch. "The Polhemus system can very accurately replicate complicated movements of martial artists, but sometimes we want those moves to exist in a creature or other being or be different from a human's movement. That's why we have skilled animators who can play with, alter, or refine it."
As motion-capture technology plays a growing role in the quest to create photo-realistic, digital humans, Hollywood's oldest visual effects company, 76-year-old Pacific Title Mirage has taken up the challenge. The company recently decided to make motion capture the foundation of its expanding visual effects/animation program with British motion-capture veteran William Plant as its new president of production.
Plant quickly set up a large, Ascension-based body-capture stage and supervised the creation of a proprietary performance-control system operated by a joystick for one hand and a unique, multi-accessed input device for the other. The tool allows artists to puppeteer simple facial movement for certain animated characters and is resolution-independent. The body stage and the performance-control system permit the company to direct animated TV and commercial sequences much like live-action-in realtime and under one roof.
"The idea is to use pieces of different technologies, including some we build ourselves, to stage live performances of animated characters in a regular production cycle," says Plant. "We recently shot two Captain Crunch commercials in a single day with 320 performance-capture takes. We then applied that data to lower-resolution, cel-animated 2D characters, since Captain Crunch is not a 3D character."
For more complex characters, Pacific Title Mirage integrates performance capture with the company's proprietary LifeF/x digital animation technology, originally developed for medical use. A recent demo conducted for Millimeter showed an extremely convincing, moving, digital replica of a famous actor's face created for an upcoming feature film. LifeF/x appears to be pushing closer to the coveted goal of creating lifelike, animated human faces.
According to Dr. Mark Sagar and Dr. Paul Charette, the company's co-directors of research and development, LifeF/x uses data taken from an optical, facial motion-capture session conducted by filming the actor with several high-definition cameras. No sensors are required since special software analyzes the high-def images and extracts data to digitally build geometric replicas of the complex movements of the actor's face. Animators build those movements into the computer model of that face at the same time the model itself is constructed. The CG facial model relies on data from several sources to construct a face that both looks and acts like the performer. Those sources include a Cyberscan of the face, scanned photos or maquettes of the individual, and data from a generic, CG human facial model, all combined with the motion-capture data.
"The cool thing is that this system drives realistic tissue motion into the computer model," Charette explains. "It is possibleto essentially fake the same look for very short viewing periods by utilizing low-resolution geometry and then doing real-time texture updates. But this method, instead of texturing a subtle facial movement, actually builds that movement with real geometry. The wrinkles you see on this CG face are actual geometry, not texture."
The technology also permits captured movements to be altered by animators or even inserted into other computer models by altering data or the model itself, as opposed to key-framing changes. In other words, one can direct the entire animated performance even after the motion-capture session has ended.
"We can age the actor, make him younger, turn him into an alien, and still have those subtle facial movements that make the animated face seem to be alive," Sagar says.
