Multimedia
Blogs
Forum
Affordable HD
Whitepapers
Newsletters
Events
Advertisers
Blogcast
MillimeterKeeping Track: Case Studies in Asset Management
Jul 1, 2000 12:00 PM, Audrey Doyle
These days, one person seldom, if ever, creates visual effects from start to finish. The effects are too complex. Instead, imagery and clips move from one department to the next within a facility's divisions, and specialists in each department-all of whom are networked to each other and to a central server-alter the imagery based on their areas of expertise.
Because all these artists generate multiple versions of images and clips, keeping track of the latest iterations during the course of a project can be a daunting task. Storing and locating those digital image files and their associated meta-data after the project is complete can be just as difficult. Therefore, facilities are increasingly recognizing the need for systems that enable them to track, transfer, archive, and manage the digital image files they create. And they're addressing that need in different ways, based on their unique requirements.
The Big Picture: Disney's DALS Most large facilities have had infrastructure systems in place for several years to help them manage their assets and workflow. Usually, these facilities won't discuss details of their systems-for competitive reasons and because they don't want to appear to be endorsing a vendor's product. However, they do acknowledge that their systems are based on commercial and proprietary software powered by the latest in hardware and networking technologies.
At Walt Disney Feature Animation, a team of database, network, and systems-integration experts designed and developed DALS (Disney Animation Logistics System), which they continue to maintain and enhance. According to Graham Allan, manager of systems applications, DALS development began in the late 1980s but really evolved in the early '90s, when Disney and Pixar developed the Academy Award-winning CAPS 2D Computer Animation Production System. "Once we started using CAPS in production, we recognized that soon, that pile of digital images we were creating was going to get out of hand, and we were going to need a way to track and archive them," he says.
According to Allan, Disney's infrastructure had to meet three main requirements. For one, the Feature Animation division has more than 700 artistic workstations. These desktop UNIX machines, all of which run CAPS, are spread among four locations: two in Burbank and one each in Orlando and Paris. "On any given project, we needed to be able to store several million digital files online at a time, and they had to be quickly accessible," he says.
Also, Disney artists can generate several hundred thousand new image files on a busy day, and the facility wanted to be able to write those files to long-term archives as well as protect them via fast, short-term backups. Finally, the system needed to integrate seamlessly with the CAPS graphics applications so that it was more or less invisible to the artists.
Because nothing was available commercially to satisfy its needs, Disney built its own system from commercial and proprietary technologies. The most important component, Allan says, is its file server technology, which is almost exclusively UNIX-based, boasts several terabytes of disk space, and incorporates RAID 5 storage.
"One of our tenets here is redundancy," says Allan. "We have studios in California, Florida, and Paris. There's no point in the day when we're not working somewhere, so our systems must be reliable." To that end, DALS consists of two file servers placed side by side and connected to the disk units. "Currently they share the load, but if one server goes down, the other takes over," he says. So far, the only time Disney has come close to reaching capacity was in 1996, during production of The Hunchback of Notre Dame. "We try to stay at least one step-preferably four or five steps-ahead of our file servers by constantly monitoring load and performance," Allan says.
Another important component is the DALS database system, which consists of two databases. One is internal to and supports DALS. It uses a commercial database engine with proprietary Disney software and scripts to track the location of all files on the network. A central database table tracks file attributes (server and volume location, file size, owner, date, etc.) and groups files in folders according to each film's scenes. The database tracks the grouping along with metadata about the scenes-the animators' names, the length of the scene, whether the scene data is online. "If an artist at a workstation needs, say, the ink plane for Tarzan cel 1 in scene 3 to match a color, that workstation knows which file server and disk unit the file lives on," Allan explains. "This database tracks all that."
The second database is for the CAPS applications-the painting, compositing, and effects software, including Deep Canvas. Each application has specific metadata storage requirements and, therefore, builds a per-scene folder database as a simple ASCII file that can be stored, tracked, and manipulated similar to the image files.
Also integral to DALS are two networks, which connect the database and file servers to the artist workstations. A 10Base-T Ethernet network supports basic systems and administration servers, while a high-speed, fiber-based network transfers image data via ATM protocol. "We decided about five years ago to go this route," recalls Allan. "It was a bit risky because ATM hadn't really taken off, but we needed something that was high-bandwidth and scalable to many hundreds of artistic workstations. ATM turned out to be an excellent choice."
DALS also incorporates what Allan refers to as the facility's proprietary "virtual studio software," so-called because it optimizes communications and data transfer among workstations. Also, the software verifies access, and it tracks and logs all transactions. "With this software, artists can call up artwork regardless of where they are," he says. "An artist in Florida can work on a scene whose data physically lives in California without noticing it's at a different site because our software hides that completely."
The last main component of DALS is the proprietary archiving software, which uses the same client-server transaction model as CAPS. "Long-term data preservation is crucial to us. We're still re-using artwork from Snow White for publicity, commercials, and consumer products," Allan says. "Sixty years from now we'll probably be re-using artwork from Dinosaur and Fantasia 2000. So we had to build a system that would preserve those archives for a long time." According to Allan, scenes are saved on commercial CDs and data tape recording devices and are then deleted from the online disk arrays. Proprietary software arranges the metadata on the CDs and tapes, and all the media receive tracking IDs that are stored in the central DALS database for fast and easy retrieval.
Solutions at Smaller Facilities Although Disney has had its system in place for several years, most small facilities are only now implementing full-fledged asset tracking and management. "Up until now, we've only been able to do it to a limited extent," admits John Bennett, head of software at The Moving Picture Company, a London-based post house. "Although we've always believed organization is important, asset management and tracking wasn't a priority for us."
But last year, MPC, which employs approximately 60 artists, moved to a new location outfitted with a high-speed network. In addition to hooking all its artists up to the network, MPC also had started to see a greater number of larger projects that were computer-based. More people were working on any given job, and the number of files per job had increased drastically. "We started looking into asset management when we realized that for a job to run smoothly logistically, we had to know where each clip was and what stage it was at," he says.
The story is similar at mid-size facilities such as Klasky Csupo in Hollywood. "We've gotten a lot busier in the last few years, and we're working with more digital imagery now than ever before," says Jerry Mills, director of digital technology. "The problem is finding it."
According to Bennett and Mills, both facilities have relied on the project-management tools embedded within their primary content-creation systems: Alias|Wavefront's Maya and Discreet's Inferno at MPC, and Toon Boom Technologies' U.S. Animation digital ink and paint software at Klasky Csupo. But both say this approach makes it difficult to locate the latest iteration of an image or sequence from a company-wide perspective. "The tools in Inferno, for example, are fine for tracking work done in Inferno but not work done in the 3D applications," says Bennett.
"One of the main reasons we're using U.S. Animation is its strong database management tools," says Mills. "Imports, exports, archives, and so on are easily managed within the software. What it does not offer are intuitive search capabilities. You have to know the name or address of a file in order to locate it, which is not asset management."
As a result, it's common at Klasky Csupo for animators to create elements from scratch, even if variations of those elements already exist from a previous project. "People tend to keep their own archives here, which is kind of self-defeating because although the information is stored, a lot of times nobody except the person who stored it can find their way back to it," says Mills. Therefore, a lot of elements are re-created unnecessarily. "Say you need an image of a telephone. We probably have 500 drawings of telephones. But I bet the next time someone needs a telephone, an artist will draw a new one because it's too time consuming and difficult to find that asset."
Although Klasky Csupo has only begun investigating commercial solutions, including those from Cinebase, the folks at MPC have found their solution in Cakes, a Web-based asset tracking and production management system developed by production facility London-based Smoke & Mirrors and available commercially from S&M subsidiary London-based Unique ID.
Users enter Cakes via their Web browser and a security password. They can then access information and view data stored in four databases.
The Contacts database contains data on the clients, agencies, creatives, technicians, and production staff involved in each production.
The Media database is where media associated with jobs are logged and separated into two groups: virtual media, including rendered frames, files, and other databases; and physical media, including D1 or Digital Beta videotapes, data tapes such as Exabytes, DLTs, and DTFs, film, and physical artwork. Physical media is entered into the Media database using a microchip readable by a Cakes reader. Similar to a barcode, the microchip is attached to each asset and provides it with a unique identity number for tracking purposes.
Through the Jobs database, Cakes securely streams media to clients over the Internet so that they can monitor work-in-progress from their own offices and keeps track of what media clients have viewed and when. It also can provide communication for those involved in a project via an e-mail-based workflow-management system, and it stores the documents relating to a job.
The Dispatch database keeps track of where media has come from and where it's going as projects are shared between locations. Both the media and its metadata travel electronically via the Internet. Because users view the databases via their Web-browser front end, the software has a common look and runs the same on any computer within the building, regardless of platform.
According to Bennett, about 18 months ago MPC began investigating solutions and decided on Cakes because of its ability to handle clips and track physical media. "Whenever you play a piece of tape, Cakes automatically makes a clip of what you played," he says. "So if I record some videotape into my compositing machine, Cakes updates the Media database. Then, if I alter that piece of videotape, Cakes updates the same database." So, without having to manually enter anything into a server or database, users can view on their computers updated proxies of everything on the videotape by running the bar code through the Cakes reader.
The proxies also include timecode information. "Imagine you're in a library containing 100,000 tapes, and you need to find a particular clip. Instead of narrowing your choice down to 10 tapes and viewing them, hoping that one has what you're looking for, you can search for the job by product name in the Media database," Bennett explains. "The system tells you which tapes contain that job, and by running those tapes over the reader, you can view the proxy and its timecode location. It's an easy, fast, and organized way to work." MPC is using Cakes with an NFS UNIX- and NT-based file system, with approximately 1.5 terabytes of disk space and incorporating RAID storage. The database server and file servers are connected to the Cakes workstations on the same network, which is a mixture of Ethernet, gigabit Ethernet, and HIPPI. "High-end client-attend machines must be fast, so they get on HIPPI," states Bennett. "The machines that don't need to be as fast run Ethernet." For archiving, the facility uses D1 tape for video, and data tape, such as Ampex's DST or Sony's DTF, for 3D CG and film images.
It's clear from talking with these facilities that asset management is just as crucial to their daily operations as the tools they've been using to create their content. "From our standpoint, asset management is as essential to a facility as having a telephone," says Mills. "Getting a system in place is now top priority here."
"It's very important to us that we now have a standard, logistical approach to doing all our jobs," says Bennett. "Now, much of the panic of not knowing whether an effect is finished or where certain elements are, has changed from people running through corridors and phoning each other up to simply looking through a database on their desktop."
Continue the discussion on “Crosstalk” the Millimeter Forum.
