Scorsese's Color Homage
Jan 1, 2005 12:00 PM, By Michael Goldman
Director Enters Digital Realm to Craft The Aviator's Vintage, Dye-Transfer Palette
If you are ever lucky enough to find yourself in Martin Scorsese's private screening room discussing the history of color feature film processes, he will no doubt school you on such movies as Follow Thru, an obscure 1930 film about golf that illustrates the limitations of the early Technicolor two-strip, dye-transfer process by showing golf courses with blue grass. He might also show you clips from other two-strip films, like 1934's La Cucaracha and three-strip movies such as The Divorce of Lady X (1938), Blithe Spirit (1945), and Leave Her to Heaven (1945).
Scorsese will also likely lecture you about visual and visceral differences between the early American Technicolor dye-transfer process and the British version, and he will probably show you how color palettes and styles varied wildly in those early years from studio to studio and project to project.
For those collaborating with Scorsese on his Howard Hughes film, The Aviator, such screenings and intellectual discussions were an ongoing ritual. But those sessions were not mere academic exercises — they were necessary, given Scorsese's determination to use The Aviator to pay homage to the early Technicolor two-strip and three-strip processes.
“I was particularly fascinated by color when I went to the movies as a young person,” Scorsese recently told Millimeter. In particular, he says he was obsessed with the notion that movies offered viewers “different forms of color” during his formative years in the mid-1940s.
“There were different processes around back then,” Scorsese says. “Each process was different, and therefore, each film was different. That period of filmmaking and film viewing was formative for me, in a very primal way, and those images remain imprinted in my mind. They were colors that were almost palpable, designed to evoke a particular emotional state, and it was all pioneered during the great period of Hollywood filmmaking in the '20s and '30s. It was a period of experimentation — grass looking blue-gray rather than green, skies that were slightly aqua rather than blue, metal shining blue instead of white. I've been interested in all that for a long time. Since the subject matter of this movie begins with Howard Hughes the filmmaker during this period, I thought it would be interesting to explore adding these color schemes to [The Aviator].”
Scorsese's plan was to craft the first half of the movie to mimic the Technicolor two-strip, dye-transfer process. Then, during the second half of the film, he wanted to alter the palette to emulate the three-strip process that took over in the mid-1930s.
Although Technicolor briefly revived three-strip, dye-transfer printing in the 1950s and again in the 1990s, the dye-transfer approach overall was permanently decommissioned in 2002, according to Technicolor. Therefore, Scorsese had to find collaborators who could figure out ways to digitally replicate the sub-textural tones that the original dye-transfer processes brought to the older films.
Ironically, this craving to replicate a vintage look from the analog era ended up causing Scorsese, a so-called traditionalist, to dive into the world of digital filmmaking for the first time.
“Early on, I realized I would have to use digital rushes for the first time,” he says. “It didn't make sense to cut the film using the normal color process of today, become accustomed to that look, and then, after the film is cut, try the two-strip look knowing I would probably prefer the way I had become accustomed to seeing it. It also required us to do a digital intermediate for the first time, which I loved, and the movie also has far more visual effects [395 shots] than I'm used to. I designed shots my typical way, much like I did the fighting sequences in Raging Bull, for instance. But what was new was the ability to take my notes and drawings to [visual effects supervisor Rob Legato] and have him work it all out with computer previsualizations. These are things that others have done, but it was all new for me.”
In the project's earliest stages, Scorsese asked Legato, an Oscar-winning visual effects supervisor, to join his team to figure out a workable digital path to emulate the dye-transfer process. Legato began researching various approaches to the problem, and eventually signed on as visual effects supervisor and second-unit director.
“I may have filled a niche for Marty in the digital process,” Legato explains. “The first part of that niche became clear in our first meeting — figure out how to re-create two-strip and three-strip. But I was also there to handle the effects, an area where Marty is less familiar, and really, to just find seamless ways to accomplish shots — to film things that normally can't be filmed. At the end of the day, we combined all sorts of different techniques — CG with models, hanging miniatures, forced perspective, radio-controlled planes, etc. What was particularly enjoyable is that Marty is interested only in the result — he gave us a lot of freedom to go out, shoot elements, and figure out solutions.”
Joining Legato early on were DP Bob Richardson, ASC, and editor Thelma Schoonmaker, ACE. Over time, their discussions centered around which two-strip and three-strip process to emulate, a question that was not fully answered until the film's DI was finished a year later.
“There are so few films that come out of that nitrate world that even exist in good quality,” Richardson says. “Many are in disrepair, so even seeing excellent examples of two-color was not an easy thing to do. But even the ones we could see, each one was different. The color, the registration — it's all different. Marty had a specific look in mind, where the grass is blue, for instance, and we stopped only when he felt we had it. We knew we'd have to find a way to approximate the look generally in dailies, which is why we had to do HD dailies. We knew we could capture some of it correctly this way, but not all of it. [Technicolor] kept working to develop a technique to bake it into the final imagery, and that process continued throughout production.”
It quickly became apparent to Scorsese that his decision to emulate dye-transfer processes would impact virtually every aspect of the project.
“The color of the sets, the costumes, makeup, every object was affected,” says Scorsese. “For instance, Katharine Hepburn's dress in one scene, where Hughes takes her flying — that dress was really peach. But in two-color, it comes out sort of beige. This had a great deal of impact on [production designer] Dante Ferretti and our set and costume teams, as well.”
Indeed, as was true in the original dye-transfer era, art direction choices heavily influenced the final two-strip and three-strip imagery versions seen in The Aviator. Ferretti, at Scorsese's behest, therefore tried to emulate the design approach of William Cameron Menzies' sci-fi work, to serve a particular creative goal — “that art-deco thing that Menzies had in Things to Come,” according to Scorsese.
“We wrestled with this design approach a bit, in the sense that Menzies was doing futuristic stuff, so it didn't necessarily look totally real, and we were trying to be totally real in showing what Hughes' planes and designs looked like,” Legato explains. “But Marty made an interesting correlation that Menzies emphasized the ships and buildings in Things to Come with use of color, light, and composition that made them stand out in a certain way. Marty said that Howard Hughes viewed his planes, which were gigantic and futuristic for their time, in the same way, and we wanted to get that across in the design.”
Legato utilized Adobe After Effects (version 6.5) and Photoshop CS on his Toshiba PC laptop and Macintosh G4 Titanium computer — he even consulted with Adobe engineers at one point to stretch the software for his purposes — to figure out what color combinations would yield realistic emulations of two-strip and three-strip on the big screen. He also consulted with experts at Technicolor, including retired dye-transfer expert Dr. Dick Goldberg, to learn how dye packs were originally mixed and matched (see “Seeking Vintage Color” in December's Millimeter).
To achieve two-strip, Legato created digital color procedures employing digital filters that emulated the deep red and green filters used during original photography in the two-strip era. Once filtered, the frames were converted to the CMYK format, where the yellow layer (blue in RGB space) was removed. Then, to bring skin tone back to normal, yellow was added to the magenta layer, turning it orange, while a smaller amount of yellow was added to the cyan layer. According to Legato, this combination emulated the same color configurations that were used in the Technicolor dye printing processes during the '20s and early '30s. The final result was later achieved by creating special look-up tables (LUTs) and algorithms (written by Josh Pines, Technicolor Digital Intermediate VP of research and development) that permitted the image-processing software to emulate Legato's formula, and painstakingly applying that data to the imagery during a special rendering process that immediately followed the DI phase.
Three-strip was achieved using the same approach, except Legato added digital filters that emulated not only deep red and green but also blue. Those filters were then combined to create, for instance, a purer red than could be photographed conventionally. This digital construction of new red, green, and blue layers emulated the three black-and-white Technicolor matrices used for the original, three-step process.
“I tried to be faithful to the way they really did this during the dye-transfer era,” says Legato. “I made up the template, relying on Photoshop's ability to re-create CMYK colors, and then re-created that on [transparent acetate material] using an Epson printer. That was the template that Josh Pines later used to build LUTs for the DI.”
Color correcting the movie through what Legato calls “this prism that only shows two colors” for the DI would later fall to TDI colorist Stephen Nakamura, under Richardson's supervision. Before that stage, however, filmmakers faced the challenge of giving Scorsese a high-quality approximation of this look for HD.
This problem was solved by colorist Steve Arkle, commonly known as Sparkle, Techicolor Creative Services, Hollywood, who came up with a telecine solution. (Some dailies from scenes shot in Montreal were color corrected at Technicolor Montreal, but all two-strip dailies sequences were processed in Los Angeles.)
“We transferred the film to HD D5 tape — a 24fps progressive signal — and simultaneously created an SD format version for [Schoonmaker] to use during the editorial process,” says Sparkle. “The fun came in finding a way to approximate Rob Legato's process. I normally run a [Cintel] Millennium telecine, and I'm well acquainted with it. During tests, the point was made that they only had red and green emulsion layers during the days of the dye-transfer process. I realized that, on the Millennium, I could shut down the blue channel and replace it with the green channel. You can physically do that using the machine's regular controls, so there was no need to reconfigure the device. Within hours, everyone agreed that this was good enough to serve as a basis for the two-strip look. That was not the final look, obviously — that couldn't happen until Rob created the template for the LUTs they would later use for the DI. But the result was close enough. The final look is probably more saturated than what we had during the dailies process, but we were able to consolidate the basic look for everyone involved.”
Sparkle then tweaked the approach to achieve the three-strip look. “I used the same controls on the Millennium to find a proper mix of the various channels to emulate three-strip,” he explains. “We supplied a mix in the blue channel, so that it was not just red-green-blue, but rather, it was red-green with a little bit more blue and a little bit more green thrown in. The difficulty was that it made it tough to really control skin tones, so there was some time spent figuring out how to compensate in color correction.”
Throughout production, Scorsese and his team were able to view dailies using an NEC 1K projector in his New York studio. Richardson adds that while dailies fell short of the exact, final look, they were “crucial to refining the look because it allowed us to go back to sequences we shot earlier and alter them as the technique improved throughout production.”
Filmmakers used Final Cut Pro HD (version 4) to cut dailies in New York, while an assistant generated a LightWorks Edit Decision List (EDL) for Schoonmaker, because she cuts on a LightWorks (version 1.5) system. (Legato's group, headquartered at Sony Pictures Imageworks, Culver City, Calif., used an identically calibrated Final Cut Pro system to cut various versions of effects shots before sending them to New York to be added to the evolving cut.)
“This workflow let us constantly screen off HD, and we conformed the HD in Final Cut Pro ourselves here in New York, which saved us a lot of money,” Schoonmaker explains. “I must say, it was the first time I had worked with digital dailies for an entire film, and it was very helpful compared to using a beat-up work print. An old, beat-up work print can impact your feelings about the film and, therefore, how you cut the film. Here, we already had the basic two-strip color look in HD, and we had a beautiful image that was consistent with what we were working toward in terms of the final image. I was constantly editing while they were shooting, replacing previz elements with final shots as they came through.”
Production and Effects
Meanwhile, the crew also faced numerous production challenges, like how to shoot complicated sequences within their timeframe and budget. The film's opening sequence was particularly challenging. It shows Hughes filming his famous World War I aerial epic, Hell's Angels, based on camera angles and coverage actually used in that film.
Since flying dozens of vintage airplanes into the sky and filming them in mock combat, as Hughes did back in the '20s, was not a viable option, Scorsese turned to Legato. First, Legato previzzed Scorsese's choreographed vision for the sequence, which includes what Legato calls “a signature Scorsese shot” — going from wide to tight on Hughes (Leonardo DiCaprio) in a vintage plane high in the sky, operating a camera as other planes whiz past his head.
“When Hughes shot this originally, he had something like 60 planes in the sky, all choreographed, doing pretty wild things,” says Legato. “There was no way we could film that way, obviously, but Marty wanted to re-create that moment. We designed many shots right out of the original Hell's Angels movie, and then we placed [DiCaprio] right in the middle of it. We wanted to illustrate how things were on the very day Hughes was filming those shots.
“I had people take [Hell's Angels] and copy it, frame-for-frame, to see how the planes flew. That literally became a three-dimensional version of the background of the scene, and then we choreographed shots to fly within that. So I created a virtual world in Maya, separately choreographed the planes, and then as we move the camera back wider, it can go anywhere it wants within this virtual world. Once I did the pre-choreography in CG form, we could extract from that what physical things we would need to get those shots. In other words: you start in 3D, and then take the 3D back into something you can physically photograph, and then bring those elements back into the virtual world. We shot DiCaprio, for instance, greenscreen and then SPI added the virtual elements, like CG clouds and planes.”
Legato adds that clouds were crucial to the scene because this part of the story revolves around Hughes' obsession with having the right clouds in the background before he would film his dogfights — a process that took months to achieve. SPI, under digital effects supervisor Peter Travers' direction, therefore had to re-create those particular clouds seen in the original Hell's Angels movie.
“In CG, they re-built the cloud shapes from particles and then applied photographic textures from real clouds to those particle shapes,” says Legato. “That gave the clouds depth as they moved through space. This technique gave us the flexibility to really art-direct the clouds. They could be added, subtracted, and reshaped at will for each shot.”
The movie also includes an historically accurate mixture of newsreel footage and new footage designed to match vintage clips showing the Hell's Angels premiere at Grauman's Chinese Theater in Hollywood. Scorsese wanted to include old stock footage shots from the front of the theater during the actual premiere, and then had that footage colorized to match new footage.
“Marty would never consent to colorizing somebody's actual movie, but in this case, it was newsreel footage and it was appropriate for what we were creating, so we added color to the stock shots,” Legato explains. “Then, we photographed a partial exterior Grauman's set on a stage in Montreal. Later, in Los Angeles, we shot a scaled miniature set of Hollywood Boulevard and all the buildings from Grauman's east for six blocks. We then added CG planes and miniature searchlights, doing everything we could to match the newsreel stuff. Eventually, our artists added more detail to the backgrounds, but the cars and banners and things are all miniatures. It sums up the approach for the effects shots — shoot as much live as you can, keep building and collecting elements, and then put it all together in the compositing phase.”
Other sequences such as the crash of Hughes' experimental spy-plane, the XF-11, in Beverly Hills and the historic first and only flight of his Hercules cargo plane (the Spruce Goose), were classic multimedia affairs, relying on models and miniatures created by New Deal Studios, Los Angeles, mixed with CG, matte paintings, and live-action material.
“The Beverly Hills crash was done with radio-controlled models and miniatures photographed from a helicopter and some CG shots,” explains Legato. “One of the ideas we came up with is the simplicity of photographing a regular radio-controlled model — once they get up in the air, you have no idea how big the plane is. If you shoot models on a stage, you have to put them under motion control. That leads to a limitation, and it becomes a bit mechanical. If you do it totally CG, you have to animate all the wind resistance and stuff. But shooting models while they are flying and building them big enough to film from a helicopter — that's a much more organic way to do it. We just did these aerial days, shot the hell out of it, and then assembled the best elements later. We used a SpiderCam with a gyro head rigged to fly at 70mph as far as his plane did when it plummeted into the house. We hung the rig between two towers, starting 270ft. in the air and ending up at 50ft in the air, traveling more than a quarter mile. Matt Gratzner [from New Deal Studios] and his team built models for the actual impact into the houses. We also photographed other elements in Beverly Hills, and we filmed the model on a barren area near a golf course. We later put together that combination of things to re-create this actual moment from history.
“The Hercules was a hanging miniature — we shot it out in a field, and that's the real sun, the real sky from that day that you see in the movie,” he continues. “That's the same model we used for all shots where you see the Hercules flying or sitting on the water.”
Legato led a small Aviator visual effects unit ensconced at SPI, where a central Medéa Raid 1.2TB server stored evolving HD effects shots. Eventually, Legato also installed a home-built RAID PC server with 3.2TB of slow storage that grew to 5TB in the basement of his home to warehouse scans, temp shots, and finals. He emphasizes that this configuration “kept the effects approach small,” with data shipped from facilities to SPI and from SPI to Schoonmaker's Final Cut Pro HD system in New York for dailies on LaCie 250GB FireWire 800 drives.
For outsourced shots, Legato chose to rely on boutique facilities. Digital Backlot, Playa del Rey, Calif., for instance, handled compositing and matte painting on a couple of sequences; CafeFX, Santa Maria, Calif., created a CG model of the H1 airplane and did some compositing; Pixel Playground, Los Angeles, did Flame compositing and CG element creation for the Sikorsky flight sequence; Buzz Films of Montreal and Ockhams Razor, Los Angeles, contributed Flame composites; and DNA, Los Angeles, created CG window views and CG tracking, painting, and compositing for a handful of shots.
“These are mainly two- or three-person shops,” says Legato. “We wanted to keep with the philosophy of hand-crafting the film. FireWire drives back and forth, Cineon frames out, color correction all done by our small team, and so on.”
In terms of tools, Legato first designed a previsualization workflow that featured Alias MoCap software as an interface for camera data capture, Maya for scene setup and most rendering, LightWave for modeling and some rendering, Digital Fusion and After Effects for compositing, Avid Xpress Pro and Final Cut Pro HD for editing, the Beaver Project plug-in for scene conversion between Maya and LightWave and back again, and Iridas FrameCycler for sequence viewing. In addition, digital previz supervisor Oliver Hotz wrote custom code to permit Maya to allow artists to convert any previz scene into separate control files for the motion-control camera and the motion base.
“This allowed us, on-set, to quickly accommodate any changes to previz shots, without complicating the setup for the motion-control shoot,” adds Legato. “The software was able to balance the limitations of the physical rigs of the camera and motion base.”
For final shots, the project employed the same, basic configuration of LightWave for modeling and textures, Maya for animation, Beaver Project to convert those files back and forth, and LightWave for final rendering. Legato also credits Steve Worley's FPrime LightWave plug-in with speeding up workflow, and FrameCycler for speeding up the team's ability to view sequences. Most painting and compositing were done in Flame, but Photoshop and After Effects were used for format conversions, some compositing, and color correction work.
Eventually, the digital intermediate phase arrived — a first-time experience for Scorsese.
“What I now realize is that [the DI] gave me more freedom than I ever had before,” says Scorsese. “Digitally color-timing the picture made it more of a painterly situation. It gave me a painterly method to change colors and darken and brighten certain parts of the frame. Even beyond the two-strip and three-strip look, I found the DI liberating.”
Before that liberation could begin, however, Technicolor Digital Intermediates had to collaborate with Legato on figuring out how to import his digital color formula into the DI suite in realtime in order to let Scorsese and Richardson make necessary creative decisions.
Pines took the image-processing template developed by Legato and figured out how to download that data into the company's Christie DLP projector's image-processing software. This allowed filmmakers to view imagery “through” the specialized LUTs prior to those color schemes being baked onto the images during final rendering.
“We could do incredible image manipulation by doing this process through the image-processing software,” Pines explains. “The bad news was that we didn't initially have a way to make it interactive — we had to wait for the images to be rendered. But we came up with a system that permitted us to characterize this non-realtime process in a way so as to be able to view it in realtime. The way we did it was by taking advantage of the sophisticated image-processing electronics inside our Christie 2K projectors. In other words, during the DI, everybody viewed the color correction passes instantly going through additional image processing inside the projector that emulated what two-strip and three-strip looked like on film.”
Later, Pines' algorithms were then applied to the frames during the rendering phase — the so-called “baking” process.
“After color correction, the two-strip or three-strip look would be baked in before the images were sent to the [Arri] film recorder,” Pines adds. “What is significant beyond achieving the specific look that [Scorsese] wanted, was the fact that we could take arbitrary color space manipulations that would later be applied overall to the entire movie, and we could see them interactively in the DI room. It's a way to create specialty boutique looks involving color.”
For Nakamura, however, The Aviator represented “the most difficult DI of my career.” A key reason for this was the fact that, to achieve two-strip to Scorsese's satisfaction, TDI had to “double bake” the images, which meant two color correction passes on certain sequences.
“We discovered that viewing the images with the two-strip LUT alone, the colors were not as enhanced,” says Nakamura. “So Rob Legato suggested that we first apply the three-strip LUT to those images and then the two-strip LUT. To get two-strip color correction done, we first color corrected the original negative through the three-strip LUT, and then we rendered that out as a color corrected file. That file then became the new file that I worked with to apply the two-strip LUT. That's what I mean by ‘double baking.’”
At the end of the process, however, filmmakers discovered this approach had unintended consequences. According to Nakamura, a handful of artifacts suddenly appeared in certain shots after the entire movie had been filmed out and answer-printed at Technicolor, North Hollywood.
“Normally, we color correct, render, and then shoot that out to film,” he says. “But in this case, we had to bake in the two-strip and three-strip color tables as a post process after color correction. That process probably led to these mystery artifacts.” The Aviator therefore became the first film in Nakamura's experience to be filmed out twice, resulting in the creation of two original negatives.
Beyond the various issues related to baking two-strip and three-strip into the movie, Nakamura adds that The Aviator posed several creative challenges from a DI perspective. Chief among those was the Coconut Grove scene in which Hughes engages in a business meeting with aviation rival Juan Trippe (Alec Baldwin), at the famed Coconut Grove in Hollywood. Richardson and Scorsese were not happy with the scene's lighting, so they asked Nakamura to see what he could do.
“We went with a design style of that period — bright colors,” Scorsese says. “When I watched the scene later, I realized there were different looks to three-strip, and unfortunately, this version we created on-set looked to me like Fox Technicolor of the period — those Carmen Miranda films. That took the edge off the tension at the table between the characters.”
Nakamura relied on Power Windows in his da Vinci 2K color corrector to fix the problem. “I put Windows over every character in the shot and basically color corrected them individually, dropping color down in everything else in the shot,” he says. “I would latch onto a color I could find on or near the characters, and bring things down elsewhere. I also tried to create more dimension in the faces of the characters, using my highlight key through the de-focus board to color correct any highlights Bob Richardson captured with key light on-set. That gave a better dimension to the lighting. It's unusual for every character in a scene to need that kind of treatment. There were several other shots like that, where Bob Richardson had me strategically re-create or enhance lighting in the DI. I almost felt like part of his lighting crew.”
Overall, Scorsese's collaborators believe they have developed a methodology for emulating not only two-strip and three-strip, but other vintage looks using digital tools.
“This project came up with an equivalency for dye transfer,” says visual effects producer Ron Ames, Legato's partner on the project. “It might not be perfectly true to original dye transfer, but ultimately, we can now emulate a look created with film stock and processes that no longer exist using today's stock and digital technology.”
And the movie may have another important significance — it might be the project that makes Scorsese a digital convert.
“The entire art form is changing, so it makes sense to use new tools,” Scorsese says. “Films as we know them will no longer be made the same way now that we have the kind of control with digital intermediates and digital rushes. I haven't shot with a digital camera yet — lots of guys know more about it than I do. I will always prefer film. But on the other hand, as I look back [on The Aviator], I realize I made many judgments on this film based solely on digital rushes and digital projection. That has made me start thinking about [shooting digitally]. I look forward to the day when I can try a digital camera. I'm sure I will shoot an all-digital movie eventually. It's a different tool, and it's about time I tried it.”
Continue the discussion on “Crosstalk” the Millimeter Forum.