Rodriguez and 3D Post
Jun 1, 2005 12:00 PM, By Michael Goldman
Better Anaglyphs and Revised Workflow
Robert Rodriguez says that even as production of Shark Boy and Lava Girl in 3D entered the 11th hour, he decided to make a major change in the postproduction process. His goal was to improve the left-eye/right-eye anaglyphs that transform the movie's images into stereoscopic 3D. The process involved moving production of the anaglyphs to Montreal-based, 3D cinema technology company Sensio as time was running out. But what the heck, Rodridguez recently suggested to Millimeter — his goal for the movie was always to “make 3D really pop, without sacrificing color. I think there is a lot more we can do in 3D now that we have the process down a lot more than when we did Spy Kids 3D (in 2003).”
On recommendation from his longtime collaborator, Keefe Boerner, the film's visual effects producer, Rodriguez agreed to move the anaglyph portion of the process from Post Logic, Hollywood to Sensio. There, filmmakers, at press time, were taking advantage of that company's proprietary formula designed to maintain more overall color in the red and cyan anaglyphs, while offering a more comfortable 3D viewing experience overall with strong contrast (see below). That change, combined with several improvements in how Rodriguez's crew from his Texas-based HD filmmaking factory Troublemaker Studios shot the movie, exemplifies why Rodriguez has become one of only two big-name filmmakers (the other is James Cameron) actively pioneering new digital 3D filmmaking workflow for theatrical features.
Even before his 11th hour switch, Rodriguez and his team had to push Shark Boy and Lava Girl through their pipeline while still finishing up work on Sin City. Rodriguez insists, however, that segueing Troublemaker from Sin City into Shark Boy and Lava Girl in 3D was not as logistically daunting as it might have initially appeared, partly because he simply “turns all that stuff” over to Boerner, whom Rodriguez calls “a freaking robot” when it came to managing the ebb and flow of massive amounts of HDCAM and CG data floating through virtual space between Troublemaker and 11 different visual effects facilities.
For Shark Boy and Lava Girl, Rodriguez says that his team basically followed the Spy Kids 3D template but with a number of improvements, including some that came deep into the final days of postproduction.
“On the basic, conceptual level, we learned a lot from doing [Spy Kids 3D] that we applied to this project,” Rodriguez says. “But here, to a large degree, we did it differently; we used newer [Sony HDC-950] cameras and figured out new 3D techniques to put the characters into more extreme space. Last time [on Spy Kids 3D], we had to make judgment calls every time we shot an actor in front of a greenscreen in terms of where to place them in space. This time, 3D is not just the environment, since the actors are more dynamic, with special powers, and we are supposed to exaggerate their movements more and pull them out more than we did on [Spy Kids 3D]. So we were able to use more sophisticated 3D guidelines to stretch things further.
“Plus, the [3D Reality Camera rig, designed by Vince Pace and James Cameron] is much better than it was a couple of years ago. And we were able to record to the new Sony SR-1 portable HD decks [the SRPC-1 HD processor]. In fact, I believe we had the first ones available [two, actually] in the United States when we started this project. That allowed us to record two streams of HD 422 onto a single tape and to watch 3D playback on set and freeze frames while doing so. These are things we couldn't do a couple years ago.”
The Reality rig links two HDC-950 cameras in one rig so that the camera sensors are placed 70mm apart in order to capture left-eye and right-eye imagery on, in this case, a greenscreen stage. The production had access to two of the rigs at all times and four HDC-950 cameras. Two were rented from Pace's company, Pace Technologies, and Rodriguez owns two. Also, two HDW-F900 cameras owned by Rodriguez were separately used for the movie's 2D sequences. Boerner says that improvements to the rig, the HDC-950 cameras, and corresponding Fujinon lenses, as well as the availability of the Sony SR decks, all made Shark Boy and Lava Girl a more ambitious 3D movie than Spy Kids 3D.
“We always had two 3D rigs available — one was always on a crane, and one was always on a dolly,” Boerner says. “We never had downtime resulting if Robert decided to change from a dolly shot to a crane move. And the HDCAM-SR decks made a big difference. We recorded to two of them during production, synched together with jam-synch timecode. Since, when shooting 3D, you are separately trying to capture the left-eye and the right-eye perspectives, you are shooting, essentially, in stereo, using one rig at a time, with both streams of HD recorded to the same HDCAM-SR tape.”
As he did on Spy Kids 3D, Rodriguez brought Shark Boy and Lava Girl to Post Logic, Hollywood, to build all the disparate elements into a single, stereoscopic, 3D movie. That first meant replicating Rodriguez's offline edit (he cut the film in standard definition on an Avid Meridian system, with 3TB of Unity storage) and seamlessly adding effects shots from the various vendors before conforming and color-correcting left-eye and right-eye versions of the imagery.
Initially, as was the case on Spy Kids 3D, the plan from there was to have Post Logic also build the digital anaglyphs which, when filmed out (at EFilm, Hollywood), would produce the stereoscopic, 3D version of the movie that most filmgoers will see theatrically while wearing 3D glasses. Shortly before press time, however, Rodriguez and Boerner decided to bring the anaglyph portion of the job to Sensio instead. Filmmakers made the late change after studying the effectiveness of Sensio's new proprietary algorithm for creating the anaglyphs.
“Basically, we have two conforms happening at once — a separate left-eye master and a separate right-eye master that Post Logic conforms with matching timecode,” Boerner explains. “All tapes from the vendors arrive there with a left-eye tape and a right-eye tape of delivery, with timecode that matches the shots between the tapes. The editor at Post Logic [Johnson] takes the Avid EDL and conforms the left-eye version and then matches the right-eye version to that.”
Johnson used a Quantel IQ (version 2.1, rev. 18) system for the conforming process. Post Logic colorist John Persichetti then color-corrected everything in his da Vinci 2k system, again working on the left-eye version and then applying identical settings to the right-eye version.
Johnson adds that the conform process was particularly painstaking on the project, since there was, in essence, twice the workload for all 3D elements.
“Some visual effects vendors delivered to us on HDCAM-SR in RGB color space, while others sent FireWire drives, which were then archived [on Post Logic's Flame editing system] and sent over on HDCAM-SR tape, as well,” he says. “Troublemaker sends me [Rodriguez's] EDL, and I first line up the left-eye version with the offline. Then, I compare the left eye and right eye to each other. Occasionally, a vendor might have inserted the wrong shot, or laid down the left eye where the right eye should be, or vice versa. Because of that, I have to go through all left eye and right eye cuts and make sure they all line up properly.”
Using a DVS DDR ClipStation Pro HD system, Johnson funneled shots to Persichetti for color correction, and then the imagery was sent to Sensio on HDCAM-SR tape for the anaglyph process to begin. Persichetti says he learned a great deal from Spy Kids 3 about the parameters within which he needed to work while color-correcting 3D reels.
“What we determined is that whatever color correction I do on one eye must be done for the other eye in exactly the same way,” says Persichetti. “Therefore, I primarily work on the left eye, since that is the ‘normal’ eye in the sense that the left-eye camera photographed the images straight on. I color correct for the left eye and then apply those settings to the right eye. You can imbalance the 3D if you don't do that. For 3D generally, I'm limited in the amount of heavy color correction I can do. If I manipulate the image too drastically, that impacts the quality of the anaglyph and throws off the reds and blues. Yellow and green are easier to boost and change without affecting the 3D than reds and blues. So a lot of the color correction is about balancing between shots, since they came from so many different vendors.”
To create the anaglyphs, Pierre Routhier, Sensio's vice president of engineering and the company's stereoscopic engineer on the project, used what he calls an “adaptive set of algorithms” to manage different scenes. Routhier explains that the anaglyphs, in their simplest form, involve turning the left eye red, while turning the red channel off for the right eye, with green and blue (cyan) mixed equal parts, before the left- and right-eye versions are combined. Of course, there is more to it than that.
“When using this method to encode a scene where a red balloon floats in a blue sky, the viewer would see a red balloon in a black sky in one eye and a black balloon in a blue sky in the other,” Routhier elaborates. “The clashing effect is catastrophic. This is called retinal rivalry, and is what used to make anaglyph 3D uncomfortable and difficult to watch. Our approach to this problem is a radically different way of combining the color information from the left- and right-eye images.”
That new process involves a special algorithm written by Nicholas Routhier, Sensio's president and Pierre's brother. According to Boerner, this new algorithm helped filmmakers maintain more overall color while offering a more comfortable 3D viewing experience with strong contrast.
“For this new process, all the color correction is done on the original 2D version of the film, and both reels are then sent to Sensio on HDCAM-SR tape,” Boerner explains. “The material is than transferred at Technicolor [Montreal], where the film is broken down to image sequences using [another DVS Clipster editing] station. Pierre then processes the image sequences using Sensio's proprietary algorithms. The reels will then be outputted back to an HDCAM-SR tape and sent to EFilm for printing. The Sensio algorithms also give us control over the red saturation of the picture, meaning we have additional artistic control over the balancing of comfort versus color perception. They even personalize the color encoding to match the exact 3D glasses we use for the movie, which significantly improves the 3D effect.”
Boerner adds that by using this approach, the filmmakers did not have to worry about the colors used by visual effects houses to compensate for the anaglyph effects, saving significant time by avoiding multiple re-edits. “In a movie like Shark Boy and Lava Girl, where cyan and red are common themes associated with the characters, traditional anaglyph encoding would have been a color correction nightmare,” he points out.
Continue the discussion on “Crosstalk” the Millimeter Forum.