Adrift in a Sea of Pixels
Nov 1, 2004 12:00 PM, By Scott Billups
Shooting Test Imagery for Microsoft's SteM Codec
The first time I met Bill Gates, I was richer than he was. I just cashed out of my 85-person ad agency and was nosing around for some good investments when I met this guy who was leveraging all of his worldly possessions on something called an operating system, whatever that was.
Flash-forward a few years — OK, a few decades — and I find myself receiving an email from Microsoft asking if I'd be interested in shooting a STeM (Standardized Test and evaluation Material) series. I've lensed quite a few comparative tests and product demonstration pieces over the years, so the request was nothing new. If you walked the floor of NAB this year, you saw my test footage projected from JVC's D-ILA projector, Texas Instruments' DLP projector, Mitsubishi's flat screens, and other distribution mechanisms from manufacturers trying to demonstrate the full capabilities of their products.
This project, however, was different. It wasn't about creating optimum content for dedicated hardware. Microsoft was looking to build a catalogue of high-definition footage for testing Windows Media High Definition Video (WMV HD) compression codec. Microsoft's WMV technology already drives everything from portable media players to televisions to digital cinema projectors. It has been submitted to SMPTE for standardization under the new moniker VC-1, and has been endorsed as a required technology for the two leading next-generation high-definition DVD formats: DVD-HD and Blu-Ray.
They wanted me to create range of test material that included resolutions and compression schemes well outside the parameters of any existing STeM series. For example, the STeM material produced by the Digital Cinema Initiative is a superb source of high-quality material for compression testing, which Microsoft used as part of its own internal development process. However, Microsoft does not own the rights to redistribute the material as part of its own Conformance-Test Suite (CTS). Additionally, the DCI STeM material was all shot on 35mm film, so it includes artifacts unique to film and is not suitable for testing the compression of interlaced content.
A few weeks later I met with Bill Crow, Microsoft's project leader, and Alexandra Lief from Alley Kat Productions, Los Angeles, who had been hired to produce and direct the tests.
Bill, ever the consummate engineer, started off with a long list of technical requirements that included content in all the supported interlaced and progressive ATSC formats and film resolution, plus PAL-compatible HD formats. He also wanted the widest possible array of content types — interiors, exteriors, day, night, fast motion, slow pans and zooms, bright colors, under-saturated, high detail, continuous tones, sports action, talking heads, and so on.
Unlike the usual request to make everything look fabulous, Microsoft also wanted all of the visual elements that present the worst challenges for digital encoding and compression, such as over-saturated colors, high-contrast diagonal edges and lines, fine geometric patterns, overexposed speculars and highlights, and dark shadow areas, just to name a few.
In addition to the technical requirements, Microsoft needed to end up with a final, cohesive product that could be used for presentations and demonstrations. Alexandra had proposed a music video format that would include extensive footage from aboard the newly refurbished, square-masted, tall ship Lady Washington, which was used to portray the HMS Interceptor in the Pirates of the Caribbean feature film. Together with a day shoot at a beach location north of Malibu and a day of various lighting scenarios in Microsoft's own luxurious studios, Alexandra figured that we would be able to capture the required broad spectrum of conditions and looks.
“Microsoft could have pointed a camera at a parking lot and pressed record while scrolling through format settings and menus as too many manufacturers do for test footage,” she says. “We wanted to create a catalogue of content that reflected professional film-making in its true environs. It needed to illustrate the full scope of settings, so we needed to include the challenges presented both in a bluescreen, build-it-from-scratch stage setting, and the widely unpredictable world of location shooting.”
After a location scout to the majestic Lady Washington (meticulously restored and crewed by volunteers from the non-profit Gray's Harbor Historical Seaport Authority), it was obvious that the extensive assortment of rich woods, worn hemp, polished brass, and miles of rigging would provide a broad pallet of challenges for whichever recording system we chose. Since uncompressed, 10-bit, 4:4:4, 24p was one of the format requirements, our choice of motion imagers, or cameras, was limited to the Sony HDW-950 and the Thomson Viper FilmStream camera.
Since both of the HDW-950s that we had access to were still in the clutches of George Lucas, we gladly used a Thomson Viper from Burbank-based Plus8 and a crispy set of DigiPrime lenses from Band Pro, also based in Burbank. Regrettably, the marvelously lightweight Arri 235 was in short supply and unavailable to round out our camera needs. But Oppenheimer Camera, Seattle, provided us with the tried-and-true Arri 435 35mm film camera outfitted with comparably matched primes and filters.
Unlike previous generations of HD camcorders, newer HD cameras create such huge amounts of data that they need to stream the image out to ancillary recording systems, such as Direct Disk (D2D) or RAM to get the full bandwidth. For our data recorder, we chose the new S.two D.Mag portable, digital field recorder that has the capability to record every flavor of HD up to and including Viper's extremely cumbersome (2.2Gbps) uncompressed, 10-bit, 4:4:4, 24p approach. (For a look at how the Viper FilmStream Camera and the S.two D.Mag field recorder were used in shooting a major release, see “Building Collateral,” Millimeter, August 2004.)
The first location of our ersatz music video took place at the remote El Matador beach in the furthest reaches of Malibu. Alexandra figured that it would be a pretty good approximation of the shooting conditions we'd find aboard the ship, such as proximity to salt water, inaccessibility to electrical power, high-contrast lighting, and matching temperature conditions.
With two high-end camera systems and a growing list of requirements, the need for an additional operator who also was equally comfortable with film or HD became our top priority. The name at the top of my short list was Joe DiGennaro. Luckily, he was available. Joe has as much experience with the Viper as anyone, maybe more, so our concern with the camera could be put to rest while we tried to develop a comfort zone with the available recording mechanisms. Joe and our first AC, Vince Mata, also a Viper veteran, started building our system at Plus8 Digital. Their first discovery was that the S.two needed 24V of DC power. Unfortunately, Plus8 did not have the required batteries, so they couldn't test the drainage rate firsthand. According to Ted White, VP sales, North America, at S.two Corporation, Genoa, Nev., a 17amp/hour battery would yield about 45 minutes of recording time.
Did this mean true record time or run time? We couldn't be sure. Spinning discs take a lot of energy to get moving, but once at speed, little power is needed to keep up the inertia. Even so, at 36 minutes for each digital mag, it looked as though we would need a new battery for each D.Mag reload.
Alex didn't expect to shoot more than an hour's worth of material, so Joe called another rental house and ordered six batteries to have on hand for the beach location day. He did this as a precaution against the power-consumption estimate and the propensity for directors to underestimate their shooting ratio.
As fate would have it, the batteries didn't show up until late in the day. When they did finally show up, they weren't fully charged, so Joe was not able to do a complete drain test before Plus8 closed. We were able to get the S.two spinning and recording under battery power, so we felt fairly confident despite only being able to perform an empirical test. We were soon to learn a valuable lesson about fair confidence.
The next morning, we gathered at the top of El Matador, where Alex had arranged for us to set up operations out of a 30ft. motor home. The actual shoreline was about a quarter-mile below us, at the bottom of a hill steep enough to be called a cliff.
Over the next three hours of schlepping and shooting, we cranked out five or six major setups of our two actors frolicking in the shallows and having a gloriously romantic day at the shore. After acquiring perhaps 20 minutes of material, we were only mildly surprised to discover that, indeed, spinning hard drive disks is a strenuous activity for a camera battery.
By this time, we had already sucked the life out of three of our 24V blocks, and still had 45 minutes (a conservative estimate) of material to capture. We would be out of battery power by 3 p.m., and since the recycling time on the batteries was at least six hours, the first recharge to cycle through wouldn't be ready until 4:30 p.m.
One look up the face of the cliff and we could see the absurdity of trying to run a power cord down from the motor home, so we decided to shut down the S.two between setups in an attempt to save power. This turned out to be a terrible idea because the surge required to reboot the computer made an even greater demand on the batteries. By 2p.m., the whole shoot became a beached whale.
After hauling our gear back up the steps, we powered up the S.two with the generator from the motor home. We picked some creative camera angles and employed other bits of movie magic to make a pretty convincing series of shots at the top of the cliff, which, for all intents and purposes, passed for the shoreline when seen on camera.
We considered our first day to be a huge success because it proved, by catastrophic result, that a different recording method had to be employed on board the ship.
The Drawing Board
Back to the drawing board at Plus8. All signs pointed to Sony's soon-to-be-released SRW1 portable tape-based, 10-bit, 4:4:4 data recorder. So, with the full clout of one of the industry's top rental facilities behind us, the full resources of the world's largest software company, and every last drop of journalistic clout that I could muster, we actually got a soft promise from Sony to supply us with a working, preproduction version of the SRW1.
Of course, this promise came with an awful lot of “ifs,” such as if Sony could complete the finishing touches on the SRW1 in time. We would be the first production to make practical field use of the unit, but only if Sony could deliver it. The unit would have to be shipped directly from Japan, which meant it had to clear customs — a process that could take weeks to complete. Plus, if we successfully received it on time, there would be little option for backup in the field.
Plus8 was able to get us the new Sony SRW5000 so that we had a backup plan, just in case. A huge monster of a studio recording deck, and the only reliable method of capturing significant amounts of 10-bit, 4:4:4, the SRW5000's biggest problem was that it was only available with AC power and not at all weatherproof. But, at least it was available.
With so few options, we had to rely on faith. Our schedule allowed us one pre-light day in the studio and one studio shoot day. That meant we wouldn't actually need the SRW1 until day three of our shoot. We could commit the SRW5000 to the studio day, while praying that the SRW1 arrived in time. If not, our basic plan was to outfit the 5000 with a garbage-bag raincoat and pray that we could wrestle reliable AC from our ship's aging generator.
On checkout day for the main shoot, all we had was the assurance that the SRW1 was, indeed, on our side of the Pacific Ocean, but there was no guarantee that it could be fast-tracked through the import process.
Studio Shoot: Seattle
Our first setup was a high and wide shot of an office cubicle set. Lighting was tweaked, a satisfactory rehearsal completed, and we were ready to give it a shot. But literally just before we started to record, the SRW1 magically arrived. The software incarnation of our little prototype, known as version .023. Talk about shaky confidence.
Amy Beauford, Microsoft's resident compression diva, and Bill Crow cracked open the operations manual for the SRW1 and began the shakedown as we started recording onto the 5000. We introduced the SRW1 into the recording loop about an hour later and never looked back at the SRW5000.
A compact little box, the unit is idiot-proof in design, with simple commands from a remote LCD control panel to trigger the recording. It runs off industry standard 14V Anton Bauer bricks, and does not have any huge power consumption issues. Bill and Amy did uncover a few minor software glitches as we switched between recording formats, but these were easily worked around. Besides, most normal shoots would not switch between 1080i and 4:4:4 RGB recording with any frequency.
The rest of the day went smoothly. Joe and I switched between the Arri 435 and the Viper camera. As we began striking the set and packing up the gear, I took a few grip items that we had borrowed back to the adjacent soundstage. Who comes walking up at that point? Bill Gates, looking happier than a monkey in a banana tree. It seems as though Microsoft stock had generated a substantial dividend, and he had just gotten word that he now had another $30 billion dollars in his pocket. It's always nice to see an old acquaintance doing well.
Location Shoot: Deception Pass
We drove about an hour north of Seattle to Anacortes, where we discovered that our 65ft. chase boat with an electrical generator was only a 40ft. cruiser with a 10-amp power inverter. By the time we stowed our equipment, the cabins were chock full of cases, the galley was stuffed full of battery-charging stations, and Amy had appropriated the dinette as our video village. There would be precious little room to actually open cases and deploy the camera gear, let alone sleep.
With the Lady Washington already under sail, we rushed to Deception Pass, where Alex, Bill, and Joe set up on the top of the steel bridge that spans the strait, while Bob Weback, the Seattle-based camera assistant, and I headed down the cliffs with the 35mm camera.
With a 100ft.-tall main mast, the timing of our shot had to match the tide perfectly in order for Lady Washington to cross under the bridge. We'd have two chances to get the shots we needed if the ship didn't get trapped on the wrong side of the bridge for half a day.
The Lady Washington appeared on the horizon and made several tacks along the shore, waiting for the ideal time to make her crossing through the strait. The sun sank until it cast a brilliant glare on the surface of the water. The tide had ebbed to a point where the ship could risk a crossing. She came about one last time, and then, as her crew scrambled up the ratlines to the yardarms, Joe and I rolled cameras. The spectacle of a tall ship under full sail, with the ship's crew working in a frenzy as it passed directly beneath our cameras, was a sight to behold.
One of our main objectives was to capture the highest quality images possible, so we were using prime lenses on both cameras. The actual speed of the Lady, once the sails were full of wind, caught both Joe and me by surprise as we changed lenses at a frantic pace.
It took about 15 minutes for the Lady Washington to hoist sails and come about. The second pass was even more stunning than the first as she sailed by us at the exact stroke of the magic hour. Joe and I held our shots until the ship was a speck on the coral-colored horizon, and then we wrapped up and drove to where the chase boat was waiting for us.
Ship to Ship: Puget Sound
Sadly, shooting stars will only get you so far. We had generated a substantial wish list for equipment, but we were not able to get everything we had hoped for.
Our first wish was for a gyro head to stabilize the camera while shooting on board the chase boat. If one merely attaches a tripod to the deck of a ship, the orientation of the camera stays fixed with the vessel, and the horizon will appear to pitch and yaw in the finished image. It looks nothing like the reality of being on the ocean, and it's a very disconcerting thing to watch on screen. A gyroscopically stabilized tripod head will orient the camera to the horizon, no matter how far akimbo the rolling of the waves rocks the vessel. Unfortunately, there was no such device available in the Seattle area, and shipping one in would jack the rental price up too much for the budget of our little music video shoot.
Our solution: Joe Gyro, the human Gumby doll. Using some climbing gear and a little ingenuity, we suspended Joe between the gunwales of the chase vessel as he shot the boat-to-boat photography handheld. With his hips gyrating like Elvis and his legs going goofy-foot like a left-handed surfer, our little rig helped him maintain the horizon as the boat rocked beneath him. It was quite a workout, but it made for some exciting shots as we spent the day gathering spectacular footage of the tall ship amid the lush backdrop of Mount Baker and the Puget Sound.
We began our second day at sea by pulling alongside and using an ancient block and tackle from the overhead rigging to help us hoist equipment cases safely aboard the tall ship as we headed for open water.
First item across was the SRW1 for Amy to set up in the ship's hold below deck. Now that we finally had generator power at our disposal, Amy would be able to employ a 23in. Sony 16:9 LCD monitor to view our images.
While Amy and Bill worked on assembling the recording station, Alex and I gave Joe moral support while he climbed up the main mast to get the first series of shots. Once Joe was firmly strapped into position, we hoisted the Viper up to him using the block and tackle once again. A few minutes later, Amy emerged from below with a concerned look on her face. We had a problem.
Even though we were using a different Viper body than we had used at the beach, every parameter of camera setup was the same as it was on prior days. Amy was now seeing a fringe of magenta color around every specular reflection. This was the first time we were using the LCD monitor outside of a studio environment, so we eliminated the possibility of it being an issue with the large-screen device by setting up the Astro alongside.
The problem appeared on both screens. Apparently, it had been missed because we were depending on only the Astro monitor in full daylight ambiance the day before. Now that Amy was able to view the images on a big monitor, down in the dark of the ship's hold, the subtle aberration became visible. The images we shot in the studio did not show the anomaly, because there were no extreme highlight areas in those scenes (interior nightclub with color-gelled sources and a fluorescent-lit office).
The trouble seemed to appear when the highlight areas topped 90 percent luminance. Fortunately, we had cell phone coverage, so we put a call in to George Palmer, an engineer with as much Viper experience as anyone on the planet. Yes, he had seen the problem before. No, there wasn't a way to fix it while floating on the ocean.
Why hadn't this glitch appeared during the equipment checkout? For the same reason it was invisible during our studio shoot: We never tested the dynamic range completely through overexposure. It wasn't until the camera was confronted with the huge contrast range of bright sun and deep shadow that the symptoms appeared.
In any case, we learned a valuable lesson. “From now on, if I plan to shoot with any sort of electronic capture system outdoors, I'll be doing my checkout in the parking lot of the rental house, in full sunlight,” is how Joe put it.
After a bit of head-scratching, we decided that the only option was to just scale down the exposure, relying on the dynamic range of a 10-bit log file to save the day. We would have to do some gamma correction in post, but at least we would avoid the color fringe problem. We put another call in to George, who agreed with our proposed solution. So we reset the zebras in the camera to 75 percent and continued to shoot approximately one stop underexposed. We would later discover that, given our situation, this was our only workable solution.
We spent the rest of the afternoon shooting scenes for the music video and other specific sample images for Microsoft to use for testing compression. One shot, in particular, framed the rigging of the ship in the foreground, creating high-contrast diagonal lines across the frame, as we sailed past a hillside covered with evergreen trees. The jagged edge of the mountainside, against the bright sky, moving past the ropes created a moiré pattern to the naked eye, let alone any artifacts inherent in an electronically captured image. Joyous exclamations from Amy, Alex, and Bill in the ship's hold let us know that everything was looking good.
Day Six on the Lady
For the first shot of the day, we put Joe out on the bowsprit (the long spike that sticks out on the front of the boat), looking back at the deck from out over the water. Once he was in position, we hoisted the camera out to him using our now familiar antique block and tackle as the ship's crew manned the windlass and started raising the anchor.
With Joe firmly in place, Bob and I climbed aboard a small dinghy with the Arri to shoot the anchor as it broke surface. Once it was fished into its final position against the hull of the ship, Joe and I both switched out to wider lenses to catch the mainsail being unfurled. As the ship began to catch the wind and pick up speed, and the bow started dipping and lunging over the waves, Joe began hooting like Slim Pickens riding the A-bomb.
Again, caught by the amazing acceleration of this aging replica, our little dinghy struggled to catch up as the majestic ship sailed away.
By sunset that evening, we had captured more than enough material for both the music video and the other specific shots needed for the Microsoft STeM. We had been blessed with consistently beautiful days adrift in a sea of pixels.
Microsoft intends to make the footage available to everyone, free of charge for testing and standardization purposes. At press time, it was vaulted at FotoKem, Burbank, Calif.
Scott Billups has had an extensive career in writing, directing, visual effects, and cinematography. His latest book, Digital Moviemaking, offers an in-depth look at modern digital movie production. He has also written on digital production techniques and cinematography for a range of publications. Find out more at www.pixelmonger.com.
Continue the discussion on “Crosstalk” the Millimeter Forum.