• Blogs
• Forums
• Tradeshows
• Partners
• White Papers
• Events

 

Visual History

Dec 1, 2005 12:00 PM, By Tom Patrick McAuliffe

Historical organizations fight to preserve firsthand stories of the past using video and archiving technologies.

Web Expanded
Photo Gallery

Sidebar
Managing Memories

Survivors of the Nazi death camps of World War II relate their journeys as tears slowly roll down their faces. Oral histories of the ancient hula dance and Hawaiian culture are handed down from elders to today's children. Veterans of wars from World War II to the Iraqi conflict tell painful stories of war, service, and bravery. Growing numbers of historical organizations are documenting these vital stories using video technologies — before the storytellers pass away and their stories are lost forever. By recording stories and creating searchable databases of them using metadata and other cataloging technologies, these groups make firsthand accounts available to the generations of today and tomorrow.

Among the historical groups at the forefront of this movement are the USC Shoah Foundation Institute for Visual History and Education, the Hula Preservation Society in Hawaii, and the Veterans History Project of the Library of Congress in Washington, D.C. All three have harnessed the video medium in various ways, and are also using the Internet and metadata technology to preserve recountings of past events for future generations.

Bearing witness

Survivors of the Shoah Visual History Foundation was established by Steven Spielberg in 1994 after the filming of the movie Schindler's List. The original mission of the organization was to videotape the testimonies of Holocaust survivors and witnesses before that generation passes on. Now, with nearly 52,000 videotaped survivor and witness testimonies from 57 countries and in 32 languages, the Shoah foundation is the largest visual history archive in the world.

The Shoah foundation has three main goals: to preserve and provide access to the archive, to build and support educational programs, and to develop educational products based on the testimonies collected. Today, 43 institutions in 16 countries provide educators, researchers, and the general public with remote access to collections of visual history testimonies from the archive. The Shoah foundation's educational programs and resources reach nearly 2 million students around the world. The foundation has also produced various documentaries, including The Last Days, a 1999 Academy Award-winner for Best Feature Documentary.

Sam Gustman, chief technology officer at the Shoah foundation, explains why video is a superior medium for this project. “Human emotion cannot be conveyed as strongly with text or audio. To see the face of a survivor describing the horrors of the Holocaust cannot be conveyed with anything other than or better than video.

“Video was chosen over audio for the interviews because the message is much more powerful when interviewees can explain their story with visual emotion,” Gustman explains. “The testimonies sometimes also include archival photos and location footage where the experiences took place.”

Gustman was responsible for development, production, and support of the technology used to collect 120,000 hours of testimonies, as well as the systems for digitizing, cataloging, distributing, and interfacing with the 180TB video archive.

The Shoah archive and interface represent an enormous investment in time, and were put together during a period when video technology was rapidly evolving. The project started in 1994, before the Internet was widely available — when state-of-the-art compression was MPEG-1, when database- and keyword search-driven interfaces were used mostly by researchers at universities, and when Betacam was the most accessible acquisition format.

After its founding, the first and most urgent goal of the group was simply to collect testimonies. From 1994 through 2000, coordinated through offices around the world, volunteer interviewers and videographers went to the homes of interviewees to collect stories on video. The week before an interviewee would be videotaped, the interviewer would call or meet with the survivor or witness to fill out a survey, which was used by the interviewer to prepare for geographic questions and other specifics that may come up during the interview.

But the team couldn't afford to wait until all the testimonies were gathered to start the cataloging and database-building process. So each survey was also used as a top-level record for creating metadata for the online search system. Keywords from the surveys were cataloged to later allow viewers to search for the video by the same words.

“The video was shot in Betacam SP with the average length of an interview being about 2.5 hours,” Gustman says. “The video and the survey were then sent back to the Shoah foundation for duplication and digitization with a digital Betacam — two VHS copies and a 3mbps MPEG copy made of each testimony.”

At the time, the team estimated that the duplication of all the video alone would take something like eight years. To shorten this, Shoah hired Electrosonic Systems to custom design duplication racks and software that could simultaneously (or separately) create all the video format copies and digitize the footage to MPEG-1 (later MPEG-2 when it became available).

From 1998 through 2005, cataloging professionals watched the digitized testimonies, attaching keywords and naming segments of video to create a searchable digital video library. Indexing workstations were then created to display the video and populate the digital library using keywords from a Z39.19 compliant thesaurus, people's names from the surveys, and snapshots of pictures interviewees showed during their testimonies.

“Video for access is different than video for preservation,” says Gustman. “We currently supply the testimonies on DVD for collections and access requests.”

But tape, not DVD is used for long-term storage. The footage is archived, stored, and preserved on Beta SP and Digital Betacam. The tapes are stored by commercial storage company Iron Mountain at a huge climate-controlled, secure facility in southern California, amongst other locations.

While some groups transfer to DVD for preservation, believing that tape does not last as long as discs, Gustman says DVD is questionable as a long-term storage medium. He cites reports by the federal agency the National Institute of Standards and Technology (NIST). NIST reports indicate that few life expectancy studies on disc storage have been published by independent laboratories. “That steered us away from DVD at the moment,” Gustman says.

An accelerated aging study at NIST estimated the life expectancy of one type of DVD-R for authoring disc to be 30 years if stored at 77°F and 50% relative humidity. This testing is in the preliminary stages, and more needs to be done, NIST says.

In the meantime, Shoah has embraced the Web and streaming as major distribution mediums. “We have about 1,000 users on the Internet2 website over a year, and over 200,000 users on our standard World Wide Web site,” says Gustman. “Regular Internet users cannot use our Internet2 website. Only USC, Rice, Yale, and the University of Michigan can at the moment. 120,000 hours are available on our Internet2 website. Only about three hours of clips are available on our standard website.”

The final digital library accessible on Internet2 consists of 200TB of MPEG material and metadata stored in a Sybase database using SGI, EMC, and ADIC storage systems. Plans are to move the data to a Sybase, Sun, and StorageTek infrastructure at the organization's new home.

Recently, the Shoah foundation and the University of Southern California announced the creation of the USC Shoah Foundation Institute for Visual History and Education, to be housed at the university's College of Letters, Arts, and Sciences. “The move to USC will ensure the long-term preservation and archive access for educational purposes,” Gustman says.

From its vast video archive to the various educational programs it will offer from its new USC home, the Shoah foundation continues to ensure that video of these vital histories will be preserved for future generations.

Preserving cultural heritage

For centuries, native Hawaiians have handed down their history from one generation to another only through chants, stories, and the dancing of hula. As native elders are dying, knowledge about ancient hula is increasingly threatened. Since 2000, the Hula Preservation Society (HPS) has documented more than 100 forms of ancient hula, with more than 300 hours of oral histories from hula dancers and teachers recorded on digital video. This preservation work is literally a race against time.

“The Hula Preservation Society is an educational not-for-profit organization dedicated to preserving the oral histories of hula masters alive today and perpetuating their knowledge through video technology,” says Maile Loo, a co-founder of HPS who serves as producer and lead interviewer.

Loo is a student of “Auntie” Nona Beamer, an authority on ancient hula and Hawaiian culture and the inspiration behind HPS. Loo combined her artificial intelligence studies at Stanford University with her hula background, and conceptualized a searchable cultural online database based on the video footage being gathered.

HPS captures its interviews using standard, affordable video and audio equipment. “Video is a great tool to archive the visual,” Loo says. “Perhaps more importantly, video is able to capture the subtleties of feelings we see on the faces of our elders as they describe experiences and opinions. We know that being able to both see and hear their stories will be an enormous resource for generations to come.”

“Given the nature of the content, no other media was considered. Video was chosen because it was cost effective and offered ready accessibility to recording and playback devices. It was a perfect match.”

For video services, HPS turned to a local pro, Gene Kois, owner of Specific Video in Kailua, Hawaii. Kois has shot video for clients around the world for 30 years.

“You have to admire an organization like HPS and their approach to technology,” says Kois. “They're a small nonprofit effort supported by grants and donations, yet they've been able to acquire a very nice video recording system, and their efforts have resulted in a mind boggling amount of important oral histories being documented rather than lost forever.

“They just don't have the time nor resources to spend getting caught up running from one technology du jour to the next. Their focus is totally on content, and they use the equipment they have to its fullest. That's very admirable.”

When it comes to video production, Kois thinks of himself as a visual interpreter. “I view my job as translating what the client wants to say into a video message. I come up with options on how to visually say it, develop budgets and workable schedules, and then oversee the project to ensure the goals are met,” Kois says.

“Inhouse, I can shoot on BetacamSP or DV, and through various arrangements with other production people and rental houses, I have access to just about any format. My main camera is a Sony D50 with either a BetaSP or DV back, but I also love my trusty old Sony TRV900 MiniDV cam. It just keeps on working. With HPS, I've shot extensively with their Sony DSR-PD150 and DSR-PD170 cameras.”

Kois also says audio is a critical element. “I really depend on Lectrosonic wireless mics,” he says. “I also depend on having a good audio operator working with me. Using two small Sony 170 cameras, I have used up to four wireless systems at once, assigning each to its own channel, and not using any field mixer. It works quite well.”

For editing, Kois mostly uses Apple Final Cut Pro. But he says that if there's enough money in the budget, he prefers to rough cut on his system and finish up at an outside commercial production house. “I find it rather tedious to devote the needed attention to the technical details of nonlinear editing. Plus I recognize that I'm not a great graphic artist. So whenever possible, I do an offline on my system and then finish up with a professional post facility like DBBC in Honolulu. The editor there, Daniel Bernardoni, has an Affinity system by Accom, and a very beefy Final Cut Pro Mac system.”

With a short time frame before this generation of elder hula masters passes on, HPS must focus on capturing the stories of their lives. But HPS is also laying the groundwork for providing access to this collection of irreplaceable cultural information.

“Currently, each video is transcribed word for word and then logged into the elders individual content database,” says Loo. “The originals are stored in a temperature-controlled archive room at our office. Each video is also backed up onto the identical format as the original, and these backups are stored at a separate secure location.”

A number of video and identical audio clips are currently stored as Windows Media files. “When we're able to take the online library out of prototype phase into full functionality, searchability, and content scope, the site will provide our video resources in a variety of formats,” Loo says.

Some edited clips are available at the HPS website, but because it's still a prototype, the clips are searchable only through text linked to it and not through metadata. The organization's current emphasis is on gathering footage.

“As resources in the form of time, funding, and people power increase, we'll be able develop our online presence and upload more clips online,” Loo says. “Ideally, this library will eventually be filled with streaming videos that anyone, including the hundreds of thousands of hula practitioners around the globe, can access.”

Honoring and remembering

Congress created the Veterans History Project (VHP) at the U.S. Library of Congress in 2000. Since then, the project has documented more than 25,000 stories on video from civilians and veterans of World Wars I and II, and the Korean, Vietnam, and Persian Gulf wars. With more than 18 million war veterans living today, according to the U.S. Department of Veterans Affairs, about 1,700 die every day and many have never recorded their wartime recollections.

So far, VHP volunteer videographers across the United States have collected thousands of hours of stories, including some from World War I vets, of which there are only about 100 still alive in the world. With only 20 full-time staff, VHP has received more than 40,000 individual submissions (it also accepts written and audio taped stories) increasing its archive by tens of thousands of items. The project receives more than 200 submissions a week from volunteer interviewers and family members of veterans around the country. It has also published two books in conjunction with National Geographic.

“The Veterans History Project is important because it's a people project that documents and preserves the contributions made by service men and women in wartime,” says Diane Kresh, director of the VHP.

While the project accepts written and spoken stories from vets and civilians, video is a popular and growing part of the collection. “As a professional folklorist, I use video frequently to document oral histories,” says Peter Bartis, senior program officer at VHP. “The Veterans History Project's objective is to collect first-person unedited footage. We mostly use our two Sony DSR-PD150 DV video cameras.

“We discourage editing, but we know that many of our participants do edit copy footage as a special project for community presentation.”

One of the challenges VHP faces is the sheer volume of video submissions it receives. Video interviews donated to VHP come in many formats — from VHS to DVD to Hi-8 to Betacam — created on both consumer and commercial equipment.

“Accepting this wide variety of formats ensures that anyone can participate, even if they don't have access to the latest recording equipment,” says VHP archivist Eileen Simon. Each video submission must be archived, stored, and preserved.

“Each collection is assigned a unique identification number for tracking,” Simon explains. “Our team of processing technicians records information about the interviewee, such as the war and branch of the military they served in, rank, unit, locations of service, medals, etc., and it's input into a database.

“When available, information about the content of the interview is also recorded so that we know the format of the recording, approximate length, and major topics in the interview.”

The tapes are then stored in ID number order by format in archival media boxes and kept in a secure, climate-controlled area.

Perhaps the primary challenge for any collection, and especially one as large as this, is making the information widely available. Making video searchable is a time-consuming task.

“All interviews are currently cataloged in an XML-based online database by the interviewee. So if you want to find veterans that are talking about their experiences in, say, the Battle of the Bulge, you would have to search for veterans by unit and then listen to the files,” says Neil Huntley, VHP's digital conversion specialist. “We're in the process of creating transcriptions of interviews and marking them up in XML so people can search them.”

VHP's digital initiative program uses the Metadata Encoding and Transmission Standard (METS), along with standard library practices for recording metadata for every object in a digital collection.

“We're also doing research into voice recognition software products such as Virage for use in making our webcast presentations compliant,” Huntley says. “The idea is to be able to take the output of voice recognition/captioning software and use it in some type of search interface.”

In addition to creating a searchable history collection with the latest technologies, Huntley is the point man for the Library of Congress streaming projects.

“We're adding new collections on a quarterly basis. Currently we have over 1,000 hours of audio and video interviews being streamed via Real Media Player,” he says. “We digitize the audio and video either inhouse or by our current contractor Vidipax. Audio is digitized as a .wav file at 24/96 and video as MPEG-2 files at 3mbps 704×480. These become digital masters that are archived on one of our servers. We make Real derivatives, which are uploaded to our streaming media server. From there, it's just a matter of creating a record in our XML database and the link for the file will appear in the veterans collection online.”

The system is flexible and other media types can be easily added if needed, though currently it is not offering Flash, QuickTime, or Windows Media content. The Library of Congress is also researching use of JPEG2000 and Motion JPEG2000, along with MPEG-4, Huntley says.

VHP's work to offer media online and in its archives increases our access to firsthand accounts of wartime by America's service men and women. Its presence guarantees that the importance of previous generations' sacrifices will not be forgotten or lost through the passage of time.

For the future

Video offers so many advantages to preservation programs like these. Whether they're at the higher end of funding and technology like the USC Shoah Visual History Foundation, or a more modestly funded effort like the Hula Preservation Society, or publicly mandated like the Veterans History Project, these groups have all harnessed the power of video, the Internet, and database technologies to preserve and educate.

These video collections and others like them provide everyone from scholars and researchers to elementary school students with valuable vehicles for education and cultural preservation.

“These stories become part of the fabric of American history, and whether it's video or audio, the more people that become involved in projects like these, the richer and more diverse the collections,” says Kresh of the VHP. “And time is of the essence as older generations pass on. We are visual people these days, and video helps us preserve and share this history.”

---

For more information please visit:

Survivors of the Shoah Visual History Foundation
www.vhf.org

The Hula Preservation Society
hulapreservation.org

Veterans History Project of the Library of Congress
www.loc.gov/vets

---

Video Systems contributing writer Tom Patrick McAuliffe is a journalist, entertainer, and video creator based in Hawaii.

---

Managing Memories

With thousands of videos, audio tapes, and pages of written testimony, managing their collections is a formidable challenge for these organizations. Once testimony and stories are documented, the content is prepared and preserved for distribution and storage. Interviews donated to the organizations come in many formats, from VHS and DV to DVD and Hi-8 and BetacamSP for video; reel to reel, cassette, or DAT audio tapes for audio; and .doc Word files for print testimony.

Once content is acquired, it is viewed or listened to, the content is cataloged with a description and is prepared for further use. Each story is assigned an identification number for tracking (later this number will serve for storage in the same way a call number labels a book on a shelf) and each tape can be found via its control number using a centralized database. When available, information about the interview content is also recorded so database users can find out the format, approximate length, and major topics in the interview.

In more sophisticated instances, this information is converted to metadata, invisible searchable information that is associated with each story. This allows viewers, researchers, and educators to more effectively search a collection's content. For example, using metadata, users could query a database looking for a specific location, event, action, or person.

“This information allows us to search for and retrieve interviews that address particular subjects for researchers,” explains Eileen Simon, Veterans History Project archivist. “This is why we urge participants to index their interviews once they're completed by replaying the recording and making notes on content.”

Some of the solutions used to embed metadata language include: Xt Metadata Editor (Xtme); the SGML language using a text editor; and MP, which is a compiler used to parse formal metadata. But metadata protocol is like a computer language, so you can't buy a software solution to perform encoding of the information. It must be done by hand.

Once examined and cataloged, video and audio are then digitized via computer into various digital formats like MPEG-2, MPEG-4, Windows Media, or QuickTime. Using software like Autodesk Cleaner, Sorenson Squeeze, or RealProducer from RealNetworks, users can hook up their video sources directly to their computers via IEEE-1394, or, with a hardware video card, users can connect analog video sources via YUV component, Y/C, or composite signals. Digital Voodoo's SD|Flex card is a good example of this type of input/output (I/O) solution. The card has standard definition and high definition capabilities with AES/EBU audio. It offers users the choice of SDI, Component (YUV), S-Video (Y/C), or composite video in and out. Companies like Canopus, Bluefish 444, and Pinnacle Systems also offer such hardware solutions.

The original submitted tape is converted to tape formats like BetacamSP or Digital Betacam for storage. For stories recorded with audio only, digital audio tape or CDs can be used. DVDs can be authored using applications like Adobe Encore DVD, Sonic DVDit, or Apple DVDStudio Pro. Once created, DVDs can be duplicated with hardware solutions like the Bravo XR Disc Publisher from Primera Technology and Sony DVDirect.

At this point the video, audio, and written testimonies are ready for distribution via traditional tape and disc methods, books and print articles, or via Internet streaming and web download on demand. To preserve these collections, all three of the organizations in this article have chosen videotape as an archival medium. Once cataloged, the video, audio, and paper documents are usually duplicated. The original and a copy are placed in storage. Storage may be onsite or in a secure off site location.

The sheer volume of content, the need for user accessibility, and the goal of preserving this content for the future makes a well thought-out workflow essential for these history projects.
— TP

---

feedback

To comment on this article, email the Video Systems editorial staff at vsfeedback@primediabusiness.com.

© 2008 Penton Media, Inc.