• Blogs
• Forums
• Tradeshows
• Partners
• White Papers
• Podcast Show Notes
• Events

Unleashing The Digital Postproduction Infrastructure

Jun 1, 2005 2:37 PM, By Exavio Inc.
3121 Jay Street
Santa Clara, CA 95054
Phone: (408) 213-5500

www.exavio.com

Abstract

---

"Unleashing The Digital Postproduction Infrastructure" examines today’s architectures for digital processing for postproduction and broadcast applications and provides recommendations for improving the overall performance and efficiency through an evolutionary, non-disruptive implementation plan. Production and broadcast data centers have tremendous amounts of capital invested in their equipment and when application processing performance fails to meet end users’ demands, managers are faced with difficult choices to achieve improved performance levels at acceptable costs. Scalability options are limited and do not address the most significant problem—sub-optimal performance for data access in the face of growing media resolution and user and storage requirements. This issue becomes the primary reason for media infrastructure inefficiency, even when high-speed interconnects such as Gigabit Ethernet or Fibre channel are used to connect clients, servers, and storage. As media data center managers confront this problem, a cost-effective solution is emerging that delivers immediate return on investment. This paper examines the technical advantages of the emerging Exavio ExaMax 9000 I/O Accelerator platform for production workflow, digital intermediaries, digital media delivery, and storage—all while preserving the existing investment in storage, server, and network infrastructure.

Discussion

---

Introduction—Digital Infrastructure for Postproduction

Most of today’s films and television programs are produced as a shared effort or moving into a shared environment. To handle different aspects of this creative process, multiple creative professionals use specialized computer-based applications for effects, non-linear editing, color correction, special effect creation, rendering, and so on.

Traditionally, the production process followed a sequential workflow, and content was copied from one machine to another—as media (film, audio, or video) and, more recently, as data. Increasingly, production houses are looking for an end-to-end digital infrastructure to simplify the production cycle and enable their creative staff to be more productive, focusing on eliminating file copying while maintaining reliable delivery of content to the artists when they need it. Using a shared storage and workflow environment completely eliminates the need for sequential data transfer among different workstations. A substantial amount of time and effort is saved for something that is, and should not be, an integral part of the content production and creation process.

Figure 1 shows the three components of digital infrastructure of production environments and clients, servers, and storage. Clients include film scanners and nonlinear editors; graphics, effects, and compositing workstations; color correction and retouching systems; and video servers.

The primary role for servers in a media facility is to store digital media content for recording and playback using built-in or direct attach storage (DAS). In addition, network-based storage systems comprised of network attached (NAS) and storage area network (SAN) variants may be used to keep digital media data instantly at hand using high-speed networks and hard disk drives.

Infrastructure Challenges for Optimal Production

Today’s digital production environment is very much akin to software development in the 1980s, prior to the advent of distributed development infrastructure. At that time, each software developer typically worked in isolation on a single component of the overall project. Such methodology limited the complexity of software projects, with long development and testing cycles being the norm. Today, distributed development tools enable greatly improved scope and scale of software development—a software development project may include thousands of programmers collaborating across a local or wide-area network.

Production infrastructure components are facing similar challenges which directly impact complexity, cost, and performance. The following requirements are particularly important for postproduction clients:

• Newer formats are emerging that require more network bandwidth than ever before or even possible. For example, HD digital film intermediary streams require approximately 160MBps, 320MBps, and 1200MBps bandwidth per client. In practice, Gigabit Ethernet delivers about 60MBps to 70MBps, while Fibre Channel networking can move 75MBps to 80MBps across a 1Gb connection and 150MBps to 160MBps across a 2Gb connection. To be able to ingest such high-speed data directly off the source without any data loss imposes an immediate throughput requirement on the storage systems.
• High-performance, seamless, and robust schemes for file sharing among a range of heterogeneous production clients, including Windows and Macintosh workstations and Linux and UNIX servers, are critical requirements. File copying from client to client is an inefficient use of the resources. For example, transferring a one-hour, 2K-resolution sequence (over 1TB) can take many hours to many days, depending on network congestion.

There are two specific challenges pertaining to disk drive performance and bus-based architecture that limit the ability of DAS servers to provide a central, scalable, and reliable media delivery function.

• A key goal of a truly collaborative production environment is that all users have realtime access to shared media content. While storage capacity and processing power have increased by hundreds or even thousands of times, disk drive performance has only increased a modest fourfold due to mechanical limitations that hamper seek, read, and write times. This can be a serious problem when multiple copies of an asset are requested—reducing server network I/O throughput by as much as 90 percent.
• In traditional bus-based server architectures, processing a media client servicing a request requires data traveling multiple times between the CPU and system memory, which severely limits overall network I/O performance. Typically, when a network I/O device initiates a data transfer, the CPU contacts the disk controller, which supplies the data to server memory. The CPU then tells the network I/O device where the data can be located in memory and the network I/O device then requests the data from memory, completing the multi-step data transfer.

While network-based storage systems, such as NAS and SAN-based storage systems, enable storage consolidation, they continue to provide sub-optimal performance due to a related set of unmet needs.

• While SAN and NAS storage address the problem of DAS-driven data duplication, they also permit clients across local and wide-area networks (LAN/WAN) to access a single pool of media data, which exacerbates the disk drive I/O bottleneck. The use of higher-performance drives, such as Fibre Channel drives, only marginally helps address this problem without keeping up with the increasing network I/O throughput requirements.

While open Linux-based servers are becoming more widely accepted for postproduction computation, such as for 3D rendering and non-interactive restoration applications, proprietary servers utilizing UNIX variants continue to be used for media delivery. There are a number of important challenges facing such server systems in providing a central, scalable, and reliable media storage and delivery function:

• Most media delivery servers are at or near maximum disk drive storage and network I/O capacity and, unfortunately, investing in additional CPU capacity (using "vertical" or "horizontal" scaling) yields only marginal gains in overall performance, and at costs that increase geometrically. At the root of the media server I/O bottleneck are the facts that disk drive storage, network bandwidth, and traffic loads have, and will continue to, consistently outpaced CPU performance increases, and that a large portion of server CPU cycles are being used to process the communication protocol overhead.
• There are several bottlenecks in today’s media servers. The first is the access throughput to the content on disk drive storage. Specifically, disk drive based storage system fundamentally limits the speed at which data is written to, or read from, the system. For multiple read-and-write sessions to the disk drives, the throughput can degrade rapidly because of the random disk read/write actuator movement, sometimes by as much as 90 percent.
• The next bottleneck is the host system bus bandwidth. Specifically, the most common bus interface is the PCI or PCI-X bus, which has a native bandwidth of up to 8Gbps. Factoring in the built-in overhead to support the bus protocols, the net achievable bus bandwidth is usually no more than half of that, at 4Gbps. In addition, on its way to the network output ports, the data must transit the bus at least twice, possibly through the CPUs, to the network output, which reduces the achievable throughput to half of 4Gbps, at 2Gbps.
• The last bottleneck is the network protocol processing. For example, the TCP/ IP-based protocol can easily eat up a major chunk of the CPU cycles, sometimes up to 50 percent of a P4 class CPU. An off-load protocol processing engine can help alleviate the problem. But because the above bottlenecks are in a serial relationship, removing or expanding a single bottleneck does not necessarily increase the overall performance; only an entirely new way of architecting the solution can eliminate the bottlenecks.
• Postproduction client applications require that data be delivered to, or from, storage in isochronous fashion, or at a fixed rate, with little scope for interruption or variation. This requirement is incompatible with the use of standard gigabit networking because it is almost impossible to guarantee isochronous data delivery over TCP/IP networks.

Media storage systems also suffer from a related set of unmet needs. These include:

• General-purpose storage systems deployed in production environments are not optimized for media traffic characteristics and, therefore, provide sub-optimal performance for production applications. Media data consists of large, multi-gigabyte files primarily accessed in sustained and high-speed read/write fashion, while standard storage systems and access methods are optimized for small file data accessed in transactional fashion.
• Existing storage systems cannot seamlessly and cost-effectively meet the exponential growth in media storage capacity and lifecycle management requirements driven by newer resolution types, such as HD, 2K, and 4K. The traditional approach of monolithic online, near-line, and archival storage system hinders taking advantage of cost-effective drive technologies, such as Serial ATA (SATA), and data management automation. At the same time, the traditional approach cannot provide the necessary level of I/O performance.

For the first time in the industry, a media-optimized delivery server and storage solution is available that does not disrupt the existing postproduction digital infrastructure and provides significantly better performance and management at a fraction of the cost—all while preserving the existing investment in your server, network, and storage infrastructure. This solution allows you to upgrade your postproduction data center to accommodate additional user and storage requirements in an evolutionary fashion using existing management tools and interfaces. In addition, this solution requires no modifications to existing applications.

Evaluation of Media Delivery Alternatives

In production environments where both human and monetary resources are constrained, a simple solution is required for addressing media delivery server I/O problems: a non-disruptive approach that allows better utilization of the existing server and storage infrastructure, and at the same time dramatically increasing the system performance with new add-on technology. Today, media production IT managers have four options when they want to improve the performance of their media delivery server infrastructure:

• Overprovision
  Continue to use the same approach to boost server performance—add more DAS servers to share the load. Individual servers have dedicated storage devices attached to them, providing direct data access to built-in storage. As a result, content may need to be replicated and exchange of content may require file transfer or removable media delivery.
• Speed Ethernet and Fibre Channel I/O connectivity within the existing data center
  Increase the speed of each connection from xGbps to yGbps and add TCP/IP Offload Engines (TOEs) to all network interface cards (NICs) in each existing server. This will solve the server network I/O problem without addressing the storage I/O problem.
• Replace existing server-to-server and server-to-storage connections with new technology
  As new interconnect technologies such as InfiniBan emerge, effect a wholesale replacement of existing Ethernet and Fibre Channel infrastructure with the new technology
• Utilize emerging server platforms optimized for media delivery
  Deploy "next generation" specialized media delivery servers that integrate server blades using fast internal connection schemes and unified architectures
• Maintain existing Ethernet and Fibre Channel networks
• Mix and match I/O options, including 1/10Gigabit Ethernet, 1G/2G Fibre Channel, and 4X/12X InfibiBand, as server network I/O connectivity requirements grow
• Preserve investment in management software and staff training.

The first option, DAS over provisioning, represents the status quo—it has proven overly expensive both in terms of capital equipment as well as personnel, and has delivered only marginal performance improvement. Most importantly, it does not allow for cost-effective scalable growth of the network and storage system as the performance requirement grows. DAS is the simplest storage model with storage directly and exclusively attached to a media server to assure consistent data performance.

However, there are two significant disadvantages to DAS in that storage cannot easily be seen or shared by other users within the facility, creating a bottleneck and cumbersome management in face of capacity growth. If another user requires access to the data stored on a DAS server, time-consuming LAN-based copying must be employed. As storage requirements grow, so do burdensome DAS administrative tasks, such as backup and changing and adding storage devices. As a result, DAS typically has far lower storage utilization than other shared storage models. The second option, installing higher-bandwidth Fibre Channel and Gigabit Ethernet I/O connections plus Gigabit Ethernet TOE (TCP Offload Engine) devices, would seem to present an attractive alternative. However, higher bandwidth networks can provide no improvement since they do nothing to mitigate the media server I/O bottleneck. These would simply serve to reduce network utilization (same data throughput over a higher bandwidth connection) and further magnify the impact of the underlying storage I/O bandwidth bottleneck.

The third option, replacing the existing data centers’ Ethernet and Fibre Channel infrastructure with a new technology such as InfiniBand, may solve the media server interconnect I/O bottleneck problem. This is a feasible solution for data centers that are being created from the ground up; however, there are very few of these. For existing data centers attempting to supplant existing Ethernet and Fibre Channel infrastructure with InfiniBand would be extremely disruptive. Even if disruption were not an issue, the evolution of an InfiniBand ecosystem has not progressed sufficiently to make the risk of wholesale Ethernet replacement acceptable.

The last option, moving server network I/O from individual media delivery servers to an media-optimized server platform, makes the most sense for today’s data center managers concerned about greater performance, no disruption, immediate ROI, and ability to more easily adopt evolving data center network technologies (e.g., 10Gbps Ethernet and 2Gbps, 4Gbps, or 10Gbps Fibre Channel).

In essence, such a platform allows moving server network I/O from individual servers to an optimized platform that consists of server blades with an interface to a high-speed switching fabric, all integrated into a single chassis. Deployment of such a media delivery platform addresses today’s performance problems and is the industry’s first real and complete solution that can be introduced in a seamless and non-disruptive manner.

Comparing Media Storage Options

A key media IT imperative is to evaluate storage approaches that address bottlenecks to to the delivery of near-instantaneous shared access to data from multiple computers and multiple processes within a workflow, while increasing operating leverage through simplified management.

The limitations of production computing environments become increasingly apparent as they incorporate network storage solutions to improve performance for shared access to growing data. There are three major options available to production IT for improving storage access performance, scalability, and management:

1. Use of NAS storage systems (file servers).

• Deploy Gigabit Ethernet and CIFS/NFS-based file servers to enable facility-wide data sharing among heterogeneous media clients.

2. Deploy SAN file systems.

• Layer a shared file system atop a Fibre Channel SAN for high-performance file sharing among heterogeneous clients.

3. Utilize emerging storage systems optimized for media data access and management.

• Deploy storage systems allowing high-performance file sharing among NAS and SAN clients
• Cost-effective media data lifecycle management is assured including built-in support for tiered storage

Standard file servers represent a feasible way to serve files to media clients, such as rendering application servers, whose network requirements are not particularly demanding. In addition, NAS systems, due to the overhead of TCP/IP, NFS, and CIFS processing, are limited in the number of clients that can be supported with a reasonable quality. If more clients are needed, then the data storage must be replicated, which leads to management overhead or compromised Quality of Service (QOS). In addition, network file system approaches require substantial amount of network protocol and file system processing overhead.

While Fibre Channel SANs optimize the efficient transfer of block data, which is critically important for media tasks, SAN file systems provide value-added functionality for robust SAN-based file sharing. Figure 2 depicts SAN file systems, including client software running on SAN-attached servers and an out-of-band metadata server for mediating read/write file access to SAN storage resources.

As sites deploy SAN file systems, they frequently run into two limitations: a performance-limiting approach for integrating NAS clients and closed, vendor-specific approaches. In addition, deployment of SAN based solutions does not eliminate the fact that the disk drive I/O bottleneck remains in place.

The typical approach for integrating all NAS clients into a SAN file system environment is through the SAN-attached metadata server acting as a "NAS head". While this approach may suffice for a small number of NAS clients sharing a FC link to SAN-based data access, it offers sub-optimal performance as the number of NAS clients grow. In addition, SAN file systems require vendor specific SAN infrastructure and storage systems, locking users into closed, expensive offerings.

The final option provides the best, most cost-effective approach for client access and management of growing media storage capacity. Such systems enable high-performance file sharing among SAN and NAS-enabled clients optimized for read-intensive network I/O bound media traffic, provide automated and centralized management of online, near-line, and archival storage, as well as extensibility in supporting new server and storage networking options. Unlike, the SAN file system option described previously, high-performance, low-overhead connectivity is employed to integrate NAS and SAN networks. In addition, such systems are open, enabling their use with multi-vendor SAN switching and media storage infrastructure.

Enter Exavio — Solving Today’s Production Network and Storage Problems While Preparing for Tomorrow

Exavio’s ExaMax 9000 I/O Accelerator and their ExaVault Media Storage System combination is the first product suite expressly designed to address the media delivery and storage challenges of today’s production environments and maximize the value of available resources in a non-disruptive, evolutionary fashion. As shown in Figure 3, the ExaMax 9000 I/O Accelerator unites network and storage I/O into a media-optimized delivery server that removes network overhead, simplifies existing network cabling, and facilitates infusion of data and storage networking technology upgrades, while ExaVault provides a highly cost-effective storage platform for full media lifecycle data management. These benefits are provided with no changes to the application software, while maintaining full compatibility with existing user and application software interfaces.

The ExaMax 9000 I/O Accelerator solution is an optimized media delivery platform that uses blade architecture and an advanced non-blocking fabric to deliver full line-rate support for up to 36 ports for media delivery or storage access at Gigabit rates or higher. The ExaMax 9000 seamlessly supports third party shared file systems and provides high-throughput client access to content residing on the storage systems connected to it. This is achieved through a combination of aggregation, caching and intelligent content processing capabilities implemented within the ExaMax 9000. In addition, the ExaMax 9000 enables integration of SAN and NAS devices. This bridges the gap between performance-oriented SAN and the simplicity and lower cost of NAS.

Features include:

• Each Examax 9000 interface can be configured to provide NAS (CIFS or NFS) or 2G FC SAN support. The use of dedicated processors on a per-interface basis minimizes network overhead. The ExaMax 9000 FC ports can utilize existing FC SAN storage as well as Exavio ExaVault media-optimized storage.
• The ExaMax 9000 supports the unique ExaFS file system that allows seamless and high- performance file sharing among NAS and SAN clients. Unlike traditional SAN file system approaches, the ExaMax 9000 allows NAS and SAN clients to be full peers with additional blades and ports to be configured on an as-needed basis. ExaFS includes FATpipe capability for aggregation of multiple Gigabit Ethernet or FC ports on a per-client basis for realtime viewing of high-resolution streams, such as HD digital file intermediary streams.
• The ExaMax 9000 allows automated Quality of Service schemes for prioritized and guaranteed media delivery on a policy-based per-client basis.
• The ExaMax 9000 architecture includes a large RAM cache. A dynamic cache algorithm completely removes the bottlenecks created by spindle speed, spindle contention, and the limited access times of the disk drives. The large cache drastically amplifies the throughput by absorbing large amounts data in the cache in realtime while accessing the data onto the storage device in the background.
• While providing vastly improved I/O throughput with its dynamic caching algorithm, the ExaMax 9000 cache can also be configured as a static cache, which, in effect, performs as a cache LUN. The cache then becomes a super-fast logical drive. Under the same clustering file system, multiple users can share the files residing on this LUN at a super-fast access speed. Both dynamic and static portions of the cache can co-exist in the system, and the size of dynamic and static cache is user-configurable.

Exavio Value Proposition Summary

The Exavio ExaMax 9000 I/O Accelerator and ExaVault media delivery and storage solution provides a number of significant benefits and attractive features.

• Improved operational efficiency. By combining server and storage I/O into the ExaMax 9000, there are fewer connection points, fewer cables, fewer adapter cards, and easier upgrades to existing networks. In addition, for some data centers, more productive servers can result in fewer servers, which reduce acquisition and ongoing maintenance and management expenses. The cost, scalability, and performance of ExaVault make it a highly attractive integrated platform for full media lifecycle management and consolidation.
• Increased server performance and network utilization. The full overhead of network I/O processing is removed from media application servers. Media network delivery and storage I/O is consolidated in the ExaMax 9000. Both network and storage I/O networks are connected to the ExaMax 9000, which interface to application servers using a non-proprietary GbE or 2G FC paths. This removes network overhead and significantly increases effective I/O.
• Implementation with little to no risk of disruption to media data center infrastructure and practices. The Exavio solution was designed from the beginning with simplicity in mind. Rather than introduce new interfaces to configure equipment, the ExaMax 9000 and ExaVault Media Storage system use existing NAS and SAN configuration and management interfaces. In addition, server applications are totally unaware that their I/O has been combined into ExaMax 9000.

Conclusion

---

A complete media-optimized delivery and storage solution offers an economically compelling solution for the single most important factor leading to data center inefficiencies: sub-optimal data access performance in face of growing media resolution, storage capacity, and user requirements. In developing the industry’s first media delivery and storage platform, Exavio offers postproduction IT managers a new level of cost effectiveness, performance, and manageability, without disrupting their existing data center infrastructure or architecture. Exavio’s ExaMax 9000 I/O Acclerator and ExaVault Media Storage product suite can be deployed today to yield immediate, quantifiable price/performance improvements.

Continue the discussion on “Crosstalk” the Millimeter Forum.
© 2010 Penton Media, Inc.