The Customer

Our case subject is a prominent China-based media conglomerate, which actively serves its national audience base with broadcast content over television, newspaper, Internet and new media. To protect its privacy, our customer has requested that we write this story under a disguised name. We honor that request and from here on out, will simply refer to it as CSTV, short for Case Study Television Station.

In the 40+ years since inception, CSTV has gradually expanded to operating15 channels, covering a wide array of interests such as news, sitcom, living, law and justice, community, science, international, business, agriculture, shopping, and more. In order to support the daily programming of these various channels, CSTV has an ever-growing demand on its NLE (Non-Linear Editing) system.

Background: Cloud Computing Technology in the TV Broadcasting Industry

In the 1980’s, the IT infrastructure went through a major transition from mainframe computers to client/server structure. In more recent years, the advent of the cloud computing technology and its popularity, has given IT infrastructure another structural transformation. To view cloud computing at the 30,000FT level, it is simply the centralization of traditional computing and storage technologies through networking and management, which forms a public or private resource pool that offers on-demand services to users.

Applying this exact concept, the television industry should streamline its supporting IT infrastructure, by integration of computing resources through virtualization technology, combined with FC (Fibre Channel) and SAN (Storage Area Network) technologies; which would then provide the base for improving its editing and production systems to achieve better resource utilization, higher availability, more stability and structural flexibility.

The application of cloud computing technology is not limited to hightech industries; it is a new technology that the TV industry must also understand and make use of.

Challenge: High-Availability Access and Protection for Digital TV Production Operations

CSTV was planning to upgrade its NLE system to a cloud-based structure, in order to meet its growing demands on product operations such as HD raw material ingestion, video and audio content editing, program auditing, and file-based program play-out. The goal for the new system was to implement a network-based, digitized and procedurized process model, to carry out program production and play-out operations; so as to fully satisfy the heavy loads of postproduction work generated from multiple channels and programs, which also demand flexibility in work spaces and locations. A cloud-based NLE system would extend the production workspace into office spaces, realize the separation of operation from equipment and editing from rendering, and thus improve the efficiency of resource utilization. The project also serves the purpose of pioneering new technologies for CSTV’s future datacenter expansion.

A typical TV program production workflow is that, digital video content must be uploaded from cameras, digitally edited, rendered and then sent to the play-out system from which they are broadcast. Ensuring on-time, uninterrupted broadcasts requires skilled production and workflow management; as well as ensuring performance and availability of systems for ingesting, editing, rendering, and playing-out of program content.

The system’s high availability (HA) must be addressed at all levels, including application, host, network, and storage. A sound HA system should deploy software-clustered hosts and virtual host technology, to ensure the availability of hosts and continuity of applications; as well as fully redundant SAN switches to ensure the availability of network paths. In addition, the storage must be enabled to deliver equivalent availability, so that an end-to-end redundancy from hosts through network to storage may be achieved.

While the applications of host virtualization and networking technologies for HA purpose have largely reached maturity, the same could not be said about storage HA. Therefore selecting a best-fit HA SAN architecture was identified as one of the key requirements for the project team.

Solution: HA-AP HA SAN Architecture

It was clear from the very beginning what CSTV wanted with its new HA SAN architecture:

1. Data must be protected at all times, which ensures the safety of its media asset
2. Data must be available at all times, which ensures business continuity
3. Data access must support sufficient bandwidth, which ensures overall system performance
4. Storage architecture must be open and scalable, which ensures flexibility of future system upgrade

After reviewing options that would fit these objectives, the project team narrowed the choices down to Loxoll’s HA-AP Appliance and EMC VPLEX solutions. Upon further examination, the team found that although both solutions provide storage HA function by mirroring data, however in more than one way, they are actually based on two very different designs and technologies:

1. Solution Type. The VPLEX is a virtual storage solution that also offers HA function, the storage virtualization nature dictates that the original LUN is either “stamped” or re-packaged before it is mapped and presented to the servers. Since servers do not see the original LUN, the LUN would become inaccessible if the gateway should fail.
   On the other hand, the HA-AP is a physical hardware data router built to perform only HA function. HA-AP retains LUN data in its unmodified, native format on disk. If necessary, direct storage system access could be restored by simply removing HA-AP from the data path.
2. Hardware Configuration. The VPLEX is a clustered Linux x86 PC system with multi-core highpower processor, large high-speed cache memory, and 8GB/sec. HBA card. This type of system structure is complex, requires dedicated administrative server and metadata LUN. It would seem that longer installation cycle, more tedious setup and configuration tasks, more complicated maintenance procedures, and higher stability and security risks are all valid concerns with such a system.
   Instead, the HA-AP’s single-purpose, modular enterprise-grade integral design and embedded operating system can deliver reliable HA SAN without any of these drawbacks.
3. Synchronized Write Caching. The VPLEX’s virtualization approach requires high CPU power and large cache memory to process data. The most critical technical challenge is that in order to avoid data corruption, data consistency of cache memory amongst all engines must be maintained at all times.
   In comparison, the HA-AP’s data-routing approach accomplishes LUN synchronization through analyzing standard SCSI3 read/write commands, and dispatching/re-directing data accordingly. Engine cache memory is used for command queuing purpose only, no write-caching is performed and therefore no cache synchronization is necessary, so there is no concern for data integrity.
4. Failover. Upon system component failure, the VPLEX’s clustered Linux structure requires the analysis of various factors in reaction to failure. It usually takes 40 seconds or more to complete the failover, which is more than likely to have a negative impact on the applications.
   Failover. Upon system component failure, the VPLEX’s clustered Linux structure requires the analysis of various factors in reaction to failure. It usually takes 40 seconds or more to complete the failover, which is more than likely to have a negative impact on the applications.

The project team carefully evaluated these findings and came to the conclusion that, the Loxoll HA-AP Appliance would better satisfy CSTV’s objectives set for the new NLE system. With that decision, all pieces of the HA puzzle were then in their right places:

• Two 3PAR StoreServ 7450 all-flash array storage systems at the back-end
• An Oracle RAC database and the cloud-base NLE system on the front-end 
• The Oracle database includes 3 physical servers
• The NLE platform includes 48 physical servers and about 400 VMware-based virtual clients
• A clustered 4-engine Loxoll HA-AP Appliance connecting the front and back ends
• Total data at about 30TB

Benefits: Affordable Zero-downtime Business Continuity

The system has been installed, tested, and in production with highly satisfactory results. The SI’s manager-in-charge, Mr. Zhu, passed on these customer comments, “The Loxoll HA-AP Appliance not only meets our functional and performance requirements, handles failover smoothly without disruption to the applications; the fact that data would remain immediately accessible should HA-AP be removed, is an unique feature that adds extra assurance to data availability.”

Mr. Zhu also drew his own conclusion, “The Loxoll HA-AP Appliance is a very affordable, high priceperformance HA SAN solution. It assures zero-downtime business continuity for our customers.”