Report from the SMPTE High Frame Rates Study Group
by Michael Karagosian
published in the September 2012 SMPTE Journal
This intermim report summarizes work to date undertaken in the SMPTE TC-21DC SG High Frame Rates for 3D and 2D D-Cinema Applications. From this study, one can conclude that a compression rate of 500 Mb/s is the norm available in many present day IMBs. However, there is considerable evidence that more than 500 Mb/s will be needed for higher frame rates above 2K 48fps/eye. It is the long term work of this Study Group to ascertain recommended maximum compression bit rates for future equipment. Further, more recent discussions indicate that there is desire in the creative community to achieve higher resolutions and frame rates up to 4K 60fps/eye. (4K 120fps). A final report from the Study Group is expected in late 2013.
The Digital Cinema High Frame Rate (HFR) Study Group is co-chaired by Michael Karagosian (MKPE), David Stump, ASC, and Kommer Kleijn, SBC. The Study Group conducted an "interim" review of issues that must be dealt with in today's cinema distribution environment when releasing HFR movies and trailers. Numerous issues exist that affect the mastering, fulfillment, and exhibition of HFR content. It was found that the fewest number of impacts occur in mastering, and that the most complex issues arise in exhibition.
A summary of the group's findings at the time that this report was prepared:
Mastering
Additions to the standard edit rates and frame rates will be required, similar to the work undertaken with the standardization of additional DCP frame rates in ST428-11:2009, ST429-13:2010 and ISO 26428-11:2011. Additional labels (ULs) to identify high frame rate bit streams will also be necessary. The UL matter is an important consideration, as multiple bit stream rates are likely to be introduced as this technology matures. To accommodate current production, a prototype mastering specification titled P-HFR has been circulated in the industry. Please contact the chairperson of 21DC if this specification is of interest to you.
Distribution, Ingest, and Storage
The digital cinema fulfillment channels are already filled with numerous versions of a release. It’s of great interest to distributors to reduce, or at least not grow, the number of versions required per movie. For this reason, the Study Group explored the possibility of enabling single inventory DCPs that can play on both HFR and lower frame systems. However, manufacturers report that even if it were possible to upgrade products to play HFR content in a manner compatible with legacy content, it would not be possible to do so for all legacy products.
In addition to concerns over multiple inventory content, distributors will experience larger file sizes with HFR content. Typically, 90% of a digital cinema composition is occupied by image data. Assuming that the maximum compression bit rate scales with frame rate, as the frame rate doubles from 24 to 48 frames/sec, the file size of the composition could grow by 190%. If frame rate is increased from 24 to 60 frames/sec, the file size of the composition could grow by 235%. The growth in file sizes also impacts the time required for content transmission, ingest, and movement, as well as impacts storage requirements.
Projection Systems
Not all digital cinema projection systems will accept high frame content. Further, systems that utilize dual-link HD-SDI connections will not have the ability to conduct the sizable image streams required. While the entire data path must support the bit rates needed, of particular concern is that the JPEG2000 decoder in the Image Media Block (IMB) must support higher bit rates than that specified by DCI and ISO. Notably, the P-HFR prototype mastering specification limits the maximum total image bit rate to 500 Mbits/sec, within the maximum limits of current day HFR-capable IMBs. However, there is question as to whether this bit rate is sufficient for future HFR productions, which is the long-term subject matter of the HFR Study Group as described later in this report. (Notably, the P-HFR specification assigns a unique UL to the essence file to identify the specification to which it adheres, providing the mechanism for future higher bit rate distributions to be easily identified.)
In a world with HFR content, a movie trailer sequence might have a mix of legacy 2-D content, legacy 3D 24 frames/sec per eye content, and higher frame rate 2D and 3D content. When playing 3D content of different frame rates, not all 3D add-on systems will respond instantaneously. Delays of 10 sec between compositions are possible while the electro-mechanical components of some 3D add-on systems stabilize. With sequential 3D and HFR, projection flash rates must change with frame rates, and mechanisms to support such changes are needed. For example, sequential 24 frames/sec per eye 3D requires triple-flashing of the image, while 48 frames/sec per eye 3D requires double-flashing.
More worrisome is that systems today are not designed to successfully manage all possible format changes through configuration of the system's show playlist. The issues encountered on present-day equipment when managing shows containing mixed frame rates have been observed to confuse even industry experts. Smarter systems are needed to support the complex nature of forthcoming shows.
On the brighter side, some, if not all, of these problems can be mitigated by streaming mixed frame rate shows over a fixed high frame rate stream to the projector. The fixed frame rate stream would emerge from the media block, eliminating the need for the flashing of left-right images in sequential 3D projectors. More study of this technique is needed.
Forensic Marking
HFR-specific forensic marking technology will be required for digital cinema. The sole supplier of this technology in digital cinema today is working towards a code release in summer of 2012 for 48 frames/sec per eye content.
Subtitling of 2D and 3D Content
Industry tests indicate that some manufacturers of digital cinema equipment have difficulty projecting open subtitles and captions with DCI-compliant 48 frames/sec 2D content. Subtitling with HFR content deserves more study.
3D Add-on Systems/Glasses
3D add-on systems divide into three categories: those that utilize different polarization of light per eye, those based on differently filtered spectrums per eye, and those that shutter sequentially projected images to each eye. Most 3D add-on systems employ some form of electro-mechanical device that requires time to stabilize with a change in frame rate. Such stabilization times may require as long as 10 sec between compositions. Tests conducted by multiple entities show that the majority, if not all, off-the-shelf 3D add-on technologies are operable at higher frame rates.
Color Space and Compression Bandwidth
Digital cinema employs the XYZ absolute color space. However, JPEG 2000 encoders employ the Irreversible Color Transform (ICT) to convert XYZ into a color space that can be more efficiently compressed for the distribution stage. ICT encoding produces a luminance component Y, and chrominance components Cx and Cz.
Luminance describes the brightness of the pixel while the chrominance carries information about its hue. Psycho-visual experiments demonstrate that the human eye is more sensitive to luminance than chrominance, which means that we may neglect larger changes in the chrominance without affecting our perception of the image. As a result, the maximum bandwidth required for luminance component is typically higher than that required for chrominance components.
The current JPEG2000 standard requires products to support up to 250 Mbits/sec for all components combined, and up to 200 Mbits/sec (80%), per component, for frame rates of 60 frames/sec or less. No distinction is made in the component bandwidths required for absolute XYZ color space and the YCxCz color space. As the frame rate increases, the maximum compression bit rate must also increase to preserve image quality. For this reason, HFR content will require a higher maximum compression bit rate. However, when the YCxCz color space is employed, it has been suggested that a lower maximum bandwidth requirement for chrominance components be considered from that of the luminance component.
Further Work
In addition to the study of HFR-induced impacts in the distribution and exhibition workflow, the HFR Study Group is also engaged in the study of compression artifacts at high frame rates. The planning and budgeting of a special shoot of HFR test material is underway, as well as the development of an HFR "super-system" capable of viewing compressed 2K HFR content at up to 1 Gbits/sec compressed bit rates. (The current industry standard is to limit the compressed bit rate to 250 Mbits/sec.) It is the goal of the Study Group to enable others around the world to conduct duplicate studies using the same material and viewing system. Those interested in participating in this study are welcome to contact the Study Group chairs.