The IMF UG held a plugfest on June 26, 2024 at Universal Studios in Los Angeles (USA), with remote participation through shared cloud storage and virtual breakout sessions. The plugfest gathered more than 20 participants from the content creator, user, and implementer communities.

Throughout the day, participants created, exchanged, and consumed IMF content. Cross-checking IMF content between implementations ensures the standard is supported and helps reveal potential improvements to both implementations and the standard.

This report summarizes the use cases studied during the plugfest.

High Throughput JPEG 2000 (HTJ2K)

HTJ2K significantly improves the encoding and decoding speed of JPEG 2000, reducing processing costs and allowing real time playback on consumer desktops and laptops. This use case involved QC and playback of compositions that used HTJ2K-encoded video.

Use case and source content provided by NBCUniversal and The Walt Disney Studios.

Photon

Photon is an open-source library for parsing, interpreting and validating constituent files that make an Interoperable Master Package (IMP). These include: 

• AssetMap (SMPTE ST 2067-2)
• PackingList (SMPTE ST 2067-2)
• Composition Playlist (SMPTE ST 2067-3)
• IMF track files (SMPTE ST 2067-5)

Currently, Photon provides support for IMF Application #2E (SMPTE ST 2067-21) and Application #5 ACES (SMPTE ST 2067-50), and the Immersive Audio Bitstream (IAB) Plug-in (SMPTE ST 2067-201).

For this plugfest, a pull request against Photon was created by Pierre-Anthony Lemieux that included additional checks for HTJ2K codestream header metadata. This build was used for the validation of the plugfest content.

Test content created with Colorfront, MTI, Blackmagic Design, and Telestream technologies.

No-Proxy Streaming

Using no-proxy HTJ2K MXF JavaScript player, files are read from an AWS S3 bucket with pre-signed URLs. The MXFs are played directly in the browser using JavaScript, there is no proxy involved. If connection is limited, it will fetch only the low-resolution wavelet information and show a low-resolution image during playback. Pausing playback will fetch all the data for that frame. For lossless IMFs, this is the lossless data, for lossy IMFs this is the high quality you would experience from a normal full decode.

Traditional IMF J2K part 1 encoding uses the CPRL progression order, for this player to work the no-proxy player needs HTJ2K with RPCL progression order. The no-proxy player is currently limited to 4:4:4 sources. While the 4:2:2 source will decode, it will only show the actual Y, and half-vertical with Cb and Cr that is associated with 4:2:2.

Observations

The goal of this plugfest was to focus on creation and interchange of HTJ2K within IMF. The focus has increasingly pointed to the need for the community to have consistent ways of validating content.

1. The plugfest showed that the lack of a common validation tool and/or test vectors introduces friction. The IMFUG should consider an effort to support a common validation strategy and/or toolset that includes codestream analysis.
2. HTJ2K no-proxy streaming functions as expected when source materials are encoded with RPCL progression order. Some HTJ2K content was still found to have the CPRL and LRCP progression orders which did not work.
3. Colorfront enhanced their validation toolset to include codestream inspection with comparison of the essence and metadata values. This helped the plugfest identify additional issues with submitted packages.
4. Although most solutions have the ability to create HTJ2K IMF, errors were still present that included:
   • CPRL and LRCP progression order
   • Missing TLM marker
   • Incorrect schema definition in CPL
   • Invalid HTJ2K tile part count
   • MXF essence and metadata values not matching

Future plugfests

We are already working on future plugfests, possibly as early as November 2024. In the meantime, testing should never stop, and the content used during the plugfest remains available to all IMF UG members.

Join the UG today to get access to past plugfest content and help the community plan future plugfests!

Acknowledgments

Many thanks to Pierre-Anthony Lemieux, Harvey Landy, Mike Krause, and Dave Deelo for their contributions to the program committee; to NBCUniversal who hosted the event; and to Amazon Studios who provides cloud services for the IMF UG.

Attending IBC 2024? Get to know the Interoperable Master Format (IMF) better at following exhibitors!

Ad Signal

6.C28b

Ad Signal has a range of unique SaaS products. Our Match product deduplicates content regardless of format, codec and resolution. Our patent-pending Compose product is able to create IMF from traditional content fully automatically; paired with Match we identify all of the unique content at the best quality to keep and Compose turns these assets into a single IMF package with all the required CPLs etc. We believe this is the only product capable of doing this to date.

Contact: tom.dunning@ad-signal.io or Meet at IBC

Tedial

1.B18

The smartWork platform offers comprehensive IMF support from acquisition to delivery. With smartWork, customers can select third-party systems tailored to their specific needs, such as mastering, transcoding, QC, and AI tools. By natively storing IMF, smartWork | EVO maximizes standard benefits while simplifying user interaction with the package’s internal structure.

Contact: sales@tedial.com

Ateliere Creative Technologies

5.C24

Ateliere Creative Technologies revolutionizes the media supply chain by delivering AI-driven, cloud-native solutions that streamline content creation, management, and distribution from concept to consumer. Our platform, Ateliere Connect, automates media workflows, reduces storage costs, and accelerates time-to-market, enabling clients to maximize their media profitability. A key component of our service is our integration of the Interoperable Master Format. By leveraging IMF’s component-based architecture, Ateliere helps media companies significantly reduce their online storage footprint, enhance content management, and simplify global distribution, ensuring seamless delivery across all platforms.

Rohde & Schwarz

7.B21

The R&S®CLIPSTER is the gold standard solution for mastering and distribution of feature films and episodic TV. It provides a powerful way to edit any type of media, in any resolution, and create high-quality professional deliverables that meet stringent, professional delivery specifications. Featuring a highly refined IMF delivery wizard R&S®CLIPSTER is able to simplify the creation of IMF’s, so that the outputs are constrained to comply with standards and ensure interoperability. R&S®CLIPSTER makes versioning IMF’s quick and easy and features a dedicated tool to merge and combine multiple IMFs into a single package.

Contact: matthew.ellsworth@rohde-schwarz.com

Colorfront

Private Demo Suite – Okura Hotel

Colorfront, headquartered in Budapest with offices in Los Angeles and sales partners worldwide, is a leader in IMF and DCP mastering and known for its award-winning on-set dailies and transcoding systems used by companies of all sizes for Hollywood blockbusters, high-end TV, and OTT entertainment. Founded in 2000 by Mark and Aron Jaszberenyi, the company combines expertise in image color science and software development, earning an Academy Award for its innovations. Today, Colorfront is renowned for its camera-to-post products and offers advanced cloud services alongside a state-of-the-art DI and post-production facility in Budapest.

Contact: brandon@colorfront.com

Marquise Technologies

7.D47

Marquise Technologies has developed a comprehensive product range for IMF workflows, including the mastering of any IMF application packages, the versioning, the validation and the QC. Marquise Technologies’ mastering & transcoding solutions also permit the creation of digital cinema.broadcast and online deliverables from any IMF master.For IMF master creations or complex versioning MIST Studio is the tool of choice. ICE Studio is the QC solution with version’s comparison, package validations and content analysis capabilities. ICE Prime is the ideal desktop player for IMF playback and control, TORNADO allows volume transcoding to and from IMF, batch validation and analysis.

Contact: lstoll@marquise-tech.com or Meet with us

Venera Technologies

7.D45

Venera Technologies provides cutting-edge AI and ML enabled file-based auto QC solutions to the digital media industry, tailored to the evolving requirements of its customers and the industry. Venera provides a comprehensive list of IMF/DCP QC checks in both its on-prem A/V auto QC solution (Pulsar) and its native-cloud A/V auto QC solution (Quasar). As an active member of the IMF User Group, Venera is the only auto QC vendor with an extensive category of checks for verification of variety of IMF packages.

Contact: Fereidoon@veneratech.com  or Meet at IBC

Fraunhofer Digital Media

8.B80

With IMF Studio, the Fraunhofer Institute for Integrated Circuits and easyDCP provide proven easyDCP technology tailored to the needs of anybody who masters, creates and inspects IMF packages and exports content to other formats.

IMF Studio consists of our IMF Creator app as well as the IMF Player for quality control on PC and Mac, and comes with flexible licensing options that fit your needs.

Contact: philipp.eibl@iis.fraunhofer.de 

Amazon Web Services

5.C90

AWS helps media, entertainment, games, and sports customers transform the way they create, distribute, and monetize content with the most purpose-built services and capabilities for media and entertainment of any cloud. AWS offers M&E customers nine dedicated media services, 23 AWS industry solutions, and the ability to engage with more than 500 AWS for M&E Partners. AWS services and solutions for media and entertainment, games, and sports reinvent and transform the customer journey. Meet with AWS Elemental MediaConvert experts in the Builder Zone of our AWS stand (5.C90) at the RAI. MediaConvert provides native support for IMF, allowing media companies to easily incorporate IMF workflows into their video processing pipelines and distribution processes.

Details: AWS@IBC or Request a meeting or Download the attendee guide (PDF)

Telestream

7.B11

Telestream offers comprehensive solutions for reading (decoding), creating (encoding), and verifying IMF Packages. Vantage Post Producer can decode simple or complex packages and produce flattened outputs, Vantage Transcode Pro can create simple IMF packages from most media files including support for HTJ2K and IAB. For QC Qualify and Vidchecker can QC IMF Packages to ensure compliance.

Contact: Richard Andes

The IMF UG held a hybrid plugfest on December 13 and 14, 2023, with physical locations in Los Angeles (USA) and Cologne (Germany), and remote participation through shared cloud storage, virtual breakout sessions and instant messaging. The plugfest, the first since February 2020, gathered more than 50 participants from the content creator, user, and implementer communities.

Over 2 days, participants created, exchanged, and consumed IMF content. Cross-checking IMF content between implementations ensures the standard is supported and helps reveal potential improvements to both implementations and the standard.

This report summarizes the use cases studied during the plugfest.

Lossless conversion from MOV to IMF

IMF Application ProRes (SMPTE RDD 45) enables lossless conversion of existing ProRes QuickTime files to IMF. For this use case, the ProRes video contained in MOV files was rewrapped into IMF Compositions without any decoding and re-encoding, demonstrating lossless conversion and eliminating generation losses.

Use case and source content provided by BBC.

Deduplication

Deduplication refers to the reuse of identical essence in multi-version titles, also known as “versioning”. A typical ProRes workflow produces one flat MOV file for every version of a title, even if only a few frames differ between those versions. For this use case, five MOV files were provided: one was identified as the source version and the others as derived versions with different inserts, cuts and pre-rolls. From these five flat files, five IMF composition playlists, each corresponding to one of the versions, and a collection of video components were created. One of the components contained all the frames of the source version while the others contained only inserts and pre-rolls. By allowing portions of the source version components to be reused across composition playlists, IMF achieved a 75% reduction in storage compared to the five flat files.

Use case and source content provided by RTL.

Immersive Audio Bitstream (IAB)

IAB is a mature format for encoding immersive sound fields as a collection of audio channels and audio objects. This use case involved successful QC and playback of an IMF composition whose audio essence consisted solely of IAB essence and did not contain multichannel audio essence.

Use case and source content provided by Sony Pictures Entertainment.

Audio Definition Model (ADM)

S-ADM (Serialized Audio Definition Model) and ADM (Audio Definition Model) are recent ITU-R Recommendations for interactive and immersive audio applications associating metadata with audio essence. The use case involved creating IMF compositions starting from audio essence with S-ADM and ADM metadata wrapped in MXF files.

Use case and source content provided by Fraunhofer IIS.

High Throughput JPEG 2000 (HTJ2K)

HTJ2K significantly improves the encoding and decoding speed of JPEG 2000, reducing processing costs and allowing real time playback on consumer desktops and laptops. This use case involved QC and playback of compositions that used HTJ2K-encoded video.

Use case and source content provided by Warner Bros Discovery, Sony Pictures Entertainment, NBC Universal, and Walt Disney Studios.

ISXD

Carrying metadata as a separate component allows its addition, modification, and removal without having to modify associated essence components. This use case involved QC and playback of a composition that contained Dolby Vision metadata stored in a separate track file conforming to the Isochronous Stream of XML Documents (ISXD) plug-in.

Use case and source content provided by Walt Disney Studios.

Stereoscopic 3D

Stereoscopic 3D video has been a core, but infrequently exercised, capability of IMF since its inception. This use case involved QC and playback of a Stereoscopic 3D composition.

Use case and source content provided by Walt Disney Studios.

Lossless To Lossy Transcode

IMF Application 2E supports both lossless and lossy JPEG 2000 video essence. This use case involved the transcoding of lossless JPEG 2000 (Main level 7 Sublevel 0) compositions (HDR and SDR) to lossy JPEG 2000 (Main level 6 Sublevel 4 and Main level 6 Sublevel 3) compositions, which were subsequently subjected to QC and playback.

Use case and source content provided by Sony Pictures Entertainment.

Observations

As IMF matures, the focus has increasingly turned to operational improvements and incremental upgrades.

• The plugfest highlighted the importance of a common validation strategy: different versions of the widely used Photon OSS validator performed differently when faced with newer use cases.
• The plugfest was an opportunity to experiment with new capabilities such as S-ADM and ADM, which are expected to be leading topics of future plugfests.
• Support for IMF across proprietary and open-source tools can always be improved. There is, for example, an opportunity to expand support for IMF beyond multichannel audio and monoscopic video in the popular open-source FFMPEG toolkit.

Future plugfests

We are already working on future plugfests. In the meantime, testing should never stop, and the content used during the plugfest remains available to all IMF UG members.

Join the UG today to get access to past plugfest content and help the community plan future plugfests!

Acknowledgments

Many thanks to David Gageos, Mike Krause, Wolfgang Ruppel, Brian Holter and Dave Deelo for their contributions to the program committee; to Walt Disney Studios and RTL, who hosted the event; and to Colorfront who sponsored lunch at the Los Angeles location.