The IMF UG held a plugfest on June 26, 2024 at Universal Studios in Los Angeles (USA), with remote participation through shared cloud storage and virtual breakout sessions. The plugfest gathered more than 20 participants from the content creator, user, and implementer communities.

Throughout the day, participants created, exchanged, and consumed IMF content. Cross-checking IMF content between implementations ensures the standard is supported and helps reveal potential improvements to both implementations and the standard.

This report summarizes the use cases studied during the plugfest.

High Throughput JPEG 2000 (HTJ2K)

HTJ2K significantly improves the encoding and decoding speed of JPEG 2000, reducing processing costs and allowing real time playback on consumer desktops and laptops. This use case involved QC and playback of compositions that used HTJ2K-encoded video.

Use case and source content provided by NBCUniversal and The Walt Disney Studios.

Photon

Photon is an open-source library for parsing, interpreting and validating constituent files that make an Interoperable Master Package (IMP). These include: 

• AssetMap (SMPTE ST 2067-2)
• PackingList (SMPTE ST 2067-2)
• Composition Playlist (SMPTE ST 2067-3)
• IMF track files (SMPTE ST 2067-5)

Currently, Photon provides support for IMF Application #2E (SMPTE ST 2067-21) and Application #5 ACES (SMPTE ST 2067-50), and the Immersive Audio Bitstream (IAB) Plug-in (SMPTE ST 2067-201).

For this plugfest, a pull request against Photon was created by Pierre-Anthony Lemieux that included additional checks for HTJ2K codestream header metadata. This build was used for the validation of the plugfest content.

Test content created with Colorfront, MTI, Blackmagic Design, and Telestream technologies.

No-Proxy Streaming

Using no-proxy HTJ2K MXF JavaScript player, files are read from an AWS S3 bucket with pre-signed URLs. The MXFs are played directly in the browser using JavaScript, there is no proxy involved. If connection is limited, it will fetch only the low-resolution wavelet information and show a low-resolution image during playback. Pausing playback will fetch all the data for that frame. For lossless IMFs, this is the lossless data, for lossy IMFs this is the high quality you would experience from a normal full decode.

Traditional IMF J2K part 1 encoding uses the CPRL progression order, for this player to work the no-proxy player needs HTJ2K with RPCL progression order. The no-proxy player is currently limited to 4:4:4 sources. While the 4:2:2 source will decode, it will only show the actual Y, and half-vertical with Cb and Cr that is associated with 4:2:2.

Observations

The goal of this plugfest was to focus on creation and interchange of HTJ2K within IMF. The focus has increasingly pointed to the need for the community to have consistent ways of validating content.

1. The plugfest showed that the lack of a common validation tool and/or test vectors introduces friction. The IMFUG should consider an effort to support a common validation strategy and/or toolset that includes codestream analysis.
2. HTJ2K no-proxy streaming functions as expected when source materials are encoded with RPCL progression order. Some HTJ2K content was still found to have the CPRL and LRCP progression orders which did not work.
3. Colorfront enhanced their validation toolset to include codestream inspection with comparison of the essence and metadata values. This helped the plugfest identify additional issues with submitted packages.
4. Although most solutions have the ability to create HTJ2K IMF, errors were still present that included:
   • CPRL and LRCP progression order
   • Missing TLM marker
   • Incorrect schema definition in CPL
   • Invalid HTJ2K tile part count
   • MXF essence and metadata values not matching

Future plugfests

We are already working on future plugfests, possibly as early as November 2024. In the meantime, testing should never stop, and the content used during the plugfest remains available to all IMF UG members.

Join the UG today to get access to past plugfest content and help the community plan future plugfests!

Acknowledgments

Many thanks to Pierre-Anthony Lemieux, Harvey Landy, Mike Krause, and Dave Deelo for their contributions to the program committee; to NBCUniversal who hosted the event; and to Amazon Studios who provides cloud services for the IMF UG.

Attending IBC 2024? Get to know the Interoperable Master Format (IMF) better at following exhibitors!

Ad Signal

6.C28b

Ad Signal has a range of unique SaaS products. Our Match product deduplicates content regardless of format, codec and resolution. Our patent-pending Compose product is able to create IMF from traditional content fully automatically; paired with Match we identify all of the unique content at the best quality to keep and Compose turns these assets into a single IMF package with all the required CPLs etc. We believe this is the only product capable of doing this to date.

Contact: tom.dunning@ad-signal.io or Meet at IBC

Tedial

1.B18

The smartWork platform offers comprehensive IMF support from acquisition to delivery. With smartWork, customers can select third-party systems tailored to their specific needs, such as mastering, transcoding, QC, and AI tools. By natively storing IMF, smartWork | EVO maximizes standard benefits while simplifying user interaction with the package’s internal structure.

Contact: sales@tedial.com

Ateliere Creative Technologies

5.C24

Ateliere Creative Technologies revolutionizes the media supply chain by delivering AI-driven, cloud-native solutions that streamline content creation, management, and distribution from concept to consumer. Our platform, Ateliere Connect, automates media workflows, reduces storage costs, and accelerates time-to-market, enabling clients to maximize their media profitability. A key component of our service is our integration of the Interoperable Master Format. By leveraging IMF’s component-based architecture, Ateliere helps media companies significantly reduce their online storage footprint, enhance content management, and simplify global distribution, ensuring seamless delivery across all platforms.

Rohde & Schwarz

7.B21

The R&S®CLIPSTER is the gold standard solution for mastering and distribution of feature films and episodic TV. It provides a powerful way to edit any type of media, in any resolution, and create high-quality professional deliverables that meet stringent, professional delivery specifications. Featuring a highly refined IMF delivery wizard R&S®CLIPSTER is able to simplify the creation of IMF’s, so that the outputs are constrained to comply with standards and ensure interoperability. R&S®CLIPSTER makes versioning IMF’s quick and easy and features a dedicated tool to merge and combine multiple IMFs into a single package.

Contact: matthew.ellsworth@rohde-schwarz.com

Colorfront

Private Demo Suite – Okura Hotel

Colorfront, headquartered in Budapest with offices in Los Angeles and sales partners worldwide, is a leader in IMF and DCP mastering and known for its award-winning on-set dailies and transcoding systems used by companies of all sizes for Hollywood blockbusters, high-end TV, and OTT entertainment. Founded in 2000 by Mark and Aron Jaszberenyi, the company combines expertise in image color science and software development, earning an Academy Award for its innovations. Today, Colorfront is renowned for its camera-to-post products and offers advanced cloud services alongside a state-of-the-art DI and post-production facility in Budapest.

Contact: brandon@colorfront.com

Marquise Technologies

7.D47

Marquise Technologies has developed a comprehensive product range for IMF workflows, including the mastering of any IMF application packages, the versioning, the validation and the QC. Marquise Technologies’ mastering & transcoding solutions also permit the creation of digital cinema.broadcast and online deliverables from any IMF master.For IMF master creations or complex versioning MIST Studio is the tool of choice. ICE Studio is the QC solution with version’s comparison, package validations and content analysis capabilities. ICE Prime is the ideal desktop player for IMF playback and control, TORNADO allows volume transcoding to and from IMF, batch validation and analysis.

Contact: lstoll@marquise-tech.com or Meet with us

Venera Technologies

7.D45

Venera Technologies provides cutting-edge AI and ML enabled file-based auto QC solutions to the digital media industry, tailored to the evolving requirements of its customers and the industry. Venera provides a comprehensive list of IMF/DCP QC checks in both its on-prem A/V auto QC solution (Pulsar) and its native-cloud A/V auto QC solution (Quasar). As an active member of the IMF User Group, Venera is the only auto QC vendor with an extensive category of checks for verification of variety of IMF packages.

Contact: Fereidoon@veneratech.com  or Meet at IBC

Fraunhofer Digital Media

8.B80

With IMF Studio, the Fraunhofer Institute for Integrated Circuits and easyDCP provide proven easyDCP technology tailored to the needs of anybody who masters, creates and inspects IMF packages and exports content to other formats.

IMF Studio consists of our IMF Creator app as well as the IMF Player for quality control on PC and Mac, and comes with flexible licensing options that fit your needs.

Contact: philipp.eibl@iis.fraunhofer.de 

Amazon Web Services

5.C90

AWS helps media, entertainment, games, and sports customers transform the way they create, distribute, and monetize content with the most purpose-built services and capabilities for media and entertainment of any cloud. AWS offers M&E customers nine dedicated media services, 23 AWS industry solutions, and the ability to engage with more than 500 AWS for M&E Partners. AWS services and solutions for media and entertainment, games, and sports reinvent and transform the customer journey. Meet with AWS Elemental MediaConvert experts in the Builder Zone of our AWS stand (5.C90) at the RAI. MediaConvert provides native support for IMF, allowing media companies to easily incorporate IMF workflows into their video processing pipelines and distribution processes.

Details: AWS@IBC or Request a meeting or Download the attendee guide (PDF)

Telestream

7.B11

Telestream offers comprehensive solutions for reading (decoding), creating (encoding), and verifying IMF Packages. Vantage Post Producer can decode simple or complex packages and produce flattened outputs, Vantage Transcode Pro can create simple IMF packages from most media files including support for HTJ2K and IAB. For QC Qualify and Vidchecker can QC IMF Packages to ensure compliance.

Contact: Richard Andes

The IMF UG held a hybrid plugfest on December 13 and 14, 2023, with physical locations in Los Angeles (USA) and Cologne (Germany), and remote participation through shared cloud storage, virtual breakout sessions and instant messaging. The plugfest, the first since February 2020, gathered more than 50 participants from the content creator, user, and implementer communities.

Over 2 days, participants created, exchanged, and consumed IMF content. Cross-checking IMF content between implementations ensures the standard is supported and helps reveal potential improvements to both implementations and the standard.

This report summarizes the use cases studied during the plugfest.

Lossless conversion from MOV to IMF

IMF Application ProRes (SMPTE RDD 45) enables lossless conversion of existing ProRes QuickTime files to IMF. For this use case, the ProRes video contained in MOV files was rewrapped into IMF Compositions without any decoding and re-encoding, demonstrating lossless conversion and eliminating generation losses.

Use case and source content provided by BBC.

Deduplication

Deduplication refers to the reuse of identical essence in multi-version titles, also known as “versioning”. A typical ProRes workflow produces one flat MOV file for every version of a title, even if only a few frames differ between those versions. For this use case, five MOV files were provided: one was identified as the source version and the others as derived versions with different inserts, cuts and pre-rolls. From these five flat files, five IMF composition playlists, each corresponding to one of the versions, and a collection of video components were created. One of the components contained all the frames of the source version while the others contained only inserts and pre-rolls. By allowing portions of the source version components to be reused across composition playlists, IMF achieved a 75% reduction in storage compared to the five flat files.

Use case and source content provided by RTL.

Immersive Audio Bitstream (IAB)

IAB is a mature format for encoding immersive sound fields as a collection of audio channels and audio objects. This use case involved successful QC and playback of an IMF composition whose audio essence consisted solely of IAB essence and did not contain multichannel audio essence.

Use case and source content provided by Sony Pictures Entertainment.

Audio Definition Model (ADM)

S-ADM (Serialized Audio Definition Model) and ADM (Audio Definition Model) are recent ITU-R Recommendations for interactive and immersive audio applications associating metadata with audio essence. The use case involved creating IMF compositions starting from audio essence with S-ADM and ADM metadata wrapped in MXF files.

Use case and source content provided by Fraunhofer IIS.

High Throughput JPEG 2000 (HTJ2K)

HTJ2K significantly improves the encoding and decoding speed of JPEG 2000, reducing processing costs and allowing real time playback on consumer desktops and laptops. This use case involved QC and playback of compositions that used HTJ2K-encoded video.

Use case and source content provided by Warner Bros Discovery, Sony Pictures Entertainment, NBC Universal, and Walt Disney Studios.

ISXD

Carrying metadata as a separate component allows its addition, modification, and removal without having to modify associated essence components. This use case involved QC and playback of a composition that contained Dolby Vision metadata stored in a separate track file conforming to the Isochronous Stream of XML Documents (ISXD) plug-in.

Use case and source content provided by Walt Disney Studios.

Stereoscopic 3D

Stereoscopic 3D video has been a core, but infrequently exercised, capability of IMF since its inception. This use case involved QC and playback of a Stereoscopic 3D composition.

Use case and source content provided by Walt Disney Studios.

Lossless To Lossy Transcode

IMF Application 2E supports both lossless and lossy JPEG 2000 video essence. This use case involved the transcoding of lossless JPEG 2000 (Main level 7 Sublevel 0) compositions (HDR and SDR) to lossy JPEG 2000 (Main level 6 Sublevel 4 and Main level 6 Sublevel 3) compositions, which were subsequently subjected to QC and playback.

Use case and source content provided by Sony Pictures Entertainment.

Observations

As IMF matures, the focus has increasingly turned to operational improvements and incremental upgrades.

• The plugfest highlighted the importance of a common validation strategy: different versions of the widely used Photon OSS validator performed differently when faced with newer use cases.
• The plugfest was an opportunity to experiment with new capabilities such as S-ADM and ADM, which are expected to be leading topics of future plugfests.
• Support for IMF across proprietary and open-source tools can always be improved. There is, for example, an opportunity to expand support for IMF beyond multichannel audio and monoscopic video in the popular open-source FFMPEG toolkit.

Future plugfests

We are already working on future plugfests. In the meantime, testing should never stop, and the content used during the plugfest remains available to all IMF UG members.

Join the UG today to get access to past plugfest content and help the community plan future plugfests!

Acknowledgments

Many thanks to David Gageos, Mike Krause, Wolfgang Ruppel, Brian Holter and Dave Deelo for their contributions to the program committee; to Walt Disney Studios and RTL, who hosted the event; and to Colorfront who sponsored lunch at the Los Angeles location.

Do you have a content library or an implementation that speaks IMF, then join us at the December 2023 IMF plugfest, which will be held concurrently in Los Angeles, USA and Cologne, Germany.

https://www.imfug.com/imf-plugfest-2023/

Only a few days left to register!

Test content from BBC, NBC Universal, RTL Technology, Fraunhofer DMT and The Walt Disney Studios is now available, with more to come.

This is a unique opportunity to gain implementation experience with IMF and network.

The HPA IMF UG hosted a workshop at IBC 2023 where 28 stakeholders from the media supply chain, representing both users and suppliers, explored using quality control (QC) with the Interoperable Master Format (IMF). This report summarizes their discussions.

What is QC?

QC ensures consistent consumer experiences, prevents technical failures, and avoids regulatory penalties. QC involves testing a media asset against a list of requirements, ranging from objective, e.g., ensuring that the audio loudness level remains below a specified level, to subjective, e.g., detecting incorrect color conversions.

Performing QC is costly. QC involves complex operations on large media files and expert human review. It is therefore critical to avoid performing QC multiple times on the same media assets.

Failing QC is costly. Failing QC nearly always means human intervention, which does not scale, and redelivery, which takes time and incurs expenses. It is therefore critical to minimize QC failures.

Minimizing QC by sharing assets across program versions

IMFallows the re-use of media assets across versions of a program. As such, IMF makes it possible to avoid repeating QC on media assets that are shared across versions of a program. This results in significant cost savings when, for example, a small part of a program is modified to correct an error or when multiple language versions have largely identical video content.

This, however, comes with challenges. Tools must provide context is needed when performing QC on a new part of a program: while the new part might pass QC on its own, it might not when considered with the rest of the program. For example, a video frame might have been duplicated where the new part meets the original program. It is also necessary to design workflows such that QC results on the original part of the program can be trusted when performing QC on new parts.

Avoiding rejections from expected failures

Consider, for example, a program where a long silence is part of the storyline. Such a program will likely repeatedly fail QC tests designed to detect missing audio, potentially resulting in multiple rejections as the program moves across the supply chain. QC tests that a program is expected to fail should therefore be captured as machine-readable data and as early as possible in the supply chain. It is also critical that such expected-failure metadata remain synchronized with the program across the supply chain.

IMF is uniquely positioned to be the source of expected-failure metadata: such metadata is generated at mastering, where IMF is typically used, and IMF allows rich metadata to be synchronized with a program. Additional specification work however remains to fully define the syntax of expected-failure metadata.

Automating QC using machine readable reports

Automation is impossible without standardization. Today, QC reports are routinely shared in a non-machine-readable form, e.g., as a PDF file, and using vendor-specific terminology. This makes it difficult to automate QC, keep QC data synchronized with the program, and trust QC data across the supply chain. Standardizing QC data is a non-trivial problem given the historical disparities in QC practices worldwide. Efforts, such as EBU QC (https://tech.ebu.ch/qc), are underway to standardize QC tests.

Next steps

As the number of programs explodes worldwide, IMF offers the opportunity to reduce QC costs by reusing assets across versions and being the source of QC data for the supply chain. Challenges remain in standardizing QC data and designing workflows and tools that make use of it. Get involved today with the HPA IMF UG to contribute to these efforts.

Acknowledgments

Many thanks to Tedial, who kindly sponsored the location and catering, and to our program committee: Fereidoon Khosravi (chair), Andrew Dunne, Andy Quested, Brian Holter, Harvey Landy, and Bruce Devlin.

Attendees

Pixelogic Media Partners, Venera Technologies, BBC, EBU, Amazon Studios, Tedial, RTL, Marquise Technologies, Andy Quested, Mr. MXF, Lum::Invent, Netflix, MediaArea, Colorfront, Motion Picture Solutions, Deluxe, Sky UK, nomalab, BFI, Bitmax, Walt Disney Studios, Sky UK, Ateliere Creative Technologies, Dolby Laboratories, DI Factory.

Subscribe

Sign-up for our mailing list and stay up-to-date with the latest on IMF and component-based workflows!

Email address:

Leave this field empty if you’re human:

The HPA IMF UG hosted a workshop at IBC 2022 where 34 stakeholders from the broadcasting community, representing both users and suppliers, explored using the Interoperable Master Format (IMF) as the source content format in broadcast workflows. This report summarizes their discussions.

UHD Transition

The broadcast community is facing a transition from HD to UHD programming. This transition requires a new program exchange format but also offers the opportunity to upgrade workflows so that they are more efficient today and more resilient to future requirements. For example, the time and cost involved with re-delivering complete programs when only a small part is defective is amplified by the larger size of UHD programs.

The Interoperable Master Format (IMF) is a worldwide standard for exchanging and archiving TV programs, movies, and ads. It is component-based: audio, video and access services files are stored separately and synchronized on a common timeline using a single playlist file.

Strengths

IMF is deployed today, both in internal workflows and as a delivery format by large content providers and broadcasters: Netflix, BBC, Disney, Deluxe… Several attendees indicated that they are actively evaluating it.

IMF supports UHD today and is implemented by a wide range of commercial and open-source tools, including mastering, QC and transcoding tools. For example, the recent addition of an IMF demuxer to the FFmpeg, Avisynth and VapourSynth open-source toolkits has significantly reduced the barrier to adopting IMF.

IMF allows the re-use of components across deliveries and versions, reducing QC, making deliveries faster and requiring less storage for archives. One participant reported that they had saved 40,000₤ in QC costs alone by avoiding having to QC multiple times components shared by multiple deliveries. Component-based delivery and storage is also particularly attractive for cloud-based applications where bandwidth is limited.

Challenges

While the IMF technology is mature, there is still little operational familiarity with component-based media processing and IMF is often used as a flat linear mezzanine, creating an obstacle to realizing the full benefits of IMF. For example, while IMF carries rich metadata that allows the location of color bars or candidate commercial insertion points to be identified, the automated use of such metadata during playout is not common practice.

Participants highlighted that it is preferable to modify one’s workflow to take advantage of component-based media, and thus of IMF, instead of shoehorning IMF into existing linear workflows. Such a transition should be planned over 2 to 3 years and account for education and onboarding.

It was also noted that several traditional playout and media server vendors do not support IMF, despite IMF using many of the technologies (playlist and MXF files) that they are familiar with.

Opportunities

It is unlikely that IMF will displace current formats and practices for HD. It was however emphasized that the future is not flat, and that IMF allows a broadcaster to both transition to UHD today and be prepared for future technologies. The future might include, for example, object-based media as identified by Ofcom, rights metadata or access services such as sign language.

Now is therefore a good opportunity for the broadcasting community to collaborate around developing best practices for component-based workflows and develop delivery specifications around IMF.

Links

IMF Product and services	https://www.imfug.com/suppliers/
IMF Open-source	https://www.imfug.com/open-source/
IMF Explainer	https://www.imfug.com/explainer/

Attendees

BBC, Colorfront, BFI, Ateliere, DI-Factory, Mr. MXF, NRJ, Netflix, Sky UK, Deluxe, Venera Technologies, Sky Italy, CST, Vidispine, Arte, Amazon, filter Media, Tedial, Black Photon, RTL, Marquise Technologies, Lum Invent, Aveco, ITV, Via Play, RAI, Fraunhofer IIS, Dolby, Rohde & Schwarz, MediA Digital Nutty, BBC Studios.

Acknowledgments

Many thanks to Ateliere and Deluxe, who kindly sponsored the location and catering, and to our program committee: Andrew Dunne (chair), Julián Fernández-Campón, Kirk Bradford, Laurence Stoll, Mark Pascoe and Simon Thompson.

Subscribe

Sign-up for our mailing list and stay up-to-date with the latest on IMF and component-based workflows!

Email address:

Leave this field empty if you’re human: