For over a decade, Meta relied on a heavily modified internal fork of FFmpeg to handle the unique demands of processing billions of video uploads daily for VOD and livestreaming. This fork provided critical optimizations absent in upstream FFmpeg, but it came with a steep cost: divergence from the main project, missed upstream improvements, and significant maintenance overhead. This article explores the strategic engineering effort to bridge that gap, collaborate with the FFmpeg community, and finally deprecate the internal fork—a move that improved efficiency for Meta and the entire open-source ecosystem. You can read the original engineering case study for more details on the FFmpeg at Meta project.
The Core Challenges of a Diverging Fork
Maintaining an internal fork created two major problems:
- Feature Divergence: The fork had custom features (parallel multi-lane encoding, real-time quality metrics) while upstream FFmpeg added new codecs, formats, and reliability fixes we needed.
- Rebasing Hell: Safely merging upstream changes into our fork to avoid regressions became increasingly complex and risky.
The Strategic Solution: Upstreaming for Mutual Benefit
Instead of perpetually maintaining the fork, Meta's Video Engineering team partnered with FFmpeg developers, FFlabs, and VideoLAN to upstream the core features. This required significant refactoring of FFmpeg's core architecture.
- Multi-Lane, Threaded Transcoding: Prior to FFmpeg 6.0/8.0, encoding multiple output streams (e.g., for DASH) in one command was serialized per frame. Our internal fork ran encoders in parallel. This design influenced the upstream community, leading to a decades-spanning refactor that now provides more efficient encoding for all FFmpeg users.
- Real-Time Quality Metrics ("In-Loop" Decoding): For livestreaming, we needed to compute metrics like VMAF during transcoding, not after. This required inserting a decoder after each encoder in the processing graph. This capability, known as "in-loop" decoding, was upstreamed and is available from FFmpeg 7.0.
Critical Perspectives and Strategic Decisions
When Not to Upstream: Not all internal modifications are suitable for the open-source project. A key example is support for Meta's Scalable Video Processor (MSVP), a custom ASIC. While integrated via FFmpeg's standard hardware acceleration APIs, contributing this code upstream would burden maintainers with supporting hardware they cannot test. Such highly infrastructure-specific patches are kept internal, with Meta assuming the cost of rebasing them.
The Trade-off: This highlights a crucial balance in corporate open-source strategy: contribute features with broad community impact, but internalize those tied to proprietary infrastructure. This approach, similar to how companies integrate custom AI hardware like the Maia 200 AI accelerator, ensures the main project remains lean and universally applicable.
Lessons Learned and Next Steps
Key Takeaways for Engineering Teams:
- Evaluate Fork Longevity Early: The cost of maintaining a fork grows exponentially. Proactively seek upstream collaboration.
- Contribute Generically Useful Features: Focus efforts on upstreaming changes that solve problems for others, not just your own stack.
- Leverage Standard APIs: Building custom hardware support (like MSVP) on standard FFmpeg APIs minimizes friction and keeps the door open for future upstreaming.
The Future of Media Processing at Scale: With the internal fork retired, Meta's teams can now focus on innovating with the community, not alongside it. The next frontier involves leveraging even more advanced tooling for high-frequency tasks, much like the efficiency gains promised by new terminal-based AI assistants such as those explored in our look at Gemini CLI for developer workflows.
Your Next Step: If you're dealing with media processing at scale, audit your dependencies. Are you maintaining patches that could benefit the wider community? Engaging with open-source projects is not just altruism—it's a strategic engineering efficiency play.