OpenDrives Q&A: What’s the Future of Storage Technology for Media Workflows?

MASV and OpenDrives recently announced an integration designed to streamline file transfer workflows by embedding MASV’s reliable, ultra-fast transfer capabilities directly into the OpenDrives Atlas storage platform.

This powerful integration enables media professionals dealing with massive files to effortlessly combine MASV’s speed and ease of use with OpenDrives’ high-performance storage solutions.

To get the inside story on how the integration was developed—and to explore where storage technology for media workflows is heading—we sat down with James DiVito, VP of Product and Solutions, and Roger Beck, Solution Architect at OpenDrives. Let’s dive in.

このインタビューは、長さと明瞭さのために軽く編集されている。

• How did OpenDrives get its start, and what challenges were you trying to solve?
• What sets OpenDrives apart from other storage for M&E?
• What challenges did you face integrating MASV Agent into your Atlas platform?
• Why is it important to integrate file transfer capabilities directly into storage?
• How does the performance of OpenDrives enhance file transfer reliability and efficiency?
• How does OpenDrives’ file system maintain data integrity and ensure consistent performance?
• How has OpenDrives adapted to meet the growing demands of film and post-production?
• What trends do you see shaping the future of media storage and data movement?
• Given the rise of distributed teams and remote collaboration, how crucial will storage performance be?
• これで終わりだ

How did OpenDrives get its start, and what challenges were you trying to solve?

James DiVito: Our founders, including an individual named Jeff Brew, who was at the time David Fincher’s chief senior engineer, were looking for media solutions to solve their needs for post-production. This was around 2011 and a couple of years after that, 4K was all the talk and they looked around. A lot of the storage solutions that were out there were just not performant enough or just way too expensive or complex.

So Jeff and the other co-founders built their own and word started getting out on the street. People started asking, ‘Hey, can you make me one of those systems that you were building?’ And then they just had an idea: ‘You know what, I think this is a product.’ So at that point, we started making it more product-ized and started OpenDrives as we see it today.

What sets OpenDrives apart from other storage for M&E?

JD: I would say it is the fact that we focus on media. It’s where we come from, it’s in our blood, and we want to solve those problems. We were the ones who were up late at night, getting deliverables done, rendering, cutting, knowing that we had delivery deadlines. And so to this day, being able to speak in not just input/output operations per second (IOPS), but EXR, DPX, and different kinds of codecs, and understanding these workflows, is what continues to drive our team.

What challenges did you face integrating MASV Agent into your Atlas platform?

Roger Beck: When we got to the Agent from MASV, the nice part was actually that MASV maintains the Docker containerization. We love Docker containerization: It aligns with the future of OpenDrives. And it decouples application upgrades from our software upgrades.

MASV Agent is very well-documented. We had a few issues, but they were overcome quickly by reaching out to your support. We don’t just take the Docker container and give it to our customers. We build a recipe around it, which makes it easier for the customer to digest without actually having any knowledge of Docker or Kubernetes. The customer enters just security, ID, and the path to deploy the MASV Agent in a couple of seconds to have a full setup running, Agent running, and without touching the command line.

JD: That ease-of-use component you mentioned is a significant driver for OpenDrives. Coming from that media background, we wanna keep creatives creative. Integrations with lots of steps make it untenable, and people with multiple hats within organizations may be hesitant to move forward with those integrations. This is click, fill in a couple of spots, press the button, and you’re off to the races.

Why is it important to integrate file transfer capabilities directly into storage?

RB: Bringing the application closer to the storage removes any additional layers: You don’t need to dedicate the client, and you do not need a network protocol. You increase the performance because you have native access to the storage. This is highly beneficial for outside broadcasting (OB) events, where space is at a premium. By embedding the client directly, you achieve better performance and save valuable real estate by eliminating the need for dedicated equipment.

Could you explain how the performance of OpenDrives enhances file transfer reliability and efficiency?

JD: The design of OpenDrives is very flash first, meaning we use the highest performance tiers of the system and then rely on the slower tiers like the HDD storage for more of the capacity. This really blends well with the MASV Agent. When I was doing some testing with the Agent and the work that Roger had done—we do a lot of our development through VMs (virtual machines). So the infrastructure and the actual production stuff isn’t there. I saw some good performance and some good ease of use for deployment.

But the real kicker was when we did a test together on a real production system that we have in the lab, and it flew. The combination of MASV and the OpenDrives system pushes that bottleneck to the WAN connection. I think that’ll always be the limitation. We’re designed for these high-speed, sequential, high-throughput transfers, and the combination of MASV and OpenDrives allows that absorption, or transfer, out into the system or from the system very cleanly.

How does OpenDrives’ file system maintain data integrity and ensure consistent performance?

JD: The file system underneath OpenDrives uses a checksum-based file system. And what that means is that we have a checksum of everything. So if a client now requests a block and there’s something corrupted due to a signal error, or bit rot, it will be corrected, written down to the level of the file system, and handed to the client. So the silent data corruption doesn’t occur.

In addition we do have snapshots, a bit like Time Machine on Mac, but just in the blink of an eye you get a snapshot of everything with no performance impact. We are the file system, we are the core of everything. We cannot lose a bit.

RB: Our primary focus is to reduce latency, and we do it by managing the cache. We have found a way to really work out on the cache side, taking the content in from the ingesting clients, from the MASV transfer and syncing it to the storage when there’s not a peak time. At the same time, a different algorithm goes out and reads the data from the requesting client, and we do a lot of aggressive prefetching.

This means we go out and read the next block before and after in the hope that you will need it. So if the editor hits play, the asset or the blocks already come from memory and are no longer from disk. We always try to be a step ahead. And if the first block was successful, we read more and more blocks to be really aggressive and be predictive.

How has OpenDrives adapted to meet the growing demands of film and post-production?

JD: AI is being thrown around everywhere. And a lot of people are trying to figure out what that means, what it means per industry, and what it means for them.

To me, the future of data services is rich metadata. Not just: Here’s the file, here’s some information, or here’s some XMP headers. We’re seeing a lot more of: I want to be able to search on what someone said or who’s in the content, or how can I access the past five years of this type of data with this feel or this look. And I think that is the next phase in what creatives will be looking for.

What trends do you see shaping the future of media storage and data movement?

JD:  The trends we’ve seen are that there was an immediate turn to be remote due to the pandemic. I think a lot of people moved things to the cloud. When people started coming back into the office, they turned on some of that gear that they couldn’t mess with before. So we’re starting to see a lot of disaggregated storage silos. Not just storage, but also systems and data being disaggregated all over the place.

What we will see is this trend of having the right data in the right place at the right time, and making the best of both worlds, whether that be a hybrid cloud or on-prem. It’s really about accessibility. There’s a bit of a trend, especially this year, in repatriation: There’s now this stepping back and finding what’s the most cost-effective and workflow-effective use of time and infrastructure.

So we’re going to see that blend, and we’re going to see a lot of these technologies like AI and automation ensure that data is where it needs to be, and that users don’t have to be super technical to access the content they’re trying to access.

Given the rise of distributed teams and remote collaboration, how crucial will storage performance be?

RB: If you had asked me this 15 years ago, it would be the same answer. Because while we create better SSDs, NVMes, that limitless performance on the storage side, if they’re possible, you have other layers in between, you have the internet in between.

So we got used to working remotely. Many people would love to continue remotely. So you have to push the content through one or two edge devices somewhere, but you cannot push petabytes through the cloud. You can, but it will be slow and it will be expensive. It doesn’t make sense. You need a way to access the machine another way.

And by doing so, you have, I think, nowadays three, probably four different workflows nowadays. One’s gonna be on-premises, one’s completely in the cloud, where you spin up virtual desktop infrastructure (VDI). And then comes the hybrid workflow. Because we tend to, especially in M&E, push the limits, push the edge of what can be done.