Decentralized Storage: Never Lose a File Again?

by | September 16, 2022

As a video professional, there’s nothing like that feeling of dread when you lose a file. Maybe you accidentally deleted it off your system or your computer gets damaged and you lose access to everything. Either way, the file is gone.

If it’s something like a motion graphics template file, it’s not the worst thing. But, if it’s some raw footage from set, that’s where things can get messy.

Hopefully, you have an extra backup in place, either on an external hard drive or cloud storage. If you don’t, a reshoot is in order, and that involves bringing an entire crew back to set which slows down production and increases costs.

Wouldn’t it be nice if you never had to worry about losing a file ever again? To not create manual redundancies across drives and different storage providers; to always have a backup in place?

That is the guarantee of decentralized storage. A product of Web3 technologies, decentralized storage stores data across a series of ‘nodes’ all around the world, without a central location or server (hence the name ‘decentralized’). In essence, your files gain multiple redundancies across a peer-to-peer network, so that if one node goes down, there are plenty of other nodes to carry on and keep your files intact.

Sound too good to be true?

We asked the team at Filebase, a decentralized storage provider, to provide a breakdown the tech, function, and use case of decentralized storage.

Table of Contents

Send Large Video Files

Send and receive up to 5 TB of large media assets over the cloud with MASV file transfer.

1. What is Decentralized Storage?

Decentralized Storage blog image

Decentralized storage is a method of data storage where data files are stored across a series of geographically distributed nodes that are connected using peer-to-peer (P2P) networking.

Decentralized storage can refer to data stored on:

  1. A blockchain network that uses P2P networking, or
  2. Data stored using the distributed P2P protocol IPFS.

What is IPFS?

The InterPlanetary File System (IPFS) is a P2P communication protocol used to retrieve a file based on its unique content identifier (CID), which is a cryptographic hash that is generated and assigned to the file, based on its content.

CIDs can be used in conjunction with IPFS Gateways to access data from anywhere by anyone.

2. How Does Decentralized Storage Work?

  1. When a file is uploaded to the IPFS network, the file is split into multiple pieces and given a unique value known as an IPFS CID.
  2. To request a file, a user queries the file’s CID through an IPFS URL or IPFS HTTP gateway. This query is passed to the network’s distributed hash table, which is a type of database that maps CIDs to the node that is storing the associated file.
  3. The node storing the file is sent a request to retrieve the file.
  4. When a file is retrieved, the node that retrieves it keeps it in its cache storage to be accessed by other nodes that request it.
  5. The node’s cache is emptied periodically through a process known as the IPFS garbage collection process. To avoid files being removed, they must be pinned to the network to be permanently stored.

What is IPFS pinning?

Pinning a file on IPFS protects it from being removed during the garbage collection process, which is a resource maintenance process that clears the cache of an IPFS node regularly to free up resources. Unless a file is pinned, it will only be kept in a node’s cache storage until it is regularly removed.

By default, Filebase pins all IPFS files with 3 redundant replications stored across locations in the US and Europe for assured resilience.

Related: Creating the Metaverse with Volumetric Video and Virtual Production

Easily Send Large Videos Fast

Deliver raw footage, high-resolution proxies, DCPs, and more with MASV.

3. Decentralized Storage vs Cloud Storage: How Do They Differ?

Traditional cloud storage stores a data file on a single file server that is hosted in one geographic location. Decentralized storage breaks a file into multiple pieces and then distributes it across a variety of geographic locations.

So instead of uploading a single video file to many different cloud storage providers as a backup, you upload it once with a decentralized paradigm and automatically disperse it to multiple locations.

Decentralized storage has no single point of failure and can tolerate multiple simultaneous outages. In comparison, traditional cloud storage has multiple single points of failure that can cripple infrastructures when outages occur.

If it’s not in the cloud, then where is it?

Traditionally, ‘the cloud’ refers to large data centers that contain thousands of servers used for data storage. Decentralized storage doesn’t use data centers; instead, it utilizes unused storage space across the globe through P2P networks.

These networks are comprised of nodes, which are compute resources that are manually added to the network and willingly provide its local resources such as storage and processor power to be used by the network to store data and process transactions.

With this in mind, decentralized storage can be considered ‘the cloud’ since it is universally accessible, but it does not refer to the same ‘cloud’ that traditional centralized storage uses.

4. Why is Decentralized Storage Important?

Data stored on IPFS offers three main advantages when compared to centralized storage:

  • Reliability: Decentralized peer-to-peer networks have no single point of failure like centralized cloud storage does. Multiple nodes can be affected by outages or data corruption and the network will still function as if all the nodes were online.
  • Interoperability: Data stored using IPFS benefits from flexible interoperability. It can be accessed through web browsers using IPFS gateways and the data’s IPFS CID without needing authentication or specified access to the file. This provides widespread functionality through various workflows, codebases, SDKs, and tools.
  • Redundancy: When data is stored on the IPFS network, any node on the network can store and retrieve that file when requested. As a result, multiple nodes on the network can store a file at once. To assure that a file is stored permanently by a node, however, the file must be pinned on the network.

And unlike other Web3 technologies, which strive to provide new functionality and concepts like NFT collections, storage is essential to everything in the digital world. Decentralized storage simply aims to provide a more secure, reliable, and flexible alternative to cloud storage that is used by everyday users, developers, and enterprises daily.

Related: 6 AI tools for Filmmaking You Need to Know About

Speed Up Your Remote Production Workflow

High-speed large file transfers for media professionals.

5. How Do I Gain Access to My Files Stored on IPFS?

To view or download a file stored on IPFS, you can use an IPFS gateway URL with the file’s IPFS CID. This gateway can be accessed from anywhere by anyone without needing authentication or authorization.

You can also use providers like Filebase to interact with IPFS in the same way that you use traditional cloud storage. Filebase provides an easy-to-use, drag-and-drop web-console dashboard for uploading files onto the IPFS network and also provides an S3-compatible API that can be used in hundreds of tools, SDKs, and code bases.

6. What Happens if a Decentralized Storage Provider ‘Goes Down’?

When a node on a decentralized storage network goes down, the other nodes on the network are able to provide access to the network’s stored files.

One, or even tens of nodes being down on a P2P network doesn’t affect the ability of users to continue to use the network. For a decentralized storage provider to truly be ‘down’, more than the majority of the nodes on the network would need to be simultaneously offline, which is extremely unlikely due to their diverse geographic locations.

Don't Babysit File Transfers

MASV will deliver your files. Guaranteed. If not, you get your money back.

7. How Much Does Decentralized Storage Cost?

On average, decentralized storage costs a fraction of what centralized storage providers charge. It uses storage space that already exists in the world, so it doesn’t require costs for adding new hard drives or servers and doesn’t incur costs like data center management, upkeep, staffing, or real estate costs.

On average, most customers experience an 80% reduction in overall storage costs.

If you want to get started with a Web3 IPFS storage system, sign up for a free Filebase account and use code WEB3WELCOME to get your first 1TB free for the first month.

Your Files, When You Need Them

Decentralized storage is a great way for filmmakers, creators, and anyone, really to work with peace of mind that their critical files are stored and ready to access at a moment’s notice.

Even if something catastrophic happens, like a hard drive is damaged or a cloud server goes down, decentralized storage’s peer-to-peer network and geo-based redundancies will have your files when you need them.

Another way to have your files when you, or someone you know, need them is to use MASV; the fast, easy, and secure way to transfer large video files.

MASV is a ISO 27001 certified file transfer solution capable of delivering up to 5 TB, per file, over the cloud, to anywhere in the world. Sending a file with MASV is as easy as sending an email with the added speed, stability, and performance of an enterprise-grade tool.

You can even share files directly to a cloud storage rather than an individual.

But what about decentralized storage? We got that covered too. Read how Filebase integrated with MASV using our simple S3 storage configuration. We also support direct delivery to media asset management tools and object storage.

Sign up for a free trial of MASV today to send those heavy media assets to and from clients and collaborators. We’ll even throw in 100 GB to get you started.

MASV File Transfer

Get 100 GB to use with the fastest, large file transfer service available today, MASV.