Decentralized data storage is essential when developing DeFi applications. It is the only way to achieve true decentralization. Organizing this kind of data storage is no easy task. But the result is definitely worth the effort. In this article, we look at IPFS, cloud-based decentralized technologies, and the various protocols and coins associated with decentralized storage.
Using IPFS
The first solution for decentralized storage was IPFS (Inter Planetary File System). This system was launched back in 2016 and was initially only used in conjunction with the Ethereum network. In many ways, it is similar to BitTorrent. The IPFS protocol operates with separate units of information, called blocks. A block may contain a part of a file or references to other blocks. Blocks, in turn, are used to build files and a directory structure. A cryptographic technique called the directed acyclic graph is used for this purpose.
Content addressing using Content Identifiers (CID) ensures the immutability of the content, which is stored in a decentralized form. There is no consensus mechanism at the IPFS level. However, there is one at higher levels of abstraction, such as FileCoin and IPFS cluster. An IPFS cluster relies on CRDT (Conflict-free Replicated Data Type) technology. Content is addressed and linked using Merkle Directed Acyclic Graphing technique.
For interfacing with IPFS, there are tools such as:
It should be remembered that files not used and not saved by anyone in this system disappear. Your decentralized hosted website can be accessed either via IPNS or directly. To access the site directly, you will need to set up DNS records. The address that will be obtained will not be real to remember. However, this problem can be solved by using bookmarking and link-shortening services.
The addresses are basically linked to a hash of the contents of the file. This may not be convenient to use, because when the file changes, the path to the file changes too. For this reason, IPNS is designed to display dynamic content. IPNS is an address system for displaying content that changes, such as blog posts or news feeds.
Coins and protocols for decentralized storage
IPFS users have no financial incentive to provide their disk space to other network participants. But has quite a few coins and protocols which provide rewards for users. This is not always a good idea. On the one hand the users who are giving away disk space are interested in the value of the coin. On the other hand, users who need decentralized storage want this service to cost them as little as possible. This is the main contradiction that limits the growth of decentralized coins and protocols.
SC coins and the Sia protocol
One of the first coins associated with decentralized data storage was Sia. This project was founded in 2013 and began to be actively used in 2015. SC coins could be ‘mined’ using free hard disk space. In doing so, user data is distributed from several sources at once. The information is accessed using private encryption keys. When any of the Sia nodes becomes unavailable, data is recovered from the duplicate nodes without any interruption. Smart contracts enable communication between file owners and users who store files on their computers. This peer-to-peer (p2p) network has no single point of failure and is arranged as follows:
Features of FileCoin and further development of this network
FileCoin uses IPFS as the basis. FIL tokens circulate in the FileCoin network. Two consensus mechanisms are used: Proof-of-Replication and Proof-of-Spacetime. The Proof-of-Replication mechanism verifies that a miner has actually stored the number of copies of data that it claims to have on its media. The Proof-of-Spacetime mechanism is responsible for ensuring that the data storage time of the miner is indeed what it says it is. The FileCoin system has a competition of miners who try to offer users the lowest rates for storing data.
FileCoin is based on Chainsafe, an application with a simple interface that provides access to 20 gigabytes of disk space for decentralized data storage. And a system such as NFT Storage, based on FileCoin, is used by the well-known NFT trading platforms OpenSea and Magic Eden.
Image source: Messari Research Group
Interestingly, FileCoin is currently used mainly for storing encyclopedic and reference data.
And here, the founders of the project have cited an extensive list of startups that could use decentralized data storage. The FileCoin system is being actively developed. The most anticipated innovation in this network is the introduction of FEVM – Filecoin Ethereum Virtual Machine, which will allow smart contracts to be imported from the Ethereum network. Also, FEVM will allow us to overcome some existing limitations of Filecoin. For example, with FEVM it will be possible to store files forever or automatically create the required number of copies of files.
What is Storj?
The native currency of the Storj network is called STORJ. This decentralized storage network fragments files into discrete pieces. All these pieces of files are stored on multiple nodes, and it is a much more cost-effective approach to storing data than replication. The file is divided into 80 segments and only 30 segments are needed to fully recover the data. The files are continuously audited so that missing pieces of data can be recovered. Data storage costs around $0.004 per gigabyte per month. There is compatibility with AWS S3 and a high degree of privacy thanks to strong AES-256-GCM data encryption.
Torrents for files – BTT Coin
The BTT Coin is released by a project everyone knows, called BitTorrent Inc. Founded in 2001, BitTorrent was the first and still is the largest file exchange. DLive, BitTorrent Speed, BitTorrent File System, and many other applications are based on BTT. User data is not stored on BitTorrent servers. Instead, it is stored encrypted in BitTorrent applications. Communication between the owners of the files and the users who provide storage to the sites takes place using smart contracts. The cost of storage, which is paid for in BTT coins, is around $.00675 USD/GB/month. On the technical side, BTT is similar to IPFS in many ways.
How files are stored on Arweave
Arweave, formerly Archain, is a network for the decentralized storage of data on users’ computers. It is made up of two layers. The first layer is Blockweave, which allows blocks to be linked together. The Proof of Access consensus protocol is used to verify data integrity. Each block is linked to the previous block and to one random block. The second tier is Permaweb, an inexpensive and persistent web page and website hosting service. Data storage in this system is rewarded with AR tokens. There is no subscription in this system, and there is only a one-time fee for perpetual data storage, it costs around $8 per gigabyte with no storage time limit.
However, all of the above coins and protocols have now stalled in their development. They have to compete with each other as well as with established data storage services such as Dropbox. Their development is also limited by the need for specific blockchains, tokens and payment bridges to communicate with these networks. That said, the popularity of the token or coin itself has nothing to do with the popularity of the decentralized hosting itself. The alternative is cloud-based decentralized data storage.
Cloud decentralized data storage
The great thing about cloud decentralized storage is that it’s growing in popularity, and the overall capitalisation of the field is increasing. More and more IoT devices and mobile gadgets are using cloud-based decentralized data storage. Cloud storage is becoming the cornerstone of many systems, the hub on which all work is wrapped up. Client-side encryption alone cannot compete with centralized storage services. There is also cloud-based, decentralized storage that relies on the familiar IPFS.
In 2021, the total cloud computing market was estimated to be worth more than $300 billion. With cloud technology, we can use Google’s email and documents anywhere in the world without having to worry about overflowing that storage in any way. Currently, Microsoft Azure, Amazon AWS and Google Cloud dominate among cloud computing providers.
The use of decentralization and blockchain technologies allows resources to be used more efficiently.
Right now, around 2% of global energy consumption is spent on data storage, with cloud providers typically using 10-15% of their capacity. For example, during the day and evening, when people are all doing business and playing games and watching media, the cloud computing center utilization increases to half of the possible capacity. When people are watching popular shows and sporting events, the hardware utilization is even higher. However, in the early morning hours, activity is below a few percent and the facility as a whole is virtually idle, consuming a lot of energy for cooling. Thanks to the decentralization of cloud computing, data that can wait a few seconds for processing can be processed from another country. And blockchain technology is used to account for hardware usage. In this way, all cloud storage is gradually becoming decentralized. This is another important implementation of blockchain technology.
