How Codex Enables a Durable Archive for Ethereum
The public testnet for the Codex storage network and decentralised durability engine (DDE) is now live. Anyone can run a node and help build the future of distributed censorship-resistant data storage.
The Codex whitepaper is also now available, detailing the vision behind the decentralised storage protocol as a robust and durable alternative to centralised storage providers that is also resistant to censorship.
With Ethereum’s roadmap directed towards rapid scaling, this is the ideal moment for a decentralised storage protocol to step up and address the long-term data storage concerns involved in the explosion of transaction data expected as the network grows.
Codex is designed as a step towards building a decentralised, resilient, and economically viable storage layer critical for the broader decentralised ecosystem. This makes it the ideal contender for serving the inevitable archival data storage needs of the Ethereum network.
You can try out the testnet now by downloading the Codex Testnet Starter.
Why Ethereum Needs Decentralised Archival Storage
As we have outlined in a previous article, Vitalik Buterin's “Endgame” for Ethereum scaling, which describes centralised block processing and decentralised validation, inherently requires a decentralised storage platform for archival blockchain data.
The network’s scaling roadmap would see the chain size continue to grow at an accelerating rate. This increases storage requirements for node operators and subsequently increases the risk of the network’s state history becoming available through fewer operators as it becomes unfeasible for them to store the full chain.
This trend would see the public record of truth for historical Ethereum transactions become increasingly centralised and therefore susceptible to risks such as manipulation or data loss. The current full chain sync size for node operators is over 1TB and growing.
However, these node operators discard the network's complete state history for efficiency, storing only a record of historical blockchain data and occasional snapshots. They can use these to regenerate older states on request, although this is computationally expensive. For example, accessing a historical state on a full node might require a node to execute all past transactions from the genesis block until that state is reached.
Archive nodes solve this problem by keeping a full history of the Ethereum network's previous states for quick reference. These nodes offer a trade-off that sacrifices disk space for a much more efficient lookup of historical data. Many full nodes may also opt to store the full archival history of Ethereum states to improve the efficiency of applications or tools they are running on the network.
For those operating archive nodes, the ballooning storage requirements accompanying Ethereum’s rapid scaling are even more concerning than for full-node operators.
As Ethereum’s state history continues to grow in size and accelerates, it will become prohibitively difficult to run an archive node. This already requires around 20TB of storage space, and it is easy to see how this could become a centralised operation, with only a few full node operators storing a complete history of Ethereum states.
The centralisation of archival data storage creates risk for the many dApps, rollups, and other projects that rely on this conveniently referencable information, increasing exposure to manipulation by bad actors, data corruption, or censorship.
Codex as a Censorship-Resistant Record of Ethereum’s History
Codex offers a durable, decentralised protocol for data storage that archive nodes could leverage to mitigate the risks of centralisation, allowing them to store archival blockchain data more safely and easily.
With erasure coding for redundancy and storage proofs, as well as zero-knowledge proofs and lazy repair for tunable durability guarantees, Codex can ensure that the archival network data relied on by Ethereum dApps will remain intact and reliably available while being distributed amongst operators running relatively modest hardware.
Using a decentralised data storage protocol like Codex, archive nodes can offload the cost of storing Ethereum’s historical states to a distributed network of providers, enabling a marketplace for archival Ethereum data storage where those hosting the data are rewarded for acting as distributed storage for an archive node.
In this way, the same robust incentives that drive staking participation through decentralised staking pools could be applied to the provision of archival data storage for Ethereum.
Rather than implementing a full archive node, specific applications may also be able to use Codex to store a history of transactions relevant to their dApp or service. This would allow them to easily reference historical states of the Ethereum network without relying on the data provided by another archive node, reducing centralised risk.
With Ethereum's rapid scaling, the need for a robust storage system is becoming increasingly critical, as current methods risk centralising historical transaction data. Codex presents a vital solution for Ethereum's archival storage challenges, offering a decentralised, censorship-resistant protocol that enhances data durability.
By fostering a distributed storage network and accompanying digital marketplace, Codex not only mitigates risks but also incentivises participation, ultimately supporting the long-term integrity of the Ethereum ecosystem.
Get Started with Codex
If you want to contribute to building the distributed storage platform that could help scale Ethereum while keeping it decentralised and open to all, you can install and try out the Codex testnet today.
Download the Codex Testnet Starter.
For help setting up a Codex testnet node, read our quickstart guide or head to the #join-testnet channel in the Codex Discord to start the onboarding process.
It is important to note that this testnet is an alpha implementation of the Codex protocol, and users should expect bugs and issues to arise during testing. If you encounter bugs in the Codex testnet, please report the issues on Discord.