Exploring FileHog: Decentralised Personal Storage with Codex

Exploring FileHog: Decentralised Personal Storage with Codex

In a recent Learn-Up session, the Codex team explored the specification for FileHog, a straightforward tool designed to upload entire folders to the Codex decentralised storage network. 

While the concept may sound simple, the demonstration revealed a thoughtfully constructed approach to applying Codex in real-world scenarios. 

In particular, this specification provides a way for users to store personal data securely, privately, and without relying on centralised cloud services.

To learn more about FileHog and its application of the Codex Data Durability Engine to provide durable personal data storage, check out the video walkthrough or read on below:

What is FileHog and How Does it Work?

FileHog describes the specification for a standalone tool that enables users to select a folder on their computer that they would like to persist durably, such as one containing family photos or video files, and upload its contents to the Codex network. 

The primary aim is to deliver long-term durability and data privacy, ensuring that even if a local drive fails, the files remain accessible. Crucially, the system removes any dependency on third-party cloud platforms.

Originally developed as part of Codex’s internal stress-testing suite, the specification for FileHog is designed to be implemented by non-technical users. Once it is running, it handles uploads and monitoring in the background without any further input required.

At its core, FileHog watches a specified target folder and uploads all its contents to a Codex storage node. 

Metadata about each upload, such as content IDs (CIDs), timestamps, and purchase IDs, is stored in an output folder, which acts as a retrieval log. This folder contains everything a user would need to recover their files later.

After it’s launched, FileHog continuously monitors the folder for changes. When new files are added or existing files are modified, the tool detects the updates and handles uploading. It also keeps an eye on storage contracts created in the Codex network; if contracts expire, fail, or complete, FileHog renews them automatically, ensuring that the data remains available for as long as the user desires.

Design, Structure, and Storage Efficiency

FileHog currently operates within a number of design constraints for simplicity and ease of deployment. For example, it assumes that once added, files in the target folder are not deleted or altered.

The application supports files between 1 MB and 1 GB in size, and users must supply their own Codex endpoints - the preconfigured Codex nodes to which Filehog will send data.

Note that the current Codex 0.2.4 binary releases and previous versions do not encrypt data, but symmetric encryption for Codex nodes is expected to be phased in as part of the migration to an Incentivised Testnet later this year. When this is implemented, we will publicly announce which binary releases support symmetric encryption. In the meantime, users are advised to pre-encrypt any sensitive data themselves before uploading.

FileHog is being designed to work on all major operating systems without a graphical interface, which is ideal for server environments or automated backup systems. The end goal of the specification is to enable apps that offer simple, automated off-site storage which does not rely on traditional cloud providers.

An important design discussion during the demonstration centred around whether FileHog should upload each file separately or bundle files into archives (such as tarballs). Uploading files individually allows users to retrieve specific files without downloading everything, but may lead to inefficient storage if each file demands its own contract.

On the other hand, bundling files can reduce overhead but sacrifices selective access. Proposed compromises include treating subfolders as individual datasets and creating a dynamic size threshold above which files are bundled.

An Ideal Entry Point for Contributors

FileHog offers an exciting platform for future innovation. Features such as access control, multi-user file sharing, or advanced encryption settings could be developed on top of this straightforward architecture.

FileHog is a great entry point for developers and other contributors looking to get involved with Codex. The specification is public, and contributions from external and open-source developers are encouraged. 

Those who build meaningful features or demonstrate thoughtful implementation could also potentially take on larger roles within the project.

A simple and accessible use case, FileHog makes Codex tangible. It offers a real-world solution to a universal problem: safe, reliable, and durable storage for personal files.

Codex is currently operating a public Non-Incentivised Testnet, where anyone can install and run a node using the peer-to-peer Altruistic Mode.

If you want to help build the future of decentralised storage or integrate Codex into your application, get started now at docs.codex.storage.

Follow us on social media, join our Discord, and subscribe to our newsletter to get the latest updates from Codex.