Will Urbit be a general file server?

urbit system

After learning what Urbit is, a common question that arises is whether Urbit has the necessary infrastructure to become a general file server, allowing the sharing and storage of images, videos and any type of file in a decentralized way, serving as an alternative to systems like IPFS.

The answer is yes, Urbit will one day have these features. Today Urbit's infrastructure is not complete, but some applications such as DEXes (for example Uniswap) can already have their virtual interface stored in Urbit today.

A Closer Technical Look

urbit logo

Everything in Urbit is represented as a noun. Nouns persist orthogonally (the entire state of your Urbit is a single value), and we serialize them and send them over the network.

There are basically two kinds of limitations that have effectively prevented arbitrary file storage/sharing: lack of tooling and runtime memory limits. Until about a year ago, Vere could only address 2 GiBs worth of nouns. That's since been doubled twice to 8 GiB. Some fairly straightforward work could double that once more, while getting further would involve switching to 64 bit (32-bit assumptions are deeply embedded in the codespace, so this would involve a fair amount of grunt work).

The Ares project (the new Urbit runtime) is starting out with 64-bit references, so it won't have those same limitations. Vere's default memory arena (the "loom") is still 2 GB. We only recently shipped "demand paging", which makes large persistent states much more efficient (can be cheaply/transparently moved from disk to memory and back). So we should be able to raise the default settings to the max of 8 GB without breaking anyone's setup (which why they were previously opt-in). There needs to be good support for getting things in and out of Urbit (not rocket science, it just doesn't exist in a particularly general or usable way right now, but we've gotten much closer in the last couple years).

Urbit will be a great, general purpose file server, and well before 5 years. Urbit is designed for you (or whoever is running your ship on your behalf) to host/control your data directly. We (or anyone) could build an IPFS-like layer on top of urbit, but data non-locality and the associated incentive structures are'nt necessarily an intrinsic part of Urbit's roadmap.

On the other hand, the lack of tooling: to use urbit as a general file server, it needs to be straightforward to see and manage what you have, how much memory are you using, how much free space do you have, what files should you delete to free up space as needed.

Recently, the Urbit Improvement Proposal (UIP) system was implemented. That is now the way core capabilities and features get on the roadmap.

More about the Ares Project:

The article from Urbit.org provides a light technical description of Ares, the new Urbit runtime. Here's a breakdown of how Ares will handle the sharing and storage of images and videos, as well as its limitations:

What is Ares?

Ares is a new Urbit runtime designed to be faster and manage a much larger amount of data than previous versions. With Ares, users will be able to host photos and videos on their Urbit ships. The project is a collaboration between the Urbit Foundation, Tlon, and Zorp, and it consists of three main components:

Code Generation:

Modern CPUs are fast, but previous Urbit runtimes couldn't fully utilize this speed. Ares introduces a codegen system that compiles Nock (Urbit’s assembly language) to machine code that the CPU can predict well.

The new code generation process in Ares uses a system called “subject knowledge analysis.” This system analyzes how an Urbit subject might change during a Nock computation, allowing the runtime to understand the subject's tree shape and its data dependencies. This knowledge enables the machine code to run faster and more efficiently.

2stackz Noun Allocator:

Ares introduces the 2stackz noun allocator, which means Urbit will operate as a virtual machine with two stacks and no heap. Traditional programs have a stack (for execution) and a heap (for storing data). Ares eliminates the need for a heap because of the constrained nature of Nock. Instead, it splits the function call stack in two, using the top of either stack as a temporary heap.

This change makes the allocation and freeing of nouns (atoms or cells) in the Urbit runtime faster and more efficient.

Persistent Memory Arena (PMA):

The PMA is designed to manage large amounts of data within Urbit without slowing it down.nIf an Urbit has terabytes of data, only a small portion can be in the hardware’s RAM at any time. The rest resides on the disk. The motherboard moves data between RAM and disk as needed, a process known as paging.

The PMA is optimized for Urbit's single-level store, meaning there's no distinction between data in RAM and data on disk. The system manages which parts of the data are in RAM and which are on the disk. This allows Urbit's state to grow without being limited by the hardware’s RAM.

Limitations:

While Ares is designed to handle large amounts of data, there will still be constraints based on the hardware's RAM. The PMA can optimize data management, but it can't overcome the inherent limitations of the hardware. The article mentions that Ares is still a work in progress, implying that there might be unforeseen challenges or issues that arise as it's further developed and rolled out.

In summary, Ares is a significant update to the Urbit runtime, aiming to make it faster and more efficient, especially in terms of handling large data like images and videos. However, like any software, it will have its limitations, particularly those imposed by the underlying hardware.