abstract: The ability to precisely visualize the atomic geometry of the interactions between a drug and its protein target in structural models is critical in predicting the correct modifications in previously identified inhibitors to create more effective next generation drugs. It is currently common practice among medicinal chemists while attempting the above to access the information contained in three-dimensional structures by using two-dimensional projections, which can preclude disclosure of useful features. A more precise visualization of the three-dimensional configuration of the atomic geometry in the models can be achieved through the implementation of immersive virtual reality (VR). In this work, we present a freely available software pipeline for visualising protein structures through VR. New customer hardware, such as the HTC Vive and the Oculus Rift utilized in this study, are available at reasonable prices. Moreover, we have combined VR visualization with fast algorithms for simulating intramolecular motions of protein flexibility, in an effort to further improve structure-lead drug design by exposing molecular interactions that might be hidden in the less informative static models.
author: Erick Martins Ratamero; Dom Bellini; Christopher G. Dowson; Rudolf A. Römer
bibliography: Mendeley_Proteins.bib
date: $Revision: 1.0$, compiled 2024-10-03
title: Touching proteins with virtual bare hands: how to visualize protein-drug complexes and their dynamics in virtual reality

# Introduction

Proteins are three-dimensional (3D) objects and, for the last century, spatially-resolved structural models of proteins and other biologically relevant molecules have been provided by various experimental techniques. X-ray crystallography was particularly instrumental in this revolution with the very first structure of a protein resolved by this method in the 1950s . Since then, X-ray crystallography has led to the building of detailed protein models and was instrumental for a number of important advances, with Watson and Crick's accurate 3D model of the DNA structure as a prominent example . The 3D characteristics of protein molecules are important in aiding our comprehension of many biological processes. Furthermore, beyond the classical "structure implies function" approach, it is now also becoming increasingly clear that protein dynamics is key to understanding protein function . One of the ways we could potentially access this information is by interacting with, manipulating and visualising static and dynamic models of such proteins in 3D. These might be constructed as real objects or exist in a virtual reality (VR) environment. A large part of scientific and medicinal research on drugs, such as, e.g., understanding antimicrobial resistance (AMR), revolves around clarifying the ways drugs bind to proteins and vice versa. In the, e.g., AMR context, viewing protein structures and their dynamics and understanding how mutations can lead to conformational changes and, thus, changes in binding regions that are relevant to AMR, is essential.

Since computers became ubiquitous, the development of *computer models* for proteins, as opposed to physical ones, has became progressively easier and cheaper. With that came the possibility to show proteins in stereoscopic 3D view. Many such projects were developed, as TAMS, for example, which used polarized slide projectors to display stereoscopic 3D images . Many tools exist to make viewing protein structures possible, such as PyMol , VMD , Rasmol and Chimera . In recent years, there was a further push to develop systems based on web browsers, such as iView and Jmol . To some degree, every one of those tools can produce stereoscopic 3D images, be it through passive 3D (using chromatic distortion glasses), active 3D (with shutter glasses synchronised to the image displaying device) and autostereoscopic 3D (no headgear required) . Though their method changes significantly, they all aim for the same near-3D effect for the end user. Besides technical drawbacks that either limit resolution or require expensive equipment, these attempts at providing 3D perception when analysing protein structures lack *immersion* into a different environment and true 3D depth perception.

VR allows us to address this lack of immersion, and introduce a level of interaction between user and visualisation tool that was not possible before. While the usage of VR in research is not new, the current levels of performance with affordable cost definitely are. Implementing a VR can be achieved through many different techniques. On one side of the complexity scale, we could point towards whole-room arrangements like CAVEs (cave automatic virtual environments) , while extremely simple solutions like Google Cardboard would lie at the other end of that scale. Somewhere in between we find modern head-mounted devices (HMDs), like the Oculus Rift and the HTC Vive . These create a stereoscopic 3D effect through the usage of LCD displays and lenses that allow the system to display different images for each eye, with slightly different points of view that mimic the position of the eyes of a virtual observer. These HMDs are relatively easy to use and to program, with excellent display quality and affordable prices. The prominence of such HMDs amongst the gaming community is particularly useful for researchers. Since VR gaming has gained popularity, the tools for programming software to use HMD capabilities have become better, more streamlined and easier to use for people even without a background in graphics programming. Initiatives such as SteamVR and VR addons for the Unity3D game engine decrease the amount of work necessary to build a VR application considerably.

This paper reports on a protocol for introducing protein structures into VR programs, using a combination of widely and freely available software and custom-built scripts and programs. Besides being an useful tool for researchers to visualise conformational changes in proteins, VR also provides a great outreach opportunity to motivate the general public to understand proteins and their relationship to biochemical research and its applications in drug discovery better. We aim with this paper to give our readers the necessary tools to start their own VR applications. Freely available VR setups for HTC Vive and Oculus Rift can be downloaded from Ref. .

# Background and Motivation

Exploring VR environments is not a recent pursuit. The Sensorama , first proposed in 1955 and constructed in 1962, is one of the earliest examples of such attempts. The late 1980s was the first period when VR applications and technology drew real interest, with fast growth in adoption during that time. However, the display technology lacked definition, and most products were dismissed as being low quality and unresponsive, both in terms of response time and tactile feedback . The responses to these problems were products with exorbitant costs that were not accessible for many research groups and most consumers. As inaccessible as they were, these products created excitement around VR, while the technology was still not ready to deliver results for most people, and the interest in VR decreased towards the end of the 1990s. At the same time, the usage of video game technology for research and clinical purposes ("serious games") started gaining popularity . This movement drove a resurgence in interest for VR technologies; as an example, visualisation of volumetric data and engineering assemblies were two of the subjects of VR development in recent years. The current VR "boom" differs from the one during the 80's and 90's. This time, consumer-level hardware is relatively affordable and presents high-quality displays, and game software technology is sufficiently developed to make creating new VR applications simple and fast, even for scientists without training in graphics programming.

In our previous work on AMR, we have developed methods and algorithms for simulating the mobility of proteins based on flexibility . This method requires four orders of magnitude less computing power when compared with molecular dynamics, and yields a series of conformers that describe the large-scale motions of a protein. During this project, it became immediately apparent how difficult it was to distinguish between consecutive conformers and visualise the difference between them. It was, then, not easy to see which motions were actually taking place in the protein structure, and near impossible to derive any biological insight from that. Our initial approach was to generate videos from the series of conformers. This approach clarified the kinds of motion that were happening in larger time scales, and it allowed us to make use of our flexibility-based simulations to better understand the interactions of the proteins being studied. However, it was still difficult to identify motions in specific residues and domains of the proteins. Specific points of view were necessary for generating videos, and those would always make certain areas clearer, while others were obscured. In this context, an interactive solution where the protein appears as an animated 3D object that can be freely rotated and manipulated is desirable. Thus, VR emerges as an ideal solution for visualising protein dynamics. It allows us to clearly communicate the large-scale motions taking place on the protein structure, while making interaction possible for looking closer at specific areas of interest, such as binding sites.

# Workflow

Our choice for an HMD for VR is the HTC Vive . It uses two infrared emitters ("lighthouses") in the corners of the play area, which generate infrared beams in sweeping patterns. The headset possess multiple infrared photo diodes, which will detect these beams, and the position of the headset in the room can be reconstructed from the time differences between signals received at each diode . The user can move around the room freely, with the only inconvenience being the cables attached to the headset. Furthermore, the HTC Vive can also track controllers to grab and manipulate objects in virtual environments, and can be precisely tracked and displayed inside the virtual environment as well. This allows for an immersive, interactive experience, where the limits are only the HMD cable extension and the corners of the delimited play area defined by the position of the lighthouses.

Our application uses Unity3D, a programming and execution environment . This is a game engine, which allows us to implement ideas quickly and easily. Since it provides standard VR features such as graphics, physics (gravity and rigid-body modelling) and lighting, people without programming experience in these areas can also develop their ideas with minimal training involved. Whenever the standard features are not enough, Unity3D also allows us to write scripts (in C#, in our case) to treat special cases or implement features that are not readily available. One of these examples is presented as Fig. . Here, we have written some code to implement protein animation by replacing the structure being presented multiple times per second. For interfacing with the virtual reality equipment, we use the SteamVR abstraction layer. By using such a tool, it is possible to create a single project that works seamlessly on the two most popular HMDs (Oculus Rift and HTC Vive), without any code duplication. However, using such a tool for creating our applications brings downsides as well. Not every file format can be easily imported into Unity3D; specifically, importing 3D models with correct colours can be tricky. Since protein structures will be represented as 3D models, this required a specific set up to allow us to visualise these structures inside Unity3D.

We present our current workflow for displaying protein structures in VR in Fig. . The initial inputs to our application are Protein Data Bank (PDB) files. These are text files that describe the geometric structure of molecules. It allows for description of atomic coordinates, rotamers, secondary structures and connections between atoms. In our case, we are mainly concerned with the 3D position of each atom in the protein at each simulation time step; these positions need to be translated to an animated 3D protein structure.

Representing individual atoms in a protein does not indicate clearly 3D features such as twist and fold. Therefore, we will use the ribbon diagram representation to visualise secondary structures. These are representations where the polypeptide backbone is interpolated by a smooth curve, generating helices, sheets and loops. The result is a simple, yet informative diagram where 3D information about the protein can be easily displayed. In our case, we use the Visual Molecular Dynamics (VMD) program for creating representations. Other representations such as the full "sphere" atom representation can of course be chosen as well if desired. The next step is generating 3D models for the ribbon representations. VMD allows us to create OBJ files; this is an open file format for 3D geometry, storing information about individual vertices, vertex normals, faces of polygons and textures associated with them. This way, we can export the 3D ribbon representations (and any other 3D structure, for that matter) from VMD while keeping any visual information such as colouring associated with it. Finally, it is necessary to import the generated OBJ files into Unity3D. Fortunately, Unity3D imports OBJ files natively, and integrates textures without any extra effort being necessary. Therefore, the VMD-generated structures can immediately become physical objects in the VR environment. Unity3D allows us to attach collider objects and rigid-body physics to the protein objects, integrating them into the physics framework of the application seamlessly.

It is also possible to introduce dynamic structures into VR. We have use data generated by our simulation method in the form of multiple PDB files acting as snapshots of the protein conformation over time. Next, we load each individual PDB file into our pipeline, obtaining 3D models for each frame. We then use a script in Unity3D to flip through the models over time (cp. Fig. ), creating an animation that shows the conformational changes calculated by our simulations.

#  Case Studies

The structural models used in this study for the implementation of 3D visualization in VR are those of four bacterial proteins from different cellular compartments: cytoplasm, inner membrane, periplasm and outer membrane. MurC is a cytoplasmic ligase involved in the construction of the pentapeptide stem, which is a central component of the peptidoglycan (Fig. a). MraY is an integral cytoplasmic membrane enzyme that catalyzes the formation of lipid II by transferring the pentapeptide to the lipid carrier, which is another essential step in peptidoglycan biosynthesis (Fig. b). Penicillin-binding protein 1b (PBP1b) is a bifunctional enzyme containing both a transglycosilation (TG) and a transpeptidation (TP) domain, which is able to polymerize lipid II to form the peptidoglycan mesh (Fig. c). OmpF is an integral outer membrane channel exploited by most antimicrobial drugs to enter the organism on their way to the target (Fig. d). All structures are in complex with either the substrate or antimicrobial drugs that inhibit bacterial cell wall (peptidoglycan) synthesis. As such, they are currently studied targets for understanding AMR resistance mechanisms .

Initially, intrinsic motions of these proteins were simulated as presented above by analysing protein flexibility, defining movement modes through elastic network modeling and generating conformers based on that information and steric interactions. These simulations produced a series of PDB files detailing the state of protein structures at certain "snapshot" moments (e.g., every 100th conformer out of $5000$ overall for each mode of motion). The output of these simulations are PDB files containing the motions of protein atomic coordinations over time. Unfortunately, the PDB format is not directly supported by the VR software Unity3D. Therefore, VMD was subsequently used to transform PDB files into OBJ files. TCL scripts for VMD were written to go through output folders from the simulations and automatically generate OBJ files of each conformer. When writing these TCL scripts, care was taken to ensure that positive and negative directions of motion blend together seamlessly. An example of such script for a specific protein is presented as Fig. . Here, every PDB file from folders with the "pos" and "neg" tags for positive and negative directions of protein movement gets loaded onto VMD, specific colouring and representation options for that protein are chosen and finally each frame is rendered and stored as a Wavefront (OBJ) file. A dataset containing all source code used in this project is available in a public repository for download . Through this process, a series of OBJ files are obtained that act as 3D "frames" of animation. Upon importing these files into Unity3D, the only task left is to ensure that these frames are shown sequentially to impart upon the user the illusion of an animated protein. In this study we do so by updating the displayed object every $1/30$ of a second. Though theoretically possible, it was decided against updating the collision structure of the object at each frame, as this would consume significant computational resources, even if it would allow for "correct" physics in VR at all times. Instead, we have chosen to define a bounding box spanning the whole range of movement for the protein as a constant collision structure.

A "template" room was constructed for visualising proteins structures in Unity3D. This saves time when a quick visualisation is necessary by importing the relevant OBJ files into an existing "template" project where lighting and physics of the virtual room have previously been defined, and into which VR structures have already been hooked. Use of a "template" project reduces the tasks to simply importing OBJ files to create "father" objects for the 3D models and to define rigid body physics and collider boxes. If necessary, a script can be attached to that object for animation. Depending on the number of snapshots in PDB files, the process from simulation outputs to ready-to-use VR application can be as short as 5 minutes.

Snapshots of the VR application being used and presented here are shown in figures (a) to (d), where a user was interacting with the four proteins discussed above in a VR environment. Conveying a VR experience in pictures is a difficult task; readers with access to the HTC Vice or the Oculus Rift can download our preconfigured VR rooms from . Fig. (a) shows the template room containing the four protein structures . Some physical objects such as cubes and balls have been added to emphasize the sense of "reality" and physicality around the protein models. This figure shows how is also possible to see both controllers being held by an user, with tool tips added for introducing new players to the controls. Fig. (b) shows a snapshot of a protein structure being held by an user. In this case, the user is holding up dimeric MraY, with the same structure and coloring as in Fig. (a). A close look to the drug and its interactions with protein residues can be taken as desired by "grabbing" the protein with the controller and moving it towards the viewer; tilting the controller allows to rotate the protein structure in an easy and intuitive way.

Further detail on specific residues and drug-binding pockets can be seen by simply approaching the areas of interest, or through clipping the protein structure through the view point. Fig. (c) shows a detail of ampicillin bound to the TP domain of PBP1b. This is the same region presented in light blue in Fig. (d). This figure shows how much and how simple VR allows for in-depth analysis of specific areas of a protein structures. Finally, in Fig. (d), we present a detail of an area of contact between MurC, the substrate and the cofactor, including the two catalytic manganese ions. In the VR environment, investigation of protein-ligand interactions is straightforward, since complete control of the 3D position and rotation of the protein model is mapped to movement of the controllers.

# Conclusions

In this work, we present a way to visualise and interact with both static structure and dynamics of proteins by using VR. A software pipeline was constructed that enables non-expert researchers to easily embed protein structures into VR programs using a combination of widely available software and custom-built codes. The immersion and interactivity that VR brings can significantly change the level of details accessible to a researcher when analysing protein-ligand interactions or conformational changes. While it is difficult to convey this with static figures as in Fig. , it is interesting to report our personal experiences with immersive 3D VR. For example, despite having worked with these protein structures for a while and having observed the drug-bound catalytic pockets of these proteins for many times on 2D rendering softwares such as PyMol, only when we observed the same in VR we realized that the 3D configuration of the catalytic pocket was significantly different to the picture conveyed by the 2D projections and that we had built in our minds. These differences could translate into designing more effective modifications into the drug.

With the decreasing costs for customer VR hardware and an established workflow for importing structures into VR, immersive 3D visualization should become viable for an increasing number of research groups. It is a first step for developing VR-based ways of interacting with proteins and probing their properties. At the moment, there is still the limitation in both 2D and 3D protein visualization software packages that pre-computed conformers are necessary and the interaction between user and structure is only at a non-interactive visualisation level. In the future, physical interactions may become possible, where the user could bend the protein structure or add new functional groups to the ligand while background simulations calculate in real-time whether that changes are mechanically stable or energetically favorable. In such a scenario, immersive VR environments can aid much better than other 2D or 3D visualizations in guiding molecule manipulation.

As well as VR being a very useful tool for visualisation of both protein conformational changes and drug design, there is also the element of fantastic potentials for outreach and engagement with the general public.

# Acknowledgments

We gratefully acknowledge funding via the EPSRC's *Cross-scale prediction of Antimicrobial Resistance: from molecules to populations* network (EP/M027503/1). UK research data statement: data is available at .


abstract: Nakamoto consensus, the protocol underlying Bitcoin, has the potential to secure a new class of systems which agree on non-mathematical truths. As an example of this capability, we propose a design for a trustless, data availability oracle. This exposition reduces the problem of determining whether or not a registered datum is publicly available to the problem of constructing a network in which either almost all nodes can download a given datum, or almost none of them can.
author: Jason Teutsch
*TrueBit Estsblishment*
[`email@example.com`](mailto:firstname.lastname@example.com)
bibliography: decentralized_oracles.bib
date: December 25, 2017[^1]
title: On decentralized oracles for data availability

# Silo effect

Traditional blockchains, like Bitcoin and Ethereum , live a lonely life. By design, miners, which constitute the "eyes and ears" of each network, have no formal means to reach consensus about external events. While Oraclize  and Reality Keys  successfully pipe Internet information into Ethereum, their bridges rely on trusted authorities. These efficient, oracle methods, and even aggregating methods like Chainlink  suffice for many applications, however they also introduce central failure points in Ethereum's otherwise autonomous, reputationless system.

Unstoppable, decentralized applications, such as the original Livepeer protocol  demand not only trustless computation oracles, like TrueBit , but also un-erasable data storage. Ethereum provides scarce but immutable "on-chain" storage, however prohibitive expense limits on-chain storage to only the most laconic of messages. Ethereum's security depends on every miner storing every byte of blockchain data until the end of time, hence substantive data uploads on this system remain unwieldy. Moreover, propagating more than a little data in each block could induce rational miners to skip block verification and even break the underlying consensus, a phenomenon known as the Verifier's Dilemma . Indeed, rational miners might not wait for downloads. We therefore consider off-chain alternatives.

We set ourselves the task of constructing a trustless oracle which can confirm off-chain data availability. This use case fundamentally differs from that of traditional, cloud storage platforms, such as Dropbox , Storj , Sia , and Maidsafe , which place users in control of the data that they upload. Dropbox, in particular, allows uploaders to decide whether and when to share their data with others. In a data availability scenario, on the other hand, disappearance of data can destroy high-stakes financial transactions as well as critical computations. The network must therefore counteract users' incentives to *pretend* that they have "published" data. For illustration, a Bitcoin transaction should not magically disappear after the system confirms its funds as spent, and neither should the input for a TrueBit computation vanish before the system has had time to process it.

To distinguish the present approach from existing projects like Filecoin  and Swarm , we explicitly isolate the data availability problem from other commonly associated concepts such as data sharding, privacy, computation, scalability, and Ethereum. We aim for a dead simple, modular design which is amenable to a clean and rigorous security analysis.

#### Overview of technical contributions.

Under the peer-to-peer network assumptions of Section , the proposed construction in Section  achieves a trustless, Nakamoto-based system which correctly reports on whether or not registered data are available during a given epoch. Our construction maintains integrity without resorting to either distinguished nodes or a reputation scheme. Any anonymous node can participate in the system with a deposit, and like Bitcoin, security improves as more honest and rational miners nodes join the network. The main idea is straightforward: use Nakamoto consensus  to create a report of what's available and what isn't.

In the present system, every functioning Miner node stores and propagates each and every registered datum. We do not attempt to "shard" data among various parties. Rather than attempting an infinitely scalable solution, we instead content ourselves with designing a system where each node securely stores some orders of magnitude more data than "on-chain" storage permits. Unlike a blockchain, which permanently stores all data forever, the present system releases data after its registration period ends, hence its storage space is reusable. We shall further explore scaling methods in Section .

Building on an independent blockchain offers some flexibility over Ethereum. As noted earlier, Ethereum has limited storage space and its miners do not see external inputs. Thus our data availability system requires some consensus beyond Ethereum mining. Other approaches to trusted data feeds include data attestation via trusted hardware  and, since the original publication of this manuscript, decentralized, on-chain voting via smart contracts .

# Warm-up: fiat-crypto exchange rates

Verifying an Ethereum transaction amounts to checking signatures, logical conditions, and self-evident mathematical truths. Thus Ethereum miners, who are tasked with verifying these transactions, need only pay attention to local announcements on the blockchain. One can, however, imagine that miners could agree on other kinds of objective, global facts, whose validity depends on time. In this section, we outline a simple "consensus computer"  application which extends beyond agreement on mathematical facts.

Consider the following concrete example. Some decentralized applications require a bound on the exchange rate between a stable, fiat currency and a native, cryptocurrency token. Since miners in traditional blockchains do not observe fiat exchanges, some other mechanism must necessarily feed this information to the blockchain. Rather than relying on an authority, Teutsch and Reitwießner proposed to use a blockchain consensus protocol to agree on external exchange costs . We shall now extrapolate on this idea and then, in Section , transmogrify it into a data availability protocol.

Network nodes agree on exchange rates using a variant of Nakamoto consensus , the protocol underlying Bitcoin. Rather than collating financial transactions into blocks, each Miner simply includes, in addition to a proof-of-work, the delimiters of a real interval allegedly containing the value of the current exchange rate. A block, then, is valid if both the proof-of-work is correct and its interval contains the *true* exchange rate accepted by a majority of other Miners. By convention, Miners keep an eye on real exchange rates and mine on top of blocks they perceive as valid.

How large should the exchange rate interval be? While in general, Miners might be able to agree, say, that the price of an ether lies between 650 and 700 USD, during moments of wild volatility or low liquidity, the market may not afford such precision. For this reason, each Miner independently chooses the size of of the interval and receives a block reward inversely related to the size of the interval. Smaller intervals have higher reward potential, however they also have a greater chance of being ignored by Miners who may perceive the associated block as invalid.

# The consistency problem

Let us now return to our original problem. We seek a decentralized system which correctly reports on data availability. Our consensus protocol construction hinges on the following crucial assumption about the underlying peer-to-peer network.

**Consistency Axiom 1**. Either almost all nodes in the network can download a given datum, or almost none of them can.

The World Wide Web, for example, largely exhibits this property. Either the website truebit.io is "up" and everyone in the world can see it, or else most people agree that the site is "down" regardless of their Internet access point. For the purpose of this exposition, we shall assume that a peer-to-peer network exists and exhibits this desired consistency property. We shall not concern ourselves here with the construction of the peer-to-peer layer but rather explore a cryptoeconomic structure on top of it.

The Consistency Axiom above permits us to make the following well-defined notion.

**Definition 1**. Let $x$ be a datum. If most nodes can see $x$, then $x$ is *publicly available*. If few nodes can see $x$, then $x$ is *not publicly available*.

Due to the gap afforded by the Consistency Axiom, the above definition covers all possible cases for all data. We now introduce a secondary network assumption.

**Upload Axiom 1**. Any computer can join the network and, with high probability, propagate data into a publicly available state.

Without the Upload Axiom, our Consistency Axiom might vacuously describe a network without any nodes. A new upload need not become instantly available, however we shall assume a bounded lag on its propagation. We conclude this discussion with one final requirement.

**Directory Axiom 1**. Nodes can efficiently determine whether or a datum $x$ is publicly available from $\mathop{\mathrm{hash}}(x)$. If $x$ is available, then $\mathop{\mathrm{hash}}(x)$ tells where to download $x$.

Thus any node which wishes to upload or download a publicly available datum can use the datum's hash as a directory. The IPFS  filesystem, for example, satisfies the Directory Axiom. Aside from exceptions described in Section , the system relies on Nakamoto consensus for security.

# A consensus protocol for data availability

Under the axioms of Section , we now devise a decentralized system which indicates whether or not a registered datum is publicly available. We assume familiarity with Nakamoto consensus . The network consists of two types of parties: Storers who wish to provably publish data and Miners who both confirm and guarantee availability of data. Let $c$ be the *network lag time*, or number of blocks to confirm a storage request for a datum and propagate it through the network. For simplicity of presentation, we assume that this time bound suffices for all datums, regardless of size. The Upload Axiom from Section  now allows us to introduce the following upload interface.

Storer interface.

:   A Storer who wishes to publish a datum $x$ broadcasts a *registration* of $x$ to the network consisting of the following components:

    1.  $\mathop{\mathrm{hash}}(x)$ which, by the Directory Axiom, doubles as an address for downloads,

    2.  the number of block epochs for which $x$ should be publicly available (excluding network lag time $c$), and

    3.  a *reporting fee* payable to Miners based on the size of $x$ and the registration duration specified in item 2.

    We say that datum $x$ is *registered at time $t$* if the blockchain contains a registration for $x$ that persists at block epoch $t$. Note that registered data may or may not be publicly available. The Storer must propagate his registered datum, lest the network report it as unavailable.

    **Definition 2**. A *report*, denoted $R(x)$, is a formal, plaintext assertion that datum $x$ is publicly available. A Miner who includes $R(x)$ in the $t^\text{th}$ block signals that, "datum $x$ is publicly available in both block epochs $t$ and $t+1$." A set of reports $S$ in block $t$ is called *complete* if for every registered datum $x$, $R(x) \in S$ if and only if $x$ is publicly available in block epochs $t$ and $t+1$.

Using Definition , we shall obligate each Miner who propagates a block claiming that $x$ is publicly available at time $t$ to further ensure that $x$ remains available at time $t+1$. We shall assume that a single miner propagating $x$ suffices to make the datum publicly available, hence an honest Miner can always ensure $R(x)$ holds and guarantee herself a block reward. See "Space economy" in Section  for further details.

Miner interface.

:   Any Miner who wishes to join the network first identifies the "longest" blockchain, namely the one containing the greatest proof-of-work . The miner obtains the longest blockchain from peer nodes and retraces it from its genesis block, observing each Storer transaction along the way in sequence, and noting which Storer requests are still registered at the present moment. The Miner attempts to download, store, and propagate all currently registered data. The Miner locally considers any data successfully downloaded to be publicly available. She can determine validity of the current header block valid, according to the criteria below, after silently observing the chain for $c$ epochs. We assume that the initial, altrustic Miners in the system converge to a consistent world view during the first $c$ blocks following the consensus genesis.

    **Definition 3**. A *valid* block at time $t$ consists of the following elements:

    1.  the complete set of reports at time $t$ (taking into consideration network lag time $c$),

    2.  a collection of new, cryptographically signed Storer requests, and

    3.  a value *nonce* witnessing a proof-of-work. More specifically, the concatenation of the following components must hash to a small value:

        1.  the mining nonce ,

        2.  items 1. and 2. above.

        3.  a private key at which to receive the block reward and network fees, and

        4.  the hash of the previous block header.

    **Main Loop 1**. The protocol steps for the Miner now roughly follow Nakamoto consensus .

    1.  In each block epoch, the first miner to find a valid block broadcasts it to the network and receives a block reward plus applicable reporting fees.

    2.  Upon verifying a new valid block, each Miner downloads, stores, and propagates all data registrations contained in the new block. The Miner locally considers any data successfully downloaded to be publicly available and propagates the data as expediently as possible.

    3.  The mining race begins anew on top of the new block.

    Miners always mine on the "longest" chain whose most recent $c$ blocks are all valid. "Longest" here formally means the chain with the greatest proof-of-work, since block difficulty may change over time. Miners need not store data which ceases to maintain registered status.

Unlike Bitcoin, the present blockchain construction has no notion of validity for individual "transactions." Indeed reports are only valid as sets. We remark that the validity of a block also depends on time because data presence can vary. We thus realize a powerful application of the consensus computer  which both requires and permits agreement on facts external to mathematics.

# Security analysis

In this section, we argue that the blockchain construction in Section  accurately reports data availability. More specifically, we argue that a report for a registered datum $x$ appears in the $t^\text{th}$ block if and only if $x$ was publicly available during the $t^\text{th}$ block epoch. In other words, the system records both when a registered datum is publicly available and when it isn't. The "Main Loop" in Section  dictates that Miners always broadcast valid blocks consisting of complete reports. The security doubts we must resolve in order to establish our desired property above are twofold:

1.  Can rational Miners and Storers gain by deviating from the consensus protocol?

2.  Does the consensus withstand peer-to-peer layer failures?

We shall assume that an adversary of the first type wishes to either:

1.  convene a published report that some datum was publicly available when it actually wasn't, or

2.  receive a block reward without actually committing resources to the network.

Nakamoto consensus largely inhibits attacks of types 1a) and 1b). The Consistency Axiom permits us to circumvent potential attack vectors of the second type, however precise protocol adaptations for handling these attacks depends on the specific implementation of the underlying peer-to-peer network. It remains a crucial open problem to design and construct a peer-to-peer layer satisfying the three axioms in Section .

#### Data withholding.

A Storer who registers a datum $x$ could potentially take any of the following actions:

1.  neglect to publish $x$ itself,

2.  delay his public reveal of $x$ until the final moments of $c$-block propagation period, or

3.  publish $x$ at first but then hide this datum from the network.

In case 1, Miners never see $x$, unless it was otherwise publicly available, and hence they never report $x$ in a block. Since the blockchain witnesses no report of $x$ as publicly available, this type 1a) attack fails. Cases 2 and 3 do not impact functionality of the network, care of the Consistency Axiom. Indeed, either $x$ becomes publicly available, in which case most Miners would report it as such, or else it would not be publicly available, in which case they would not. Miners have incentive to propagate data (Section ), which reduces the chances that $x$ could transition from publicly available to not publicly available during its registration period. In short, the Storer cannot incite a fallacious or controversial report on the blockchain.

#### Space economy.

Although the consensus protocol in Section  obligates Miners to store and propagate data, a Miner might neglect this obligation in attempt to conserve resources in accordance with attack type 1b). A Miner who fails to upload and propagate publicly available data risks orphaning his block as other Miners who cannot access $x$ may perceive the block as invalid. Similarly, the Miner who mines the subsequent block at time $t+1$ bears responsibility for the availability of $x$ until the next block epoch $t+2$. This overlapping storage requirement reinforces the Consistency Axiom by incentivizing Miners to download and propagate all registered, publicly available data. Indeed, a Miner who witnesses but does not propagate data must rely on the faithfulness of other anonymous Miners and Storers to maintain validity of her reports in the successive mining race.

We remark that with some small probability, a Miner may not be able to see a publicly available datum and therefore might unknowingly propagate an invalid block. This invalid block would then not be accepted by other Miners, who have the "true" world view. As in Bitcoin, timely, well-intentioned blocks occasionally "uncle," or never reach the blockchain. Confirmed reports, however, remain consistent with publicly available data because the majority of Miners share correct perceptions.

#### Mining pools.

Miners often share computing resources and block rewards via *pools* in order to collectively reduce income variance. The pool *operator*, who coordinates a pool's cooperation efforts, typically chooses the reports for all members in the pool. Under such circumstances, pool members would rely on their pool operator to determine whether a datum is publicly available, thereby reducing the total number of eyes on the peer-to-peer network and weakening consistency. A simple solution is to require all mining pool members to choose their own reports rather than relying on operators for selection. To this end, the underlying consensus mechanism could, for example, mandate universal participation in SmartPool .

# Scalable implementation

How much data could a system like the one proposed here actually monitor in practice? While the actual capacity depends on the structure of the underlying peer-to-peer network, the present storage mechanism clearly has some finite limit for the same reasons that Bitcoin and Ethereum have bounded transaction volumes . The finite limit of this system should, however, greatly exceed what the system could securely store directly on the blockchain itself. What about more storage — does this construction scale? Fortunately, two independent storage systems can store twice as much as a single one!

In theory, one can store an unlimited amount of data through system replications, however, if one wishes to use the availability of data in some root system, such as Ethereum, the root system's underlying consensus protocol must keep an eye on each individual, data availability system. Suppose that a Task Giver in TrueBit were to provide a computational task whose off-chain input Solvers and Verifiers could not see (i.e. they witness only a hash on-chain). Then the Task Giver could potentially provide a bogus solution to his own task, and neither Solvers nor Verifiers would have any recourse to object against the data unavailability unless the authoritative Judges, or miners, collectively expand their myopic, on-chain, world view.

Finally, we remark that maintaining the same level of proof-of-work security on two independent blockchains requires twice as much mining resources as a single blockchain. Thus a hierarchical system such as Plasma , in which the integrity of each child blockchain relies on proper function of its parent, could provide a useful scaling model for data availability. Each leaf in Plasma's blockchain hierarchy would manage a modest amount of data, while the root system would only monitor data availability disputes for escalated situations.

#### Acknowledgements.

Thanks to Andreas Veneris for useful comments which are reflected in the updated version.

[^1]: This version, updated in 2019, includes minor changes, and corrections.