abstract: This paper offers a critique of the Bayesian interpretation of quantum mechanics with particular focus on a paper by Caves, Fuchs, and Schack containing a critique of the "objective preparations view" or OPV. It also aims to carry the discussion beyond the hardened positions of Bayesians and proponents of the OPV. Several claims made by Caves et al. are rebutted, including the claim that different pure states may legitimately be assigned to the same system at the same time, and the claim that the quantum nature of a preparation device cannot legitimately be ignored. Both Bayesians and proponents of the OPV regard the time dependence of a quantum state as the continuous dependence on time of an evolving state of some kind. This leads to a false dilemma: quantum states are either objective states of nature or subjective states of belief. In reality they are neither. The present paper views the aforesaid dependence as a dependence on the time of the measurement to whose possible outcomes the quantum state serves to assign probabilities. This makes it possible to recognize the full implications of the only testable feature of the theory, viz., the probabilities it assigns to measurement outcomes. Most important among these are the objective fuzziness of all relative positions and momenta and the consequent incomplete spatiotemporal differentiation of the physical world. The latter makes it possible to draw a clear distinction between the macroscopic and the microscopic. This in turn makes it possible to understand the special status of measurements in all standard formulations of the theory. Whereas Bayesians have written contemptuously about the "folly" of conjoining "objective" to "probability," there are various reasons why quantum-mechanical probabilities can be considered objective, not least the fact that they are needed to quantify an objective fuzziness. But this cannot be appreciated without giving thought to the makeup of the world, which Bayesians refuse to do. Doing this on the basis of how quantum mechanics assigns probabilities, one finds that what constitutes the macroworld is a single Ultimate Reality, about which we know nothing, except that it manifests the macroworld or manifests itself as the macroworld. The so-called microworld is neither a world nor a part of any world but instead is instrumental in the manifestation of the macroworld. Quantum mechanics affords us a glimpse "behind" the manifested world, at stages in the process of manifestation, but it does not allow us to describe what lies "behind" the manifested world except in terms of the finished product—the manifested world, for without the manifested world there is nothing in whose terms we could describe its manifestation.
author: U. Mohrhoff
Sri Aurobindo International Centre of Education
Pondicherry 605002 India
firstname.lastname@example.com
title: Objective probability and quantum fuzziness

# Introduction

The present paper has two objectives. Fuchs  and Caves, Fuchs, and Schack  have presented a case for a Bayesian interpretation of quantum mechanics. My first objective is to offer a critique of the Bayesian interpretation with particular focus on the authors' recent paper  ("CFS" in what follows). CFS contrast their subjective stance on both quantum uncertainty *and* certainty with what they call the "objective-preparations view" (OPV). According to the latter, a system's quantum state is determined by a sufficiently detailed, agent-independent classical description of a preparation device, this being itself an agent-independent physical system. My second objective is to carry the discussion beyond these hardened opposing positions.

The first section states what is *not* at issue. Points of divergence are spelled out in Sec. , the key issue being the nature of the time dependence of quantum states. CFS share with the proponents of the OPV the cognate notions that quantum states *evolve* and that quantum-mechanical probabilities are *absolute*. Here the case is made that quantum-mechanical probabilities are conditional rather than absolute, and that the time dependence of a quantum state is the dependence of an algorithm, which serves to assign probabilities to the possible outcomes of a measurement, on the time of the measurement.

CFS go so far as to assert that for sufficiently divergent prior beliefs, "two agents might even legitimately assign different pure states." This brings up the question, considered in Sec. , of how much state assignments can legitimately differ. A further claim made by CFS is that proponents of the OPV ignore the essential quantum nature of preparation devices (and that it is wrong to do so). They illustrate this claim with the help of a quantum circuit containing qbits and c-not gates. The holes in their argument are pointed out in Sec. , and in Sec.  it is argued that a device is a legitimate preparation device if and only if its quantum-mechanical nature is irrelevant to its functioning as a preparation device.

The present paper's constructive part begins with Sec. . While quantum Bayesians keep the notorious measurement problem safely locked away in "a single black-boxed piece of hardware: the 1-Qbit measurement gate" , most others believe that "to solve this problem means to design an interpretation in which measurement processes are not different in principle from ordinary physical interactions," as an anonymous referee once put it to me. [^1] The way I see it, to solve this problem means to design an interpretation in which the central role played by measurements in standard axiomatization of quantum mechanics is recognized and explained. If this were easy, it would have been done long ago. As was pointed out by Dieks , the difficulty is that

> the outcome of foundational work in the last couple of decades has been that interpretations which try to accommodate classical intuitions are impossible, on the grounds that theories that incorporate such intuitions necessarily lead to empirical predictions which are at variance with the quantum mechanical predictions. However, this is a negative result that only provides us with a starting-point for what really has to be done: something conceptually new has to be found, different from what we are familiar with. It is clear that this constructive task is a particularly difficult one, in which huge barriers (partly of a psychological nature) have to be overcome.

I believe that in attempting this constructive task, the following points, which are outlined in Sec. , must be borne in mind :

-   The intrinsically and completely differentiated arena of physical events called "spacetime" is a figment of our mathematical imagination.

-   Spacetime coordinates only exist as features of the quantum-mechanical correlation laws. While not objective *per se*, they are (to a limited extent) capable of being objectified or becoming objective via outcome-indicating events or states of affairs—the only points of contact that exist between the correlation laws and the physical world. [^2]

-   The spatiotemporal aspects of the physical world consist of the more or less *fuzzy* relative positions and relative times that exist between physical objects, physical events, or physical states of affairs.

-   These relative positions and relative times exist to the extent that they are indicated by, or can be inferred from, physical objects, physical events, or physical states of affairs.

-   The spatiotemporal differentiation of the physical world is incomplete.

Outcome-indicating events and states of affairs occur or obtain in the macroworld. What counts as "macroscopic" is rigorously defined (without going beyond the conceptual tools of standard quantum mechanics) in Sec. . This definition, which strongly relies on the incompleteness of the world's spatiotemporal differentiation, makes it legitimate to attribute to the macroworld a reality independent of anything external to it.

How does one define and quantify a fuzzy observable? The answer, spelled out in Sec. , is: by assigning probabilities to the possible outcomes of a measurement of the same. This is also the reason why the probabilities of measurement outcomes play a central role in standard axiomatization of the theory. And why are observables such as relative positions and relative momenta fuzzy? The answer to this question is that the fuzziness of those observables is required for the stability of matter. This answer disposes of arguments by CFS to the effect that if quantum uncertainty is subjective, then so is certainty. On the contrary, quantum uncertainty—a bad translation of Heisenberg's term *Unschäre*, the literal meaning of which is "fuzziness"—can be as objective as certainty. After all, the stability of a material object hinges on the objective fuzziness of its internal relative positions and momenta, rather than on our subjective uncertainty about the values of these observables.

In Sec. several good reasons are given why quantum-mechanical probabilities may be considered objective. In addition, an argument by Fuchs and Schack , designed to illustrate "the folly of trying to have two kinds of probabilities in quantum mechanics" (one subjective and one objective), is refuted.

Perhaps the most disconcerting aspect of quantum mechanics—which only comes to light if, instead of trying to explain it away, one tries to make sense of the central role played by measurements—is the supervenience of the microscopic on the macroscopic. A set of properties A supervenes on a set of properties B if and only if any two objects which share all properties in B must also share all properties in A. The properties of the microworld depend in just this way on the properties of the macroworld. Hence we cannot think of particles, atoms, and such as constituents of the macroworld. Then what constitutes the macroworld? This question is addressed in Sec. . The conclusion reached is that the number of "ultimate constituents of matter" is exactly one. This conclusion is supported by a time-honored ontology: ultimately there exists a One Being, and the macroworld is its manifestation. The so-called "microworld" is therefore neither a world nor a part of any world. Rather, it is instrumental in the manifestation of the macroworld. Quantum mechanics affords us a glimpse "behind" the manifested world at formless particles, non-visualizable atoms, and partly visualizable molecules which, instead of being the world's constituent parts or structures, are instrumental in its manifestation. But—and this is *why* the microscopic supervenes on the macroscopic—we cannot describe what lies "behind" the manifested world except in terms of the finished product—the manifested world.

Section , finally, spells out the reason why the manifestation of the world cannot be described except in terms of the manifested world. It is simply that without the manifested world there is nothing in whose terms we could describe its manifestation. Section summarizes and concludes.

# Points of agreement

Before embarking on the critical part of this paper, I wish to point out what is *not* at issue. I agree with CFS that

-   there are no preassigned values to, and no instruction sets behind, quantum measurement outcomes; this includes outcomes that are certain;

-   differently put, there is nothing intrinsic to a quantum system—no element of reality, no independently possessed property—that guarantees a particular measurement outcome;

-   there is no local *and* realistic explanation for the correlations predicted by quantum mechanics;

-   state assignments, including pure-state ones, have no objective status;

-   the role of physical law in a quantum mechanical world is to relate probabilities, not to determine them;

-   the Bayesian interpretation of probability is superior to both the frequentist and the potentiality/propensity interpretations. [^3]

# Points of divergence

A quantum state $\rho(t)$ is an algorithm that serves to assign probabilities to the possible outcomes of any measurement that may be made at the time $t$. In keeping with this commonplace, I take the view that the time dependence of $\rho(t)$ is a dependence on the time of a measurement—the measurement to whose possible outcomes $\rho$ serves to assign probabilities. It is *not* the time dependence of an evolving state of any kind. This disposes of the question whether a quantum state has two modes of evolution or only one; it has *none*. [^4]

What is at issue here is *not* (at any rate, not primarily) the role of probability in quantum mechanics but *the principle of evolution*. By this I mean the notion that physics can be divided into a kinematical part, which concerns the description of a system at an instant of time, and a dynamical part, which concerns the evolution of a system from earlier to later times. While relativity made it seem as if the principle of local action was a consequence of the principle of evolution, quantum mechanics appears to rule out not only local realistic interpretations of the predicted and observed correlations between measurements performed in spacelike relation but also realistic interpretations of the principle of evolution.

In keeping with this principle, the wave function is widely regarded as the primary affair, which the propagator $\langle x_f,t_f\vert x_i,t_i\rangle$ serves to propagate through time *à la* [^5] $$\psi(x_f,t_f)=\int\!dx_i\,\langle x_f,t_f\vert x_i,t_i\rangle\,\psi(x_i,t_i).$$ This way of thinking makes it seem as if quantum-mechanical probabilities were determined by a wave function or a quantum state, rather than by measurement outcomes via computational devices called "wave functions" or "quantum states." If in addition one believes that quantum states exist and evolve in the absence of measurements, and that measurements merely *contribute* to determine them, then one treats quantum-mechanical probabilities as *absolute* probabilities.

Conversely, the assumption that quantum-mechanical probabilities are absolute, leads to (i) the spurious question of what it is that determines them (over and above measurement outcomes) and (ii) the facile answer that they are determined by a wave function or a quantum state that is *not* fully determined by measurement outcomes. This in turn leads to (iii) the spurious question about the nature of quantum states and (iv) the false dilemma that quantum states are *either* states of nature *or* states of knowledge or belief. One horn of this dilemma is the OPV, the other is the belief that quantum states exist and evolve in the absence of measurements *as prior probabilities in the beliefs of agents*. While this notion can make it easier to accept the idea that measurements contribute to determine the evolving states of quantum systems, it is nevertheless unwarranted, inasmuch as it involves the belief in the existence of evolving quantum states.

To my mind, the laws of quantum mechanics encapsulate correlations between measurement outcomes—diachronic correlations between outcomes of measurements performed on the same system at different times as well as synchronic correlations between outcomes of measurements performed on different systems in spacelike relation. We can use these correlations to assign probabilities to possible measurement outcomes *on the basis of actual measurement outcomes*, just as in a hypothetical classical world we can use the classical laws to predict later states on the basis of earlier ones. Our use of the correlations for the purpose of predicting probabilities may be considered subjective, but the correlations themselves are as objective as the classical laws are in a classical world. [^6]

If the laws of quantum mechanics are correlation laws—correlating measurement outcomes synchronically as well as diachronically—then quantum-mechanical probabilities are conditional rather than absolute. Conversely, if quantum-mechanical probabilities are conditional, as has been stressed by Primas , [^7] then we have no need for an evolving quantum state, be it a state of nature or a state of belief. In the following passage CFS appear to support this conclusion:

> Imagine a scientist who performs a sequence of $Z$ measurements on a qbit. Quantum mechanics, plus his experience and prior judgment and perhaps the outcomes of a long sequence of previous measurements, make him certain that the outcomes will all be "up." Now he performs the measurements, and he always gets the result "up." Shouldn't the agent be surprised that he keeps getting the outcome "up"? Doesn't this mean that it is a fact, rather than a mere belief, that the outcomes of his experiment will be "up"?… The answer to the first question is easy: Surprised? To the contrary, he would bet his life on it. Since the agent was certain that he would get the outcome "up" every time, he is not going to be surprised when that happens. Given his prior belief, only observing "down" would surprise him, since he was certain this would not happen, though nature might choose to surprise him anyway. The answer to the second question is similarly straightforward. According to our assumption, the agent has put together all his experience, prior beliefs, previous measurement outcomes, his knowledge of physics and in particular quantum theory, all to predict a run of "up" outcomes. Why would he want any further explanation? What could be added to his belief of certainty? He has consulted the world in every way he can to reach this belief; the world offers no further stamp of approval for his belief beyond all the factors that he has already considered.

In other words, we do not need a state of affairs that is external to the agent, that depends on the first measurement outcome, and that is responsible for the outcomes of the subsequent measurements. We do not need an evolving state of affairs that *mediates* an influence of the first measurement outcome on the subsequent measurement outcomes. In fact, if mediating states of affairs fail to account for the synchronic correlations, how can we expect them to account for the diachronic ones?

Instead of drawing the obvious conclusion that quantum states encapsulate *unmediated* correlations between measurement outcomes, CFS do not challenge the notions at the roots of the above dilemma. They accept the notion that quantum states evolve, for how else could they assert that a pre-measurement system-apparatus state arises from a "unitary interaction" between the system and the apparatus? They accept that quantum-mechanical probabilities are absolute rather than always conditional on measurement outcomes, for how else could they assert that "facts alone never determine a quantum state"? They merely deny the external, agent-independent existence of quantum states. In other words, they merely internalize the evolving quantum states of their opponents. Whereas for proponents of the OPV a quantum state exists in advance of measurements as a state of nature, for CFS it exists in advance of measurements in the form of a prior belief. "Any usage of probability theory starts from a *prior probability assignment*" (original emphasis). And whereas that state of nature is unknown, this prior belief can be neither verified nor falsified. "The question of whether a prior probability assignment is true or false cannot be answered." The Bayesian internalization of quantum states may relieve symptoms of the disease but it is far from being a cure.

# How much can state assignments differ?

CFS go so far as to assert that for sufficiently divergent prior beliefs, "two agents might even legitimately assign different pure states". [^8] This brings up the question of how much state assignments can legitimately differ. This question has been addressed by Brun *et al.* , who showed that several density matrices are mutually compatible if and only if the supports of all them have at least one state in common, or equivalently, if and only if all of them have expansions of the form $$\rho=\sum_i p_i\,\vert v_i\rangle\langle v_i\vert,\qquad p_i>0$$ with at least one state common to all expansions. It follows at once that two pure-state density matrices are compatible if and only if they are identical.

Brun *et al.* define a set of density matrices to be compatible when there could be circumstances under which they would represent the knowledge different people have of one and the same physical system. Caves, Fuchs, and Schack , who call the belief that an outcome is impossible a "firm belief," have shown that this definition of compatibility is equivalent to the existence of a density operator that does not contradict the firm beliefs of any party. Hence if two agents can "legitimately assign different pure states," as CFS claim, such a density operator does not exist. *Any* density operator then contradicts someone's belief that an outcome is impossible (i.e., it assigns a probability greater than 0 to at least one outcome to which at least one party assigns probability 0).

# An argument based on qbits

Carl Sagan, echoing Hume and Laplace, popularized the slogan that "extraordinary claims require extraordinary proof." The argument offered by CFS in support of their extraordinary claim that two agents can legitimately assign different pure states, however, is not likely to convince anyone who is not already convinced of the conclusion. The argument begins by considering a quantum circuit containing a system qbit initially in the state $\alpha\vert 0\rangle+\beta\vert 1\rangle$ and an "apparatus" qbit initially in the state $\vert 0\rangle$. A controlled NOT (c-not) gate produces the entangled state $\alpha\vert 00\rangle+\beta\vert 11\rangle$. A "measurement" of the apparatus qbit then flips the system qbit if the outcome is 1, otherwise it does nothing. The system qbit is thus prepared in the state $\vert 0\rangle$.

CFS next consider a modified circuit in which the "measurement device" is replaced by a second c-not gate. This likewise prepares the system qbit in the state $\vert 0\rangle$ but leaves the apparatus qbit in the state $\alpha\vert 0\rangle+\beta\vert 1\rangle$. If this modified circuit is used but the initial state of the apparatus qbit is $\vert 1\rangle$, the system qbit is prepared in the state $\vert 1\rangle$, and the apparatus qbit is left in the state $\alpha\vert 1\rangle+\beta\vert 0\rangle$. The system qbit is thus prepared in the initial apparatus state, whatever that state has been.

CFS insist on "the *essential* quantum nature of the preparation device" (original emphasis). In conjuction with their Bayesian view of quantum states, this "means that the prepared quantum state always depends on prior beliefs in the guise of a quantum operation that describes the preparation device." The operation of a preparation device "always depends on prior beliefs about the device, in particular, its initial quantum state."

In actual fact, even if the operation of a preparation device did sometimes depend on the initial state of the device, this won't always be the case. If, for instance, in the above example we simply exchange the roles of system and apparatus, then the final apparatus state is identical to the initial system state, whatever that state has been. Hence one might simply point out that the device chosen by CFS is not a legitimate preparation device. A preparation device is legitimate only if its initial state does *not* affect the state prepared and, as we have just seen, such devices exist.

One might also object against the use of a two-state system as a measurement device. How will one ever know whether a failure of the device to change its state indicates an actual outcome or simply a failure of the device? (There is no such thing as a 100% efficient device.) To get around this problem—and to forestall qbit-based shenanigans such as the above—a measurement device is usually expected to have a neutral state in addition to its outcome-indicating states.

# The preparation device: classical or quantum?

Another tenet of the view advocated by CFS is that the assumption that a preparation device can be given a complete classical description "neglects that any such device is quantum mechanical and thus cannot be specified completely in terms of classical facts." It may or may not be the case that a preparation device can be given a completely classical description, but it is besides the point, inasmuch as a device is a legitimate preparation device only if its quantum-mechanical nature is irrelevant to its functioning as a preparation device.

As an illustration, consider a particle of spin-$\frac12$ that has been prepared, with the help of a Stern-Gerlach filter, in the state $\vert+z\rangle$. For a zero Hamiltonian this means that a subsequent measurement of the particle's spin component with respect to the $z'$ axis, performed with a Stern-Gerlach splitter and two detectors, will yield "up" or "down" with probabilities [^9] $|\langle+z'\vert+z\rangle|^2$ and $|\langle-z'\vert+z\rangle|^2$, respectively. In order to say this, neither the preparation device nor the measurement device needs to be given any but a classical description. The essential piece of information is the angle between the respective directions of two magnetic field gradients. All we need to know about the detectors is that a response ("click") indicates the arrival of a particle in its sensitive region. The prior beliefs of agents nowhere enter the picture.

All that quantum mechanics allows us to calculate is correlations between measurement outcomes. To be able to make predictions (or retrodictions, for that matter), all we need and all we have is (i) these correlations and (ii) actual measurement outcomes. Think of density operators as machines with inputs and outputs: insert a measurement M by specifying its possible outcomes, insert the time of M, insert at least one relevant actual outcome of another measurement, press compute—and out pop the probabilities of the possible outcomes of M. The *only* possible description of a quantum system—at any rate, the only one that enters into the calculation of probabilities and can therefore be tested—consists of quantum-mechanical correlation laws and actual measurement outcomes, the former given in mathematical language, the latter given in *classical* terms. If there is anything more, it will involve hidden variables and/or a cryptodeterminism of some kind. Speculations involving such things invariably raise questions that they fail to answer—e.g., "Why are Bohmian trajectories unobservable?"—even though there are straightforward answers—e.g., "because they don't exist."

Come to think of it, why is it that, according to CFS, the question of whether a prior probability assignment is true or false, cannot be answered? The answer is that without factual input, the quantum-mechanical correlation laws are useless. If you press compute without inserting at least one measurement outcome, the probabilities that pop out will evince an equal lack of information, in keeping with the Principle of Indifference.

# The limited reality of spatiotemporal distinctions

The disease alluded to at the end of Sec.  is the notion that a time coordinate serves to label objectively distinct simultaneities—infinitely thin slices of spacetime, if you like. In other words, it is the belief that physical events and states of affairs happen or obtain in an intrinsically and completely differentiated arena called "spacetime." The principle of evolution mentioned in that section is a direct consequence of this belief, inasmuch as it underwrites the notion that quantum states are evolving instantaneous states.

What if that intrinsically and completely differentiated arena is a figment of our mathematical imagination? In this case spacetime coordinates only exist as features of the quantum-mechanical correlation laws. While not objective *per se*, they are (to a limited extent) capable of being objectified or becoming objective via outcome-indicating events or states of affairs—the only points of contact that exist between the correlation laws and the physical world. The spatiotemporal aspects of the physical world then consist of the more or less *fuzzy* relative positions and relative times that exist between physical objects, physical events, and physical states of affairs. And these relative positions and relative times exist to the extent that they are indicated by, or can be inferred from physical objects, physical events, or physical states of affairs. [^10]

If I am on the right track, then the spatiotemporal differentiation of reality doesn't go "all the way down." To see this, remember that the exact localization of a particle implies an infinite momentum dispersion and thus an infinite mean energy. (In a relativistic world, the attempt to produce a strictly localized particle instead results in the creation of particle-antiparticle pairs.) Therefore no material object ever has a sharp position (relative to any other object). Now let $I\hspace{-0.25em}R^3$($O$) be the set of exact positions relative to some object $O$. We can conceive of a partition of $I\hspace{-0.25em}R^3$($O$) into finite regions that are so small that none of them is the sensitive region of an actually existing detector. [^11] Hence we can conceive of a partition of $I\hspace{-0.25em}R^3$($O$) into sufficiently small but finite regions $R_k$ of which the following is true: there is no object $Q$ and no region $R_k$ such that the proposition "$Q$ is inside $R_k$" has a truth value. In other words, there is no object $Q$ and no region $R_k$ such that $R_k$ exists for $Q$. But a region of space that does not exist for any material object, does not exist at all. The regions $R_k$—or, what comes to the same, the distinctions we make between them—correspond to nothing in the physical world. They exist solely in our heads. It follows that the spatial differentiation of the physical world is incomplete—it doesn't go all the way down.

What holds for the world's spatial differentiation also holds for its temporal differentiation. The times at which observables possess values, like the values themselves, must be indicated in order to exist or be possessed. Clocks are needed not only to indicate time but also, and in the first place, to make times available for attribution to indicated values. Since clocks indicate times by the positions of their hands, [^12] the world's incomplete spatial differentiation implies its incomplete temporal differentiation.

# The macroworld

The possibility of obtaining evidence of the departure of an object $O$ from its classically predictable position [^13] calls for detectors whose position probability distributions are narrower than $O$'s—detectors that can probe the region over which $O$'s fuzzy position extends. For most objects with sufficiently sharp positions, such detectors do not exist. For the objects I have in mind, the probability of obtaining evidence of departures from the classically predictable motion is very low. Hence *among* these objects there will be many of which the following is true: every one of their indicated positions is consistent with every prediction that can be made on the basis of previously indicated properties and a classical law of motion. These are the objects that deserve to be labeled *macroscopic*. To permit a macroscopic object—e.g., the proverbial pointer needle—to indicate the value of an observable, one exception has to be made: its position may change unpredictably if and when it serves to indicate a property or a value.

We are now in position to define the *macroworld* unambiguously as the totality of relative positions obtaining between macroscopic objects. Let's shorten this to "macroscopic positions." By definition, macroscopic positions never evince their fuzziness (in the only way they could, through departures from classically predicted values). This makes it legitimate to attribute to the macroworld—not individually to each macroscopic position but to the macroworld in its entirety, and not merely "for all practical purposes" but strictly—a reality independent of anything external to it (such as the consciousness of an observer ). But it also entails the supervenience of the microscopic on the macroscopic.

In philosophy, supervenience is a relationship of dependence typically obtaining between sets of properties. According to one standard definition, a set of properties A supervenes on a set of properties B if and only if any two objects which share all properties in B must also share all properties in A. Beyond that, the nature of this dependence (logical, nomological, or other) remains unspecified. A typical example is the notion that psychological properties supervene on physical properties (usually goings-on in a brain), which does not imply that the former depend on the latter logically or nomologically. What I mean by the supervenience of the microscopic on the macroscopic is that the properties of the microworld depend in just this way on the properties of the macroworld—rather than the other way round, as we are wont to think. [^14]

# Probability, objective fuzziness, and the stability of matter

The Bayesian approach to quantum semantics is to take the Bayesian interpretation of probability for granted and to see what light it throws on quantum mechanics. To my mind, this puts the cart before the horse. As Appleby  remarked, "\[w\]hereas the interpretation of quantum mechanics has only been puzzling us for about 75 years, the interpretation of probability has been doing so for more than 300 years." Quantum mechanics, properly understood, may be better equipped to throw light on the meaning of probability than probability theory is equipped to throw light on the interpretation of quantum mechanics. In any event, the Bayesian interpretation of probability appears to me to be singularly ill-equipped for this task.

One only has to ask: what accounts for the stability of matter? Specifically, what accounts for the existence of spatially extended objects

-   that are composed of a large but finite number of unextended objects (particles that do not "occupy" space),

-   that "occupy" finite volumes of space, and

-   that neither explode nor collapse the moment they are formed?

The existence of such objects hinges on the *objective fuzziness* of the relative positions and relative momenta of their constituents. [^15] This, rather than our *subjective uncertainty* about the values of these observables, is what "fluffs out" matter.

The mathematical theory of probability relies on a primitive notion of probability to make contact with reality . Instead of treating probability as a primitive notion, we may regard possibility as primitive and look upon probability as a quantification of possibility. Any further interpretation ought to take account of the context in which probabilities are assigned. The one-size-fits-all approach fails to do justice to the disparate roles that probability plays in the classical and quantum contexts.

One could take a further step, citing a close connection between possibility and ignorance: as long as there is ignorance, there are alternative possibilities, and as long as there are alternative possibilities, there is ignorance. But a purely subjective interpretation of probability masks a fundamental difference between the reasons for our ignorance. In a classical world, ignorance results from a practical lack of access to existing facts or data (which could in principle be used to retrodict the past as well as predict the future). [^16] In our quantum world, ignorance results from the non-existence of facts or data sufficient to predict the future or retrodict the past.

Why does quantum mechanics reduce us to predicting probabilities? I do not believe that this question can be answered without reference to the constituents of this world—which is what quantum Bayesians refuse to do. (Whatever the constituents may turn out to be, they are not quantum-computational gates.)

The answer I suggest is that the fuzziness (or indefiniteness, or indeterminacy) of all relative position and momenta is an objective feature of the physical world. [^17] The question then is: how does one define and quantify a fuzzy observable? And the answer is: by assigning probabilities to the possible outcomes of a measurement. To be precise, the proper way to define and quantify a fuzzy observable is to assign probabilities to the possible outcomes of *unperformed* measurements.

Suppose we have measured an observable $Q$ and obtained the outcome $q$. At the time of the measurement, the value of $Q$ is $q$. What can we say about the value, at this time, of an observable $Q'$ that is incompatible with $Q$? We can say that it is fuzzy. And how do we describe its fuzziness? We describe it by assigning probabilities to the possible outcomes of an unperformed measurement of $Q'$ on the basis of the actual outcome of the measurement of $Q$.

# Objective probability

Contrary to the caveats of CFS, there are several good reasons for thinking of quantum-mechanical probabilities as objective: [^18]

-   They are assigned on the basis of (i) objective, value-indicating events or states of affairs and (ii) objective physical laws.

-   They do not reduce to Bayesian degrees of belief. (As said, the stability of atomic hydrogen rests on the objective fuzziness of its internal relative position and momentum, not on anyone's belief about the values of these observables.)

-   They play an essential role in the description of physics reality.

-   In particular, they are needed to define and quantify the objective fuzziness of observables.

I take this opportunity to address an argument against what Fuchs and Schack  have called "the folly of trying to have two kinds of probabilities in quantum mechanics"— one subjective and one objective.

The general state of a spin-$\frac12$ particle can be written in terms of the Pauli matrices and the unit operator $I$ as $$\rho=\frac12(I+{\mathbf S}\cdot\sigma),$$ where $|{\mathbf S}|\leq1$. If $|{\mathbf S}|<1$, there is an infinite number of decompositions of $\rho$ having the form $$\rho=\sum_j p_j\,\vert{\mathbf n}_j\rangle\langle{\mathbf n}_j\vert\,,$$ where $|{\mathbf n}_j|=1$ and $\{p_j\}$ is a probability distribution. Fuchs and Schack appear to think that we are to regard the probabilities $p_j$—which in one decomposition may equal $\frac34$ and $\frac14$, respectively, and in another may both equal $\frac12$—"as subjective expressions of ignorance about which eigenstate is the 'true' state of the particle." This is not correct. The probabilities $p_j$ are ignorance probabilities if and only if a measurement with the possible outcomes $\vert{\mathbf n}_j\rangle\langle{\mathbf n}_j\vert$ has been made but its outcome is not taken into account. In this case the decomposition of $\rho$ is uniquely determined by this measurement, and there is no ambiguity whatever about which probabilities are grounded in mere ignorance and which are grounded in objective fuzziness.

On the other hand, if no measurement is made, then $\rho$ itself is the "true" state of the particle (that is, the most informative probability algorithm), so the probabilities $p_j$ cannot be "subjective expressions of ignorance about which eigenstate is the 'true' state of the particle." The conclusion of Fuchs and Schack—that "if a density operator is even partially a reflection of one's state of belief, the multiplicity of ensemble decompositions means that a pure state must also be a state of belief"—thus falls apart.

# In search of the ultimate constituent(s)

The conclusion of Sec.  was that the properties of the microworld supervene on the properties of the macroworld. This means that we cannot think of particles, atoms, and such as constituents of the macroworld. Then what is it that constitutes the macroworld? And first of all, what does it *mean* to constitute something?

We are in the habit of thinking that small things constitutes large things, and that wholes are made up of distinct individual parts. Quantum mechanics teaches us—at any rate, it ought to have taught us by now—that this kind of bottom-up modeling has passed its expiry date. The conclusion of Sec.  was that if we imagine the world partitioned into smaller and smaller spatial regions, there comes a point beyond which these regions, or the corresponding distinctions, no longer correspond to anything in the actual world. By the same token, if we keep dividing a material object, its so-called "constituents" lose their individuality, as the following will show.

To begin with, if the properties of the microworld supervene on the properties of the macroworld, then what are the things we call "particles"? In the first place, they are correlations between "detector clicks". [^19] Suppose that we perform a series of position measurements, and that every position measurement yields exactly one outcome (i.e., each time exactly one detector clicks). Then we have a conservation law, and we are entitled to infer the existence of an entity $O$ which persists through time, to think of the clicks given off by the detectors as indicating the successive positions of this entity, to think of the behavior of the detectors as position measurements, and to think of the detectors as detectors.

If each time exactly two detectors click, we are entitled to infer the existence of two persistent entities. Or are we? If there isn't another conservation law providing the entities with persistent identity tags, then the question of which is which—Which of the particles detected at $t_1$ is identical with which of the particles detected at $t_2$?—has no answer. In other words, the question is meaningless.

Here as elsewhere, the challenge is to learn to think in ways that do not lead to meaningless questions. What we have in this case is a *single* entity with the property of being in *two* places whenever we check—not a system "made up" of two things but one thing with the property of being in *two* places every time a position measurement is performed. If we choose this way of thinking, then the meaningless question "which is which?" can no longer be asked.

This conclusion can also be reached by other routes. Consider, for instance, a particle that lacks internal relations. What is it "in itself," out of relation to its external relations? The answer is: nothing, except possibly a substance without properties. [^20] The reason this is so is that the properties of particles are either kinematical relations, such as positions or momenta, or parameters characterizing dynamical relations, such as the various kinds of charge, or they have objective significance independent of conventions only as dimensionless ratios (e.g., mass ratios). [^21]

But according to the philosophical principle known as "the identity of indiscernibles," $A$ and $B$ are one and the same thing just in case there is no difference between $A$ and $B$. Not only is there no difference between two particles lacking internal relations considered "in themselves," but nothing corresponds to the distinction we make between *this* particle and *that* particle *over and above* the distinction between *this* property and *that* property. What follows from this is the *numerical identity* of all particles lacking internal relations, when each is considered by itself. If we think of particles lacking internal relations as the "ultimate constituents of matter," then there is a clear sense in which the actual number of "ultimate constituents of matter" equals 1.

The bottom line: a quantum system is always *one*. The number of its so-called "constituent" particles is just one of its measurable properties (which, in a relativistic setting, can come out different every time we check). Hence if I permit myself to think of the entire physical world as a quantum system and to ask about its constituents, I find that it has just one—a single intrinsically propertyless substance. In addition I am now in a position to account for the origin of both matter and space: by entering into spatial relations with itself, this single intrinsically propertyless substance gives rise to space, conceived of as the totality of spatial relations, and it gives rise to matter, conceived of as the corresponding apparent multitude of relata—"apparent" because the relations are *self*-relations.

# Understanding the supervenience of "micro" on "macro"

The conclusions we have reached are supported by a time-honored ontology: ultimately there exists a One Being, and the world is its manifestation. The question, then, is: how does this One Being manifest the world, or manifest itself as the world? And quantum mechanics suggests an answer. The "microworld" is neither a world nor a part of any world but instead is *instrumental* in the manifestation of the world (the macroworld, to be precise). Quantum mechanics affords us a glimpse "behind" the manifested world at formless particles, non-visualizable atoms, and partly visualizable molecules , which, instead of being the world's constituent parts or structures, are instrumental in its manifestation. But—and this is the punch line—we cannot describe what lies "behind" the manifested world except in terms of the finished product—the manifested world. [^22]

But why cannot we describe the manifestation of the world except in terms of the manifested world? The reason is that the spatial distinctions we make—such as the distinction between inside $R$ and outside $R$—are warranted only if, and only to the extent that, they are physically realized (made real), for instance by $R$'s being the sensitive region of a detector. To be able to say truthfully that a particle is inside a region $R$, we need a detector, and this not merely in order that the particle's presence in $R$ can be indicated but, in the first place, in order that $R$ be real and the property of being in $R$ be available for attribution to the particle.

Furthermore, in order that $R$ be real, it must be definable in terms of macroscopic positions. By the same token, in order that the property of *being in* $R$ be attributable to the particle, it must be indicated by an unpredictable transition in the value of an otherwise deterministically evolving macroscopic position—that of the proverbial pointer needle. And in order that the property of being in $R$ *at the time* $t$ be attributable to the particle, $t$ too must be indicated by a macroscopic position. All of this is entailed by the identification of the macroworld as the direct referent of "reality" or of "the (physical) world."

# Conclusion

The purpose of this paper was twofold: (i) to offer a critique of the Bayesian interpretation of quantum mechanics with particular focus on a paper by CFS (which contains a critique of the "objective preparations view" or OPV), and to carry the discussion beyond the hardened positions of the Bayesians and the proponents of the OPV. Several claims made by CFS were rebutted, including the claim that different pure states may legitimately be assigned to the same system at the same time, and the claim that the quantum nature of a preparation device cannot legitimately be ignored.

If a quantum state is more than a probability algorithm, it is not testably so. It is therefore only prudent to view the time dependence of a quantum state as a dependence on the time of the measurement to whose possible outcomes the quantum state serves to assign probabilities. For both Bayesians and proponents of the OPV, on the other hand, the time dependence of a quantum state is the continuous dependence on time of an evolving state of some kind. This makes it impossible to recognize the full implications of the one and only testable feature of quantum mechanics—the probability distributions it yields. For, as shown, continuous state evolution entails a completely differentiated time and therefore a completely different spacetime, whereas an analysis of quantum-mechanical probability distributions yields the opposite: the spatiotemporal differentiation of the physical world does not go "all the way down."

Why are measurements essential ingredients of all standard formulations of quantum mechanics? While Bayesians keep the measurement problem safely locked away in "a single black-boxed piece of hardware" , most others try to gloss it over or explain it away. To be able to understand the special status of measurements, one needs a clear distinction between what counts as macroscopic and what does not. As shown, it is the incomplete spatiotemporal differentiation of the physical world that makes this distinction possible. This incomplete differentiation is entailed by an objective fuzziness, which is required for the stability of matter, and which explains why the fundamental theoretical framework of physics is a probability calculus.

Bohr was right: out of relation to measurements, the values of physical observables are undefined. They are undefined because they supervene on value-indicating events or states of affairs. What constitutes the macroworld is not particles, atoms, or such but a Single Being, an Ultimate Reality, about which we know nothing, except that it manifests the macroworld or manifests itself as the macroworld. The microworld is neither a world nor a part of any world but instead is instrumental in the manifestation of the macroworld. Quantum mechanics affords us a glimpse "behind" the manifested world, at stages in the process of manifestation, but it does not allow us to describe what lies "behind" the manifested world except in terms of the finished product—the manifested world. This is the reason why the properties of the microworld supervene on those of the macroworld. The reason why the manifestation of the world cannot be described except in terms of the manifested world, is simply that without the manifested world there is nothing in whose terms we could describe its manifestation.

[^1]: Since quantum mechanics describes interactions in terms of correlations between the probabilities of the possible outcomes of *measurements* performed on the interacting systems, I wonder what the referee could have meant by an "ordinary physical interaction."

[^2]: A measurement is an event or state of affairs from which something can be inferred about something else. (Attempted but unsuccessful measurements are not included in this definition.) Do we also need to define events and states of affairs? Obviously not. For one thing, we know them when we see them. For another, because quantum mechanics presupposes their existence or occurrence, it cannot account for it, anymore than we can explain why there is anything, rather than nothing at all.

[^3]: Relative frequencies, belonging as they do to the domain of facts or events, are different from probabilities, and the interpretation of probabilities as potentialities  or propensities  contravenes the conditional (as against absolute) nature of quantum-mechanical probabilities pointed out in the next section.

[^4]: To forestall a possible objection: if the Hamiltonian depends on time, it includes effects of *classical* boundary conditions. By no means does its possible time dependence support the view that quantum states evolve.

[^5]: In the case of a nonrelativistic system consisting of a fixed number $n$ of particles, $x$ stands for a set of $3n$ coordinates.

[^6]: I am not saying that a mathematical construct like the Faraday tensor $\mathbf{F}$ is anything more than a computational tool. I am only saying that $\mathbf{F}$ is a useful mathematical tool for formulating an objective correlation law. The essential difference I see between the laws of classical physics and those of quantum physics is the absence of principles limiting the accuracy of predictions in classical physics as against the presence of such principles in quantum physics.

[^7]: Primas  has also drawn attention to an axiomatic alternative to Kolmogorov's  formulation of probability theory, due to Rényi . Whereas in Kolmogorov's theory absolute probabilities have primacy over conditional ones, Rényi's theory is based entirely on conditional probabilities. Primas states that every result of Kolmogorov's theory has a translation into Rényi's.

[^8]: Fuchs and Shack write that "there is no fact of nature to prohibit two different agents from using distinct pure states $\vert\psi\rangle$ and $\vert\phi\rangle$ for a single quantum system." Of course, if quantum states are only beliefs, they can be mutually inconsistent, as beliefs often are—even if they are held by the same person.

[^9]: If we take into account that no detector is 100% efficient, these probabilities are the respective limits of $n_u/n$ and $n_d/n$ as $n\rightarrow\infty$, where $n$ is the total number of detector "clicks" and $n_u$ and $n_d$ are the respective numbers of "up" and "down" clicks.

[^10]: This means, *inter alia*, that if physical objects, events, and states of affairs did not exist, neither would the spacetime "arena." The existence of the world's spatiotemporal properties stands or falls with the existence of objects, events, or states of affairs to which they can be attributed.

[^11]: By a *detector* I mean an object with a sensitive region $R$, capable of indicating the presence in $R$ of another object of a more or less specific type. An object $A$ is capable of *indicating* the possession of a property (by another object) or a value (by an observable) if this can be inferred from a property—or a change in the properties—of $A$.

[^12]: Digital clocks indicate times by transitions from one reading to another, without hands. The uncertainty principle for energy and time, however, implies that such a transition cannot occur at an exact time, except in the unphysical limit of infinite mean energy .

[^13]: A "classically predictable position" is a position that can be predicted on the basis of (i) a classical law of motion and (ii) all relevant value-indicating events.

[^14]: We may now think of measurements as departures of macroscopic positions from their classical law of motion, indicating goings-on in the microworld rather than being caused by them.

[^15]: By itself, the fuzziness of a relative momentum causes the corresponding relative position to become more fuzzy. In a stable material object, this inherent tendency of relative positions to become more fuzzy is counterbalanced by the electrostatic attraction between oppositely charged particles, which causes relative positions to become less fuzzy.

[^16]: In a cryptodeterministic quantum world, ignorance would result from a theoretical or in-principle lack of access to existing facts or data.

[^17]: I used to consider it superfluous to stress that I am talking about a fuzziness in the world rather than a fuzziness in the mind, but referees's objections against my purported use of "fuzzy logic" have taught me otherwise. As I have argued elsewhere , most interpretations of quantum mechanics suffer from a conceptual fuzziness that can be avoided by taking the fuzziness out of the mind and placing it into the world.

[^18]: I agree, though, that there are also *bad* reasons, e.g., the statistical interpretation of quantum mechanics, the frequentist interpretation of probability, or the propensity interpretation of (quantum-mechanical) probability assignments.

[^19]: This means, among many other things, that particles do not *cause* detector clicks. An $\alpha$-detector, for example, does not click because an $\alpha$ particle has entered its sensitive region. Rather, the click of an $\alpha$ detector is the reason why an $\alpha$ particle is in the detector's sensitive region.

[^20]: Whereas a property is that in the world to which a logical or grammatical predicate can refer, a substance is that in the world to which only a logical or grammatical subject can refer.

[^21]: Spin is both relational (inasmuch as its components are defined relative to a reference frame) and characteristic of dynamical relations (inasmuch as it affects a particle's momentum probability distribution in the presence of an electric current). The use of "dynamical" is not an endorsement of the notion of quantum state evolution but a reference to the dependence of probabilities on the times of measurements.

[^22]: Perhaps Bohr himself didn't realize how right he was when he insisted that, out of relation to experimental arrangements, the properties of quantum systems are undefined.


abstract: At HEP experiments, processing terabytes of structured numerical event data to a few statistical summaries is a common task. This step involves selecting events and objects within the event, reconstructing high-level variables, evaluating multivariate classifiers with up to hundreds of variations and creating thousands of low-dimensional histograms. Currently, this is done using multi-step workflows and batch jobs. Based on the CMS search for $H(\mu \mu)$, we demonstrate that it is possible to carry out significant parts of a real collider analysis at a rate of up to a million events per second on a single multicore server with optional GPU acceleration. This is achieved by representing HEP event data as memory-mappable sparse arrays, and by expressing common analysis operations as kernels that can be parallelized across the data using multithreading. We find that only a small number of relatively simple kernels are needed to implement significant parts of this Higgs analysis. Therefore, analysis of real collider datasets of billions events could be done within minutes to a few hours using simple multithreaded codes, reducing the need for managing distributed workflows in the exploratory phase. This approach could speed up the cycle for delivering physics results at HEP experiments. We release the `hepaccelerate` prototype library as a demonstrator of such accelerated computational kernels. We look forward to discussion, further development and use of efficient and easy-to-use software for terabyte-scale high-level data analysis in the physical sciences.
author: J. Pata, M. Spiropulu
California Institute of Technology
title: hepaccelerate: Fast Analysis of Columnar Collider Data

# Introduction

At the CMS experiment of the Large Hadron Collider, the final stage of data processing is typically carried out over several terabytes of numerical data residing on a shared cluster of servers used for batch processing. The data consist of columns of features for the recorded particles such as electrons, muons, jets and photons for each event in billions of rows, one row per event. In addition to the columns of purely kinematic information of particle momentum, each particle carries a number of features that describe the reconstruction details and other high-level properties of the reconstructed particles. For example, for muons, we might record the number of tracker layers where an associated hit was found, whether or not it reached the outer muon chambers and in MC simulation the index of the associated generator-level particle, thereby cross-linking two collections. Typical compressed event sizes for such reduced data formats at CMS are a few kilobytes per event. In practice, mixed precision floating points are used to store data only up to experimental precision, such that the number of features per event is on the order of a few hundred.

A typical physics analysis at the LHC such as the precision measurement of a particle property or the search for a new phenomenon involves billions of real and simulated events. The final result typically requires tens to hundreds of iterations over this dataset while the analysis is ongoing over the period of months to a year. For each iteration of the analysis, hundreds of batch jobs of custom reduction software is run over these data. The maintenance and operation of this data pipeline takes up valuable researcher time and can result in delays between iterations of the analysis, slowing down innovation in experimental methods.

We demonstrate that by efficiently accessing and caching only the needed columns from the data, sizeable batches of data can be loaded from disk to memory at the speed of several MHz (millions of events per second). By processing the data using efficient vectorizable kernels on the arrays, applying multithreading and GPU acceleration to the final stage of data analysis becomes natural, such that data can also be processed at MHz-level speeds already on a single server. This means that a complete physics analysis of billions of events can be run as a single integrated Python code on a single machine with a rate of about a billion events per hour. Although we use a specific and relatively simple CMS analysis as an example, the method based on array computing with accelerated kernels is generic and can be used for other collider analyses. The purpose of this paper is to detail the issue of processing terabyte-scale data in HEP efficiently. We release the `hepaccelerate` library for further discussion .

In the following parts, we explore this approach based on columnar data analysis in more detail. In section , we describe the structure of the data and discuss how data sparsity is handled efficiently. We introduce physics-specific computational kernels in section  and describe the measured performance under a variety of conditions in section . Finally, we conclude with a summary and outlook in section .

# Data structure

We can represent HEP collider data in the form of two-dimensional matrices, where $N$ rows correspond to events and columns correspond to features in the event such as the momentum components of all measured particles. However, due to the nature of the underlying physics processes that produce a varying number of particles per event, the number of features varies from one collider event to another, such that a fixed-size two dimensional representation is not memory-efficient. Therefore, the standard HEP software framework based on ROOT includes mechanisms for representing dynamically-sized arrays as well as complete C++ classes with arbitrary structure as the feature columns and a mechanism for serializing and deserializing these dynamic arrays .

Based on the approach first introduced in the `uproot`  and `awkward-array`  python libraries, many existing HEP data files with a varying number of particles per event can be represented and efficiently loaded as sparse arrays with an underlying one-dimensional array for a single feature. Event boundaries are encoded in an offset array that records the particle count per event. Therefore, the full particle structure of events can be represented by a contiguous offset array of size $N$ and a contiguous offset array of size $M$ for each particle feature. This can easily be extended to event formats where the event contains particle collections of different types, e.g. jets, electrons and muons. By using the data and offset arrays as the basis for computation, efficient computational kernels can be implemented and evaluated on the data. We illustrate the jagged data structure on figure .

In practice, analysis-level HEP event data are stored in compressed ROOT files storing raw features in a so-called "flat analysis ntuple" form. Out of hundreds of stored features, a typical analysis might use approximately 50 to 100, discarding the rest and only using them rarely for certain auxiliary calibration purposes. When the same features are accessed multiple times, the cost of decompressing the data can be significant. Therefore, in order to maximize the computational efficiency of the analysis, we have implemented a simple cache based on memory mapped files for the feature and offset data. The efficiency of the uncompressed cache depends on the ratio of features used for analysis. We find that for the representative CMS analysis, the average uncompressed cache size is approximately 0.5 kB/event after choosing only the necessary columns, down from approximately 2 kB/event in compressed form. The choice of compression algorithms can and should be addressed further in optimizing the file formats for cold storage and analysis .

# Computational kernels

In the context of this report, a kernel is a function that is mapped across an array to transform the underlying data. A simple kernel could compute the square root of all the values in the array. More complicated kernels such as convolutions might involve relations between neighbouring array elements or involve multiple inputs and outputs of varying sizes. When the individual kernel calls across the data are independent of each other, these kernels can be evaluated in parallel over the data using SIMD processors. The use of efficient computational kernels with an easy-to-use API has proven to be successful for machine learning frameworks such as `tensorflow`. Therefore, we propose to implement HEP analyses similarly using common kernels that are steered from a single high-level code.

In the following, we will demonstrate that by using the data and offset arrays as described in section , common HEP operations such as computing the total scalar sum of momenta of all selected particles in an event can be formulated as kernels and dispatched to vector processing units (CPUs, GPUs), efficiently processing the event data in parallel. The central result of this paper is that only a small number of simple kernels, easily implemented in e.g. Python or C, are needed to implement a realistic HEP analysis.

As mentioned above, a prototypical HEP-specific kernel would be to find the scalar sum $H_T$ of all particles passing some quality criteria in an event. We show the Python implementation for this on listing . This kernel takes as input the $M$-element data array of all particle transverse momenta `pt_data` and an $N+1$-element array of the event offsets. In addition, as we wish to include only selected particles in selected events in this analysis, we use an $N$-element boolean mask for events and $M$-element boolean mask for the particles that have passed selection. These masks can be propagated to further functions, making it possible to efficiently chain computations without resorting to expensive data copying. Finally, the output is stored in a preallocated array of size $N$ that is initialized to zeroes.

``` python
def sum_ht(
  pt_data, offsets,
  mask_rows, mask_content,
  out):

  N = len(offsets) - 1
  M = len(data)

  #loop over events in parallel
  for iev in prange(N):
    if not mask_rows[iev]:
      continue

    #indices of the particles in this event
    i0 = offsets[iev]
    i1 = offsets[iev + 1]

    #loop over particles in this event
    for ielem in range(i0, i1):
      if mask_content[ielem]:
        out[iev] += pt_data[ielem]
```

This kernel generically reduces a data array using a pairwise operator $(+)$ within the offsets and can be reused in other cases, such as counting the number of particles per event passing a certain selection. Other kernels that turn out to be useful are related to finding the minimum or maximum value within the offsets or retrieving or setting the $m$-th value of an array within the event offsets.

The generic kernels we have implemented for the $H(\mu\mu)$ analysis are the following:

-   `get_in_offsets`: given jagged data with offsets, retrieves the $n$-th elements for all rows. This can be used to create a contiguous array of e.g. the leading jet $p_T$ for further numerical analysis.

-   `set_in_offsets`: as above, but sets the $n$-th element to a value. This can be used to selectively mask objects in a collection, e.g. to select the first two jets ordered by $p_T$.

-   `sum_in_offsets`: given jagged data with offsets, calculates the sum of the values within the rows. As we have described above, this can be used to compute a total within events, either to count objects passing selection criteria by summing masks or to compute observables such as $H_T$.

-   `max_in_offsets`: as above, but calculates the maximum of the values within rows.

-   `min_in_offsets`: as above, but calculates the minimum.

-   `fill_histogram`: given a data array, a weight array, histogram bin edges and contents, fills the weighted data to the histogram. This is used to create 1-dimensional histograms that are common in HEP. Extension to multidimensional histograms is straightforward.

-   `get_bin_contents`: given a data array and a lookup histogram, retrieves the bin contents corresponding to each data array element. This is used for implementing histogram-based reweighting.

This demonstrates that a small number of dedicated kernels can be used to offload a significant part of the analysis. There are also a number of less generic analysis operations which do not easily decompose into other fundamental array operations, but are still useful for HEP analysis. A particular example would be to find the first two muons of opposite charge in the event, or to perform $\Delta R(\eta, \phi)$-matching between two collections of particles. In the standard approach, the physicist might simply write down procedural code in the form of a nested loop over the particles in the event which is terminated when the matching criterion is satisfied. These functions can similarly be expressed in the form of a dedicated kernels that do a single pass over the data and that are easy to write down and debug procedurally for the physicist before being dispatched to SIMD processors. We choose to implement these specifically rather than rely on more general array operations that perform cross-collection joins for speed and simplicity of implementation. These specialized kernels are as follows:

-   `mask_deltar_first`: given two collections of objects, masks all objects in the first collections that are closer than a predefined value in $\Delta R^2 = \Delta \eta^2 + \Delta \phi^2$ to an object in the second collection

-   `select_opposite_sign_muons`: given a collection of objects with charge (i.e. muons), masks all but the first two objects ordered by $p_T$ which are of opposite charge

With a complete set of about 10 kernels, implemented in Python and just-in-time compiled to either multithreaded CPU or GPU/CUDA code using the Numba package , a standard HEP analysis such as the search for $H(\mu \mu)$ can be carried out from a simple controller script. We have chosen Python and Numba to implement the kernels in the spirit of quickly prototyping this idea, but this approach is not restricted to a particular programming language. The total number of lines of code for both the CPU and GPU implementations of all kernels is approximately 250, reflecting the simplicity of the implementations. Using a 24 core Intel Xeon E5-2687W v4 @ 3.00GHz with Intel Optane SSDs, networked storage using CephFS and an nVidia Geforce Titan X, we have benchmarked the performance of the kernels on preloaded data. The results are shown on figure . For complex kernels such as $\Delta R$ masking, with 24 cores we observe a speedup of approximately 5 times over single-core performance. On the GPU, we find a total speedup of approximately 15x over single-core performance.

# Analysis benchmark

Besides testing the individual kernels in a controlled setting, we benchmark a complete physics analysis based on multithreaded kernels using a CMS analysis for $H(\mu\mu)$. This analysis requires trigger selection, jet and lepton selection, matching of leptons to trigger objects as well as generator-level objects and of jets to leptons. The dimuon candidate is constructed from muons that pass quality criteria. Events are categorized based on the number of additional leptons and jets and the distribution of several control variables as well as the variable of interest, the dimuon invariant mass, is saved to histograms for further analysis. We also evaluate a high-level pretrained DNN discriminator using `tensorflow` based on about 25 input variables. Pileup reweighting is applied on the fly to correct the MC distribution of the number of reconstructed vertices to the one observed in data, with variated histograms being used to account for systematic uncertainties. We use existing CMS code for lepton scale factor and Rochester corrections, which is integrated using Python CFFI and multi-threaded using OpenMP. In general, the most computationally complex steps in high-level analysis are the evaluation of machine learning models and filling of hundreds to thousands of histograms with systematic variations. Both of these are highly efficient on SIMD processors such as GPUs. In order to emulate a more complex analysis, once the data are loaded to memory in batches, we study the scaling behaviour with respect to analysis complexity by rerunning the analysis multiple times on the same data, which reflects the possibility of easily doing parameter scans in this approach.

We use approximately 90 million MC events in 80 GB of compressed CMS NanoAOD files to benchmark the analysis, reflecting about 10% of the total needed for a single year of data taking. First, we decompress and cache data at a speed of approximately 100-1000 kHz on the benchmark machine, creating an uncompressed cache of approximately 40 GB which is stored on a local SSD. This step is crucial for increased physics throughput, but needs to be repeated only when additional branches need to be included in the analysis. We find that it is advantageous to store the input data on a fast parallel storage with a capacity of several terabytes that supports random reads at speeds exceeding 100 MB/s, but caching speeds from networked bulk storage such as HDFS-FUSE are still acceptable at around 50-100 kHz. The cache creation speed is largely IO limited and can vary significantly depending on the hardware. Further optimization of the decompression and loading step of columnar ROOT data would be beneficial.

After the branch cache is created, the physics analysis code can be run efficiently. We observe baseline event processing rates of approximately 0.8 MHz on the multicore CPU backend and approximately 0.9 MHz using the GPU backend for a single analysis iteration. It is important to note that the absolute event processing speed depends on the analysis complexity. In HEP data analysis it is common to apply some preselection or "skimming" such as trigger bit requirements to the data that are stored or cached for regular reanalysis. By doing this, we avoid loading data that we know will never be used and the effective data processing rate will thus be increased by the preselection factor, which in the case of $H(\mu\mu)$ is approximately three. We choose to report the speed with respect to raw event numbers from CMS simulation without preselection, which would artificially inflate the reported speed. For the baseline analysis, the CPU and GPU backends perform about equally. The GPU backend is advantageous in the case of more complex analyses: when the analysis complexity is scaled up by a factor of 10, we find that the GPU-based backend completes the analysis approximately 2x faster than a pure CPU implementation, and provides a 2.5-fold speedup with respect to a naive implementation. These results are summarized on figure . In the implementation of the data flow pipeline, we use a multithreaded approach where a worker thread preloads cached data to system memory and feeds it into a thread-safe queue, which is processed by either the CPU or GPU backend on a separate thread. The batch size of the preloaded data determines the memory usage of the code, which is currently around 10-15 GB total ($< 0.7$ GB/core). The complete analysis is encapsulated in a single multithreaded python script which can be steered directly from the command line and does not require running additional server software or services.

We have not performed a symmetrical benchmark with respect to current commonly used non-array-based codes for this analysis, as these cannot be easily run from a single integrated workflow and thus cannot be reliably benchmarked end-to-end. However, neglecting the time spent in batch job preparation and management, event processing speeds for ROOT-based event loop analyses are commonly in the range of 50 - 100 kHz, which also factors in data fetching and decompression. We therefore stress that the benchmark shown in this result is not aimed to be symmetrical against standard analysis codes, but rather reflects the possibilities afforded by using array-based techniques on local data for analysis, which allows for fast experimentation and iteration on sizeable datasets before resorting to extensive distributed computing infrastructure. We hope this encourages further development and optimization of HEP data analysis codes along the performance and scaling axes.

In addition to benchmarking the scaling performance on a small dataset of 90 million simulated events as above, we have found that an end-to-end physics processing of roughly a billion events of data and MC is possible within minutes. The input data consist of approximately 1.2 billion events for the 2017 data taking period in approximately 880 GB of ROOT files. After the initial column caching and decompression step, which takes about an hour for a single 24-core job on the benchmark machine, a cache of approximately 210 GB is produced, preselecting 600M events based on the isolated muon trigger bit and saving only the approximately 50 analysis-specific columns. This cache is stored on an SSD for fast access and is portable between machines. The benchmarks are summarized in table .

|       Platform       | Time to create caches | Time to run physics code |
|:--------------------:|:---------------------:|:------------------------:|
| 24-core server + GPU |      50 minutes       |        10 minutes        |
|    4-core laptop     |          N/A          |        30 minutes        |

Time to run the analysis on 1.2 billion raw events on a server ^a^ and on a laptop ^b^. We list the time to decompress and cache approximately 880 GB of ROOT data, which is done only on the server, and the time to run the physics analysis code on approximately 210 GB of uncompressed and preselected caches.

^a^ 24-core Intel Xeon E5-2687W v4 3.00GHz, Intel P4608 SSD, nVidia GTX Titan X
^b^ 4-core 2.3 GHz Intel i5 laptop, 8GB RAM, Thunderbolt 3 SSD

Although some computationally demanding analysis features such as jet energy scale variations are not yet implemented, the scaling behaviour suggests that even a significantly more complex analysis could be completed within a few hours using only a single machine. Both horizontal and vertical scaling are possible. The work can be distributed across multiple machines using data parallelism within environments such as Apache Spark or using the more traditional batch software. Further engineering work on multithreading and kernel optimization, GPU streaming and caching is expected to increase the achievable analysis speeds on a single machine.

# Summary and outlook

The central message of this report is that it is possible to do significant physics analysis processing from a simple code on a single machine using multithreading within minutes to hours. Using memory-mappable caching, array processing approaches and a small number of specialized kernels for jagged arrays implemented in Python using Numba, it is possible to do real physics analysis tasks with event rates reaching up to a million events per second. It is also possible to offload parts of these array computations to accelerators such as GPUs, which are highly efficient at SIMD processing. Once the batched data are in device memory, reprocessing is computationally efficient, such that multiple iterations of the analyses can be run for optimization purposes. We have demonstrated a prototypical Higgs analysis implementation using these computational kernels which can be evaluated on a billion MC and data events in less than an hour with optional GPU offloading. Several optimizations remain possible in a future work, among them optimizing the data access via efficient decompression and caching, scaling across multiple machines as well as optimizing the threading performance. We hope the approach shown here will spark discussion and further development of fast analysis tools which would be useful for scientists involved in HEP data analysis and more widely in data intensive fields.

# Acknowledgment

We would like to thank Jim Pivarski and Lindsey Gray for helpful feedback at the start of this project. We are grateful to Nan Lu and Irene Dutta for providing a reference implementation of the $H(\mu\mu)$ analysis that could be adapted to vectorized code. We would like to thank Christina Reissel for being an independent early tester of these approaches and for helpful feedback on this report. The availability of the excellent Python libraries `uproot`, `awkward`, `coffea`, `Numba`, `cupy` and `numpy` was imperative for this project and we are grateful to the developers of those projects. Part of this work was conducted at "*iBanks*", the AI GPU cluster at Caltech. We acknowledge NVIDIA, SuperMicro and the Kavli Foundation for their support of "*iBanks*".


abstract: The recent explosion of recorded digital data and its processed derivatives threatens to overwhelm researchers when analysing their experimental data or when looking up data items in archives and file systems. While current hardware developments allow to acquire, process and store 100s of terabytes of data at the cost of a modern sports car, the software systems to handle these data are lagging behind. This problem is very general and is well recognized by various scientific communities; several large projects have been initiated, e.g., DATAGRID/EGEE[^1] federates compute and storage power over the high-energy physical community, while the international astronomical community is building an Internet geared Virtual Observatory[^2] connecting archival data. These large projects either focus on a specific distribution aspect or aim to connect many sub-communities and have a relatively long trajectory for setting standards and a common layer. Here, we report "first light" of a very different solution (Valentijn & Kuijken, 2004) to the problem initiated by a smaller astronomical IT community. It provides the abstract "scientific information layer" which integrates distributed scientific analysis with distributed processing and federated archiving and publishing. By designing new abstractions and mixing in old ones, a Science Information System with fully scalable cornerstones has been achieved, transforming data systems into knowledge systems. This break-through is facilitated by the full end-to-end linking of all dependent data items, which allows full backward chaining from the observer/researcher to the experiment. Key is the notion that information is intrinsic in nature and thus is the data acquired by a scientific experiment. The new abstraction is that software systems guide the user to that intrinsic information by forcing full backward and forward chaining in the data modelling.
author: Edwin A. Valentijn, John P. McFarland, Jan Snigula$^\dag$, Kor G. Begeman, Danny R. Boxhoorn, Roeland Rengelink, Ewout Helmich, Philippe Heraudeau, Gijs Verdoes Kleijn, Ronald Vermeij, Willem-Jan Vriend, Michiel J.Tempelaar; Erik Deul, Konrad Kuijken; Massimo Capaccioli, Roberto Silvotti; Ralf Bender, Mark Neeser, Roberto Saglia; Emmanuel Bertin, Yannick Mellier
title: Astro-WISE: Chaining to the Universe

# Introduction

The classical paradigm to handle data streams of large physical experiments, such as CERN-HEP, astronomical telescopes, or scientific satellites, is characterized by different layers in the project that apply certain algorithms to the stream of data and subsequently deliver the results to the next layer, following a so-called TIER architecture. This architecture can be characterized as data-driven and "feed-forward". The construction of the different layers is often grown historically and relates to implicit or explicit project management decisions facing sociology, geography and interfacing of working groups with different expertise. Projects operated under this paradigm work with releases of datasets, which are obtained with a certain version of the code meeting certain quality control standards. Operators have a task to push the input data through the stream, often by means of a "pipeline" irrespective of whether the derived data items are actually used by the end users. Examples of successful forward workflow projects are numerous (e.g., Valentijn & Kuijken, 2004; Costa et al., 2003; Hiriart et al., 2003; Jung et al., 2003; Mehringer & Plante 2003; and Pierfederici et al., 2003). When funding for the operation centres dries up, a final release of processed data is made, and the project comes to an end.

# The Problem

This "classical paradigm" has the advantage of publishing complete sets of data with well described methods, qualifications, and calibrations. However, it has bad scalability when exposed to modern huge data streams: re-processing all the raw data and storing the results for new releases when new computational methods, calibration strategies, insights or improved code becomes available is very difficult to impossible. And, moreover, why should operators re-process all data as long as it is even not known whether there are customers for the individual products? Though the classical approach has the obvious advantage of facilitating established releases, it cannot facilitate the demands of the end-user/researcher, who actually might want to evaluate specific questions, and specific results obtained with specific methods from (a subset) in the flood of physical observational data (including even holiday digital pictures or scanned documents of national libraries) with. Key to addressing the merits of a system for a researcher is to evaluate the question of what analysis on uncertainties the researcher facilitates in that system before (s)he submits his/her 4 sigma result to a scientific journal.

Such demands combined with the increased data rates from instruments and world-wide communities ask for a different approach. A first interesting step is set by the Space Telescope Science Institute, operating the Hubble Space Telescope, where users can ask for a certain data product which will then run the Calibration pipeline on-demand using the best available calibrations (Swade et al., 2001), a case of forward chaining. But what to do when a user discovers a very faint object and wants to inspect the robustness of the result by checking the dependencies on uncertainties of calibration parameters and applied methods? The user wants to interact with the data, derive his own result following his insights and methods, and preferably this knowledge is fed back into the system at the disposal of other researchers who optionally want to take advantage of the progress of insight. In the 1980s, solutions were explored in expert systems. Nowadays, with distributed communities working on enormous data floods, new artificial intelligence-like, distributed information systems are required to facilitate the scientific endeavour.

# The Solution

A new generation of wide field astronomical imaging cameras includes the MEGACAM at the CFHT and OmegaCAM (Kuijken et al., 2004) at the VLT Survey Telescope (VST). OmegaCAM, with a one square degree field of view and a pixel size of 0.2 arc second, will deliver 256 mega-pixel images of the sky every few minutes, for 300 nights per year for many years to come. These experiments will produce 100's of terabyte per year of raw and processed data, e.g., to trace the effects of dark matter and dark energy. These requirements inspired an international consortium, Astro-WISE, to design and implement a completely different approach to connect the end-user to the experiment. Facing the problems sketched above we build an information system in which the role of the user (observer) is central. The system is built to handle queries by the user for his/her desired result, which we call a "target". A "target" could be an astrometrically and photometrically calibrated image, or the results for a set of calibration parameters, or a list of parameter values describing an astronomical object, or whatever "target" that is facilitated by the system. In the abstractions used for the software the "target" processing resembles that of the UNIX make metaphor, but in addition the dynamic aspects resembles to the "goal" in artificial intelligence systems.

# How it Works

Following a query by the user, the system checks whether the "target" has been processed before,

-   if not, it will be processed "on-the-fly", recursively following all its dependencies in an object model, similar to the Unix make metaphor. See the "target diagram" in Fig. .

-   if already existent the system will check all its dependencies up to the raw data taken at the telescope (Fig. )

    -   if all its dependencies are "up-to-date" the target object is returned.

    -   if all or some dependencies are not "up-to-date" they will be re-computed on-the-fly, again according to the 'make' metaphor, but with optional, tunable "depth".

All data production in the system follows this schema, whether it concerns an individual user asking for a single special result (target), a calibration scientist determining the instrument behaviour over long periods or a production scientist deriving results for whole nights of observing. They all add knowledge to the system. Note that, in the end, it does not matter very much whether or not a target requested by the user has been processed before, it only affects the performance of the system. This way, the workflow in the ocean of data space is fully driven by the users quest. (see <http://process.astro-wise.org/>)

This behaviour is achieved by linking (referencing) all dependent data items in the information system. This is achieved by carefully specifying the observation procedure both for science and calibration observations and subsequently designing a data model, chaining the processing targets to the raw data. Next, the data model is converted into an object model, which in turn is ported in a commercial database (using object oriented "user-defined types" in a relational database). The database thus has full awareness of all dependencies. When compute scripts are run at a computer, all Class instantiations are automatically made persistent in the database forming a dynamic archive of all targets and added knowledge, while new and nearly "sacred" raw data is ingested as long as the acquisition continues. Table summarizes some key differences between the Astro-WISE system and the classical approach.

| "Classical" Paradigm | Astro-WISE Paradigm   |
|:---------------------|:----------------------|
| forward chaining     | backward chaining     |
| "tier" architecture  | "target" architecture |
| driven by raw data   | driven by user query  |
| process in pipeline  | process on-the-fly    |
| operators push data  | users pull data       |
| results in releases  | information system    |
| static archive       | dynamic archive       |
| raw data is obsolete | raw data is sacred    |

Characteristics of the Astro-WISE Information System

Obviously, this can only be achieved by integrating all computing and storage hardware into a single information system. Figure shows the peer-to-peer network employed for Astro-WISE. The network and its hardware can be viewed as an extension of the telescope and measurement apparatus hardware, together with the abstract software, forming a unity. The user, by requesting a target, triggers the whole chain from target to raw observational data of the measurement apparatus, a real virtual observatory.

All top level software in the system is written in the Python scripting language (see <http://www.python.org/>) which in turn calls C libraries. Following the LISP language, Python achieves a very high level of integration. It facilitates another crucial unification in the system: apart from the GUIs, all usage of the system is done from a Python prompt via a Python binding to the database. Novice users, advanced users, and scientific programmers/developers alike use the same environment, allowing a continuous transition between all levels. All Python open source facilities, e.g., visualisation libraries (MatPlotLib), numerical libraries (NumPy), etc., are at direct disposal of the user and are connected to the output of user queries to the information system. Users apply scripts provided by a federated CVS distribution of the standard Astro-WISE code base, but can also modify scripts, fine-tuning them or adding methods as long as they do not violate the object model. This way, both a simple standard version and user tuned versions of processing targets can be obtained simultaneously (different versions of the code are tagged and can be traced).

The database is partitioned in instruments and projects, facilitating individual read and write privileges for persons, groups and projects. Users set a context relating to their project and thus see only their partition in the ocean of data. The smallest project, the individual or anonymous Web based user doing his/her "afternoon" experiment is facilitated in a "MyDB" context, which can be upgraded to another, appropriate context/project at the end of the session. Project leaders have the responsibility for the project quality control. The code base supports this by built-in "verify", "compare" and "inspect" methods for each Class, where "verify" involves an internal code check, "compare" involves a database query to different instantiations of the same Class and "inspect" is a visual inspection by the user. Eventually, all quality control is maintained by a time-stamping mechanism on each Class instantiation, and a GUI is build for calibration scientists to supervise and alter the timestamps, the ultimate point where human insights add knowledge to the system.

In fact, an important design criterion was to allow a complete federation of the system, facilitating different research groups spread over Europe to share scientific projects. Currently, the system connects National data centres in the Netherlands (Groningen), Italy (Naples), France (Paris), and Germany (Munich), which in turn re-distribute the system to satellites (e.g., Bonn and Heidelberg). Similar hardware is installed at the National centres: the code-base is federated, the database is replicated, and the file-servers prevent duplication of massive storage of image data. Network traffic is minimised by allowing for terabyte-sized local cache for frequently used objects. All GUIs are web based. These include the database viewer (<http://dbview.astro-wise.org/>) and the GUI to evaluate and operate target processing at any selected Linux farm of the federation. User (client) altered Python scripts are sent to the processors in the federation as so-called Python "pickles" (persistent Python objects in a string representation).

The system has been fully implemented and currently operates the raw data of various astronomical wide field imaging cameras at ESO's 2.2m telescope in Chile (300.000 CCD read-outs), the ING telescope(Canary Islands), the Subaru telescope (Japan) and OmegaCAM for ESO's VLT survey telescope (currently test data only). During the implementation, while taking care of object model purity, we experienced an avalanche of benefits for which relatively little extra effort had to be made:

-   The full linking of object dependencies allows full backwards chaining as employed by the artificial intelligence (Shachter & Heckerman, 1987 and Thompson & Thompson, 1985) and history tracking. Literally every bit of information that went into the target can be retrieved: the system provides a "tell me everything tool". In fact, this is an a priori ontological implementation, in contrast to the fashionable a posteriori, semantic web search engines.

-   Publishing on Internet/EURO-VO (the European Virtual Observatory) is done by raising a flag. It is up to project managers to do this for classical paradigm static results or for Astro-WISE paradigm dynamic results.

-   The control by the database over the parallel processing (e.g., SETI@home) on any number of nodes.The enabling of a international compute GRID.

-   Enabling global astrometric and photometric solutions with increased accuracy, redirecting global database knowledge back into the system.

-   The built-in workflow directly guides the user and no workflow systems are required.

Next to facilitating statistical studies, the system is optimized for "needle-in-the-haystack" kind of searches, finding extremely rare astronomical objects, such as moving, solar system objects, or variable objects like ultra-compact binary systems of white dwarf stars or distant supernovae. Next to all image Meta-data, the astronomical source parameters are stored in an Oracle database and Oracle partitioning is used to quickly address up to 100 Tbyte data volumes. Fast astronomical object associations (a posteriori associations) are made in "many-to-many" mode using both native database indexing and positional HTM[^3] indexing. The linking (joins and references) are maintained at an extreme level. For example, in the KIDs 1500-5000 degree survey nearly 1 Tbyte of linked data items is anticipated.

The avalanche of side products and its rapid implementation are thanks to several unifications which are achieved by merging various pointer mechanisms, such as those provided by object oriented programming (inheritance), class/attribute persistency, database internal links (joins, references), and namespace handling by the Python scripting language. In the context of physical information theory, all these links can be viewed as kinds of memory addresses and facilitate forms of information transfer, gearing the user to intrinsic information, and facilitating the handling of the dispersion caused by the measurement apparatus, camera, and the off-line computing hardware. Particularly, the option to inspect partial derivatives of dependent parameter, by re-deriving results, allows the user to inspect the information content in the data (Frieden, 1998).

# Conclusion & Acknowledgements

Our key-concept, involving novel approaches to maintain data associations in federations of integrated pipelines and archives, can be applied to arbitrary forms of digitized observational data, ranging from DNA sequences to numerical simulations and national libraries (e.g., centuries of Dutch governmental handwritten records being scanned and entered into the system will be processed with pattern recognition techniques). After all, this is all part of our Universe that we observe, and with the coming decade, will be copied into our hardware at multi-petabyte scales to be interpreted by a next generation of scientists.

The project received funding from the European Union through the EC Action *Enhancing Access to Research Infrastructures*, a FP5 RTD programme, the Netherlands Research School for Astronomy NOVA and NWO. Correspondence and requests for materials should be addressed to E.A. Valentijn, Coordinator, Astro-WISE *firstname.lastname@example.com*.

Supplementary Information can be found on <http://www.astro-wise.org/>

[^1]: <http://www.eu-egee.org/>

[^2]: <http://www.euro-vo.org/pub/> and Padovani, 2006

[^3]: <http://www.sdss.jhu.edu/htm/>