abstract: We show that various milestones of high-redshift galaxy formation, such as the formation of the first stars or the complete reionization of the intergalactic medium, occurred at different times in different regions of the universe. The predicted spread in redshift, caused by large-scale fluctuations in the number density of galaxies, is at least an order of magnitude larger than previous expectations that argued for a sharp end to reionization. This cosmic scatter in the abundance of galaxies introduces new features that affect the nature of reionization and the expectations for future probes of reionization, and may help explain the present properties of dwarf galaxies in different environments. The predictions can be tested by future numerical simulations and may be verified by upcoming observations. Current simulations, limited to relatively small volumes and periodic boundary conditions, largely omit cosmic scatter and its consequences. In particular, they artificially produce a sudden end to reionization, and they underestimate the number of galaxies by up to an order of magnitude at redshift 20.
author: Rennan Barkana; Abraham Loeb
title: Unusually Large Fluctuations in the Statistics
  of Galaxy Formation at High Redshift

# Introduction

Recent observations of the cosmic microwave background have confirmed the notion that the present large-scale structure in the universe originated from small-amplitude density fluctuations at early cosmic times. Due to the natural instability of gravity, regions that were denser than average collapsed and formed bound halos, first on small spatial scales and later on larger and larger scales. At each snapshot of this cosmic evolution, the abundance of collapsed halos, whose masses are dominated by cold dark matter, can be computed from the initial conditions using numerical simulations and can be understood using approximate analytic models . The common understanding of galaxy formation is based on the notion that the constituent stars formed out of the gas that cooled and subsequently condensed to high densities in the cores of some of these halos .

The standard analytic model for the abundance of halos considers the small density fluctuations at some early, initial time, and attempts to predict the number of halos that will form at some later time corresponding to a redshift $z$. First, the fluctuations are extrapolated to the present time using the growth rate of linear fluctuations, and then the average density is computed in spheres of various sizes. Whenever the overdensity (i.e., the density perturbation in units of the cosmic mean density) in a sphere rises above a critical threshold $\delta_c(z)$, the corresponding region is assumed to have collapsed by redshift $z$, forming a halo out of all the mass that had been included in the initial spherical region. In analyzing the statistics of such regions, the model separates the contribution of large-scale modes from that of small-scale density fluctuations. It predicts that galactic halos will form earlier in regions that are overdense on large scales , since these regions already start out from an enhanced level of density, and small-scale modes need only supply the remaining perturbation necessary to reach $\delta_c(z)$. On the other hand, large-scale voids should contain a reduced number of halos at high redshift. In this way, the analytic model describes the clustering of massive halos.

As gas falls into a dark matter halo, it can fragment into stars only if its virial temperature is above $10^4$K for cooling mediated by atomic transitions \[or $\sim 500$ K for molecular ${\rm H}_2$ cooling; see, e.g., Figure 12 in \]. The abundance of dark matter halos with a virial temperature above this cooling threshold declines sharply with increasing redshift due to the exponential cutoff in the abundance of massive halos at early cosmic times. Consequently, a small change in the collapse threshold of these rare halos, due to mild inhomogeneities on much larger spatial scales, can change the abundance of such halos dramatically. The modulation of galaxy formation by long wavelength modes of density fluctuations is therefore amplified considerably at high redshift. In this paper we show that this results in major new predictions for high-redshift observations. The implications are particularly significant for cosmic reionization and all observational probes of this epoch.

This paper is organized as follows. In § 2 we quantify the scatter in the statistics of galaxy formation produced by this amplification effect. We first explain in § 2.1 the basic physical ideas and implications using the well-established extended Press-Schechter model. We then present in § 2.2 a simple idea that yields a much more accurate model that fits an array of previous simulations at low redshift. We demonstrate the qualitative correctness of our basic assumptions as well as the quantitative accuracy of our model by matching results from recent simulations at high redshift. Since high-redshift galaxies provide the UV photons that lead to the reionization of the intergalactic medium (hereafter IGM), a large scatter is also expected in the reionization redshift within different regions in the universe. We consider this scatter and the modified character of reionization in § 3.1, and show in § 3.2 that existing numerical simulations do not include fluctuations on sufficiently large scales at high redshift. In § 3.3 we discuss the observational implications of the large cosmic scatter expected at high redshift. Finally, we summarize our main results in § 4.

# Halo Mass Function in Different Environments

## Basic Model: Amplification of Density Fluctuations

Galaxies at high redshift are believed to form in all halos above some minimum mass $M_{\rm min}$ that depends on the efficiency of atomic and molecular transitions that cool the gas within each halo. This makes useful the standard quantity of the collapse fraction $F_{\rm
col}(M_{\rm min})$, which is the fraction of mass in a given volume that is contained in halos of individual mass $M_{\rm min}$ or greater. If we set $M_{\rm min}$ to be the minimum halo mass in which efficient cooling processes are triggered, then $F_{\rm col}(M_{\rm
min})$ is the fraction of all the baryons in the universe that lie in galaxies. In a large-scale region of comoving radius $R$ with a mean overdensity $\bar{\delta}_R$, the standard result is $$F_{\rm col}(M_{\rm min})={\rm erfc}\left[
\frac{\delta_c(z)- \bar{\delta}_R} {\sqrt{2 \left[S(R_{\rm min}) -
S(R) \right]}} \right]\ , \label{eq:Fcol}$$ where $S(R)$ is the variance of fluctuations in spheres of radius $R$, and $S(R_{\rm min})$ is the variance in spheres of radius $R_{\rm min}$ corresponding to the region at the initial time that contained a mass $M_{\rm min}$. In particular, the cosmic mean value of the collapse fraction is obtained in the limit of $R \rightarrow \infty$ by setting $\bar{\delta}_R$ and $S(R)$ to zero in this expression. Throughout this section our results assume this standard model, known as the extended Press-Schechter model, which we apply to a universe with cosmological parameters matching the latest observations \[specifically, the running index model of \]. Whenever we consider a cubic region, we estimate its halo abundance by applying the model to a spherical region of equal volume. Note also that we consistently quote values of comoving distance, which equals physical distance times a factor of $(1+z)$.

Our results are based on a simple idea. At high redshift, galactic halos are rare and correspond to high peaks in the Gaussian probability distribution of initial fluctuations. A modest change in the overall density of a large region modulates the threshold for high peaks in the Gaussian density field, so that the number of galaxies is exponentially sensitive to this modulation. This amplification of large-scale modes is responsible for the large statistical fluctuations that we find.

In numerical simulations, periodic boundary conditions are usually assumed, and this forces the mean density of the box to equal the cosmic mean density. The abundance of halos as a function of mass is then biased in such a box (see Figure 1), since a similar region in the real universe will have a distribution of different overdensities $\bar{\delta}_R$. At high redshift, when galaxies correspond to high peaks, they are mostly found in regions with an enhanced large-scale density. In a periodic box, therefore, the total number of galaxies is artificially reduced, and the relative abundance of galactic halos with different masses is artificially tilted in favor of lower-mass halos. We illustrate our results for two sets of parameters, one corresponding to the first galaxies and early reionization ($z=20$) and the other to the current horizon in observations of galaxies and late reionization ($z=7$). We consider a resolution equal to that of state-of-the-art cosmological simulations that include gravity and gas hydrodynamics. Specifically, we assume that the total number of dark matter particles in the simulation is $N = 324^3$, and that the smallest halo that can form a galaxy must be resolved into 500 particles; showed that this resolution is necessary in order to determine the star formation rate in an individual halo reliably to within a factor of two. Therefore, if we assume that halos that cool via molecular hydrogen must be resolved at $z=20$ (so that $M_{\rm min}=7 \times 10^5 M_{\odot}$), and only those that cool via atomic transitions must be resolved at $z=7$ (so that $M_{\rm
min}=10^8 M_{\odot}$), then the maximum box sizes that can currently be simulated are $l_{\rm box}=1$ Mpc and $l_{\rm box}=6$ Mpc at these two redshifts, respectively.

At each redshift we only consider cubic boxes large enough so that the probability of forming a halo on the scale of the entire box is negligible. In this case, $\bar{\delta}_R$ is Gaussian distributed with zero mean and variance $S(R)$, since the no-halo condition $\sqrt{S(R)} \ll \delta_c(z)$ implies that at redshift $z$ the perturbation on the scale $R$ is still in the linear regime. We can then calculate the probability distribution of collapse fractions in a box of a given size (see Figure 2). This distribution corresponds to a real variation in the fraction of gas in galaxies within different regions of the universe at a given time. In a numerical simulation, the assumption of periodic boundary conditions eliminates the large-scale modes that cause this cosmic scatter. Note that Poisson fluctuations in the number of halos within the box would only add to the scatter, although the variations we have calculated are typically the dominant factor. For instance, in our two standard examples, the mean expected number of halos in the box is 3 at $z=20$ and 900 at $z=7$, resulting in Poisson fluctuations of a factor of about 2 and 1.03, respectively, compared to the clustering-induced scatter of a factor of about 16 and 2 in these two cases.

Within the extended Press-Schechter model, both the numerical bias and the cosmic scatter can be simply described in terms of a shift in the redshift (see Figure 3). In general, a region of radius $R$ with a mean overdensity $\bar{\delta}_R$ will contain a different collapse fraction than the cosmic mean value at a given redshift $z$. However, at some wrong redshift $z + \Delta z$ this small region will contain the cosmic mean collapse fraction at $z$. At high redshifts ($z > 3$), this shift in redshift can be easily derived from eq. 1 to be $$\Delta z = \frac{\bar{\delta}_R}{\delta_0} - (1+z)
\times \left[ 1 - \sqrt{1 - \frac{S(R)} {S(R_{\rm min})}}\ \right]\ ,$$ where $\delta_0 \equiv \delta_c(z)/(1+z)$ is approximately constant at high redshifts , and equals 1.28 for our assumed cosmological parameters. Thus, in our two standard examples, the bias is -2.6 at $z=20$ and -0.4 at $z=7$, and the one-sided $1-\sigma$ scatter is 2.4 at $z=20$ and 1.2 at $z=7$.

## Improved Model: Matching Numerical Simulations

In this subsection we develop an improved model that fits the results of numerical simulations more accurately. The model constructs the halo mass distribution (or mass function); cumulative quantities such as the collapse fraction or the total number of galaxies can then be determined from it via integration. We first define $f(\delta_c(z),S)\,
dS$ to be the mass fraction contained at $z$ within halos with mass in the range corresponding to $S$ to $S+d S$. The halo abundance is then $$\frac{dn}{dM} = \frac{\bar{\rho}_0}{M} \left|\frac{d S}{d M}
\right| f(\delta_c(z),S)\ , \label{eq:abundance}$$where $dn$ is the comoving number density of halos with masses in the range $M$ to $M+dM$. In the model of , $$f_{\rm PS}(\delta_c(z),S) =
\frac{1} {\sqrt{2 \pi}} \frac{\nu }{S} \exp\left[-\frac{\nu^2}{2}
\right]\ , \label{eq:PS}$$where $\nu=\delta_c(z)/\sqrt{S}$ is the number of standard deviations that the critical collapse overdensity represents on the mass scale $M$ corresponding to the variance $S$.

However, the Press-Schechter mass function fits numerical simulations only roughly, and in particular it substantially underestimates the abundance of the rare halos that host galaxies at high redshift. The halo mass function of adds two free parameters that allow it to fit numerical simulations much more accurately . We note that these simulations followed very large volumes at low redshift, so that cosmic scatter did not compromise their accuracy. The matching mass function is given by $$f_{\rm ST}(\delta_c(z),S) = A' \frac{\nu }{S} \sqrt{\frac{a'} {2
\pi}} \left[ 1+\frac{1}{(a' \nu^2)^{q'}} \right] \exp\left[-\frac{a'
\nu^2}{2} \right]\ , \label{eq:ST}$$with best-fit parameters $a'=0.75$ and $q'=0.3$, and where normalization to unity is ensured by taking $A'=0.322$.

In order to calculate cosmic scatter we must determine the biased halo mass function in a given volume at a given mean density. Within the extended Press-Schechter model , the halo mass distribution in a region of comoving radius $R$ with a mean overdensity $\bar{\delta}_R$ is given by $$f_{\rm
bias-PS}(\delta_c(z), \bar{\delta}_R,R,S)=f_{\rm PS}(\delta_c(z)-
\bar{\delta}_R,S-S(R))\ \label{eq:ePS}.$$The corresponding collapse fraction in this case is given simply by eq. (). Despite the relatively low accuracy of the Press-Schechter mass function, the *relative change* is predicted rather accurately by the extended Press-Schechter model. In other words, the prediction for the halo mass function in a given volume compared to the cosmic mean mass function provides a good fit to numerical simulations over a wide range of parameters .

For our improved model we adopt a hybrid approach that combines various previous models with each applied where it has been found to closely match numerical simulations. We obtain the halo mass function within a restricted volume by starting with the Sheth-Tormen formula for the cosmic mean mass function, and then adjusting it with a relative correction based on the extended Press-Schechter model. In other words, we set $$\begin{aligned}
& & f_{\rm bias}(\delta_c(z),\bar{\delta}_R,R,S)
= \nonumber \\ & & f_{\rm ST}(\delta_c(z),S)\ \times \left[ \frac{f_{\rm
PS} (\delta_c(z)-\bar{\delta}_R,S-S(R))} {f_{\rm PS}(\delta_c(z),S)}
\right]\ . \label{eq:bias}
\end{aligned}$$As noted, this model is based on fits to simulations at low redshifts, but we can check it at high redshifts as well. Figure  shows the number of galactic halos at $z \sim 15-30$ in two numerical simulations run by , and our predictions given the cosmological input parameters assumed by each simulation. The close fit to the simulated data (with no additional free parameters) suggests that our hybrid model (solid lines) improves on the extended Press-Schechter model (dashed lines), and can be used to calculate accurately the cosmic scatter in the number of galaxies at both high and low redshifts. The simulated data significantly deviate from the expected cosmic mean \[eq. (), shown by the dotted line\], due to the artificial suppression of large-scale modes outside the simulated box. We note that mentioned that the lack of large-scale modes might produce a systematically low halo abundance, particularly in the RSI model, but they did not quantify this effect.

As an additional example, we consider the highest-resolution first star simulation , which used $l_{\rm box}=128$ kpc and $M_{\rm min}=7 \times 10^5 M_{\odot}$. The first star forms within the simulated volume when the first halo of mass $M_{\rm min}$ or larger collapses within the box. To compare with the simulation, we predict the redshift at which the probability of finding at least one halo within the box equals $50\%$, accounting for Poisson fluctuations. We find that if the simulation formed a population of halos corresponding to the correct cosmic average \[as given by eq. ()\], then the first star should have formed already at $z=24.0$. The first star actually formed in the simulation box only at $z=18.2$ . Using eq. () we can account for the loss of large-scale modes beyond the periodic box, and predict a first star at $z=17.8$, a close match given the large Poisson fluctuations introduced by considering a single galaxy within the box.

The artificial bias in periodic simulation boxes can also be seen in the results of extensive numerical convergence tests carried out by . They presented a large array of numerical simulations of galaxy formation run in periodic boxes over a wide range of box size, mass resolution, and redshift. In particular, we can identify several pairs of simulations where the simulations in each pair have the same mass resolution but different box sizes; this allows us to separate the effect of large-scale numerical bias from the effect of having poorly-resolved individual halos. Specifically, their simulations Z1 and R4 \[see Table 1 in \] used the same particle mass but R4 had a box length larger by a factor of 3.375 . The simulations Q1 and D4 are similarly related, as are Q2 and D5. In each case, the smaller simulation substantially underestimated the star formation rate at high redshift \[see Figure 10 in \], Z1 by a factor of 5 at $z=15$ compared to R4, Q1 by a factor of 3 at $z=8$ compared to D4, and Q2 by a factor of 3 at $z=10$ compared to D5.

We note that there have been previous attempts to develop a model for the halo mass function in different environments, so that the model would be consistent with the Sheth-Tormen mass function of eq. () which accurately fits the cosmic mean mass function measured in numerical simulations. In order to identify the specific requirements for such a consistency, we first consider the analogous case of the extended Press-Schechter model and its relation to the Press-Schechter formula for the cosmic mean mass function. The extended Press-Schechter model is consistent with the mean mass function in the sense that $f_{\rm bias-PS}$ evaluated for an infinite box (i.e., in the limit where $\bar{\delta}_R$ and $S(R)$ both vanish) yields the Press-Schechter mass function: $$f_{\rm bias-PS}
(\delta_c(z), 0,\infty,S)=f_{\rm PS}(\delta_c(z),S)\ . \label{eq:cons}$$ This condition does not suffice, however, since there is an additional self-consistency test that any viable model must satisfy. Consider any fixed scale $R$. Suppose we consider a very large number $N$ of spheres of radius $R$ within the universe. The mean density $\bar{\delta}_R$ in each sphere is determined according to a probability distribution $p(\bar{\delta}_R)$; we assume that $R$ is large enough so that the probability of forming a halo out of all the mass on the scale $R$ is negligible, and so the distribution is a Gaussian with zero mean and variance $S(R)$ (see also §2.1). The number of galaxies in each sphere is given in the extended Press-Schechter model by eq. (). As $N \rightarrow
\infty$, the halo mass function averaged over all these spheres must approach the cosmic mean value, and it must also approach the ensemble-averaged mass function, where the averaging is performed over the probability distribution of $\bar{\delta}_R$. This yields the following self-consistency requirement, which is indeed satisfied by the extended Press-Schechter model: $$\begin{aligned}
& & \int f_{\rm
bias-PS}(\delta_c(z), \bar{\delta}_R,R,S)\, p(\bar{\delta}_R)\,
d\bar{\delta}_R = \nonumber \\ & & f_{\rm bias-PS}(\delta_c(z),
0,\infty,S)\ .  \label{eq:scons}
\end{aligned}$$

Now we again consider attempts to construct an improved model that is consistent with the Sheth-Tormen mass function. Such a model must satisfy eq. () (except with $f_{\rm ST}$ on the right-hand side), and it must also satisfy eq. () in order to be self-consistent. The latter equation must be satisfied separately for every scale $R$ large enough to avoid collapsing \[i.e., that satisfies $\sqrt{S(R)} \ll \delta_c(z)$\]. Previous proposed models satisfied simple consistency but not the self-consistency test. Our hybrid model of eq. () satisfies both requirements (with respect to the Sheth-Tormen mass function), a result that follows immediately from the fact that the extended Press-Schechter model also satisfies both requirements (with respect to the Press-Schechter mass function). Thus, our hybrid model is the first self-consistent model that is also consistent with the Sheth-Tormen mass function, at least when fluctuations are considered on large scales $R$ for which $\bar{\delta}_R$ is Gaussian distributed. As demonstrated in this section, our model also matches results from a wide array of numerical simulations .

# Implications

## The nature of reionization

The photons of the cosmic microwave background have traveled to us mostly undisturbed after neutral atoms first formed in the universe at the cosmic recombination epoch. Radiation from the first generation of stars is thought to have reionized the hydrogen throughout the universe, transforming the IGM back into a hot and highly-ionized plasma.

The popular view developed in the literature maintains that reionization ended with a fast, simultaneous, overlap stage throughout the universe. This view has been based on simple arguments and has been supported by numerical simulations with small box sizes. The underlying idea was that the ionized hydrogen () regions of individual sources began to overlap when the typical size of each bubble became comparable to the distance between nearby sources. Since these two length scales were comparable at the critical moment, there is only a single timescale in the problem – given by the growth rate of each bubble – and it determines the transition time between the initial overlap of two or three nearby bubbles, to the final stage where dozens or hundreds of individual sources overlap and produce large ionized regions. Whenever two ionized bubbles were joined, each point inside their common boundary became exposed to ionizing photons from both sources, reducing the neutral hydrogen fraction and allowing ionizing photons to travel farther before being absorbed. Thus, the ionizing intensity inside regions rose rapidly, allowing those regions to expand into high-density gas that had previously recombined fast enough to remain neutral when the ionizing intensity had been low. Since each bubble coalescence accelerates the process, it has been thought that the overlap phase has the character of a phase transition and occurs rapidly. Indeed, the best simulations of reionization to date found that the average mean free path of ionizing photons in the simulated volume rises by an order of magnitude over a redshift interval $\Delta z = 0.05$ at $z=7$.

Our results substantially modify this commonly accepted picture for the development of reionization. Overlap is still expected to occur rapidly, but only in localized high-density regions, where the ionizing intensity and the mean free path rise rapidly even while other distant regions are still mostly neutral. In other words, the size of the bubble of an individual source is about the same in different regions (since most halos have masses just above $M_{\rm
min}$), but the typical distance between nearby sources varies widely across the universe. The strong clustering of ionizing sources on length scales as large as 30–100 Mpc introduces long timescales into the reionization phase transition. The sharpness of overlap is determined not by the growth rate of bubbles around individual sources, but by the ability of large groups of sources within overdense regions to deliver ionizing photons into large underdense regions. Simply put, the common view assumes that reionization occurred in patches a few Mpc in size, and that it ended nearly simultaneously in all of them. In reality, however, these two statements are contradictory. If the patches are a few Mpc in size, then there is a very large spread in their reionization redshifts. Conversely, if the spread is small, this implies (from Figure 3) that the patches must be far larger than is commonly assumed.

Note that the recombination rate is higher in overdense regions because of their higher gas density. These regions still reionize first, though, despite the need to overcome the higher recombination rate, since the number of ionizing sources in these regions is increased even more strongly as a result of the dramatic amplification of large-scale modes discussed earlier.

## Limitations of current simulations

The shortcomings of current simulations do not amount simply to a shift of $\sim 10\%$ in redshift and the elimination of scatter, for several reasons. First, the effect that we have identified can be expressed in terms of a shift in redshift only within the context of the extended Press-Schechter model, and only if the total mass fraction in galaxies is considered and not its distribution as a function of galaxy mass. The halo mass distribution should still have the wrong shape, resulting from the fact that $\Delta z$ in eq. 2 depends on $M_{\rm min}$. Furthermore, in our more accurate hybrid model (§ 2.2), the effect on the collapse fraction is no longer exactly equivalent to a shift in redshift. In any case, a self-contained numerical simulation cannot rely on approximate models and must directly evolve a very large volume[^1].

The second reason that current simulations are limited is that at high redshift, when galaxies are still rare, the abundance of galaxies grows rapidly towards lower redshift. Therefore, a $\sim 10\%$ relative error in redshift implies that at any given redshift around $z \sim 10$–20, the simulation predicts a halo mass function that can be off by an order of magnitude for halos that host galaxies (see Figures 1 and ). This large underestimate suggests that the first generation of galaxies formed significantly earlier than indicated by recent simulations. This makes it easier to explain recent observations of the cosmic microwave background that suggest an early reionization at $z \sim 15$–20.

The third reason for the failure of simulations arises from the large cosmic scatter. This scatter can fundamentally change the character of any observable process or feedback mechanism that depends on a radiation background. Simulations in periodic boxes eliminate any large-scale scatter by assuming that the simulated volume is surrounded by identical periodic copies of itself. In the case of reionization, for instance, current simulations neglect the collective effects described above, whereby groups of sources in overdense regions may influence large surrounding underdense regions. In the case of the formation of the first stars due to molecular hydrogen cooling, the effect of the soft ultraviolet radiation from these stars, which tends to dissociate the molecular hydrogen around them , must be reassessed with cosmic scatter included.

## Observational consequences

The spatial fluctuations that we have calculated fundamentally affect current and future observations that probe reionization or the galaxy population at high redshift. For example, there are a large number of programs searching for galaxies at the highest accessible redshifts (6.5 and beyond) using their strong Ly$\alpha$ emission . These programs have previously been justified as a search for the reionization redshift, since the intrinsic emission should be absorbed more strongly by the surrounding IGM if this medium is neutral. For any particular source, it will be hard to clearly recognize this enhanced absorption because of uncertainties regarding the properties of the source and its radiative and gravitational effects on its surroundings . However, if the luminosity function of galaxies that emit Ly$\alpha$ can be observed, then faint sources, which do not significantly affect their environment, should be very strongly absorbed in the era before reionization. Reionization can then be detected statistically through the sudden jump in the number of faint sources . Our results alter the expectation for such observations. Indeed, no sharp "reionization redshift" is expected. Instead, a Ly$\alpha$ luminosity function assembled from a large area of the sky will average over the cosmic scatter of $\Delta
z \sim 1$–2 between different regions, resulting in a smooth evolution of the luminosity function over this redshift range. In addition, such a survey may be biased to give a relatively high redshift, since only the most massive galaxies can be detected, and as we have shown, these galaxies will be concentrated in overdense regions that will also get reionized relatively early.

The distribution of ionized patches during reionization will likely be probed by future observations, including small-scale anisotropies of the cosmic microwave background photons that are rescattered by the ionized patches , and observations of 21 cm emission by the spin-flip transition of the hydrogen in neutral regions . Previous analytical and numerical estimates of these signals have not included the collective effects discussed above, in which rare groups of massive galaxies may reionize large surrounding areas. These photon transfers will likely smooth out the signal even on scales significantly larger than the typical size of an bubble due to an individual galaxy. Therefore, even the characteristic angular scales that are expected to show correlations in such observations must be reassessed.

The cosmic scatter also affects observations in the present-day universe that depend on the history of reionization. For instance, photoionization heating suppresses the formation of dwarf galaxies after reionization, suggesting that the smallest galaxies seen today may have formed prior to reionization . Under the popular view that assumed a sharp end to reionization, it was expected that denser regions would have formed more galaxies by the time of reionization, possibly explaining the larger relative abundance of dwarf galaxies observed in galaxy clusters compared to lower-density regions such as our Local Group of galaxies . Our results undercut the basic assumption of this argument and suggest a different explanation altogether. Reionization occurs roughly when the number of ionizing photons produced starts to exceed the number of hydrogen atoms in the surrounding IGM. If the processes of star formation and the production of ionizing photons are equally efficient within galaxies that lie in different regions, then reionization in each region will occur when the collapse fraction reaches the same critical value, even though this will occur at different times in different regions. Since the galaxies responsible for reionization have the same masses as present-day dwarf galaxies, this estimate argues for a roughly equal abundance of dwarf galaxies in all environments today. This simple picture is, however, modified by several additional effects. First, the recombination rate is higher in overdense regions at any given time, as discussed above. Furthermore, reionization in such regions is accomplished at an earlier time when the recombination rate was higher even at the mean cosmic density; therefore, more ionizing photons must be produced in order to compensate for the enhanced recombination rate. These two effects combine to make overdense regions reionize at a higher value of $F_{\rm col}$ than underdense regions. In addition, the overdense regions, which reionize first, subsequently send their extra ionizing photons into the surrounding underdense regions, causing the latter to reionize at an even lower $F_{\rm col}$. Thus, a higher abundance of dwarf galaxies today is indeed expected in the overdense regions.

The same basic effect may be even more critical for understanding the properties of large-scale voids, 10–30 Mpc regions in the present-day universe with an average mass density that is well below the cosmic mean. In order to predict their properties, the first step is to consider the abundance of dark matter halos within them. Numerical simulations show that voids contain a lower relative abundance of rare halos , as expected from the raising of the collapse threshold for halos within a void. On the other hand, simulations show that voids actually place a larger fraction of their dark matter content in dwarf halos of mass below $10^{10} M_{\odot}$ . This can be understood within the extended Press-Schechter model. At the present time, a typical region in the universe fills halos of mass $10^{12} M_{\odot}$ and higher with most of the dark matter, and very little is left over for isolated dwarf halos. Although a large number of dwarf halos may have formed at early times in such a region, the vast majority later merged with other halos, and by the present time they survive only as substructure inside much larger halos. In a void, on the other hand, large halos are rare even today, implying that most of the dwarf halos that formed early within a void can remain as isolated dwarf halos till the present. Thus, most isolated dwarf dark matter halos in the present universe should be found within large-scale voids .

However, voids are observed to be rather deficient in dwarf galaxies as well as in larger galaxies on the scale of the Milky Way . A deficit of large galaxies is naturally expected, since the total mass density in the void is unusually low, and the fraction of this already low density that assembles in large halos is further reduced relative to higher-density regions. The absence of dwarf galaxies is harder to understand, given the higher relative abundance expected for their host dark matter halos. The standard model for galaxy formation may be consistent with the observations if some of the dwarf halos are dark and do not host stars. Large numbers of dark dwarf halos may be produced by the effect of reionization in suppressing the infall of gas into these halos. Indeed, exactly the same factors considered above, in the discussion of dwarf galaxies in clusters compared to those in small groups, apply also to voids. Thus, the voids should reionize last, but since they are most strongly affected by ionizing photons from their surroundings (which have a higher density than the voids themselves), the voids should reionize when the abundance of galaxies within them is relatively low. A quantitative analysis of how the reionization redshift varies with environment may help establish a common framework for explaining the observed properties of dwarf galaxies in environments ranging from clusters to voids.

# Conclusions

We have shown that the important milestones of high-redshift galaxy formation, such as the formation of the first stars and the completion of reionization, occurred at significantly different times in different regions of the universe. This conclusion results from the fact that the temperature threshold, above which cooling and fragmentation of gas are possible, selects out dark matter halos that become exceptionally rare at high redshifts. Consequently, density fluctuations on large scales modulate the threshold for the collapse of high density peaks on small scales in the exponential tail of the Gaussian random field of density fluctuations, and introduce a remarkably large scatter in the abundance of star-forming galaxies at early cosmic times.

We have developed an improved method to calculate the cosmic scatter (see §2.2). This yields the first self-consistent analytic model that matches the halo mass function measured in various regions in numerical simulations that covered a wide range of the parameter space of region size, mean density, and redshift.

Since the characteristic distance between nearby sources of ionizing radiation varies widely across the universe, the overlap of the regions produced by these individual sources in the IGM occurs at significantly different times in different cosmic environments. Quantitatively, we find that the spread in the redshift of reionization should be at least an order of magnitude larger than previous expectations that argued for a sharp end to reionization (see § 3.1).

Current numerical simulations that treat gravity and hydrodynamics largely eliminate this real cosmic scatter, and are artificially biased toward late galaxy formation since they exclude large-scale modes (see Figures 3 and 4). We find that galaxy formation within state-of-the-art simulations with $324^3$ particles is artificially biased to occur too late by a redshift interval $\Delta z \sim 0.5$ at $z=7$ and $\Delta z \sim 2.5$ at $z=20$. The box length used in state-of-the-art simulations of reionization is 1.5–2 orders of magnitude below the minimum size necessary to treat the scatter reliably, and so alternative computational schemes must be implemented in order to quantify the implications of the large cosmic scatter on the reionization history. This scatter should affect the statistical fluctuations in the number and clustering properties of sources in surveys with a narrow field of view (such as the Hubble Deep Field), the luminosity function of Ly$\alpha$-emitting galaxies around the reionization redshift, the fluctuations in the 21 cm flux produced by the neutral IGM, the power spectrum of the secondary anisotropies in the cosmic microwave background, and the present abundance of dwarf galaxies in various environments (see § 3.3). Simulations limited to a small box may be able to study the scatter in the number density of galaxies by varying the mean density of the box, but such simulations cannot probe the global structure of reionization since this would involve the radiative transfer of ionizing photons over distances larger than the box size.

We thank Paul Steinhardt for suggesting to apply our work to galaxy formation in voids. We acknowledge support by NSF grant AST-0204514 and NATO grant PST.CLG.979414. R.B. is grateful for the kind hospitality of the Harvard-Smithsonian CfA and the Institute for Advanced Study, and the support of an Alon Fellowship at Tel Aviv University and of Israel Science Foundation grant 28/02/01. A.L. acknowledges sabbatical support from the John Simon Guggenheim Memorial Fellowship. This work was also supported in part by NSF grant AST-0071019 and NASA grant NAG 5-13292 (for A.L.).

[^1]:  simulated a 30 Mpc box, larger than previous simulations, but only gravity was directly simulated, and the mass resolution was three orders of magnitude lower than the minimum necessary to resolve most galactic halos at high redshift.


abstract: The movement of the eyes has been the subject of intensive research as a way to elucidate inner mechanisms of cognitive processes. A cognitive task that is rather frequent in our daily life is the visual search for hidden objects. Here we investigate through eye-tracking experiments the statistical properties associated with the search of target images embedded in a landscape of distractors. Specifically, our results show that the twofold process of eye movement, composed of sequences of fixations (small steps) intercalated by saccades (longer jumps), displays characteristic statistical signatures. While the saccadic jumps follow a log normal distribution of distances, which is typical of multiplicative processes, the lengths of the smaller steps in the fixation trajectories are consistent with a power-law distribution. Moreover, the present analysis reveals a clear transition between a directional serial search to an isotropic random movement as the difficulty level of the searching task is increased.
author: Heitor F. Credidio; Elisângela N. Teixeira; Saulo D. S. Reis; André A. Moreira; José S. Andrade Jr.
date: 2024-09-30
title: Statistical patterns of visual search for hidden objects

It is a common misconception to believe that memories are stored in the brain the same way a movie is stored in a hard drive. Remembering, just like seeing and listening, is in fact an act of construction much more complex than usually thought, where vasts amounts of information are processed and interpreted by the brain in order to create what we call memories, and pretty much everything else we call reality . The field dedicated to the study of these types of processes is called Cognitive Science, which took its current form in the first half of the 20th century out of a mishmash of sciences, including, among others, Psychology, Linguistics and Computer Science. More precisely, the main challenge of the Cognitive Science is to answer questions related to the way in which the brain processes available information and how this shapes behaviour. .

As theoretical entities, cognitive processes cannot be directly observed and measured . Thus, in order to be able to study them, we need to rely on observations about the behaviour of individuals. A very often utilized approach is to follow the eye movement during cognitive tasks. By the end of the XIX century, it was still thought that the eyes smoothly scanned the line of text during reading. Louis Émile Javal , in his unprecedented study of 1879, observed that the eyes actually move in a succession of steps, called fixations, followed by jerk-like movements, called saccades, that are too fast to capture new visual information . The method of eye-tracking as a fundamental source of information about cognition was finally introduced through the seminal work of Yarbus . This study provided unambiguous demonstration for the fact that the movement of the eyes is strongly correlated with the cognitive objectives of the individual.

A cognitive process that benefits the most from the study of eye movement is the visual search for hidden objects , like when trying to find a person in a crowded place, or a 2 inches nail inside a box of nails of various sizes. An early theory related to this process is due to Treismann and Gelade, called Feature Integration Theory (FIT) . This theory deals with attention, a kind of mental focus that can be directed to a desired region of the visual scene, therefore enhancing the perceptual sensitivity in that region. The FIT proposes that visual search tasks are divided into two stages. The first is a detection stage, in which a small set of simple separable features like color, size and orientation are identified in the elements inside the optical array. This stage is a preattentive one, that is, attention need not be directed at each element of the image in order to perform detection, all feature registration takes place in parallel across the whole visual scene. In the second stage, called integration, the features identified in the previous stage are combined in order to conceive more complex characteristics. This is an attentive stage, thus it is much slower, requiring the observer to scan each element of the image serially.

It is interesting to note that the FIT resembles a broader category of paradigms, namely *the dual process models* . Under this conceptual framework, complex cognitive tasks usually consist of two systems, that essentially differ in which Kahneman referred as (1) effortless intuition, and (2) deliberate reasoning. The system (1) comprises processes that are fast, intuitive and can be performed automatically and in parallel, like when trying to identify the state of spirit of a person based on her/his facial expression. These processes are acquired through habit, being usually inflexible and hard do control or modify. The system (2), on the other hand, is characterized by slow, serial but extremely controlled processes, which is the case, for example, when one tries to solve a mathematical equation.

The FIT was thoroughly studied and expanded during subsequent years . Although regarded, in its initial form, as an oversimplification , it surely represents a formidable conceptual starting point point for research on the subject. Two particular assumptions of the FIT, namely that the whole optical array is homogeneously analyzed and that attention can be displaced independently of the eye movements (thus being called *covert attention*), are of interest to be expanded upon . It is widely known that visual acuity falls rapidly from the point of fixation , being confined to a small region called fovea. While there is little doubt about the existence of covert attention , it has been argued that situations in which covert attention performs better than overt eye movements are unusual and restricted to laboratory tests . These led to further investigation about the function of eye movements in visual search .

Eye movements are composed of fixations and saccades, but even during fixations, the eyes are not completely still. In fact, fixational eye movements (FEyeM) include drift, tremor and microsacades. The drift corresponds to the erratic and low velocity component of FEyeM. The tremor is irregular and noise-like with very high frequency, while microsacades correspond to small rapid shifts in eye position akin to saccades, but preferentially taking place on horizontal and vertical directions . Whether or not each one of these movements play an effective role in visual cognition still represents a rather controversial issue , but it is known widespread that, if the FEyeM halt, visual perception stops completely. Previous attempts to model eye movements have been mainly devoted to describe the sequence of fixations and saccades in terms of stochastic processes like regular random walks . Very often, the gaze is considered as a random walker subjected to a potential extracted from a saliency map, namely a field that depends on the particular features of the image under inspection, such as color, intensity and orientation .

Recent research on visual cognition has been directed to the development of experimental and analytical methods that can potentially elucidate the interplay between different components of cognitive activities, and how their interactions give rise to cognitive performance . While the detection and integration processes mentioned before represent basic components of visual cognition that can be investigated separately, the way they interact should be relevant for the comprehension of more intricate visual tasks. Therefore, it is of paramount interest to determine if cognitive dynamics is dominated by components or interactions. Here we show through eye-tracking experiments that the cognitive task of visual search for hidden objects displays typical statistical signatures of interaction-dominated processes. Interestingly, by increasing the difficulty level of the visual task, our results also indicate that the eye movement changes from a serial reading-like (systematic) to an isotropic (random) searching strategy.

# Results

Visual search experiments have been performed with targets hidden in two different types of disordered substrate images (see *Methods* for details). In the first, as depicted in Fig. , the subjects were asked to search for a target (number *5*) in an image with distractors (numbers *2*) placed on a regular array. Figure  shows an example of the second type of test, where we utilized images from the book series "Where's Wally?" . These last can be considered as very complex images, since distractors are irregularly placed in an off-lattice configuration and specially drawn to closely resemble the target. The resulting image designed under these conditions frequently leads to a searching task of enhanced difficulty. The analysis of the results from the two tests enabled us to identify general statistical patterns as well as particular features in the eye movement that are related with the irregularity and complexity of the image adopted in the eye-tracking experiments.

In the case of the *5-2* lattice tests, the typical trajectories shown in Fig.  indicate that, when the number of distractors is small, most subjects performed systematic searches, that is, the task is accomplished in a manner that resembles a person reading a text, for example, from left to right and/or from top to bottom. By increasing the number of distractors, a transition can be observed from this directional (systematic) trajectory to an isotropic random strategy of searching for the large majority of the experiments. Precisely, systematic patterns have been observed in two thirds of the eye-tracking recordings (42 out of 63) for difficulty 0 (see Fig. a), half of the recordings (16 out of 32) for difficulty 1 (see Fig. b), and only one fourth of the recordings (6 out of 24) for difficulty 2 (see Fig. c). No discernible systematic searches were observed in the case of "Where's Wally?" tests.

Next, we analyze the size distributions of gaze jumps calculated for the raw data obtained from eye-tracking experiments. By definition, the size of a jump in this case corresponds to the distance, measured in number of pixels, covered by the eye gaze during each recording step of the eye-tracker device, adjusted here for approximately 17 milliseconds. Strikingly, as depicted in Figs. , all tests produced alike distributions of gaze jumps, regardless of the subjects, complexity of the tests, or the search strategy (regular or random). This universal shape reflects the fixation-saccade duality of the eye movement and clearly points to a superposition of behaviours instead of a description in terms of pure monomodal distributions .

The presence of two modes separated by a slight depression that marks the overlap region can be observed in practically all jump size distributions of the raw data. Such a behaviour strongly suggests the need for a filtering process through which fixations and saccades can be adequately identified and their statistical properties independently studied. With this purpose, here we apply a modified version of the fixation filter developed by Olsson , as described in the *Methods* section. As shown in Figs.  and , the resulting distributions of jump sizes for fixational movements obtained for *5-2* and "Where's Wally?" tests, respectively, also display the same statistical signature. Precisely, for gaze steps larger than $10\,px$, the distances $\Delta r$ follow typical power-law distributions, $$P(\Delta r) \propto \Delta r^{-\alpha},
    \label{eq:powerl}$$ with a statistically identical exponent, $\alpha \approx 2.9$, for all tests (see Table ). For gaze steps smaller than $10\,px$, the distributions display approximately uniform behaviour, possibly due to the fact that, in this scale, eye tremor is of the order of drift, although this hypothesis cannot be tested with the time resolution used in our measurements . Once identified through the filtering process, the analysis of the saccadic movements in all tests reveals that the distributions of sizes for this type of eye jump can be well described in terms of a log-normal distribution, $$P(\Delta r) = \frac{1}{\Delta r\sqrt{2\pi\sigma^2}}\exp\left[{-\frac{
    \left(\log \Delta r - \mu\right)^2}{2\sigma^2}}\right],
    \label{eq:lognormal}$$ where the parameters $\mu$ and $\sigma$ correspond to the average and variance of the logarithm of the saccade length, respectively. Once more, the fact that a single distribution function can properly describe the general statistical features of different searching tests suggests that same underling mechanisms control the cognitive task under investigation. It is interesting to note, however, that the numerical values of the estimated parameters of the distributions depend on the details of the test. For instance, in the case of *5-2* tests, the mode of the distribution (the most probable length) decreases systematically with the difficulty of the searching task, indicating that saccadic movements somehow adapt to the complexity of the image.

```latex
\begin{centering}
\begin{tabular}{cc|cc}
 & \multicolumn{1}{c}{Fixations} & \multicolumn{2}{c}{Saccades}\tabularnewline
\cline{2-4}
 & $\alpha$ & $\mu$ & $\sigma^{2}$\tabularnewline
\hline
\hline
Difficulty 0 & $2.951\pm0.054$ & $4.810\pm0.050$ & $0.675\pm0.415$\tabularnewline
Difficulty 1 & $2.825\pm0.020$ & $4.594\pm0.018$ & $0.727\pm0.168$\tabularnewline
Difficulty 2 & $2.938\pm0.019$ & $4.510\pm0.019$ & $0.830\pm0.173$\tabularnewline
Wally & $3.091\pm0.017$ & $4.444\pm0.012$ & $0.698\pm0.112$\tabularnewline
\end{tabular}
\par\end{centering}
```

# Discussion

In summary, our results from eye-tracking tests in which subjects are asked to find a specific target hidden among a set of distractors (see *Methods*) reveal a gradual change on the searching strategy, from a directional reading-like (systematic) to an isotropic (random) movement as the number of distractors increases. However, regardless of the differences in image complexity, searching tasks and individual skills of the subjects, we observe universal statistical features related with the distributions of gaze jump sizes. These distributions generally show a characteristic bimodal behaviour, consequence of the intrinsic dual nature of eye movement , that alternates between saccades and fixations.

The application of a fixation filter to the raw data enables us to study separately the distributions of jump sizes for fixational and saccadic gaze steps. We find that the distribution of fixational movements show long tails which obey power-laws , while saccades, on the other hand, follow a log-normal type of behaviour. The fact that both log-normal and power-law distributions arise from multiplicative processes provide strong support to the hypothesis that the interactions between components dominates the cognition task of visual search . In a dynamics governed by interactions, the organization of the components and the way they process information are context dependent, with no particular function being encapsulated in any of the components themselves. This non-linear response to the influx of information would give rise to multiplicative distributions like the ones we disclosed here.

These observations are in evident contrast with a component based scenario, where the final performance of a given cognitive task results from the simple addition of sub-tasks that usually process information in a specialized manner. Instead of log-normal or power-law distributions, a process like this would give rise to Gaussian or other additive distributions (e.g., exponential or gamma distributions) . It is worth noting that our results are conceptually consistent with previous studies describing complex behaviour in visual cognition . As a perspective for future work, it would be interesting to relate our findings with other potential approaches based on non-cognitive random strategies, where the searching task can be the result of an optimization process .

# Methods

## Equipment

Eye movements were recorded with a Tobii T120 eye-tracking system (Tobii Technology). In this study we only consider data obtained after a valid calibration protocol is applied to both eyes of the subject. The stimuli were presented on a 17" TFT-LCD monitor with resolution $1024\times1280$ pixels and capture rate of $60$ Hz.

## Tests

Two types of tests consisting of visual searching for a hidden target randomly placed among a set of distractors were performed by 11 healthy subjects with an average age of 23 years. The stimuli of the first test consists of a square lattice composed of a single target number *5* and several number *2*'s serving as distractors. All numbers (target and distractors) are randomly colored red or green, hindering the visual detection of the target through the identification of patterns on the peripheral vision. This images were organized in three difficulty levels according to the number of distractors, labeled 0, 1 and 2 for 207, 587 and 1399 distractors, respectively.

The stimuli of the second test are scanned images from the "Where's Wally?" series of books . The complexity of these images, where a large number of distractors (background characters) are irregularly placed together with Wally, the hidden target character, explains the high difficulty involved in this visual searching task. Not all images used had an actual target, since we had no intention to track the time taken to find the target. Instead, our objective was to induce the subjects to perform the searching task as naturally as possible.

In order to stimulate subjects to search efficiently, in all tests, they were told to have a limited time to find the target, but not informed exactly how much time would be available. In the case of the *5-2* lattice tests, $1$, $1.5$ and $2$ minutes were given to search the target for the difficulties 0, 1 and 2, respectively. For the "Where's Wally" tests, the subjects had 2 min. A summary of the parameters can be found in Table .

Parameters used in the 5-2 and "Where's Wally" search tasks.

Difficulty
 # of Images
 # of Distractors

Size of Distractors

Time Available

 16 × 13
 76 px
1 min

 33 × 26
 38 px
1 min 30 s

 40 × 35
 25 px
2 min

Wally

 –
 –
2 min

## Fixation Filter

We adopted a modified version of the fixation filter developed by Olsson in order to identify which gaze points belong to fixations and which belong to saccades. The basic idea is to distinguish between segments of the signal that are moving slowly due to drift, thus identified as part of a fixational sequence, from those moving faster, constituting the saccades. This is achieved here by taking the raw signal output, $\mathbf{s}_i$, namely the position of the gaze captured at each timestamp $i$, and calculating for each point the mean position of two sliding windows of size $r$, one retarded and the other advanced, $$\label{eq:mbefore}
    \mathbf{m}^{\pm}_i = \frac{1}{r}\sum^{r}_{k=1}\mathbf{s}_{i\pm k}.$$ The distance between them is calculated as, $$\label{eq:dist}
    d_i = \left|\mathbf{m}^{+}_i - \mathbf{m}^{-}_i\right|.$$ Since each timestamp has the same duration, the displacement given by Eq.  may be analyzed in the same way as the average velocity, thus if $d_i$ is larger than its two neighbors ($d_{i-1}$ and $d_{i+1}$), and is also larger than a given velocity threshold, it is considered a peak. If two peaks are found within the interval of a single window, only the largest one is considered.

At this stage, the gaze points are divided into clusters separated by the peaks. In the original filter , the median position of each cluster is used to locate the corresponding fixation. Since we are instead interested in separating the gaze points that correspond to fixations from those that belong to saccades, the radius of gyration for each cluster $C$ is then calculated as, $$\label{eq:gyrad}
    R_g = \sqrt{\frac{1}{N}\sum_{i\in C} \left|\mathbf{s}_i - \mathbf{\bar{s}}\right|^2},$$ where $\mathbf{\bar{s}}$ is the mean position of the gaze points that belong to $C$. Steps that fall inside the circle area covered by the radius of gyration, and are centered at $\mathbf{\bar{s}}$, are considered to be fixational. The same applies to those steps that leave this area but return to it without passing through another fixation cluster. All other steps are considered saccadic jumps.

# Acknowledgments

We thank the Brazilian Agencies CNPq, CAPES, FUNCAP and FINEP, the FUNCAP/CNPq Pronex grant, and the National Institute of Science and Technology for Complex Systems in Brazil for financial support.

# Author contributions

All authors contributed equally to the paper.

# Additional information

**Competing financial interests:** The authors declare no competing financial interests.


abstract:
  .
  With ever-increasing computational demand for deep learning, it is critical to investigate the implications of the numeric representation and precision of DNN model weights and activations on computational efficiency. In this work, we explore unconventional narrow-precision floating-point representations as it relates to inference accuracy and efficiency to steer the improved design of future DNN platforms. We show that inference using these custom numeric representations on production-grade DNNs, including GoogLeNet and VGG, achieves an average speedup of 7.6$\times{}$ with less than 1% degradation in inference accuracy relative to a state-of-the-art baseline platform representing the most sophisticated hardware using single-precision floating point. To facilitate the use of such customized precision, we also present a novel technique that drastically reduces the time required to derive the optimal precision configuration.
author: Parker Hill, Babak Zamirai, Shengshuo Lu, Yu-Wei Chao, Michael Laurenzano, Mehrzad Samadi
**Marios Papaefthymiou, Scott Mahlke, Thomas Wenisch, Jia Deng, Lingjia Tang, Jason Mars**
Department of Electrical Engineering and Computer Science
University of Michigan, Ann Arbor
`{parkerhh,zamirai,luss,ywchao,mlaurenz,firstname.lastname@example.com`
`{marios,mahlke,twenisch,jiadeng,lingjia,firstname.lastname@example.com`
bibliography: bib.bib
title: Rethinking Numerical Representations for Deep Neural Networks

# Introduction

Recently, deep neural networks (DNNs) have yielded state-of-the-art performance on a wide array of AI tasks, including image classification , speech recognition , and language understanding . In addition to algorithmic innovations , a key driver behind these successes are advances in computing infrastructure that enable large-scale deep learning—the training and inference of large DNN models on massive datasets . Indeed, highly efficient GPU implementations of DNNs played a key role in the first breakthrough of deep learning for image classification . Given the ever growing amount of data available for indexing, analysis, and training, and the increasing prevalence of ever-larger DNNs as key building blocks for AI applications, it is critical to design computing platforms to support faster, more resource-efficient DNN computation.

A set of core design decisions are common to the design of these infrastructures. One such critical choice is the numerical representation and precision used in the implementation of underlying storage and computation. Several recent works have investigated the numerical representation for DNNs . One recent work found that substantially lower precision can be used for training when the correct numerical rounding method is employed . Their work resulted in the design of a very energy-efficient DNN platform.

This work and other previous numerical representation studies for DNNs have either limited themselves to a small subset of the customized precision design space or drew conclusions using only small neural networks. For example, the work from Gupta et al. 2015 evaluates 16-bit fixed-point and wider computational precision on LeNet-5  and CIFARNET . The fixed-point representation (Figure ) is only one of many possible numeric representations. Exploring a limited customized precision design space inevitably results in designs lacking in energy efficiency and computational performance. Evaluating customized precision accuracy based on small neural networks requires the assumption that much larger, production-grade neural networks would operate comparably when subjected to the same customized precision.

In this work, we explore the accuracy-efficiency trade-off made available via specialized custom-precision hardware for inference and present a method to efficiently traverse this large design space to find an optimal design. Specifically, we evaluate the impact of a wide spectrum of customized precision settings for fixed-point and floating-point representations on accuracy and computational performance. We evaluate these customized precision configurations on large, state-of-the-art neural networks. By evaluating the full computational precision design space on a spectrum of these production-grade DNNs, we find that:

1.  Precision requirements do not generalize across all neural networks. This prompts designers of future DNN infrastructures to carefully consider the applications that will be executed on their platforms, contrary to works that design for large networks and evaluate accuracy on small networks .

2.  Many large-scale DNNs require considerably more precision for fixed-point arithmetic than previously found from small-scale evaluations . For example, we find that GoogLeNet requires on the order of 40 bits when implemented with fixed-point arithmetic, as opposed to less than 16 bits for LeNet-5.

3.  Floating-point representations are more efficient than fixed-point representations when selecting optimal precision settings. For example, a 17-bit floating-point representation is acceptable for GoogLeNet, while over 40 bits are required for the fixed-point representation – a more expensive computation than the standard single precision floating-point format. Current platform designers should reconsider the use of the floating-point representations for DNN computations instead of the commonly used fixed-point representations .

To make these conclusions on large-scale customized precision design readily actionable for DNN infrastructure designers, we propose and validate a novel technique to quickly search the large customized precision design space. This technique leverages the activations in the last layer to build a model to predict accuracy based on the insight that these activations effectively capture the propagation of numerical error from computation. Using this method on deployable DNNs, including GoogLeNet  and VGG , we find that using these recommendations to introduce customized precision into a DNN accelerator fabric results in an average speedup of 7.6$\times{}$ with less than 1% degradation in inference accuracy.

# Customized Precision Hardware

We begin with an overview of the available design choices in the representation of real numbers in binary and discuss how these choices impact hardware performance.

## Design Space

We consider three aspects of customized precision number representations. First, we contrast the high-level choice between fixed-point and floating-point representations. Fixed-point binary arithmetic is computationally identical to integer arithmetic, simply changing the interpretation of each bit position. Floating-point arithmetic, however, represents the sign, mantissa, and exponent of a real number separately. Floating-point calculations involve several steps absent in integer arithmetic. In particular, addition operations require aligning the mantissas of each operand. As a result, floating-point computation units are substantially larger, slower, and more complex than integer units.

In CPUs and GPUs, available sizes for both integers and floating-point calculations are fixed according to the data types supported by the hardware. Thus, the second aspect of precision customization we examine is to consider customizing the number of bits used in representing floating-point and fixed-point numbers. Third, we may vary the interpretation of fixed-point numbers and assignment of bits to the mantissa and exponent in a floating-point value.

## Customized Precision Types

In a fixed-point representation, we select the number of bits as well as the position of the radix point, which separates integer and fractional bits, as illustrated in Figure . A bit array, $x$, encoded in fixed point with the radix point at bit $l$ (counting from the right) represents the value $2^{-l} \sum_{i=0}^{N-1}2^i\cdot x_i$. In contrast to floating point, fixed-point representations with a particular number of bits have a fixed level of precision. By varying the position of the radix point, we change the representable range.

An example floating-point representation is depicted in Figure . As shown in the figure, there are three parameters to select when designing a floating-point representation: the bit-width of the mantissa, the bit-width of the exponent, and an exponent bias. The widths of the mantissa and exponent control precision and dynamic range, respectively. The exponent bias adjusts the offset of the exponent (which is itself represented as an unsigned integer) relative to zero to facilitate positive and negative exponents. Finally, an additional bit represents the sign. Thus, a floating-point format with $N_m$ mantissa bits, $N_e$ exponent bits, and a bias of $b$, encodes the value $2^{({\sum_{i=0}^{N_e-1}2^{i}\cdot{e_i}})-b}(1+\sum_{i=1}^{N_m}2^{-i}\cdot m_i)$, where $m$ and $e$ are the segments of a bit array representing the mantissa and exponent, respectively. Note that the leading bit of the mantissa is assumed to be $1$ and hence is not explicitly stored, eliminating redundant encodings of the same value. A single-precision value in the IEEE-754 standard (i.e. `float`) comprises 23 mantissa bits, 8 exponent bits, and a sign bit. IEEE-754 standardized floating-point formats include special encodings for specific values, such as zero and infinity.

Both fixed-point and floating-point representations have limitations in terms of the precision and the dynamic ranges available given particular representations, manifesting themselves computationally as rounding and saturation errors. These errors propagate through the deep neural network in a way that is difficult to estimate holistically, prompting experimentation on the DNN itself.

## Hardware Implications

The key hardware building block for implementing DNNs is the multiply-accumulate (MAC) operation. The MAC operation implements the sum-of-products operation that is fundamental to the activation of each neuron. We show a high-level hardware block diagram of a MAC unit in Figure  (a). Figure  (b) adds detail for the addition operation, the more complex of the two operations. As seen in the figure, floating-point addition operations involve a number of sub-components that compare exponents, align mantissas, perform the addition, and normalize the result. Nearly all of the sub-components of the MAC unit scale in speed, power, and area with the bit width.

Reducing the floating-point bit width improves hardware performance in two ways. First, reduced bit width makes a computation unit faster. Binary arithmetic computations involve chains of logic operations that typically grows at least logarithmically, and sometimes linearly (e.g., the propagation of carries in an addition, see Figure  (c)), in the number of bits. Reducing the bit width reduces the length of these chains, allowing the logic to operate at a higher clock frequency. Second, reduced bit width makes a computation unit smaller and require less energy, typically linearly in the number of bits. The circuit delay and area is shown in Figure  when the mantissa bit widths are varied. As shown in the figure, scaling the length of the mantissa provides substantial opportunity because it defines the size of the internal addition unit. Similar trends follow for bit-widths in other representations. When a unit is smaller, more replicas can fit within the same chip area and power budget, all of which can operate in parallel. Hence, for computations like those in DNNs, where ample parallelism is available, area reductions translate into proportional performance improvement.

This trend of bit width versus speed, power, and area is applicable to every computation unit in hardware DNN implementations. Thus, in designing hardware that uses customized representations there is a trade-off between accuracy on the one hand and power, area, and speed on the other. Our goal is to use precision that delivers sufficient accuracy while attaining large improvements in power, area, and speed over standard floating-point designs.

# Methodology

We describe the methodology we use to evaluate the customized precision design space, using image classification tasks of varying complexity as a proxy for computer vision applications. We evaluate DNN implementations using several metrics, classification accuracy, speedup, and energy savings relative to a baseline custom hardware design that uses single-precision floating-point representations. Using the results of this analysis, we propose and validate a search technique to efficiently determine the correct customized precision design point.

## Accuracy

We evaluate accuracy by modifying the Caffe  deep learning framework to perform calculations with arbitrary fixed-point and floating-point formats. We continue to store values as C `float`s in Caffe, but truncate the mantissa and exponent to the desired format after each arithmetic operation. Accuracy, using a set of test inputs disjoint from the training input set, is then measured by running the forward pass of a DNN model with the customized format and comparing the outputs with the ground truth. We use the standard accuracy metrics that accompany the dataset for each DNN. For MNIST (LeNet-5) and CIFAR-10 (CIFARNET) we use top-1 accuracy and for ImageNet (GoogLeNet, VGG, and AlexNet) we use top-5 accuracy. Top-1 accuracy denotes the percent of inputs that the DNN predicts correctly after a single prediction attempt, while top-5 accuracy represents the percent of inputs that DNN predicts correctly after five attempts.

## Efficiency

We quantify the efficiency advantages of customized floating-point representations by designing a floating-point MAC unit in each candidate precision and determining its silicon area and delay characteristics. We then report speedup and energy savings relative to a baseline custom hardware implementation of a DNN that uses standard single-precision floating-point computations. We design each variant of the MAC unit using Synopsys Design Compiler and Synopsys PrimeTime, industry standard ASIC design tools, targeting a commercial 28nm silicon manufacturing process. The tools report the power, delay, and area characteristics of each precision variant. As shown in Figure , we compute speedups and energy savings relative to the standardized IEEE-754 floating-point representation considering both the clock frequency advantage and improved parallelism due to area reduction of the narrower bit-width MAC units. This allows customized precision designs to yield a quadratic improvement in total system throughput.

## Efficient Customized Precision Search

To exploit the benefits of customized precision, a mechanism to select the correct configuration must be introduced. There are hundreds of designs among floating-point and fixed-point formats due to designs varying by the total bit width and the allocation of those bits. This spectrum of designs strains the ability to select an optimal configuration. A straightforward approach to select the customized precision design point is to exhaustively compute the accuracy of each design with a large number of neural network inputs. This strategy requires substantial computational resources that are proportional to the size of the network and variety of output classifications. We describe our technique that significantly reduces the time required to search for the correct configuration in order to facilitate the use of customized precision.

The key insight behind our search method is that customized precision impacts the underlying internal computation, which is hidden by evaluating only the NN final accuracy metric. Thus, instead of comparing the final accuracy generated by networks with different precision configurations, we compare the original NN activations to the customized precision activations. This circumvents the need to evaluate the large number of inputs required to produce representative neural network accuracy. Furthermore, instead of examining all of the activations, we only analyze the last layer, since the last layer captures the usable output from the neural network as well as the propagation of lost accuracy. Our method summarizes the differences between the last layer of two configurations by calculating the linear coefficient of determination between the last layer activations.

A method to translate the coefficient of determination to a more desirable metric, such as end-to-end inference accuracy, is necessary. We find that a linear model provides such a transformation. The customized precision setting with the highest speedup that meets a specified accuracy threshold is then selected. In order to account for slight inaccuracies in the model, inference accuracy for a subset of configurations is evaluated. If the configuration provided by the accuracy model results in insufficient accuracy, then an additional bit is added and the process repeats. Similarly, if the accuracy threshold is met, then a bit is removed from the customized precision format.

# Experiments

In this section, we evaluate five common neural networks spanning a range of sizes and depths in the context of customized precision hardware. We explore the trade-off between accuracy and efficiency when various customized precision representations are employed. Next, we address the sources of accuracy degradation when customized precision is utilized. Finally, we examine the characteristics of our customized precision search technique.

## Experimental Setup

We evaluate the accuracy of customized precision operations on five DNNs: GoogLeNet , VGG , AlexNet , CIFARNET , and LeNet-5 . The implementations and pre-trained weights for these DNNs were taken from Caffe . The three largest DNNs (GoogLeNet, VGG, and AlexNet) represent real-world workloads, while the two smaller DNNs (CIFARNET and LeNet-5) are the largest DNNs evaluated in prior work on customized precision. For each DNN, we use the canonical benchmark validation set: ImageNet for GoogLeNet, VGG, and AlexNet; CIFAR-10 for CIFARNET; MNIST for LeNet-5. We utilize the entire validation set for all experiments, except for GoogLeNet and VGG experiments involving the entire design space. In these cases we use a randomly-selected 1% of the validation set to make the experiments tractable.

## Accuracy versus Efficiency Trade-offs

To evaluate the benefits of customized precision hardware, we swept the design space for accuracy and performance characteristics. This performance-accuracy trade off is shown in Figure . This figure shows the DNN inference accuracy across the full input set versus the speedup for each of the five DNN benchmarks. The black star represents the IEEE 754 single precision representation (i.e. the original accuracy with 1$\times$ speedup), while the red circles and blue triangles represent the complete set of our customized precision floating-point and fixed-point representations, respectively.

For GoogLeNet, VGG, and AlexNet it is clear that the floating-point format is superior to the fixed-point format. In fact, the standard single precision floating-point format is faster than all fixed-point configurations that achieve above 40% accuracy. Although fixed-point computation is simpler and faster than floating-point computation when the number of bits is fixed, customized precision floating-point representations are more efficient because less bits are needed for similar accuracy.

By comparing the results across the five different networks in Figure , it is apparent that the size and structure of the network impacts the customized precision flexibility of the network. This insight suggests that hardware designers should carefully consider which neural network(s) they expect their device to execute as one of the fundamental steps in the design process. The impact of network size on accuracy is discussed in further detail in the following section.

The specific impact of bit assignments on performance and energy efficiency are illustrated in Figure . This figure shows the the speedup and energy improvements over the single precision floating-point representation as the number of allocated bits is varied. For the floating-point representations, the number of bits allocated for the mantissa (x-axis) and exponent (y-axis) are varied. For the fixed-point representations, the number of bits allocated for the integer (x-axis) and fraction (y-axis) are varied. We highlight a region in the plot deemed to have acceptable accuracy. In this case, we define acceptable accuracy to be 99% normalized AlexNet accuracy (i.e., no less than a 1% degradation in accuracy from the IEEE 754 single precision accuracy on classification in AlexNet).

The fastest and most energy efficient representation occurs at the bottom-left corner of the region with acceptable accuracy, since a minimal number of bits are used. The configuration with the highest performance that meets this requirement is a floating-point representation with 6 exponent bits and 7 mantissa bits, which yields a 7.2$\times$ speedup and a 3.4$\times$ savings in energy over the single precision IEEE 754 floating-point format. If a more stringent accuracy requirement is necessary, 0.3% accuracy degradation, the representation with one additional bit in the mantissa can be used, which achieves a 5.7$\times$ speedup and 3.0$\times$ energy savings.

## Sources of Accumulation Error

In order to understand how customized precision degrades DNN accuracy among numeric representations, we examine the impact of various reduced precision computations on a neuron. Figure  presents the serialized accumulation of neuron inputs in the third convolution layer of AlexNet. The x-axis represents the number of inputs that have been accumulated, while the y-axis represents the current value of the running sum. The black line represents the original DNN computation, a baseline for customized precision settings to match. We find two causes of error between the customized precision fixed-point and floating-point representations, saturation and excessive rounding.

In the fixed-point case (green line, representing 16 bits with the radix point in the center), the central cause of error is from saturation at the extreme values. The running sum exceeds 255, the maximum representable value in this representation, after 60 inputs are accumulated, as seen in the figure. After reaching saturation, the positive values are discarded and the final output is unpredictable. Although floating-point representations do not saturate as easily, the floating-point configuration with 10 mantissa bits and 4 exponent bits (orange line) saturates after accumulating 1128 inputs. Again, the lost information from saturation causes an unpredictable final output.

For the next case, the floating-point configuration with 2 bits and 14 bits for the mantissa and exponent (blue line), respectively, we find that the lack of precision for large values causes excessive rounding errors. As shown in the figure, after accumulating 120 inputs, this configuration's running sum exceeds 256, which limits the minimum adjustment in magnitude to 64 (the exponent normalizes the mantissa to 256, so the two mantissa bits represent 128 and 64). Finally, one of the customized precision types that has high performance and accuracy for AlexNet, 8 mantissa bits and 6 exponent bits (red line), is shown as well. This configuration almost perfectly matches the IEEE 754 floating-point configuration, as expected based on the final output accuracy.

The other main cause of accuracy loss is from values that are too small to be encoded as a non-zero value in the chosen customized precision configuration. These values, although not critical during addition, cause significant problems when multiplied with a large value, since the output should be encoded as a non-zero value in the specific precision setting. We found that the weighted input is minimally impacted, until the precision is reduced low enough for the weight to become zero.

While it may be intuitive based on these results to apply different customized precision settings to various stages of the neural network in order to mitigate the sudden loss in accuracy, the realizable gains of multi-precision configurations present significant challenges. The variability between units will cause certain units to be unused during specific layers of the neural network causing gains to diminish (e.g., 11-bit units are idle when 16-bit units are required for a particular layer). Also, the application specific hardware design is already an extensive process and multiple customized precision configurations increases the difficulty of the hardware design and verification process.

## Customized Precision Search

Now we evaluate our proposed customized precision search method. The goal of this method is to significantly reduce the required time to navigate the customized precision design space and still provide an optimal design choice in terms of speedup, limited by an accuracy constraint.

First, we present the linear correlation-accuracy model in Figure , which shows the relationship between the normalized accuracy of each setting in the design space and the correlation between its last layer activations compared to those of the original NN. This model, although built using all of the customized precision configurations from AlexNet, CIFARNET, and LeNet-5 neural networks, produces a good fit with a correlation of 0.96. It is important that the model matches across networks and precision design choices (e.g., floating point versus fixed point), since creating this model for each DNN, individually, requires as much time as exhaustive search.

To validate our search technique, Figure  presents the accuracy-speedup trade-off curves from our method compared to the ideal design points. We first obtain optimal results via exhaustive search. We present our search with a variable number of refinement iterations, where we evaluate the accuracy of the current design point and adjust the precision if necessary. To verify robustness, the accuracy models were generated using cross-validation where all configurations in the DNN being searched are excluded (e.g., we build the AlexNet model with LeNet and CIFARNET accuracy/correlation pairs). The prediction is made using only ten randomly selected inputs, a tiny subset compared that needed for classification accuracy, some of which are even incorrectly classified by the original neural network. Thus, the cost of prediction using the model is negligible.

We observe that, in all cases, the accuracy model combined with the evaluation of just two customized precision configurations provides the same result as the exhaustive search. Evaluating two designs out of 340 is 170$\times$ faster than exhaustively evaluating all designs. When only one configuration is evaluated instead of two (i.e. a further 50% reduction is search time), the selected customized precision setting never violates the target accuracy, but concedes a small amount of performance. Finally, we note that our search mechanism, without evaluating inference accuracy for any of the design points, provides a representative prediction of the optimal customized precision setting. Although occasionally violating the target accuracy (i.e. the cases where the speedup is higher than the exhaustive search), this prediction can be used to gauge the amenability of the NN to customized precision without investing any considerable amount of time in experimentation.

We present the final speedup produced by our search method in Figure  when the algorithm is configured for 99% target accuracy and to use two samples for refinement. In all cases, the chosen customized precision configuration meets the targeted accuracy constraint. In most cases, we find that the larger networks require more precision (DNNs are sorted from left to right in descending order based on size). VGG requires less precision than expected, but VGG also uses smaller convolution kernels than all of the other DNNs except LeNet-5.

# Related Work

To the best of our knowledge, our work is the first to examine the impact of numeric representations on the accuracy-efficiency trade-offs on large-scale, deployed DNNs with over half a million neurons (GoogLeNet, VGG, AlexNet), whereas prior work has only reported results on much smaller networks such as CIFARNET and LeNet-5 . Many of these works focused on fixed-point computation due to the fixed-point representation working well on small-scale neural networks. We find very different conclusions when considering production-ready DNNs.

Other recent works have looked at alternative neural network implementations such as spiking neural networks for more efficient hardware implementation . This is a very different computational model that requires redevelopment of standard DNNs, unlike our proposed methodologies. Other works have proposed several approaches to improve performance and reduce energy consumption of deep neural networks by taking advantage of the fact that DNNs usually contain redundancies .

# Conclusion

In this work, we introduced the importance of carefully considering customized precision when realizing neural networks. We show that using the IEEE 754 single precision floating point representation in hardware results in surrendering substantial performance. On the other hand, picking a configuration that has lower precision than optimal will result in severe accuracy loss. By reconsidering the representation from the ground up in designing custom precision hardware and using our search technique, we find an average speedup across deployable DNNs, including GoogLeNet and VGG, of 7.6$\times{}$ with less than 1% degradation in inference accuracy.