abstract: We present FIRE/`Gizmo` hydrodynamic zoom-in simulations of isolated dark matter halos, two each at the mass of classical dwarf galaxies ($M_{\rm{vir}}\simeq 10^{10} \rm \, M_{\odot}$) and ultra-faint galaxies ($M_{\rm{vir}}\simeq 10^9 \rm \, M_{\odot}$), and with two feedback implementations. The resulting central galaxies lie on an extrapolated abundance matching relation from $M_{\rm \star}\simeq 10^6$ to $10^4 \rm \, M_{\odot}$ without a break. Every host is filled with subhalos, many of which form stars. Our dwarfs with $M_{\rm \star}\simeq 10^6~\rm \, M_{\odot}$ each have $1-2$ well-resolved satellites with $M_{\rm \star}= 3-200 \times 10^3 \rm \, M_{\odot}$. Even our isolated ultra-faint galaxies have star-forming subhalos. If this is representative, dwarf galaxies throughout the universe should commonly host tiny satellite galaxies of their own. We combine our results with the ELVIS simulations to show that targeting $\sim 50~\rm \, kpc$ regions around nearby isolated dwarfs could increase the chances of discovering ultra-faint galaxies by $\sim 35\%$ compared to random halo pointings, and specifically identify the region around the Phoenix dwarf galaxy as a good potential target.
  .
  The well-resolved ultra-faint galaxies in our simulations ($M_{\rm \star}\simeq 3 - 30 \times 10^3~\rm \, M_{\odot}$) form within $M_{\rm{peak}}\simeq 0.5 - 3 \times 10^9 \rm \, M_{\odot}$ halos. Each has a uniformly ancient stellar population ($> 10~\rm \, Gyr$) owing to reionization-related quenching. More massive systems, in contrast, all have late-time star formation. Our results suggest that $M_{\rm{halo}}\simeq 5 \times 10^9 \rm \, M_{\odot}$ is a probable dividing line between halos hosting reionization "fossils" and those hosting dwarfs that can continue to form stars in isolation after reionization.
author: Coral Wheeler$^1$[^1], Jose Onorbe$^{2}$, James S. Bullock$^1$, Michael Boylan-Kolchin$^{3}$ Oliver D. Elbert$^{1}$, Shea Garrison-Kimmel$^{1}$, Philip F. Hopkins$^{4}$, Dusan Keres$^{5}$
$\!\!$ $^{1}$Center for Cosmology, Department of Physics and Astronomy, University of California, Irvine, CA 92697, USA
$\!\!$ $^{2}$Max-Planck-Institut fuer Astronomie, Koenigstuhl 17, 69117 Heidelberg, Germany
$\!\!$ $^{3}$Department of Astronomy and Joint Space-Science Institute, University of Maryland, College Park, MD 20742-2421, USA
$\!\!$ $^{4}$TAPIR, Mailcode 350-17, California Institute of Technology, Pasadena, CA 91125, USA
$\!\!$ $^{5}$Department of Physics, Center for Astrophysics and Space Sciences, University of California at San Diego, 9500 Gilman
Drive, La Jolla, CA 92093
title:  Sweating the small stuff: simulating dwarf galaxies, ultra-faint dwarf galaxies, and their own tiny satellites

# Introduction

If $\Lambda$CDM is correct, then all dark matter halos hosting galaxies – from those hosting dwarfs to those hosting giant clusters – should be filled with substructure . Dark-matter only simulations over a vast range of particle masses and physical scales show that substructure persists down to the resolution limit of $\Lambda$CDM simulations . The basic expectation is that the mass function of subhalos rises steadily to masses well below the molecular cooling limit of $M_{\rm{halo}}\sim 10^6 \rm \, M_{\odot}$ , with thousands of sites for potentially star-forming satellites.

Observations of our own Milky Way, on the other hand, have revealed the presence of only $\sim 30$ confirmed satellite galaxies , the faintest of which were all discovered by the Sloan Digital Sky Survey (SDSS) . However, the SDSS covers only a fraction of the sky and is incomplete to the most distant and faintest satellites of the Milky Way. Any solution to the mismatch between the predicted abundance of subhalos and the observed counts of satellite galaxies around our own Milky Way will likely involve the discovery of ultra-faint satellites in unprobed regions of the sky, at large distances, and with surface brightnesses low enough that they lie just outside current detection limits . Over the last several years, searches by PanSTARRS and VST ATLAS have failed to lend critical support to this idea – finding far fewer dwarf galaxy satellites than expected . However, the recent discovery of up to nine new ultra-faint satellites in the southern sky by the Dark Energy Survey , the detection of three more faint dwarf satellite candidates , and ongoing efforts to discover dwarfs at large distances provide an exciting glimpse into the near future and point to a much larger population of as-yet-undiscovered dwarf galaxies in the Local Volume.

Interestingly, most of the newest Milky Way satellite candidates were all discovered near the Large Magellanic Cloud (LMC). While this could be nothing more than a location-based selection effect (the LMC happens to lie next to the region probed by the year one data release of DES), show that several of the candidates could very well be satellites of the LMC, and their discovery highlights the potential for discovering ultra-faint satellites of other more massive dwarf satellites, or of isolated dwarf galaxies in the Local Group. The scale-free nature of the subhalo mass function in $\Lambda$CDM suggests that groups of subhalos should be common . Because low-mass halos form earlier, are denser, and fall into smaller hosts before larger ones , it is likely that satellites of satellites or of lower mass isolated halos may have survived longer than their counterparts that fell directly into the Milky Way . This suggests that one way to search for ultra-faint galaxies might be as satellites of known dwarf galaxies. Associations and pairs of satellite galaxies in the Local Group have been observed for some time , and several Milky Way satellites are suspected to host their own satellites .

Once a group of satellites falls into the Milky Way, however, tidal forces will eventually disassemble the group and wipe out evidence of coherent structure . Isolated massive dwarfs in the Local Group may serve as complementary targets in the hunt for ever-fainter dwarfs (which presumably probe the lowest mass dark matter subhalos). suggest that even already-merged satellites of isolated dwarfs could be detected as "ghost halos" of ancient stellar populations surrounding their hosts. Despite the obvious challenge of detecting ultra-faints at large distances, isolated dwarfs also make efficient use of telescope pointings, removing some of the chance inherent in any random ultra-faint search without losing any of the search volume. In the near future, large-area, deep-sky surveys such as LSST, DES, PanSTARRS, and SkyMapper will push current detection limits to lower surface brightness and may be able to see ultra-faint satellites of isolated dwarf galaxies if they exist .

Dynamical measurements of most Milky Way dwarf spheroidals (dSphs) show that, despite having luminosities that vary over nearly five orders of magnitude, they have almost the same central densities, comparable to dark matter halos with $M_{\rm{peak}}\sim 3 \times 10^9~\rm \, M_{\odot}$ . This is surprising, not only because this is well above the mass where physical processes are believed to prevent star formation, but because the mass function of potentially star-forming subhalos should be heavily populated by smaller halos. Segue 2 may be the first of these long-searched-for small subhalos to be identified , as it appears to have a total dark matter mass $\lesssim 10^8 \rm \, M_{\odot}$. However, based on its metallicity, hypothesize that Segue 2 is a bare remnant of a much larger galaxy that has been severely tidally stripped. The common central density for observed dSphs might signify a low-mass cutoff in galaxy formation. Halos with slightly lower central densities may have been unable to shield themselves from the reionizing background, perhaps due to having a lower baryon fractions than their denser counterparts . Alternatively, suggest that the apparent common mass scale is a selection effect, and that a large population of unobserved "stealth galaxies" may reside in halos with masses just below those that host observed dwarfs. These low-mass halos have shallow potential wells, and therefore the galaxies that form within them have larger effective radii and lower surface brightnesses, allowing them to more easily avoid detection .

The number of these stealth galaxies expected to exist is sensitive to the presence of a low-mass cutoff in galaxy formation, as the lowest mass halos are expected to host the "puffiest" galaxies in this picture. The heating of accreted gas or the prevention of gas accretion by the ambient ionizing UV background – created when the first galaxies formed – can arrest or prevent star formation in the smallest dark matter halos, suggesting a threshold in halo mass below which all halos remain dark . It is possible, however, that some very low mass halos were able to form some Population II stars *before* reionization . If star formation ceased after this time, it would cause them to have uniformly ancient stellar populations with extremely low metallicity . Remarkably, this appears to be the case with the known ultra faint dwarfs of the Milky Way .

Several authors have shown that photoheating prevents gas from condensing in halos with masses that lie just below or at the common halo mass scale for Milky Way satellites . However, the specific timing of the onset and end of reionization, the spectrum of the background radiation, the ability of gas to temporarily shield itself from these high energy photons, and particularly the mass resolution, can have a large effect on the number and minimum halo mass of galaxies in any simulation . Moreover, prescriptions for how stars and supernovae return energy back to the interstellar medium (ISM) will affect the formation history of small galaxies in the presence of an ionizing background.

In what follows, we present a series of simulations of galaxy formation within small dark matter halos run with the `Gizmo` code in "PSPH-mode" . `Gizmo` implements the Feedback in Realistic Environments (FIRE) feedback scheme for converting gas into stars and capturing the energy fed back from those stars into the surrounding medium. Using these simulations, we find that the subhalos of the halos that surround $M_{\rm \star}\sim 10^6~\rm \, M_{\odot}$ dwarf galaxies should form stars fairly abundantly, and produce potentially observable ultra-faint satellite galaxies of known classical dwarfs in the Local Group. Further, the ultra-faint dwarfs in our simulations with $M_{\rm \star}= 3-30 \times 10^3~\rm \, M_{\odot}$ that form in isolation or as satellites within subhalos of $M_{\rm{peak}}\simeq 0.5-3 \times 10^9 \rm \, M_{\odot}$ all have completely ancient stellar populations, as is seen for the known population of the Milky Way. They also have low-surface brightnesses, as expected for "stealth galaxies" forming in halos of these low masses, but may be within the detection capabilities of future surveys.

This paper is organized as follows: in Section , we give the details of our simulations, including halo finding and merger-tree analysis. Section outlines our main results, including the $\rm M_{\rm \star}- M_{halo}$ relation for our galaxies and their star formation histories. In Section , we use the Exploring the Local Volume in Simulations (ELVIS) suite of dark-matter-only Local Group simulations to determine how common these objects should be and the likelihood of their detection. We compare to several recent works in Section , and summarize our results and conclude in Section .

# Simulations

We present six cosmological zoom-in simulations of four isolated dwarf galaxy halos using two implementations of subgrid physics. The four host halos are $\rm Dwarf \, 1$ and $\rm Dwarf \, 2$, which have $z=0$ virial masses of $M_{\rm{vir}}\simeq 10^{10}~\rm \, M_{\odot}$; and $\rm UFD \, 1$ and $\rm UFD \, 2$, which have $M_{\rm{vir}}\simeq 10^{9}~\rm \, M_{\odot}$ (see Table 1 for precise numbers). The $\rm Dwarf \, 2$ halo was simulated three times – two runs varied the specific subgrid feedback implementation, while the third used a different choice of gravitational softening (see below), resulting in changes in the star formation histories of the main galaxies (see for details). We refer to these three cases as $\rm Dwarf \, 2_{Early}$ (run with the fiducial parameters – the same parameters as $\rm Dwarf \, 1$, $\rm UFD \, 1$, and $\rm UFD \, 2$), $\rm Dwarf \, 2_{Middle}$, and $\rm Dwarf \, 2_{Late}$. The subscripts refer to when the halos formed their stars. Every run has a dark matter force softening of $25$ to $35$ pc and a dark matter particle mass of $m_p^{\rm dm} = 1.26 \times 10^3 \rm \, M_{\odot}$ except $\rm UFD \, 2$, which uses $m_p^{\rm dm} = 2.46 \times 10^{3} \rm \, M_{\odot}$. The $\rm UFD \, 1$ and three $\rm Dwarf \, 2$ simulations are identical to the "Ultrafaint" and Dwarf runs presented by . [^2]

The initial conditions were generated using `MUSIC` , and selected from a cosmological box run at low resolution to $z=0$. $\rm Dwarf \, 1$, $\rm Dwarf \, 2$, and $\rm UFD \, 1$ were all selected from $5~h^{-1} \, \rm Mpc$ boxes to have typical values of spin parameter $\lambda$, concentration, and formation time for their mass range, and also to have small Lagrangian volumes . $\rm UFD \, 2$ was selected from a $25~h^{-1} \, \rm Mpc$ box and required to have no other halos of $50\%$ or more of its mass within $4~R_{\rm{vir}}$ at $z = 0$ and a small Lagrangian volume. All dwarfs in this work are isolated. Once the halos are identified, the particles are traced back in time and an enclosing Lagrangian volume is chosen and re-simulated at higher resolution with dark matter and gas, buffered by dark matter-only regions of increasing particle mass. This process is done according to the zoom-in techniques outlined by and with the goal of minimizing low resolution particles within the halos of interest. All simulations begin at a redshift of $z=125$ and have $0\%$ contamination from low resolution particles within a distance of $1.6~R_{\rm{vir}}$ at all redshifts.

All simulations were run using the fully conservative cosmological hydrodynamic code `Gizmo` in "PSPH-mode". This code adopts the Lagrangian "pressure-entropy" formulation of the smooth particle hydrodynamic (SPH) equations, ameliorating previous difficulties SPH codes had in modeling multiphase fluids .

For feedback and star formation we use the Feedback in Realistic Environments (FIRE) scheme . FIRE tracks momentum imparted locally from stellar radiation pressure, radiation pressure on larger scales via the light that escapes star-forming regions, HII photoionization heating, supernovae (Type I and II) heating, momentum and mass loss and stellar winds from O-type and AGB stars. Each star particle has an age determined by its formation time, an initial mass function (IMF) taken from , and a metallicity inherited from its parent gas particle. The star particle then loses mass and creates metals according to the `STARBURST99` stellar population synthesis model and the input IMF. Our simulations use an ambient ionizing UV background from , which starts at $z = 10.65$ and completes reionization by $z \simeq 6$.

The FIRE implementation has as few free parameters as possible, with star formation in the multiphase ISM naturally self-regulating. Giant molecular clouds (GMCs) form, and then are subsequently heated and disrupted by the stars that form within the self-gravitating, molecular gas. In order to ensure that the gas within the GMCs reaches a density high enough to form stars ($n > 100~{\rm cm}^{-3}$), the code requires extremely high mass and spatial resolution. Every run uses a gas particle mass of $m^{\rm gas}_{p}  = 255~\rm \, M_{\odot}$ except for $\rm UFD \, 2$, which uses $m^{\rm gas}_{p} = 499~\rm \, M_{\odot}$. The gas force resolution varies from $\rm \epsilon^{min}_{gas} = 1.0 - 2.8~\rm \, pc$ in all runs except one version of $\rm Dwarf \, 2$ (see below).

The $\rm Dwarf \, 2_{Early}$ run uses the fiducial subgrid parameters. In $\rm Dwarf \, 2_{Middle}$ and $\rm Dwarf \, 2_{Late}$, mass, momentum, and energy are deposited to particles within the SPH kernel according to a mass-weighting scheme, rather than volume-weighting as in the fiducial runs. In addition to this change, $\rm Dwarf \, 2_{Middle}$ was run to test equal softening lengths for the dark matter and the gas (25 pc). All three runs of $\rm Dwarf \, 2$ end up with stellar masses in the central galaxies that differ by only $\sim 25\%$, suggesting that the major differences between the runs are more a result of stochasticity than the changes to the feedback implementation and softening length (see Section ).

We use the Amiga Halo Finder to identify gravitationally bound dark matter, stellar and gas particles in each snapshot of the simulation. Because pure particle-matching algorithms generally underperform relative to more sophisticated methods in terms of detection and removal of spurious interlopers between snapshots , we track particle overdensities between simulation snapshots using the `consistent-trees` software . This is particularly important given our focus on subhalos. Because the current stable version of `consistent-trees` – which makes use of the `ROCKSTAR` halo finder – does not track baryonic particles, we build a pipeline between `AHF` and `ROCKSTAR` that allows for a seamless transition between the `AHF` halo catalogs and the merger trees. This pipeline uses the halo phase space information combined with the dark matter particle IDs to match each halo in the catalogue to a `consistent-trees` merger tree. `ROCKSTAR` and `consistent-trees` were used to determine the peak virial mass and infall times for subhalos. All other halo, galaxy and stellar quantities were obtained from the `AHF` halo catalogs or the raw particle files.

All runs were initialized and simulated assuming the WMAP-7 cosmology $\sigma_8 = 0.801$, $\Omega_{\Lambda} = 0.734$, $\Omega_m = 0.266$, $\Omega_b = 0.0449$, $n_s = 0.963$ and $h = 0.71$ . Further detail on the FIRE/`Gizmo` code can be found in .

```latex
\begin{table*}
\centering
\begin{tabular}{| l  | c | c c c | c  | c | }
\hline
   & $\rm Dwarf \, 1$ & $\rm Dwarf \, 2_{Early}$ & $\rm Dwarf \, 2_{Middle}$  & $\rm Dwarf \, 2_{Late}$ & $\rm UFD \, 1$   & $\rm UFD \, 2$   \\
\hline\hline
% & & & & & &   \\
{\hspace{.2in}} Central & & & & & & \\
\hline
$M_{\rm{vir}}\, (10^9 \rm \, M_{\odot})$  & 11.1 & 7.8 & 7.7 & 7.6 & 2.5  & 1.1 \\
$M_{\rm \star}\, (10^3 \rm \, M_{\odot})$  & 620 & 2200 & 2700 & 2800 & 22 & 8.5  \\
\hline
% & & & & & & \\
{\hspace{.1in}} 1st Satellite & & & & & &  \\
\hline
$M_{\rm{peak}}\, (10^9 \rm \, M_{\odot})$  & 4.7 & 0.70 & 0.68  & 0.64 &  0.03 & 0.02 \\
$M_{\rm \star}\, (10^3 \rm \, M_{\odot})$  & 220 & 4.2 & 5.4 & 2.9 & (0.9) & (0.7) \\
\hline
% & & & & & & \\
{\hspace{.1in}} 2nd Satellite & & & & &  & \\
\hline
$M_{\rm{peak}}\, (10^9 \rm \, M_{\odot})$  & 0.51 &  0.04 & 0.04 & 0.04 &  0.02 & 0.007 \\
$M_{\rm \star}\, (10^3 \rm \, M_{\odot})$  & 5.0 & (0.7) & (3.5) & (0.6) & (0.6)  & (0.4) \\
\hline
\end{tabular}
\label{tab:tablebig}
\caption{Halo and stellar masses of all central galaxies and their 1st and 2nd Satellites from all six simulations. For all simulations except $\rm Dwarf \, 2_{Early}$ and $\rm Dwarf \, 2_{Late}$, the 1st and 2nd Satellites refer to the first and second most massive satellites in stellar mass. The 2nd Satellites for $\rm Dwarf \, 2_{Early}$ and $\rm Dwarf \, 2_{Late}$ are selected because their counterpart within $\rm Dwarf \, 2_{Middle}$ has $M_{\rm \star}> 3 \times 10^3~\rm \, M_{\odot}$. Both satellites in $\rm Dwarf \, 1$ and the 1st Satellites in all $\rm Dwarf \, 2$ runs are ``massive satellites" (see text). The stellar masses of all other satellites are shown in parentheses.}
\end{table*}
```

## Sample Details

Figure shows the dark matter distributions of all six runs in greyscale with the stellar mass density overlaid in color. The physical scales of the panels are identical: the white line in the lower left corner of each visualization indicates $5~\rm \, kpc$.

For our detailed analysis, we focus on objects with $M_{\rm \star}> 3 \times 10^3~\rm \, M_{\odot}$ in order to avoid effects from spurious star formation at our resolution limit. The gas density required to form stars in our simulations is quite high and can only occur in dense, molecular, self-gravitating regions. This onerous minimal criteria suggests that the formation of this minimum stellar mass represents a physical star formation episode. [^3] Furthermore, we find that below a subhalo mass of $M_{\rm{peak}}\sim 5 \times 10^8~\rm \, M_{\odot}$ our results become far less stable against the stochasticity in star formation (see below). We therefore consider only as robust those objects that form within halos that have a minimum dark matter particle number $N^{\rm dm}_{p} \geq 2 \times 10^5$ and refer to satellites that meet both this requirement and the stellar mass cut as "massive satellites".

$\rm Dwarf \, 1$ has two satellite galaxies that meet both of these requirements. Only one subhalo – the most massive in each of the runs of $\rm Dwarf \, 2$ – forms a "massive satellite". In $\rm Dwarf \, 2_{Middle}$ there is a second satellite that meets the stellar mass but not the dark matter particle number cut. Because all three runs of $\rm Dwarf \, 2$ use identical initial conditions, each subhalo in $\rm Dwarf \, 2_{Middle}$ has a corresponding subhalo in the other runs that shares a majority of the dark matter particles. The subhalos corresponding to this second satellite host galaxies below the stellar mass cut in $\rm Dwarf \, 2_{Early}$ and $\rm Dwarf \, 2_{Late}$. In Figure , stars are shown in this particular subhalo for all three runs of $\rm Dwarf \, 2$. Importantly, although the three subhalos that host the second satellites of $\rm Dwarf \, 2$ are effectively identical, they produce galaxies that differ by a factor of $\sim 6$ in stellar mass due to only minor changes in parameters between the runs. This motivates the fairly large dark matter particle number we require to consider a system well resolved.

$M_{\rm{halo}}$ and $M_{\rm \star}$ for the central galaxy (Central), most-massive satellite (1st Satellite), and second-most-massive satellite (2nd Satellite) [^4] in all six runs can be found in Table 1. In the Tables, Figures and throughout the text, $M_{\rm{halo}}$ is $M_{\rm{vir}}$ for centrals and $M_{\rm{peak}}$ for satellites. Values in parentheses indicate satellite systems that we regard as too poorly resolved to trust for detailed analysis. They are included here for completeness. The 2nd Satellites of all $\rm Dwarf \, 2$ runs fall into this category, as well as all satellites of the isolated ultra-faints $\rm UFD \, 1$ and $\rm UFD \, 2$.

In Figure , the dark matter distribution is shown for all dark matter particles located within $65\%$ of the virial radius of the halo listed in each panel. Star particles are only shown if they reside within any halo that currently has $> 3 \times 10^3~\rm \, M_{\odot}$ in stars in the inner $3~(1.5)~\rm \, kpc$ of a central (satellite) galaxy or are in a corresponding halo in the other two runs of $\rm Dwarf \, 2$. Many of the other subhalos also contain star particles (see Figure 2), but they are not shown so that we may focus on systems that are reasonably well resolved. The radial extent of the galaxies was chosen by inspecting their stellar mass profiles (each central galaxy also has a *very* diffuse stellar halo that we aim to avoid). The galaxy stellar masses are all computed using these radial limits.

# Results

In all six runs, the dark matter halos hosting the central galaxies contain significant substructure. The leftmost panel of Figure shows the current subhalo mass function for all six runs. We plot all subhalos with $M^{z=0}_{\rm halo} > 5 \times 10^5~\rm \, M_{\odot}$, which corresponds to $N^{\rm dm}_{p} \gtrsim 400$. $\rm Dwarf \, 1$, with the highest virial mass, has both the most and the most massive subhalos. The subhalo mass functions for the three runs of $\rm Dwarf \, 2$ lie predictably on top of each other. As expected, the two isolated "ultra-faints" ($M_{\rm{vir}}< 3 \times 10^9~\rm \, M_{\odot}$) form far fewer subhalos (at fixed mass) than their more massive counterparts.

The satellite stellar mass functions are shown in the middle panel of Figure . They are intriguingly steep, rising to as many as fifty tiny satellites in the case of $\rm Dwarf \, 1$. However, the majority of these star-forming satellites form only a single star particle, and this potentially exciting behavior will need to be confirmed at much higher resolution. In order to make this clear, the dark green shaded region shows the average stellar mass of a single star particle, while the light green shaded region shows the mass range for all other objects that form with $M_{\rm \star}< 3 \times 10^3~\rm \, M_{\odot}$ (fewer than 16 star particles). In all six simulations, a total of $187$ subhalos form at least one star particle, but a mere six of them form galaxies with $M_{\rm \star}> 3 \times 10^3~\rm \, M_{\odot}$ (and only five of these also meet the $N^{\rm dm}_{p}$ requirement). It is these, the most massive satellites, that will be the focus of our analysis (along with the centrals themselves).

The $M_{\rm \star}- M_{\rm{halo}}$ relation for the five resulting "massive satellites" (filled triangles), their less-well-resolved counterparts (open triangles), and all central galaxies (filled circles), is shown in the right panel of Figure . Plotted are the six central galaxies along with the 1st and 2nd Satellites of each host (detailed in Table 1) for each run. Although 18 symbols are shown, we only regard the 11 filled points as reasonably well-resolved. All of the well-resolved systems in this work lie only slightly offset from the extrapolated abundance matching relationship from , which has been constrained down to a stellar mass of $\sim 10^5 \rm \, M_{\odot}$. This abundance matching relation, obtained by updating the relation with a faint-end slope derived from the measurements of the GAMA survey , accurately reproduces the stellar mass functions of the Milky Way and Andromeda satellites, as well as the stellar mass function of the Local Field, when applied to the ELVIS simulations .

We do not find the "bending" of the $\rm M_{\rm \star}- M_{halo}$ relation reported by at $M_{\rm{halo}}\simeq 5 \times 10^9~\rm \, M_{\odot}$. Instead, the relation maintains a fairly steady power-law down to $M_{\rm{halo}}\simeq 10^9 \rm \, M_{\odot}$. We do see a qualitatively similar break to that seen by but at a significantly lower halo mass of $M_{\rm{halo}}\simeq 5 \times 10^8~\rm \, M_{\odot}$. Below this point it is clear that we are witnessing a large amount of stochasticity, the onset of which corresponds to the scale of extremely faint galaxies ($M_{\rm \star}\sim 10^3~\rm \, M_{\odot}$), as opposed to the results of , where stochasticity sets in near the masses of classical dSphs ($M_{\rm \star}= 10^5~\rm \, M_{\odot}$). However, it is in this regime where our resolution is severely limited – these galaxies have $\lesssim 15$ star particles. At this time we are unable to determine whether the apparent break and associated stochasticity in our relation at small mass are physical or simply artifacts of resolution, although it is intriguing that the break witnessed by occurs at the mass of $\sim 10$ of their baryonic particles as well. Future work at higher resolution will allow us to explore this question more fully.

The most massive satellite of $\rm Dwarf \, 1$ represents an interesting statistical rarity (see Section ) in that the mass of the subhalo hosting this satellite is just over $\rm 40\%$ of the mass of its host, and the satellite itself has $1/3$ of the stellar mass of its central galaxy. $\rm Dwarf \, 1$ is clearly undergoing a major merger (see Figure ) and the associated large satellite has properties that are distinct from the other satellites described in this work. Most notably, its stellar mass is two orders of magnitude greater than the other "massive satellites", and it's stellar and dark matter masses are both larger than those of the central hosts of $\rm UFD \, 1$ and $\rm UFD \, 2$. For identification only (with no relation to the structural properties of the satellite), we refer when necessary to this most massive satellite as "dSph", and all other satellites as "ultra-faint" satellites. The broader term "ultra-faints" is reserved for all galaxies, centrals and satellites, that have $M_{\rm \star}= 3-30 \times 10^3~\rm \, M_{\odot}$ in our simulations.

## Star Formation Histories

The top panel of Figure shows the cumulative fractional star formation histories (SFH) for $\rm Dwarf \, 1$ (solid line) and its two "massive satellites". The SFH of its large, "dSph" satellite ($M_{\rm \star}^{z=0} \simeq 2 \times 10^5 \rm \, M_{\odot}$) is shown by the dotted line. Its second, "ultra-faint" satellite ($M_{\rm \star}^{z=0} \simeq 5 \times 10^3 \rm \, M_{\odot}$) is shown by the dashed line. The shaded band corresponds to the epoch of reionization in our simulations. The bottom panel presents similar histories for the three $\rm Dwarf \, 2$ runs along with the single most massive "ultra-faint" satellite ($M_{\rm \star}^{z=0}  \simeq 3-5 \times 10^3 \rm \, M_{\odot}$) that forms in each run (dashed lines). In both panels, the short vertical lines at the top mark the first virial crossing of the satellites.

The massive "dSph" satellite of $\rm Dwarf \, 1$ demonstrates a particularly interesting SFH. It stops forming stars shortly after the epoch of reionization for almost $\rm 8~\rm \, Gyr$ (from $z \sim 3$ to $z \sim 0.25$). However, it manages to keep some of its gas on hand in a state that is just below the threshold for star formation. Upon infall into the virial radius of $\rm Dwarf \, 1$ at a lookback time of $\rm \sim 2~\rm \, Gyr$, followed by pericentric passage $\sim 500~\rm \, Myr$ later, its ISM is compressed enough to create a new burst of stars, all of which are concentrated at the center of the satellite and were formed in situ. The central galaxy of $\rm Dwarf \, 1$ itself also appears to experience a related (albeit mild) burst. This type of bimodal star formation history was suggested by as a signature of massive reionization "fossils". Our "dSph" satellite has a slightly higher $V_{\rm max}$ than the range given in that work, and its rebirth of star-formation was triggered by a merger rather than by a change in its halo concentration (as originally suggested by ); it is nevertheless similar in its qualitative nature.

In our simulations, halos with $M \simeq 10^{10} \rm \, M_{\odot}$ appear to robustly maintain star formation to $z=0$. This massive satellite at $M_{\rm{peak}}\simeq 5 \times 10^9 \rm \, M_{\odot}$ seems to mark a transition point where reionization begins to favor uniformly ancient star formation. The influence of its impending merger was enough to trigger a rebirth. The "ultra-faint" satellites shown in Figure all form in halos smaller than this, with $M_{\rm{peak}}\lesssim 2 \times 10^{9} \rm \, M_{\odot}$. They all form their stars entirely before $z \sim 2$ and are quenched over $5~\rm \, Gyr$ before their infall onto the larger hosts, indicating that their star formation is shut down by reionization rather than environmental processes. Importantly, these subsequent mergers do not trigger fresh star formation, as was seen in the more massive satellite. These small halos are either quenched during reionization or shortly thereafter, running out of fuel after the fresh gas supply was shut off by the ambient UV field.

This tendency for our ultra-faint galaxies in $M_{\rm{peak}}\lesssim 2 \times 10^{9} \rm \, M_{\odot}$ halos to quench early is illustrated more clearly in Figure , where we have plotted the cumulative fractional star formation histories for all seven "ultra-faints" formed in our simulations, e.g. all "ultra-faint" satellites from Figure (dashed lines), along with $\rm UFD \, 1$ and $\rm UFD \, 2$ (solid lines). We also include the unresolved 2nd Satellite of $\rm Dwarf \, 2_{Middle}$ (dotted line) due to it's high stellar mass and to illustrate its similar SFH. From Figure it is clear that, in every case, reionization plays a significant role in the shutting down of star formation in our "ultra-faints". Not only were $100\%$ of all stars in all seven of these objects formed before $z=2$, four of the objects form over $\rm 90\%$ of their stars before the completion of reionization at $z=6$.

# Detecting Satellites of Dwarfs

## How Common are Satellites of Dwarfs?

It is beyond the scope of this paper to investigate a statistically significant sample of hydrodynamic simulations of $M_{\rm{vir}}\simeq 10^{10}~\rm \, M_{\odot}$ dwarfs in order to estimate the frequency with which they will host satellites above a given mass. However, if we assume an $M_{\rm \star}$ - $M_{\rm{peak}}$ relation similar to that presented in Figure 2, we can make an estimate using dark-matter-only simulations. We do so using the Exploring the Local Volume in Simulations (ELVIS) suite of collisionless zoom-in simulations of Local Group-like environments .

We select all isolated ($d_{\rm halo} > 2~R_{\rm{vir}}^{\rm halo}$ for all more massive halos) dwarf-size dark matter halos ($35~{\rm km \, s}^{-1}< V_{\rm max}< 45~{\rm km \, s}^{-1}$) in the 12 Local Group-like pairs and the 24 isolated ELVIS simulations, and determine the fraction of those hosts that have $N$ subhalos with $M_{\rm{peak}}\geq 5 \times 10^8~\rm \, M_{\odot}$. This is the halo mass that corresponds to $M_{\rm \star}> 3000 \rm \, M_{\odot}$ according to Figure 2. [^5] We similarly compute the fraction of dwarf halos that have a very large subhalo with $M_{\rm{peak}}\geq 4.5 \times 10^9~\rm \, M_{\odot}$, set by the most massive satellite of $\rm Dwarf \, 1~(M_{\rm \star}\gtrsim 2 \times 10^5 \rm \, M_{\odot}$).

Figure shows the probability for an isolated dwarf halo in ELVIS to have $\rm \geq N$ satellites of at least these two peak virial masses within a projected distance of $50~\rm \, kpc$ (the typical virial radius for our dwarfs) as a function of $\rm N$. According to Figure , isolated halos with $M_{\rm{vir}}\sim 10^{10}~\rm \, M_{\odot}$ will have one or more subhalos that could host $M_{\rm \star}\gtrsim 3000 \rm \, M_{\odot}$ satellites about $\rm 35\%$ of the time. The likelihood that an isolated dwarf in the same mass range hosts a satellite as massive $M_{\rm \star}\simeq 2 \times 10^5 \rm \, M_{\odot}$, however, is just under $5\%$.

Using both SDSS and the semi-analytic models of applied to the MS-II simulation , show that the probability for a central galaxy to host a satellite a given fraction of its stellar mass, $M_{\rm \star}^{\rm sat} / M_{\rm \star}^{\rm cen}$, decreases as a strong function of $M_{\rm \star}^{\rm cen}$ until $M_{\rm \star}^{\rm cen} = 10^{10}~\rm \, M_{\odot}$, becoming independent of $M_{\rm \star}^{\rm cen}$ for centrals below the stellar mass of the Milky Way. Although the stellar masses of the central galaxies in our simulations lie far below their stellar mass range ($10^{7.5} \leq M_{\rm \star}/ \rm \, M_{\odot}\leq 10^{11}$), this decoupling of the probability from $M_{\rm \star}^{\rm cen}$ for satellites of dwarf galaxies allows us to make a comparison with the probabilities they find. Their Figure 2 suggests that for all central galaxies with $M_{\rm \star}^{\rm cen} \leq 10^{10}~\rm \, M_{\odot}$, there is a $\sim 40\% - 50\%$ chance that they host a satellite with stellar mass $M_{\rm \star}^{\rm sat} / M_{\rm \star}^{\rm cen} \sim 0.5\%$ – similar to our "ultra-faints". This is roughly consistent with the $\sim \rm 35\%$ probability we find. They also find that the likelihood for a satellite as massive in proportion to its central as "dSph" ($M_{\rm \star}^{\rm sat} / M_{\rm \star}^{\rm cen} \sim 1/3$), is vanishingly small. Again, according to their Figure 2 and assuming that the lack of dependence of this probability on $M_{\rm \star}^{\rm cen}$ for dwarfs, there is only about a $\rm 1-2\%$ chance for a central to have such a massive satellite, which is again roughly consistent with the results shown in Figure .

## Can they be detected?

Our ultra-faint galaxies with $M_{\rm \star}\simeq 3-30 \times 10^3~\rm \, M_{\odot}$ all reside within low mass halos ($M_{\rm{vir}}< 3 \times 10^9~\rm \, M_{\odot}$). They also have extremely diffuse stellar distributions, as expected for the ultra-faint, low dark-matter mass, "stealth galaxies" discussed in B10. Figure shows the projected (2D) half stellar-mass radii, $R_{1/2}$, of our simulated galaxies vs their total stellar mass. The central galaxies are shown as colored circles and the satellites as triangles (the single unresolved satellite with $M_{\rm \star}> 3 \times 10^3~\rm \, M_{\odot}$ is shown as an open triangle). The open black circles show observed Milky Way dwarfs . The black points with error bars are the half-mass radii and stellar masses of the eight recently reported ultra-faint satellite candidates from , and the single blue square with error bars is Hydra II from . The surface brightness detection limit for SDSS is shown as a solid black line in the Figure. It represents a constant peak central surface brightness for a Plummer profile $$\Sigma_{\rm peak} = \frac{L}{\pi R_{1/2}^2}0.036~L_{\odot}\rm pc^{-2}$$ and corresponds to a surface brightness of $\rm \mu_V = 30~mag~arcsec^{-2}$ for solar absolute magnitude $\rm M_{\odot,V} = 4.83$ and assuming a stellar mass-to-light ratio of $M_{\rm \star}/L = 1 (\rm \, M_{\odot}/L_{\odot})$.

Except for $\rm UFD \, 1$, all of our simulated "ultra-faint" dwarfs have surface brightnesses fainter than $\rm 30~mag~arcsec^{-2}$, and would qualify as "stealth galaxies". [^6] The dashed line in Figure shows a surface brightness of $\rm 32.5~mag~arcsec^{-2}$, a limit that will likely be achieved by upcoming surveys such as the Large Synoptic Survey Telescope (LSST). Once the full co-added LSST data is collected, satellites this faint should be able to detected out to $\rm \sim 1~\rm \, Mpc$ over half the sky .

In the more immediate future, the Dark Energy Survey (DES) is already online. DES will only be able to detect ultra-faint objects of this kind out to about $400~\rm \, kpc$, and is only slated to cover $\rm \sim 12\%$ of the sky over its first 5 years. However, DES contains within its footprint the dwarf galaxy Phoenix. Phoenix has a stellar mass of $M_{\rm \star}= 7.7 \times 10^5~\rm \, M_{\odot}$ and is about $415~\rm \, kpc$ away. It is falling into the Milky Way with a velocity of $\rm \sim -100~{\rm km \, s}^{-1}$ and is likely on first approach, meaning that it is a good analog to the isolated systems simulated in this work. This suggests that Phoenix may be an excellent dwarf galaxy candidate to host an ultra-faint satellite detectable in the immediate future. It also has an advantage over more distant dwarfs like Cetus or Aquarius in that it is close enough that low luminosity satellites such as the "ultra-faints" predicted in this work have a greater chance of being visible.

Although, according to Figure , there is only a $\rm \sim 35\%$ chance that Phoenix will host a subhalo massive enough to form a $\sim 3000 \rm \, M_{\odot}$ satellite, once the volume that lies between us and Phoenix has been factored in, we find that the probability for finding an ultrafaint in its field increases to anywhere between $\rm 50\%-65\%$ according to the ELVIS simulations and depending on the specifics of abundance matching for $M_{\rm \star}\sim 3000 \rm \, M_{\odot}$ satellites. In general, for future telescopes that have a limited number of pointings available, targeting the $\rm \sim 50~\rm \, kpc$ region around isolated dwarf galaxies should prove to be a much more efficient strategy than pointing into blank sky, as it should increase the chances of observing an ultra-faint satellite by $\sim 35\%$.

# Comparison to Previous work

run a series of 12 zoom-in simulations of Local Group-like environments at three levels of resolution, both with baryons and with dark matter only. They run each simulation with and without a cosmic UV background that turns on sharply at $z = 11.5$, and argue that the onset of the ionizing background radiation sets the mass scale at which all halos become dark. Their simulations shut down star formation in over $80\%$ of halos with present-day virial masses less than $10^9~\rm \, M_{\odot}$. Their Figure 2 does seem to show convergence in their highest resolution simulation run with reionization, but the gas particle mass in their highest resolution run ($\sim 10^4~\rm \, M_{\odot}$) is an order of magnitude higher than the total stellar mass formed in most of the the satellites that form in our simulations, indicating that they would fail to detect these galaxies. Furthermore, their reionization turns on at a higher redshift than does ours ($z = 10.65$), which may lead to a higher halo mass cutoff for star formation.

run a zoom-in simulation of a small group of seven dwarf galaxies with a range of virial masses ($4 \times 10^8~\rm \, M_{\odot}$ $\leq M_{\rm{vir}}\leq 4 \times 10^{10}~\rm \, M_{\odot}$). They find that galaxies only form in the halos with present day $M_{\rm{vir}}> 10^9~\rm \, M_{\odot}$, and of the halos that form galaxies, the two with $M_{\rm{vir}}< 10^{10}~\rm \, M_{\odot}$ only form stars long after the end of reionization. Although they do not model $\rm H_2$ cooling nor self-shielding in their simulations, they do approximate the effects of $\rm H_2$ cooling and run a parallel simulation without a cosmic UV background. In their reionization-free simulation, all dwarfs form stars, but they show that the gas in the three low mass halos never reaches the column density required for self shielding. Their baryonic particle mass is $\sim 10^3~\rm \, M_{\odot}$, the same particle mass we achieve in the low resolution runs of $\rm Dwarf \, 2$ (see for details). In our low resolution runs, each of the most massive subhalos forms an object with stellar mass similar to its high resolution counterpart, but at this resolution each satellite object consists of only $3-5$ star particles. While it is difficult to say conclusively that resolution is the main driver of their apparent halo mass limit for star formation, in general it has been the case that every time resolution is increased in simulations, the minimum virial mass for a halo that can form stars has approached lower values . In addition to enabling the formation of galaxies with $M_{\rm \star}$ lower than the low resolution baryonic particle mass, higher resolution simulations allow for a more accurate description of shielding, which will also affect the minimum halo mass that can form stars. Furthermore, resolving the formation of dense substructures that collapse under self-gravity and become self-shielding requires resolving the Jeans/Toomre mass of the galaxies, which can be as low as $\sim 1000~\rm \, M_{\odot}$ (and corresponds to a required force/gravitational softening of at least $< 10-100~\rm \, pc$) in baryon-poor dwarfs.

Our simulations do not escape the challenges of imperfect resolution, but because our particle mass is an order of magnitude lower than most simulations that attempt to form galaxies in low mass halos, we can push down the predicted low mass limit for the halos that can form ultra-faint satellites. Rather than attempt to make a prediction for the low*est* mass halo that can form a galaxy, we predict that ultra-faint dwarf galaxies *can* form in dark matter subhalos as low mass as $M_{\rm{peak}}\sim 5 \times 10^8~\rm \, M_{\odot}$. While we cannot state with certainty that these galaxies are fully converged in our high resolution runs, the stellar masses of the satellites do show a better convergence between low and high resolution runs than do the centrals. (see for a discussion of the convergence of the centrals). Additionally, at these low masses, we still see sensitivity to spatial resolution and feedback (e.g. the second satellite of $\rm Dwarf \, 2_{Middle}$). More and higher resolution runs of dwarfs with $M_{\rm{vir}}\sim 10^8 - 10^{10}~\rm \, M_{\odot}$ that vary feedback prescriptions, reionization onset and spectrum, and particle mass are needed to fully probe the low mass end of the simulated stellar mass function and the $M_{\rm \star}- M_{\rm{halo}}$ relation.

# Discussion and Conclusions

In the $\Lambda$CDM  paradigm, all dark matter halos, from those around giant galaxy clusters to those hosting ultra-faint galaxies, should be filled with subhalos. We have used ultra-high resolution ($m^{\rm gas}_{p} \approx 255~\rm \, M_{\odot}$) simulations with the PSPH version of `Gizmo` and the Feedback in Realistic Environments (FIRE) prescriptions to predict that subhalos of isolated dwarf halos should form galaxies.

Most of the subhalos around dwarf galaxies are expected to be of low-mass ($M_{\rm{peak}}\lesssim 10^9~\rm \, M_{\odot}$) and we predict that some should host ultra-faint galaxies with $M_{\rm \star}\lesssim 10^4~\rm \, M_{\odot}$. If these tiny satellites are observed, it would provide evidence that dark matter substructure persists to very small scales, as predicted in the standard paradigm.

Using the dark-matter-only simulations of the ELVIS suite, we show that each isolated field dwarf ($M_{\rm \star}\sim 10^{6}~\rm \, M_{\odot}$) galaxy in the Local Group has about a $\rm 35\%$ chance of hosting at least one satellite with $M_{\rm \star}> 3000 \rm \, M_{\odot}$. The extended $\rm \sim 50~\rm \, kpc$ regions around known field dwarfs in the Local Group should prove to be fruitful search areas for ultra-faint satellites, as each pointing towards a dwarf also contains all of the volume of the Milky Way dark matter halo along the line of sight to that region. The Phoenix dwarf galaxy in particular is an excellent target due to its proximity to the Milky Way and the high probability that it is on first infall.

Although we consider only isolated dwarfs in this work, it is worth noting that some satellites of the Milky Way may also have their own satellites. show, using the ELVIS simulations, that approximately $7\%$ of Milky Way satellites (but as many as $25\%$ depending on the infall time and mass of the group), fell in as a part of LMC-size groups, and that several of the recent DES satellites are likely satellites of the LMC. A simple calculation for a rigid satellite assuming the common halo mass scale for Milky Way dwarf spheroidals ($M_{\rm{halo}}= 3 \times 10^{9}~\rm \, M_{\odot}$) orbiting a Milky Way-size host ($M_{\rm{vir}}= 10^{12}~\rm \, M_{\odot}$) shows that Fornax could potentially hold on to ultra-faint satellites residing within its inner $17~\rm \, kpc$. Leo I and Leo II could each host ultra-faints out to $\sim 30~\rm \, kpc$, and Sculptor out to $10~\rm \, kpc$, assuming each satellite is on first infall. This suggests that these galaxies might also be suitable candidates to host "satellites of satellites".

We do not see a sharp cut-off or break in the $M_{\rm \star}- M_{\rm{halo}}$ relation, at least for $M_{\rm{peak}}> 5 \times 10^8~\rm \, M_{\odot}$. We do, however, see a sharp cut-off in halos that host galaxies with uniformly ancient stellar populations. The ultra-faint dwarfs in our simulations form most of their stars in the first billion years after the Big Bang and are subsequently deprived of the cold gas required for star formation due to the ionizing background radiation, and not by infall into a more massive dark matter halo. We predict that below a critical mass threshold ($M_{\rm{halo}}\sim 5 \times 10^9~\rm \, M_{\odot}$, $M_{\rm \star}\sim 3 \times 10^4 \rm \, M_{\odot}$) all galaxies are ubiquitously ancient and, unlike more massive galaxies ($10^7 < M_{\rm \star}/ \rm \, M_{\odot}< 10^9$) that are nearly uniformly star forming in the field , both central and satellite ultra-faint galaxies should all be quenched.

For satellites with $M_{\rm \star}> 10^5~\rm \, M_{\odot}$, it is likely that infall time is one of the primary factors in quenching their star formation. Massive satellites ($10^{8.5} < M_{\rm \star}/ \rm \, M_{\odot}< 10^{9.5}$) have been shown to have extremely long quenching timescales, probably due to the cutting off of the fresh gas supply after infall . Slightly less massive galaxies ($10^6 < M_{\rm \star}/ \rm \, M_{\odot}< 10^8$) are likely quenched over much shorter timescales by the tidal or ram-pressure forces they experience upon falling in as satellites . However, it is likely that the quenching of ultra-faints is completely independent of their infall time, and set by the timing of the onset of and the mass they had at reionization.

The precise stellar masses of the "ultra-faint" dwarfs in our simulations are likely also sensitive to the reionization redshift (with an earlier onset of reionization likely to quench star formation in higher mass halos) and may also be sensitive to star formation physics at very low metallicities. We plan to explore these dependencies in future work, but here we point out that it is possible to form ultra-faint satellites of regular dwarf galaxies and highlight that they are quenched by reionization rather than infall.

The recent discovery of up to nine Milky Way dwarf satellite galaxy candidates in the Southern Sky by DES has important consequences for our understanding of star formation in low mass galaxies. Of particular interest is DES J0344.3-4331 (Eridanus II), because of its large distance from the Milky Way ($> 330~\rm \, kpc$) . At this distance, whether the satellite is on first infall or even headed out after a pericentric passage, it was likely accreted within the last $\sim 2~\rm \, Gyr$. Given that the initial age estimate for Eridanus II shows that it likely has an ancient stellar population [^7], this would mean that it was quenched long before infall. If confirmed in follow-up observations, Eridanus II would be the first known ultra-faint galaxy shown to be quenched in the field, supporting our findings that, below a critical mass scale $\sim 5 \times 10^9 \rm \, M_{\odot}$, all galaxies host ancient stellar populations quenched by reionization-related feedback, and not by environmental processes. This would be a clear example of a reionization "fossil", as first discussed by .

# Acknowledgments

We thank M. C. Cooper for helpful discussions. This work used computational resources granted by NASA Advanced Supercomputing (NAS) Division, NASA Center for Climate Simulation, Teragrid and by the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number OCI-1053575 and ACI-1053575, the latter through allocation AST140080 (PI: Boylan-Kolchin). Support for this work was also provided by NASA through *Hubble Space Telescope* theory grants (programs AR-12836 and AR-13888) from the Space Telescope Science Institute (STScI), which is operated by the Association of Universities for Research in Astronomy (AURA), Inc., under NASA contract NAS5-26555. CW and OE were supported by *Hubble Space Telescope* grants HST-AR-13921.002-A and HST-AR-13888.003-A, and SGK by the National Science Foundation Grant AST-1009999. Support for PFH was provided by the Gordon and Betty Moore Foundation through Grant \#776 to the Caltech Moore Center for Theoretical Cosmology and Physics, an Alfred P. Sloan Research Fellowship, NASA ATP Grant NNX14AH35G, and NSF Collaborative Research Grant \#1411920. DK received support from National Science Foundation grant number AST-1412153, funds from the University of California, San Diego and XSEDE allocation TG-AST-120025.

[^1]: $\!$email@example.com

[^2]: Note that $\rm Dwarf \, 2_{Early}$ and $\rm UFD \, 1$ were also previously presented in as $\rm m9$ and $\rm m10$ respectively.

[^3]: All galaxies that meet our minimum $M_{\rm \star}$ requirement have at least 16 star particles and had $\gtrsim 3 \times 10^4$ dark matter particles at peak mass. All but one had $\gtrsim 2 \times 10^5$ dark matter particles at peak mass, corresponding to typical initial baryonic particle numbers of $\sim 3-4.5 \times 10^5$ before feedback dispelled most of these.

[^4]: In all cases except $\rm Dwarf \, 2_{Early}$ and $\rm Dwarf \, 2_{Late}$, the 2nd Satellite is the second most massive satellite in stellar mass. In these two runs only, the 2nd Satellites are selected as the satellites that correspond to the high stellar mass outlier in run $\rm Dwarf \, 2_{Middle}$. Only the 1st Satellites in $\rm Dwarf \, 2$ runs qualify as "massive satellites".

[^5]: In all six of our runs, every subhalo with $M_{\rm{peak}}\geq 5 \times 10^8~\rm \, M_{\odot}$ forms a well-resolved satellite. Although we focus here on satellites of isolated dwarfs, we note that this criteria would suggest the existence of $\sim 100$ undetected ultra-faint "stealth galaxies" within $400~\rm \, kpc$ of the Milky Way.

[^6]: We have checked that the spatial radii of these galaxies are also well-resolved (not just their masses). Obviously, the plotted radii are much larger than our minimum force softening. A more demanding criterion is the radius which, for the dissipationless version of these runs, is $\gtrsim 100~\rm \, pc$ (see ). While this suggests that we are dynamically well resolved, the effect on the baryonic component within this radius is harder to determine. In lower resolution runs, the satellites' half-mass radii do vary by about a factor of two, but in both directions (some are larger, some smaller). Higher resolution simulations would be particularly useful for solidifying the expectations for the sizes of these galaxies.

[^7]:  suggest that Eridanus II could possibly have a population of young stars in addition to the old stellar population.


# Abstract

Electron Microscopy (EM) image (or volume) segmentation has become significantly important in recent years as an instrument for connectomics. This paper proposes a novel agglomerative framework for EM segmentation. In particular, given an over-segmented image or volume, we propose a novel framework for accurately clustering regions of the same neuron. Unlike existing agglomerative methods, the proposed context-aware algorithm divides superpixels (over-segmented regions) of different biological entities into different subsets and agglomerates them separately. In addition, this paper describes a 'delayed'  scheme for agglomerative clustering that postpones some of the merge decisions, pertaining to newly formed bodies, in order to generate a more confident boundary prediction. We report significant improvements attained by the proposed approach in segmentation accuracy over existing standard methods on 2D and 3D datasets.

# Introduction

Extracting the network structure among neurons in animal brain has gained substantial attention lately in the field of neuroscience. Rapid advances in imaging technology, in particular Electron Microscopy (EM) techniques, have enabled us to trace neural bodies in unprecedented level of details. However, recording in such high resolution (at nanometer scale) generates massive amount of data that is too large to annotate manually. Automated region labeling or segmentation is considered to be the most viable strategy for generating a dense reconstruction of neural anatomy. Some recent efforts of such reconstruction yielded impressive results utilizing machine learning/computer vision tools such as image segmentation, and offered valuable biological insights to the neuroscience community .

Image segmentation for natural scenes has a long history in computer vision literature . In recent years, there have also been many fruitful attempts to automatically identify meaningful regions in EM images using segmentation techniques . Most of these studies initially apply a pixelwise[^1] classifier to determine whether or not any particular pixel belongs to the cell boundary. The quantified confidence values of the pixelwise classifier are utilized to produce an initial (over-) segmentation through methods such as Watershed .

Different approaches resort to different methods to generate the final segmentation by merging or clustering the over-segmented bodies to corresponding neuronal cells. Andres et.al.  addresses this problem by searching the optimal subset of superpixel borders that form closed surfaces. Several studies work with the watershed merge tree in order to identify the regions to be combined for the final segmentation . Some approaches applied agglomerative or hierarchical clustering   for this purpose. For anisotropic datasets, where the depth resolution ($z$-dimension) is coarser than the planar resolution ($x,y$ dimensions), segmented bodies on one section overlap with multiple regions in the adjacent sections. Therefore, a complete 3D reconstruction need to establish the correct correspondence, through an alignment or co-segmentation technique , among segmented regions across multiple planes . In this study, we restrict ourselves only to segmentation on images for anisotropic data – the subsequent alignment is out of the scope of this paper. For both isotropic and anisotropic reconstruction, the outputs of segmentation algorithms need to be corrected afterwards, either manually or by combining with a manually traced skeletonized representation .

Biologically, the interior of a neuron cell comprises several distinct sub-structures (or sub-categories) such as cytoplasm, mitochondria, vesicles etc. An ideal binary pixel classifier – which assigns a pixel to one of the two categories: cell boundary and cell interior – should label all locations within these sub-categories to cell-interior class. Several past studies  recommend increasingly complex pixelwise detector models to attain a binary prediction output. In contrast, some recent works  represent the sub-categories (cytoplasm, mitochondria etc.) of cell body by multiple classes and apply relatively simpler classifiers (in terms of model size, learning time and convenience) for this multiclass classification problem. The results of  suggest that using prior domain knowledge to divide a problem into multiple components can achieve high segmentation quality with simpler classifier models requiring less computation.

However, we believe the methods of  do not exploit the full benefit of multiclass predictions on EM data. Regardless of the quality of multiclass pixel classifier output, the algorithms in  do not distinguish between regions of one sub-structure (e.g., cytoplasm) from those of another (e.g., mitochondria) at the superpixel level. That is, the classification is divided into multiple sub-classes in pixel-level, but the subsequent fusion or superpixel clustering step does not utilize this additional information to compute the final segmentation. This often leads to sub-optimal performances by these methods. For example, Figure  shows an EM image (plane in a 3D volume) and the corresponding pixel predictions for the mitochondria sub-class. By not using this sub-class prediction explicitly in the clustering step, the final output of  failed to merge many regions into the correct cell (marked by 'S') and connected some of them to wrong cells (marked by 'M').

This paper introduces a context-aware scheme for combining over-segmented regions by utilizing the prior knowledge of sub-classes. We adopt an agglomerative or hierarchical clustering framework   due to its advantages such as low space, time complexity and flexibility to tune for over/under segmentation. We develop a two-pass agglomeration policy where the (estimated) cytoplasm regions are grouped together in the first phase and then the remaining mitochondria bodies are absorbed into the cell cytoplasm. In these two stages, the superpixels are agglomerated based on different merge criteria that are defined by different contexts, which is why we call it context-aware agglomeration. Our proposed context aware approach significantly reduces the false split and merge errors (example shown in Figure ) provided fairly accurate sub-structure detection. In addition, this strategy substantially reduces the training data requirement, as well as the predictor model complexity, which in turn offers significant increase in learning speed. The findings of this study further inspired us to design an interactive training algorithm  for region boundary predictor that does not require exhaustively labeled groundtruth. Generating such an exhaustive annotation is considered to be a bottleneck for neural reconstruction .

We also propose a modified version of the hierarchical clustering algorithm to cluster the superpixels in both phases of the context-aware framework. The proposed clustering method emphasizes on minimizing under-segmentation errors since these errors are conventionally costlier to correct than the over-segmentation errors . In order to minimize the number of false merges, we 'delay'  the merge decisions on a certain type of boundaries to be resolved at a later time. Compared to the traditional agglomerative scheme of , the proposed modification reduces the number of false merges significantly. We also attempt to analyze why our agglomeration approach performs better than the Global multicut scheme  on the dataset used for our experiments.

The paper is organized as follows. We define the problem in Section  and briefly describe the existing clustering segmentation algorithms in Section . Then we explain the proposed delayed agglomeration scheme in Section . This delayed strategy is employed in both the stages of our context-aware algorithm discussed in Section . Section  reports our experimental setup, both quantitative and qualitative results and their analyses. We conclude and discuss our findings further in Section .

# Methods

A formal definition of the problem we are addressing assumes an initial over-segmentation, comprising $N$ superpixels $\{ S_1, S_2, \dots, S_N\} \subseteq {\cal S}$, of an EM image or volume with $M$ neurites (neuronal regions) where $N \gg M$. Let $L(S)$ be the neurite region that $S$ actually belongs to. Our goal is to correctly assign these $N$ superpixels such that each $S_i, i = 1, 2, \dots, N$ is assigned to its corresponding $L(S_i)$.

We denote a boundary between two superpixels (i.e., oversegmented regions) by a pair of regions $e \triangleq \{ S_i, S_j\}$ and the set of all such boundaries by $E$. In a graph representation, each of the regions $S_i$ is considered to be a node and the boundary or face between two regions is regarded as an edge – a notation we will be using throughout the paper. Also, let the boundary label map $B: {\cal S} \times {\cal S} \to \{0, 1\}$ assign a 1 to a boundary that actually separates one neurite region from another and a 0 to the boundary incorrectly generated due to over-segmentation. The problem of correctly merging $S_i$ to its corresponding $L(S_i)$ is similar to a clustering problem where the number of clusters cannot be computed a priori. Following, we adopt an agglomerative approach for superpixel clustering.

In our context-aware scheme, the set of superpixels is divided into two subsets: 1) the ${\cal S}_c$ of potential cytoplasm superpixels, and 2) the set ${\cal S}_m$ of potential mitochondria superpixels. The set of cytoplasm superpixels is clustered first with the proposed delayed agglomeration algorithm. Agglomeration of the mitochondria superpixels is also performed by the proposed delayed method, but with a different merge criterion. In order to assist the reader to comprehend the novelty of the proposed approach, we introduce the prior studies on agglomerative clustering for EM segmentation  in Section . Afterwards, Sections  and  discuss the delayed agglomeration and the context- aware framework respectively.

## Prior Works on Agglomerative Clustering for EM Segmentation

Several existing EM segmentation approaches  tackled the problem of superpixel clustering by agglomerative hierarchical clustering, as described in Algorithm . These methods assume a superpixel boundary estimator $h :{\cal S} \times {\cal S} \to \mathbb{R}$ that assigns real valued confidences to all edges in $E$. This boundary estimator may represent the real-valued prediction of a classifier distinguishing true boundaries from the false ones , or compute the mean value of boundary pixel probabilities  or return the overlap percentage between borders of two adjacent superpixels. The value of $h(\{S_i, S_j\}) \in [0, 1]$ indicates how confident the estimator is about the existence of a true boundary between $S_i$ and $S_j$: a large $h(\{S_i, S_j\})$ implies the estimator is very confident that the boundary $\{S_i, S_j\}$ is correct while a small value implies the boundary was probably generated as an artifact of over-segmentation and therefore is false. Given such a function $h$, the hierarchical clustering algorithm iteratively merges cell boundaries in the increasing order of confidence values $h(e)$ (Line  in Algorithm ) until a stopping criterion is satisfied, e.g., $h(e) > \delta$ where $\delta$ is a pre-defined threshold. After each merge, it also updates the neighborhood structure of the merged superpixel, i.e., the neighbors of the absorbed region become the neighbors of the (newly) merged cell.

Each time a superpixel border is dissolved in standard agglomerative clustering, it modifies the characteristic representations of the pixels within the superpixels and on the boundary. This demands the confidences of the estimator function $h(e)$ on these boundaries be recomputed (Line  in Algorithm ). The edges, for which $h(e)$ decreases due to a merge, receive higher priority to be dissolved than it had before. The proposed delayed agglomeration strategy modifies this step and postpones merging these edges for a later time.

## Proposed Delayed Agglomerative Clustering

Our adaption of segmentation also starts with the boundary with lowest estimator confidence and repeatedly dissolve edges with in ascending order of $h(e)$. Recall that, $h(e)$ may measure the prediction of a superpixel boundary classifier, or the mean probability values on boundary pixels, or the fraction of overlap between the borders of two superpixels. After two regions have been joined due to a merge, the boundaries of the combined region is updated and the estimator function $h$ is applied to recompute the new confidences. The edges for which $h(e)$ decreases are set aside to be considered at a later stage. They are reexamined after all the borders, initially generated by over-segmentation process, have been checked.

This method is described in Algorithm . After region $R_j$ is absorbed into $R_i$, we do not *immediately* consider all the new boundaries $\{R_i, R_b\}$ between the recently merged $R_i$ [^2] and its updated neighbors $R_b$. We maintain a set of edges $W$ and insert the new edge $\{R_i, R_b\}$ only if its confidence increases from that of $\{R_j, R_b\}$ after $R_j$ is absorbed into $R_i$ (Line  in Algorithm ). The faces, for which $h(\{R_i, R_b\})$ decreases from previous value, are kept aside until there are no members left in $W$ and the modified confidence on $\{R_i, R_b\}$ is less than the agglomeration threshold (Line  in Algorithm ). Once all $e \in W$ have been considered for merge and $W$ is empty, these boundaries repopulate the list $W$ (Line  in Algorithm ) and renew the agglomeration process which continues until there exists no $e$ such that $h(e) \le \delta$.

Effectively, the proposed strategy 'delays'  the merging of new edges $\{R_i, R_b\}$ resulting from a merge: either due to an increase in $h(\{R_i, R_b\})$ or deliberately if $h(\{R_i, R_b\})$ decreases. This design postpones the merge decisions on the newly formed bodies for a later time to avoid propagating wrong decisions, made on smaller superpixels, to the larger ones. Our analyses support that deferring decisions on these edges significantly reduces false merges during agglomeration.

### Time Complexity

Asymptotically, the running time of the delayed algorithm remains the same as the traditional agglomerative clustering in the worst case. In a priority queue implementation, instead of adding the adjacent boundaries to the queue, the delayed algorithm stores them in a separate list. Later, building a queue from this list would require $O(n_1)$ time where the length $n_1$ of new list must be smaller than that of the previous one (which contains all edges): $n > n_1$.

In fact, our implementation is tuned to reduce the running time of delayed agglomeration. Notice that, a subset of adjacent boundaries are not pushed back or updated into the queue (Line 13 of Algorithm ). We may as well apply a simple trick to avoid updates at each merge altogether: instead of increasing key of the edges with increased $h$ value (Line 11 of Algorithm ), we can postpone the check and increase the key until it becomes a candidate for merge (Line 7 of Algorithm ) or in Line 6 when it being considered to be inserted into $W$. Thus, we can reduce the computation by $O(d nlogn)$ where $d$ is the degree of $S_2$ and $n$ is the queue size.

## Proposed Context-aware Segmentation

The proposed context-aware agglomeration has two different phases. We separate the set ${\cal S}_m$ of potential mitochondria superpixels from the set ${\cal S}_c$ of potential cytoplasm superpixels assuming the existence of an effective mitochondria superpixel detector (e.g., ). The regions in ${\cal S}_c$ are agglomerated first by the proposed delayed policy. Motivated by , a Random Forest (RF)  classifier $h_c$ is trained to act as the boundary predictor function for clustering the set ${\cal S}_c$ of cytoplasm superpixels. During $h_c$ training, mitochondria-cytoplasm borders are treated the same way as cell membrane.

In the second step, the mitochondria-cytoplasm edges are merged in the same delayed scheme as explained in Section , but with a different estimator function $h_m$. In order to absorb mitochondria into corresponding cells, we apply the delayed-agglomeration algorithm with a small alteration. The *set of candidate edges $W$ only contains the edges between mitochondria and cytoplasm*, that is, $W = \{ \{ S_c, S_m\} ~|~ \text{type}(S_c) = \text{Cyto}, \text{type}(S_m) = \text{Mito}, \text{Flag}(\{S_c, S_m\}) = \text{ACTIVE}\}$; *mitochondria-mitochondria edges are not considered for agglomeration*. Biologically, each mitochondrion should reside within a cell body. Therefore, boundary confidence for mitochondria merging should reflect how much a mitochondrion is contained within a cytoplasm. In order to quantify this, we define the overlap ratio $\rho(\{S_m, S_c\})$ to be the fraction of the total boundary of $S_m$ which separates $S_m$ from $S_c$ : $\rho(\{S_m, S_c\}) = {\text{length}(\{S_m, S_c\}) \over \sum_i \text{length}(\{S_m, S_i\})}$. For any edge $\{S_m, S_c\}$ with a mitochondria superpixel $S_m$ and a cytoplasm superpixel $S_c$, the confidence is defined as $h_m(\{S_m, S_c\}) = 1 - \rho(\{S_m, S_c\})$.

In effect, the mitochondria superpixels are combined with the cytoplasm superpixels in the descending order of the overlap ratio between these two types of regions. That is, a mitochondria superpixel is merged into the adjacent cytoplasm region with the largest overlap between their boundaries. The combined cytoplasm-mitochondria superpixel created by such merge then identifies the next mitochondria superpixel with the largest overlap to absorb in the next step. We show snapshots of this process, at different values of $\rho(S_m, S_c)$ in Figure . It is worth noting that, for 3D segmentation, the overlap is computed across many different planes on which the two cells are neighbors to each other.

# Results

We have applied the proposed method to EM images of two different modalities: isotropic Focused Ion Beam Scanning Electron Microscope (FIBSEM) data and anisotropic serial section Transmission Electron Microscopy (ssTEM) data. For both types of input data, the image (volume for the isotropic data) is first over-segmented for the agglomeration to be applied on. In the following sections, we explain our over-segmentation process and the error measures used to evaluate segmentation performance before reporting the results on FIBSEM and ssTEM data in Sections  and  respectively.

## Over-segmentation and training:

We learn a classifier to assign each individual pixel into multiple categories such as cell boundary, cytoplasm, mitochondria and mitochondria boundary etc. using the interactive tool Ilastik . Our pixelwise detector is a Random Forest (RF) classifier  trained on a few sparse samples from the dataset. The locations with lowest pixelwise cell boundary prediction are utilized as markers for the Watershed algorithm  to produce an over-segmentation of the image/volume. Unless otherwise specified, the same pixel prediction and watershed regions are provided as input to all (competing) methods.

The set ${\cal S}_m$ of probable mitochondria superpixels is populated with all regions having mean mitochondria probability (estimated by our pixelwise RF classifier trained by Ilastik) above a certain threshold. The rest of the superpixels constitute the set ${\cal S}_c$ of possible cytoplasm regions. The training set for superpixel boundary classifier $h_c$ consists of all boundaries among members of ${\cal S}_c$ as well as the mitochondria-cytoplasm borders. Similar to , each superpixel edge is represented by the statistical properties of the multiclass probabilities estimated by Ilastik. The statistical properties include mean, standard deviation, 4 quartiles of the predictions generated for the data locations on the boundary, two regions it separates as well as the differences of these region statistics. All of these features can be updated in constant time after a merge – a property which improves the efficiency of the segmentation algorithm substantially. The code and example dataset are publicly available at <https://github.com/janelia-flyem/NeuroProof.git.>

## Segmentation error measures:

We report segmentation error of both types, namely under- and over-segmentation, separately because one of these errors (under-segmentation) is costlier than the other. Split versions of variance of information (VI)  and Rand Error (RE)  were selected to evaluate segmentation errors. Given a groundtruth (GT), $GT =\{ g_1, \dots, g_M \}$, and a segmentation (SG), $SG = \{ r_1, \dots, r_P \}$, we compute the over-segmentation (OE) and under-segmentation (UE) errors by splitting the terms in VI and RE. For split-VI, the over and under-segmentation are quantified as follows.

$$\begin{aligned}
VI_{OE}= H(\text{GT} ~|~ \text{SG}) &=& - \sum_{i, j} {|g_i \cap r_j| \over Z} ~\log {{| g_i \cap r_j |} \over |g_i|}.\\
VI_{UE}= H(\text{SG} ~|~ \text{GT}) &=& - \sum_{i, j} {|r_i \cap g_j| \over Z } ~\log {{| r_i \cap g_j |} \over |r_i|}.
\end{aligned}$$

In these equations, $|\cdot|$ denotes the size, $\cap$ denotes the intersection between two regions and $Z$ is a normalizing constant. From information theoretic perspective, these two terms are conditional entropies defined over a set GT given SG, and vice versa.

We also quantify segmentation error by average percentage ($\times 10^{-5}$) of pairs of voxels falsely merged and split by any method. Formally, the over-segmentation (OE) and under-segmentation (UE) is computed based on the following formula.

$$\begin{aligned}
 RE_{OE} &=& \text{\% pixel pairs within same cluster in GT but different cluster in SG.} \\
 RE_{UE} &=& \text{\% pixel pairs within same cluster in SG but different cluster in GT.}
\end{aligned}$$

## Segmentation Performances-FIBSEM data:

**Dataset:** The first set of experiments was conducted on isotropic datasets from fruit fly visual system imaged at 10 nm isotropic resolution using FIBSEM technology. This data is segmented as a volume (i.e., 3D segmentation) and both the voxelwise multi-class predictor and the supervoxel boundary classifier are learned on one $250^3$ volume and applied on two $520^3$ test volumes.

**Competing methods:** We have compared the following algorithms in this study: 1) LASH: Standard agglomeration with an RF supervoxel classifier learned based on the iterative procedure of . 2) LASH-D: LASH classifier with delayed agglomeration (proposed extension). 3)GALA : an agglomerative method with repetitive learning phases like LASH, except it accumulates the training sets of multiple phases. 4) CADA-F: Proposed two stage delayed agglomeration with standard RF learned using training set accumulation similar to GALA. 5) CADA-L: Proposed delayed agglomeration with a depth-limited RF (depth =20) learned *without* training set accumulation. 6) Global Multicut: the optimization framework for finding a closed-surface segmentation proposed in . For , the boundary confidences were generated by the CADA-L predictor.

**Performance evaluation:** In order to compare different supervoxel clustering schemes, we trained (on one $250^3$ volume) and segmented two $520^3$ volumes 5 times and averaged their scores. We plot the average (over 5 trials) $VI_{UE}$ and $VI_{OE}$ respectively on x and y-axis respectively in plots on the left column of Figure  and for test Volumes 1 and 2. Similarly, we show the average $RE_{UE}$ and $RE_{OE}$ errors on x and y-axis respectively on right columns of Figure . In these figures, an ideal algorithm should achieve a zero value for both over and under-segmentation. For all algorithms except the Global multicut method, each point in a plot refers to the boundary confidence threshold $\delta_c \in [0.1, 0.2]$ which was used as stopping criterion for cytoplasm merging. For , we instead changed the value of the bias parameter in weight calculation within the range $[0.2, 0.9]$.

As the plots show, both the delayed agglomeration and two-phase segmentation process attained significant improvement over past methods : compare the performance of LASH (red +) with LASH-D (black x) and that of GALA (cyan \*) with CADA variants (green square and blue circle). Compared to the rest of the techniques, the two variants of proposed methods, namely CADA-L and CADA-F, appear to achieve the most favorable segmentations by reducing the over-segmentation steeply without increasing the false merge numbers much. During segmentation, the delayed version decreases the time needed for segmentation approximately 5 times among the agglomerative approaches.

It is also worth mentioning that, in a two stage segmentation scheme, the performance of a depth limited RF (i.e., CADA-L, green square), learned without accumulating training set over multiple passes, is very similar to that of the standard RF (CADA-F, blue circle) trained over cumulative learning passes. Training full-depth RF (CADA-F) with multiple passes needed several hours whereas training a depth limited single iteration (CADA-L) required $\le 5$ minutes.

In Figure , we show example outputs from the methods LASH-D, GALA , Global multicut  and CADA-L on three planes from the test volume 1 as depicted in Figure  (the algorithms were applied on 3D volume, we are showing three slices of the volume for demonstration). Three columns correspond to three planes, and each row presents the outputs of the aforementioned methods. The segmentation labeling is overlaid with artificial (randomly selected) colors. We have selected the parameter that results in the lowest false merges (under-segmentation) with a false split (over-segmentation) error below $0.7$ for all except the proposed CADA-L for which we selected the lowest over-segmentation (error value approx 0.56). The results are largely compatible with the quantitative ones, with all three methods, especially Global method, leaving many false boundaries intact. The false-splits are not limited to cytoplasm mitochondria borders, both Global and GALA over-segmented some cytoplasm regions as well. By separating these two sub-classes within cell bodies, the proposed method CADA-L was able to eliminate the false merges between them.

**Runtime comparison:** In Table , we report the running times of the context oblivious standard agglomerative clustering used in LASH ; the context-oblivious delayed agglomerated clustering used in LASH-D, GALA ; the context-aware delayed agglomeration used in CADA-L, CADA-F; and the Global Multicut  method. Both the standard and delayed agglomeration were executed up to the same threshold $\delta =0.2$. The context-aware method executes two phases of agglomeration, which is why CADA required more time than LASH-D. The Global multicut algorithm utilizes the solution of an optimization problem (requires optimization packages like CPLEX or Gurobi) in order to find the edges to merge for producing the final segmentation.

|                      Method                      | Run time (min)  |
|:------------------------------------------------:|:---------------:|
| LASH: Standard Agglomeration, context-oblivious  | $5.35 \pm 0.2$  |
| LASH-D: Delayed Agglomeration, context-oblivious | $2.72 \pm 0.06$ |
|    CADA: Delayed Agglomeration, context-aware    | $4.69 \pm 0.02$ |
|                 Global multicut                  | $7.13 \pm 1.1$  |

Runtime for different algorithms.

During the training of superpixel boundary classifier, all the algorithms except CADA-L and Global Multicut performs standard agglomeration multiple times (we repeated 5 times) in order to obtain extensive training sets. Both CADA-L and Global method exploited the same classifier learned from the initial set of boundaries existed in the over-segmented data (without training set augmentation).

In the following subsections, we attempt to analyze why (or how) the proposed strategies improve the segmentation performance over the existing approaches.

### Context-aware vs Context-oblivious agglomeration

It is perhaps intuitive that traditional context-oblivious agglomeration will result in higher degree of over-segmentation than the context-aware method. The mitochondria-cytoplasm borders indeed have strong feature similarity with cell membranes and consequently superpixel boundary predictors cannot distinguish between these two types of borders perfectly. Recall that, for segmentation, we need to dissolve the mitochondria-cytoplasm border but retain the cell boundaries. In order to substantiate our claim, we trained a superpixel boundary classifier in context-oblivious fashion (0: false cell membrane, 1:true cell membrane) and computed its confidences on these two types of boundaries. Figure  shows the histogram of confidence levels for actual cell boundaries and mitochondria-cytoplasm borders in red and blue respectively. If we wish to minimize false merges among neurons, we have to stop agglomeration at a lower value ( $\delta \le 0.3$). The overlap between the two distributions in the range $0.1~\sim0.5$ suggests that many of the mitochondria borders will not be merged and will lead to over-segmentation.

In addition, due to appearance dissimilarity, the distribution of same features computed on cytoplasm and mitochondria will be substantially different from each other. Combining these two types of feature value distribution will impede the identification of false boundaries between cytoplasm superpixels such as the one in the lower left corner of the output of GALA in Figure .

In practice, mitochondria from two different cells could also lead to false merges. Often the mitochondria regions from two cells are closely located to the cell membrane, or other mitochondria regions from neighboring cells, blurring the boundary. Figure  and  show several such locations where the existing techniques failed to avoid false merge.

### Global multicut vs Proposed:

The split-VI plot in Figures  show that both variants of the proposed CADA algorithm generates significantly low under and over-segmentation errors than those of Global  method in clustering *cytoplasm regions only (mitochondria not merged)*. In order to analyze why this happens, we save the initial confidences (predictor confidence at the beginning of agglomeration) of $h_c(e)$ on all $e$ that

-   were incorrectly split (over-segmented) by the Global method,

-   were correctly merged by proposed algorithm.

These boundary predictions were plotted on x-axis of Figure . The y-axis of Figure  corresponds to confidences $h_c(e)$ at the time $e$ was correctly merged by the proposed method. The threshold on boundary confidences to stop agglomeration was $\delta_c=0.2$.

Notice that, the agglomerative process correctly reduced the confidences of many false boundaries that received a high score by the predictor at the beginning (high x value but low y value). This refinement is possible through the evolution of the superpixels in the agglomerative process – an advantage the Global method of  cannot benefit from. The Global method , in comparison, generated many more false positive boundaries as depicted by the rectangular enclosed region of Figure  (left). If several boundaries within a chain of supervoxel faces receive very high predictor confidences, by construction, the Global method tends to retain the another boundary $e$ within the same chain with a low $h_c(e)$. Such tendency may be the reason behind the high concentration of false splits with low $h_c$ within the rectangular region in Figure  (left).

### Delayed vs standard agglomeration:

In order to illustrate the improved accuracy attained by the delayed agglomeration over the standard one, we collected all *faces that were incorrectly dissolved by standard agglomeration algorithm* (LASH) and examine their confidences under a delayed scheme (LASH-D) operating at $\delta_c
 = 0.14$ The confidences (clipped to 0.25) of these 534 edges generated by standard and delayed agglomeration are plotted in Figure  (right) in blue square and red $+$ respectively. The proposed delayed agglomeration accurately increased the confidences $h_c$ of many of these faces, among which, 41 exceeded the threshold of $0.14$ (green line) and avoided a false merge. In addition to these common supervoxel edges, the standard and delayed algorithms independently generated 163 and 4 more incorrect merges respectively.

## Segmentation Performances-ssTEM data:

This section reports the 2D segmentation results our method and others produced on a different data modality, namely ssTEM images. These images were part of those generated for the work of  and were collected from the authors. Fifteen $500 \times 500$ images were used for training both the pixel and superpixel boundaries. We follow the techniques and 2D versions of features described in Section  for ssTEM data. The same pixel prediction and watershed regions are provided as input to all competing methods. The segmentation is performed on each image (without connecting them across planes) and Figures  plots the average of split-VI and split-RE errors over 15 images of size $1000 \times 1000$ of the proposed CADA-L and GALA  methods. Our method CADA-L seems to produce less over-segmentation in almost all threshold values than that results of GALA . The result of the Global method  were too poor to show on this plot – lowest over-segmentation error at 4.11 with 0.13 under-segmentation average.

In Figure , we show input images and the segmentation results (overlaid on the image) of GALA and our methods at the same under-segmentation error. Examining the qualitative output in Figure , GALA seems to struggle to absorb the mitochondria regions despite multiple learning iterations and even merges two cells in one occasion. While a more accurate mitochondria predictor could potentially reduce the segmentation errors of the proposed method, context-invariant algorithms such as GALA would be less effective around mitochondria regions. It is worth mentioning here that, compared to GALA, CADA-L used less than half of the training examples ($42.46\%$) collected without training iterations (i.e., significantly more efficient in training).

# Discussion

We argue that, due to considerable ambiguity in appearances, it is only rational for an EM segmentation algorithm to be context-aware in each of its stages, i.e., in both pixel and superpixel levels (and in alignment for anisotropic data). The results reported in this paper support our claim that a context-aware clustering of sub-classes such as cytoplasm and mitochondria can improve segmentation accuracy significantly given fairly accurate sub-class detection. Our examination of both isotropic and anisotropic data suggests cell structures cannot be meaningfully identified without mitochondria regions and it is non-trivial to combine detection with a segmentation that ignores it (e.g., ) in order to produce the final segmentation . Our analysis also illustrates how a delayed agglomerative procedure benefits from the intermediate boundary probabilities and improves the efficiency of the segmentation process significantly.

In addition to reducing the over- and under-segmentation errors, one of the variants of our classifier, namely CADA-L, demands substantially fewer training examples (and no training iterations) than others, i.e., its training is significantly faster than others. A context-oblivious strategy gain significantly (compare LASH-D with GALA in Figure ) by accumulating training set over multiple iterations. However, in context-aware approach, one does not benefit much by accumulating the training set (CADA-F in Figure ) over a classifier trained from a single iteration (CADA-L). One possible explanation is that previous context oblivious strategies require the extra iterations to mitigate the impact of the noise introduced by mitochondria superpixels. This explanation implies that detecting the sub-classes, and considering them separately as necessary, is perhaps the key to train a boundary classifier accurately and efficiently.

We further investigated this conjecture and developed a semi-supervised active learning algorithm (interactive) to train the supevoxel boundary classifier with as few as $< 20\%$ of the total examples . The requirement of exhaustive labels is a critical bottleneck for automatic EM segmentation, especially for reconstructing larger brain regions, or whole animal brain, where one may anticipate the necessity to train several different classifiers . The interactive training of both pixels (using Ilastik  for example) and superpixel boundaries (using ) holds the promise of removing the need for such complete groundtruth and paves the for scaling up the EM reconstruction algorithms.

We have applied our context aware algorithm to segment 216 FIBSEM volumes of $520^3$ voxels each, with a 10nm isotropic resolution, from the Medulla region of fly retina. As far as we know, this is an attempt to reconstruct one of the largest volumes for such animal. Compared to the result of  on two of the $520^3$ blocks, our segmentation resulted in an estimated $\sim 30\%$ reduction in subsequent manual correction time. In addition, our segmentation were sufficiently accurate for regions that pertains to Post-synaptic densities (PSD), i.e., the synaptic partners of a cell. During the manual annotation of these PSDs, the output of our segmentation method assisted the experts to improve their performances .

# Acknowledgments

The authors wish to thank Stuart Berg and Bill Katz for their contributions in software development; Pat Rivlin and Shinye Takemura for providing the annotated dataset of  and for the discussion on the neurobilogical properties of our dataset; Matt Saunders and Lei-Ann Chnag for their effort in generating pixel classification ilps and groundtruth annotation; Don Olbris for his assistance with the visualization software Raveler.

# Supporting Information

[^1]: Unless it is obvious from the context, we denote locations on both 2D and 3D EM data as pixels in this manuscript.

[^2]: $R_i$ is now the union of previous $R_i$ and $R_j$