anchor.diff: +# Basic reporting
  +This paper presents an (apparently novel) approach to randomly sampling from the feasible set of integer partitions that is several orders of magnitude faster than an existing method.  As the authors argue, I believe that their approach will make statistical analyses such as those in Locey and White (2013) much easier to carry out, and may even allow for analyses that would previously have been impossible.
  +
  +Based on the R code provided by the authors, I believe that their approach is sound, although the manuscript does not do quite enough to convince me that the samples are unbiased.  Of course, a sampling method does not need to be completely unbiased to be useful, but quantifying those biases (if they exist) would make the paper much stronger.  I discuss this issue in more detail below under “experimental design”.
  +
  +Additionally, the manuscript’s writing is not as clear as I believe it could be, as discussed in this section below.
  +
  +Apart from these issues (and some notes about the R package, in the “general comments” section of this review), I think that this paper makes a very useful contribution and would be happy to see the method used in future research.
  +
  +———————
  +
  +As I was reading, I was initially confused about whether the manuscript was presenting one algorithm (based on partition transposition) or four (top-down, bottom-up, etc.).  The abstract and introduction could be clearer on this point.
  +
  +A visual representation of a SAD curve surrounded by similar curves sampled randomly from the feasible set would be a good way to quickly ensure that the reader understands the purpose of the methods.
  +
  +As the material in this paper will be unfamiliar to most ecologists, it is especially important to keep the terminology clear and consistent. The fact that part, element, and class seem to be used interchangeably may derail some readers, as might the fact that the notation used in the manuscript contradicts the R package’s notation (e.g. the manuscript’s “S” corresponds to the package’s “N”).
  +
  +Similarly, it would be useful to define important terms more precisely for the benefit of non-ecologist readers.  For example, a mathematician might not know that “total community abundance” corresponds to “the total number of individuals” or that “species richness” corresponds to “the number of species”.
  +
  +The organization of the first methods subsection could be improved by breaking it up into more subsections.  It may also be worth revisiting the decision to define `q` before introducing the algorithm.
  +
  +Currently, a reader can grasp relatively quickly that the goal of the algorithm is to randomly sample a partition with S as the largest element and then conjugate it, but it takes much more work to glean the details from the text. Greater separation between the method itself and the mathematics justifying it (e.g. one or more pseudocode boxes/figures that summarized the entire approach in one place) would go a long way toward solving this.  Enumerating the four different search strategies (e.g. with headings or a bulleted list) would also make it easier for the reader to tell that these are four approaches to performing the same step in the larger algorithm and how how each strategy differs from the others.
  +
  +If there is a way to use a figure to show the reader a useful way to visualize or think about the inequality used for candidate search, that could help as well.
  +
  +One final issue related to the candidate search that doesn’t seem to be addressed directly by the current manuscript is the number of candidate values that violate the inequality on page 8.  Without having thought about it deeply, it seems to me as though most of the search strategies would tend to be biased toward certain candidates (assuming there is more than one).  If such biases are likely to exist, I’d like to know a bit more about what their consequences might be.  Answering these questions might go beyond the scope of the paper, but I think that they should at least be raised as possible topics for future exploration.
  +
  +# Experimental design
  +I like that there are two independent implementations of their methods, and I like that the authors have explored a large portion of the parameter space to compare the different algorithms.  I do have two concerns about the authors’ experimental design.
  +
  +I'm not convinced that the statistical tests performed by the authors are sufficient to show that their samples are unbiased.
  +
  +My understanding is that the algorithms are designed to sample uniformly from among the partitions in the feasible set.  Given this goal, I’d like to see statistical tests of the hypothesis that the samples are uniformly distributed. This could be evaluated with the following R program, which (given values of Q, N, and n_samples) tallies the number of times each partition is sampled and performs a Chi-square test for uniformity.
  +
  +  counts = table(apply(rand_partitions(Q, N, n_samples), 2, paste0, collapse = ","))
  +  uniform_prob = rep(1 / NrParts(Q, N), length(counts))
  +  chisq.test(counts, p = uniform_prob, rescale.p = TRUE)
  +
  +A convincing case for unbiasedness would perform similar tests with all four methods, with and without zeros, across a range of Q and N values.  Ideally, n_samples would exceed the number of partitions by a reasonable factor (e.g. 5 or 10) for most of these tests to provide sufficient statistical power and to satisfy the requirements of the chi-square approximation, but I understand that this could be difficult in many parts of the parameter space.  I performed some quick tests on my laptop for small S and N and found no apparent biases, but I’d like to see something more systematic.
  +
  +The uniformity test may not be feasible in large parts of the parameter space, so tests like those currently included in the paper will still be useful in those areas.  I’d like to see a bit more detail on what exactly was being fed as data to the t-tests and KS tests, however, as well as an explanation for why the variance was chosen rather than some other property of the partitions.
  +
  +A multinomial test would probably be even better than the chi-square test, because it does not require approximations.  The EMT package seems to provide the needed functionality, but I haven't really looked into it.
  +
  +Adding uniformity tests to the package’s unit test suite would also do a great deal to ensure that the software’s integrity was maintained as development proceeded.
  +
  +With regard to timing, it is not clear to me the comparison to Locey and White (2013) is meaningful.  Were the two sets of analyses run on comparable hardware? What does the phrase “sampling effort per dataset was not controlled or accounted for” mean?  I would much rather see one apples-to-apples comparison than eight comparisons that are difficult to evaluate on the same scale.
  +
  +# Validity
  +
  +I speak R better than Python, so I limited my review to the R code.
  +
  +The unit testing in the R package is rather sparse. Testing for uniformity with all four algorithms (see comments on experimental design) seems like it could be a very good idea.
  +
  +I’d also like to see some more care with regard to numeric classes and numerical precision.  R 3.x is unfortunately still limited to 32-bit signed integers.  On my machine, that means that it cannot represent numbers larger than 2,147,483,647 exactly. As a result, the 105-digit number in the first unit test is stored as a floating point number with only 15 or 16 digits of precision on most machines.  This is a potential problem because of bogus “equalities” like this one:
  +
  +[105-digit number] == [105-digit number + 1E80]
  +
  +Unfortunately, I’m not sure what the best option is for 64-bit integers in R these days. Romain Francois’ int64 package seems to be dead.  The gmp and bit64 packages should both work, but I don’t know how fast or user-friendly they are.
  +
  +I have not examined the Python version of the code, but my understanding of Python suggests that this issue is much less likely to be a problem.
  +
  +Finally, if there isn’t one already, I’d like to see a set of tests that confirms that the R and Python implementations of candidate search both find the same candidates when given the same inputs (including the same random number).  This would provide a nice check that all four search algorithms are working as expected.
  +
  +# Other R package comments
   1, data structure and metadata
   2, conservation
   3, phenology and climate
  
positive.code: 75, disturbance
  76, diversity estimation (and stuff?)
  77, remove??
  78, \"movement, foraging, and migration\"
  79, nitrogen
  80, invasions
  81, remote sensing
  
negative.code: 54, host-parasite interactions
  55, extinction in the fossil record
  56, species co-occurrence patterns?
  57, ""environment"" [remove?]
  58, anthropogenic change
  59, decomposition and nutrient cycling
  60, ""rivers, erosion, and nutrient flows""
  61, valuing nature
  62, marine ecology
  63, biogeography and urban ecology??
  