Reservoir sampling
Introduction '''Reservoir sampling''' is a family of randomized algorithms for choosing a simple random sample, without replacement, of  items from a population of unknown size  in a single pass over the items. The size of the population  is not known to the algorithm and is typically too large for all  items to fit into main memory. The population is revealed to the algorithm over time, and the algorithm cannot look back at previous items. At any point, the current state of the algorithm must permit extraction of a simple random sample without replacement of size  over the part of the population seen so far.
Motivation Suppose we see a sequence of items, one at a time. We want to keep ten items in memory, and we want them to be selected at random from the sequence. If we know the total number of items  and can access the items arbitrarily, then the solution is easy: select 10 distinct indices  between 1 and  with equal probability, and keep the -th elements. The problem is that we do not always know the exact  in advance.
Simple: Algorithm R A simple and popular but slow algorithm, ''Algorithm R'', was created by Jeffrey Vitter.Initialize an array  indexed from  to , containing the first  items of the input . This is the ''reservoir''. For each new input , generate a random number  uniformly in . If , then set , otherwise, discard . Return  after all inputs are processed.This algorithm works by induction on .While conceptually simple and easy to understand, this algorithm needs to generate a random number for each item of the input, including the items that are discarded. The algorithm's asymptotic running time is thus . Generating this amount of randomness and the linear run time causes the algorithm to be unnecessarily slow if the input population is large.This is ''Algorithm R,'' implemented as follows:(* S has items to sample, R will contain the result *)ReservoirSample(S1..n, R1..k)  // fill the reservoir array  for i := 1 to k      Ri := Si  end  // replace elements with gradually decreasing probability  for i := k+1 to n    (* randomInteger(a, b) generates a uniform integer from the inclusive range {a, ..., b} *)    j := randomInteger(1, i)    if j
Optimal: Algorithm L If we generate  random numbers  independently, then the indices of the smallest  of them is a uniform sample of the k-subsets of . The process can be done without knowing : Keep the smallest  of  that has been seen so far, as well as , the index of the largest among them. For each new , compare it with . If , then discard , store , and set  to be the index of the largest among them. Otherwise, discard , and set .Now couple this with the stream of inputs . Every time some  is accepted, store the corresponding . Every time some  is discarded, discard the corresponding .This algorithm still needs  random numbers, thus taking  time. But it can be simplified.First simplification: it is unnecessary to test new  one by one, since the probability that the next acceptance happens at  is , that is, the interval  of acceptance follows a geometric distribution.Second simplification: it's unnecessary to remember the entire array of the smallest  of  that has been seen so far, but merely , the largest among them. This is based on three observations:* Every time some new  is selected to be entered into storage, a uniformly random entry in storage is discarded. * *  has the same distribution as , where all  independently. This can be sampled by first sampling , then taking .This is ''Algorithm L'', which is implemented as follows:(* S has items to sample, R will contain the result *)ReservoirSample(S1..n, R1..k)  // fill the reservoir array  for i = 1 to k      Ri := Si  end  (* random() generates a uniform (0,1) random number *)  W := exp(log(random())/k)  while i This algorithm computes three random numbers for each item that becomes part of the reservoir, and does not spend any time on items that do not. Its expected running time is thus , which is optimal. At the same time, it is simple to implement efficiently and does not depend on random deviates from exotic or hard-to-compute distributions.
With random sort If we associate with each item of the input a uniformly generated random number, the  items with the largest (or, equivalently, smallest) associated values form a simple random sample. A simple reservoir-sampling thus maintains the  items with the currently largest associated values in a priority queue. (*  S is a stream of items to sample  S.Current returns current item in stream  S.Next advances stream to next position  min-priority-queue supports:    Count -> number of items in priority queue    Minimum -> returns minimum key value of all items    Extract-Min() -> Remove the item with minimum key    Insert(key, Item) -> Adds item with specified key *)ReservoirSample(S1..?) H := new min-priority-queue  while S has data    r := random()   // uniformly random between 0 and 1, exclusive    if H.Count  H.Minimum        H.Extract-Min()        H.Insert(r, S.Current)    end    S.Next  end  return items in HendThe expected running time of this algorithm is  and it is relevant mainly because it can easily be extended to items with weights.
Weighted random sampling This method, also called sequential sampling, is incorrect in the sense that it does not allow to obtain a priori fixed inclusion probabilities.Some applications require items' sampling probabilities to be according to weights associated with each item. For example, it might be required to sample queries in a search engine with weight as number of times they were performed so that the sample can be analyzed for overall impact on user experience. Let the weight of item  be , and the sum of all weights be . There are two ways to interpret weights assigned to each item in the set:# In each round, the probability of every ''unselected'' item to be selected in that round is proportional to its weight relative to the weights of all unselected items. If  is the current sample, then the probability of an item  to be selected in the current round is .# The probability of each item to be included in the random sample is proportional to its relative weight, i.e., . Note that this interpretation might not be achievable in some cases, e.g., . === Algorithm A-Res ===The following algorithm was given by Efraimidis and Spirakis that uses interpretation 1:(*  S is a stream of items to sample  S.Current returns current item in stream  S.Weight  returns weight of current item in stream  S.Next advances stream to next position  The power operator is represented by ^  min-priority-queue supports:    Count -> number of items in priority queue    Minimum() -> returns minimum key value of all items    Extract-Min() -> Remove the item with minimum key    Insert(key, Item) -> Adds item with specified key *)ReservoirSample(S1..?) H := new min-priority-queue  while S has data    r := random() ^ (1/S.Weight)   // random() produces a uniformly random number in (0,1)    if H.Count  H.Minimum        H.Extract-Min()        H.Insert(r, S.Current)      end    end    S.Next  end  return items in HendThis algorithm is identical to the algorithm given in Reservoir Sampling with Random Sort except for the generation of the items' keys. The algorithm is equivalent to assigning each item a key  where  is the random number and then selecting the  items with the largest keys. Equivalently, a more numerically stable formulation of this algorithm computes the keys as  and select the  items with the ''smallest'' keys.=== Algorithm A-ExpJ ===The following algorithm is a more efficient version of A-Res, also given by Efraimidis and Spirakis:(*  S is a stream of items to sample  S.Current returns current item in stream  S.Weight  returns weight of current item in stream  S.Next advances stream to next position  The power operator is represented by ^  min-priority-queue supports:    Count -> number of items in the priority queue    Minimum -> minimum key of any item in the priority queue    Extract-Min() -> Remove the item with minimum key    Insert(Key, Item) -> Adds item with specified key *)ReservoirSampleWithJumps(S1..?) H := new min-priority-queue  while S has data and H.Count This algorithm follows the same mathematical properties that are used in A-Res, but instead of calculating the key for each item and checking whether that item should be inserted or not, it calculates an exponential jump to the next item which will be inserted. This avoids having to create random variates for each item, which may be expensive. The number of random variates required is reduced from  to  in expectation, where  is the reservoir size, and  is the number of items in the stream.=== Algorithm A-Chao ===Warning: the following description is wrong, see Chao's original paper and the discussion  here.Following algorithm was given by M. T. Chao uses interpretation 2: and Tillé (2006). (*  S has items to sample, R will contain the result  Si.Weight contains weight for each item *)WeightedReservoir-Chao(S1..n, R1..k)  WSum := 0  // fill the reservoir array  for i := 1 to k      Ri := Si      WSum := WSum + Si.Weight  end  for i := k+1 to n    WSum := WSum + Si.Weight    p := Si.Weight / WSum // probability for this item    j := random();          // uniformly random between 0 and 1    if j For each item, its relative weight is calculated and used to randomly decide if the item will be added into the reservoir. If the item is selected, then one of the existing items of the reservoir is uniformly selected and replaced with the new item. The trick here is that, if the probabilities of all items in the reservoir are already proportional to their weights, then by selecting uniformly which item to replace, the probabilities of all items remain proportional to their weight after the replacement.Note that Chao doesn't specify how to sample the first ''k'' elements.He simple assumes we have some other way of picking them in proportion to their weight.Chao: "Assume that we have a sampling plan of fixed size with respect to ''S_k'' at time ''A''; such that its first-order inclusion probability of ''X_t'' is ''π(k; i)''".==== Algorithm A-Chao with Jumps ====Similar to the other algorithms, it is possible to compute a random weight j and subtract items' probability mass values, skipping them while j > 0, reducing the number of random numbers that have to be generated. (*  S has items to sample, R will contain the result  Si.Weight contains weight for each item *)WeightedReservoir-Chao(S1..n, R1..k)  WSum := 0  // fill the reservoir array  for i := 1 to k      Ri := Si      WSum := WSum + Si.Weight  end  j := random() // uniformly random between 0 and 1  pNone := 1 // probability that no item has been selected so far (in this jump)  for i := k+1 to n    WSum := WSum + Si.Weight    p := Si.Weight / WSum // probability for this item    j -= p * pNone    pNone := pNone * (1 - p)    if j
Relation to Fisher–Yates shuffle Suppose one wanted to draw  random cards from a deck of cards.A natural approach would be to shuffle the deck and then take the top  cards.In the general case, the shuffle also needs to work even if the number of cards in the deck is not known in advance, a condition which is satisfied by the inside-out version of the Fisher–Yates shuffle:(* S has the input, R will contain the output permutation *)Shuffle(S1..n, R1..n)  R1 := S1  for i from 2 to n do      j := randomInteger(1, i)  // inclusive range      Ri := Rj      Rj := Si  endendNote that although the rest of the cards are shuffled, only the first  are important in the present context.Therefore, the array  need only track the cards in the first  positions while performing the shuffle, reducing the amount of memory needed.Truncating  to length , the algorithm is modified accordingly:(* S has items to sample, R will contain the result *)ReservoirSample(S1..n, R1..k)  R1 := S1  for i from 2 to k do      j := randomInteger(1, i)  // inclusive range      Ri := Rj      Rj := Si  end  for i from k + 1 to n do      j := randomInteger(1, i)  // inclusive range      if (j Since the order of the first  cards is immaterial, the first loop can be removed and  can be initialized to be the first  items of the input.This yields ''Algorithm R''.
Limitations Reservoir sampling makes the assumption that the desired sample fits into main memory, often implying that  is a constant independent of . In applications where we would like to select a large subset of the input list (say a third, i.e. ), other methods need to be adopted. Distributed implementations for this problem have been proposed.
References