class_docstr: An approach for optimizing submodular functions on massive data sets.
  
  GreeDi is an optimizer that was designed to work on data sets that are 
  too large to fit into memory. The approach involves first partitioning 
  the data into :math:`m` equally sized chunks without any overlap. Then, 
  :math:`l` elements are selected from each chunk using a standard optimizer 
  like naive or lazy greedy. Finally, these :math:`ml` examples are merged 
  and a standard optimizer selects :math:`k` examples from this set. In 
  this manner, the algorithm sacrifices exactness to ensure that memory 
  limitations are not an issue. 
  
  There are a few considerations to keep in mind when using GreeDi. 
  Naturally, :math:`ml` must both be larger than :math:`k` and also small 
  enough to fit into memory. The larger :math:`l`, the closer the solution 
  is to the exact solution but also the more compute time is required. 
  Conversely, the larger :math:`m` is, the less exact the solution is. 
  When using a graph-based function, increasing :math:`m` can dramatically 
  reduce the amount of computation that needs to be performed, as well as 
  the memory requirements, because the similarity matrix becomes smaller 
  in size. However, feature-based functions are likely to see less of a 
  speed improvement because the cost of evaluating an example is independent 
  of the size of ground set.
  
  .. code::python
  
          from apricot import FeatureBasedSelection
  
          X = numpy.random.randint(10, size=(10000, 100))
  
          selector = FeatureBasedSelection(100, 'sqrt', optimizer='greedi')
          selector.fit(X)
  
  Parameters
  ----------
  function : base.BaseSelection
          A submodular function that implements the `_calculate_gains` and
          `_select_next` methods. This is the function that will be
          optimized.
  
  m : int
          The number of partitions to split the data into. 
  
  l : int
          The number of exemplars to select from each partition. l*m must
          be larger than the number of exemplars k that will be selected
          later on.
  
  optimizer1 : str or base.Optimizer, optional
          The optimizer to use in the first stage of the process where l
          exemplars are selected from each partition. Default is 'lazy'.
  
  optimizer2 : str or base.Optimizer, optional
          The optimizer to use in the second stage where k exemplars are
          selected from the l*m exemplars returned from the first stage.
          Default is 'lazy'.
  
  random_state : int or RandomState or None, optional
          The random seed to use for the random selection process.
  
  verbose : bool
          Whether to display a progress bar during the optimization process.
  
  
  Attributes
  ----------
  self.function : base.BaseSelection or None
          A submodular function that implements the `_calculate_gains` and
          `_select_next` methods. This is the function that will be
          optimized. If None, will be set by the selector when passed in.
  
  self.m : int
          The number of partitions that the data will be split into.
  
  self.l : int
          The number of exemplars that will be selected from each partition.
  
  self.verbose : bool
          Whether to display a progress bar during the optimization process.
  
  self.gains_ : numpy.ndarray or None
          The gain that each example would give the last time that it was
          evaluated.
class_docstr_tok: ["An","approach","for","optimizing","submodular","functions","on","massive","data","sets",".","GreeDi","is","an","optimizer","that","was","designed","to","work","on","data","sets","that","are","too","large","to","fit","into","memory",".","The","approach","involves","first","partitioning","the","data","into",":","math",":","`","m","`","equally","sized","chunks","without","any","overlap",".","Then",",",":","math",":","`","l","`","elements","are","selected","from","each","chunk","using","a","standard","optimizer","like","naive","or","lazy","greedy",".","Finally",",","these",":","math",":","`","ml","`","examples","are","merged","and","a","standard","optimizer","selects",":","math",":","`","k","`","examples","from","this","set",".","In","this","manner",",","the","algorithm","sacrifices","exactness","to","ensure","that","memory","limitations","are","not","an","issue",".","There","are","a","few","considerations","to","keep","in","mind","when","using","GreeDi",".","Naturally",",",":","math",":","`","ml","`","must","both","be","larger","than",":","math",":","`","k","`","and","also","small","enough","to","fit","into","memory",".","The","larger",":","math",":","`","l","`",",","the","closer","the","solution","is","to","the","exact","solution","but","also","the","more","compute","time","is","required",".","Conversely",",","the","larger",":","math",":","`","m","`","is",",","the","less","exact","the","solution","is",".","When","using","a","graph-based","function",",","increasing",":","math",":","`","m","`","can","dramatically","reduce","the","amount","of","computation","that","needs","to","be","performed",",","as","well","as","the","memory","requirements",",","because","the","similarity","matrix","becomes","smaller","in","size",".","However",",","feature-based","functions","are","likely","to","see","less","of","a","speed","improvement","because","the","cost","of","evaluating","an","example","is","independent","of","the","size","of","ground","set",".","..","code",":",":python","from","apricot","import","FeatureBasedSelection","X","=","numpy.random.randint","(","10",",","size=","(","10000",",","100",")",")","selector","=","FeatureBasedSelection","(","100",",","'sqrt","'",",","optimizer='greedi","'",")","selector.fit","(","X",")","Parameters","--","--","--","--","--","function",":","base.BaseSelection","A","submodular","function","that","implements","the","`","_calculate_gains","`","and","`","_select_next","`","methods",".","This","is","the","function","that","will","be","optimized",".","m",":","int","The","number","of","partitions","to","split","the","data","into",".","l",":","int","The","number","of","exemplars","to","select","from","each","partition",".","l","*","m","must","be","larger","than","the","number","of","exemplars","k","that","will","be","selected","later","on",".","optimizer1",":","str","or","base.Optimizer",",","optional","The","optimizer","to","use","in","the","first","stage","of","the","process","where","l","exemplars","are","selected","from","each","partition",".","Default","is","'lazy","'",".","optimizer2",":","str","or","base.Optimizer",",","optional","The","optimizer","to","use","in","the","second","stage","where","k","exemplars","are","selected","from","the","l","*","m","exemplars","returned","from","the","first","stage",".","Default","is","'lazy","'",".","random_state",":","int","or","RandomState","or","None",",","optional","The","random","seed","to","use","for","the","random","selection","process",".","verbose",":","bool","Whether","to","display","a","progress","bar","during","the","optimization","process",".","Attributes","--","--","--","--","--","self.function",":","base.BaseSelection","or","None","A","submodular","function","that","implements","the","`","_calculate_gains","`","and","`","_select_next","`","methods",".","This","is","the","function","that","will","be","optimized",".","If","None",",","will","be","set","by","the","selector","when","passed","in",".","self.m",":","int","The","number","of","partitions","that","the","data","will","be","split","into",".","self.l",":","int","The","number","of","exemplars","that","will","be","selected","from","each","partition",".","self.verbose",":","bool","Whether","to","display","a","progress","bar","during","the","optimization","process",".","self.gains_",":","numpy.ndarray","or","None","The","gain","that","each","example","would","give","the","last","time","that","it","was","evaluated","."]
code: def select(self, X, k, sample_cost=None):
      if self.m is None and self.l is None:
          self.m = 8
          self.l = min(k // 4, X.shape[0] // self.m)
  
      if sample_cost is None:
          sample_cost = numpy.ones(X.shape[0], dtype="float64")
  
      if k > (self.m * self.l):
          raise ValueError("k must be smaller than m * l")
  
      partitions = numpy.arange(X.shape[0]) % self.m
      self.random_state.shuffle(partitions)
  
      optimizer1 = OPTIMIZERS[self.optimizer1](
          function=self.function, verbose=self.verbose
      )
      optimizer2 = OPTIMIZERS[self.optimizer2](
          function=self.function, verbose=self.verbose
      )
  
      rankings = []
  
      for i in range(self.m):
          idxs = numpy.where(partitions == i)[0]
  
          X_subset = X[idxs]
          if X.shape[0] == X.shape[1]:
              X_subset = X_subset[:, idxs]
  
          self.function._initialize(X_subset)
          optimizer1.select(
              X_subset, self.l, sample_cost=sample_cost[idxs]
          )
  
          rankings.append(idxs[self.function.ranking])
  
      rankings = numpy.concatenate(rankings)
  
      if self.verbose:
          self.function.pbar.close()
          self.function.pbar = tqdm(total=k)
  
      X_subset = X[rankings]
      if X.shape[0] == X.shape[1]:
          X_subset = X_subset[:, rankings]
  
      self.function._initialize(X_subset)
      optimizer2.select(X_subset, k, sample_cost=sample_cost[rankings])
      rankings = rankings[self.function.ranking]
  
      self.function._initialize(X)
      for idx in rankings:
          gain = self.function._calculate_gains(X, numpy.array([idx]))[
              0
          ]
          self.function._select_next(X[idx], gain, idx)
  
imports: import os
  import copy
  import numpy
  import scipy
  from tqdm import tqdm
  from .utils import PriorityQueue
  from .utils import check_random_state
  from .utils import _calculate_pairwise_distances
label_desc: Use image node_id 2 for calling the GreeDi obj's underlying member method code with example usage: obj.select(X, k, sample_cost) without return types