class_docstr: Subsample index generator.
  
  Generates cross-validation folds according used to create ``J``
  partitions of the data and ``v`` folds on each partition according to as
  per [#]_:
  
      1. Split ``X`` into ``J`` partitions
  
      2. For each partition:
  
          (a) For each fold ``v``, create train index of all idx not in ``v``
  
          (b) Concatenate all the fold ``v`` indices into a test index for
              fold ``v`` that spans all partitions
  
  Setting ``J = 1`` is equivalent to the :class:`FullIndexer`, which returns
  standard K-Fold train and test set indices.
  
  See Also
  --------
  :class:`FoldIndex`, :class:`BlendIndex`, :class:`Subsemble`
  
  References
  ----------
  .. [#] Sapp, S., van der Laan, M. J., & Canny, J. (2014). Subsemble: an
     ensemble method for combining subset-specific algorithm fits. Journal
     of Applied Statistics, 41(6), 1247-1259.
     http://doi.org/10.1080/02664763.2013.864263
  
  Parameters
  ----------
  partitions : int, list (default = 2)
      Number of partitions to split data in. If ``partitions=1``,
      :class:`SubsetIndex` reduces to standard K-Fold.
  
  folds : int (default = 2)
      Number of splits to create in each partition. ``folds`` can
      not be 1 if ``n_partition > 1``. Note that if ``folds = 1``,
      both the train and test set will index the full data.
  
  X : array-like of shape [n_samples,] , optional
      the training set to partition. The training label array is also,
      accepted, as only the first dimension is used. If ``X`` is not
      passed at instantiation, the ``fit`` method must be called before
      ``generate``, or ``X`` must be passed as an argument of
      ``generate``.
  
  raise_on_exception : bool (default = True)
      whether to warn on suspicious slices or raise an error.
  
  Examples
  --------
  >>> import numpy as np
  >>> from mlens.index import SubsetIndex
  >>> X = np.arange(10)
  >>> idx = SubsetIndex(3, X=X)
  >>>
  >>> print('Expected partitions of X:')
  >>> print('J = 1: {!r}'.format(X[0:4]))
  >>> print('J = 2: {!r}'.format(X[4:7]))
  >>> print('J = 3: {!r}'.format(X[7:10]))
  >>> print('SubsetIndexer partitions:')
  >>> for i, part in enumerate(idx.partition(as_array=True)):
  ...     print('J = {}: {!r}'.format(i + 1, part))
  >>> print('SubsetIndexer folds on partitions:')
  >>> for i, (tri, tei) in enumerate(idx.generate()):
  ...     fold = i % 2 + 1
  ...     part = i // 2 + 1
  ...     train = np.hstack([np.arange(t0, t1) for t0, t1 in tri])
  ...     test = np.hstack([np.arange(t0, t1) for t0, t1 in tei])
  >>>     print("J = %i | f = %i | "
  ...           "train: %15r | test: %r" % (part, fold, train, test))
  Expected partitions of X:
  J = 1: array([0, 1, 2, 3])
  J = 2: array([4, 5, 6])
  J = 3: array([7, 8, 9])
  SubsetIndexer partitions:
  J = 1: array([0, 1, 2, 3])
  J = 2: array([4, 5, 6])
  J = 3: array([7, 8, 9])
  SubsetIndexer folds on partitions:
  J = 1 | f = 1 | train:   array([2, 3]) | test: array([0, 1, 4, 5, 7, 8])
  J = 1 | f = 2 | train:   array([0, 1]) | test: array([2, 3, 6, 9])
  J = 2 | f = 1 | train:      array([6]) | test: array([0, 1, 4, 5, 7, 8])
  J = 2 | f = 2 | train:   array([4, 5]) | test: array([2, 3, 6, 9])
  J = 3 | f = 1 | train:      array([9]) | test: array([0, 1, 4, 5, 7, 8])
  J = 3 | f = 2 | train:   array([7, 8]) | test: array([2, 3, 6, 9])
class_docstr_tok: ["Subsample","index","generator",".","Generates","cross-validation","folds","according","used","to","create","``","J","``","partitions","of","the","data","and","``","v","``","folds","on","each","partition","according","to","as","per","[","#","]","_",":","1",".","Split","``","X","``","into","``","J","``","partitions","2",".","For","each","partition",":","(","a",")","For","each","fold","``","v","``",",","create","train","index","of","all","idx","not","in","``","v","``","(","b",")","Concatenate","all","the","fold","``","v","``","indices","into","a","test","index","for","fold","``","v","``","that","spans","all","partitions","Setting","``","J","=","1","``","is","equivalent","to","the",":","class",":","`","FullIndexer","`",",","which","returns","standard","K-Fold","train","and","test","set","indices",".","See","Also","--","--","--","--",":","class",":","`","FoldIndex","`",",",":","class",":","`","BlendIndex","`",",",":","class",":","`","Subsemble","`","References","--","--","--","--","--","..","[","#","]","Sapp",",","S.",",","van","der","Laan",",","M.","J.",",","&","Canny",",","J",".","(","2014",")",".","Subsemble",":","an","ensemble","method","for","combining","subset-specific","algorithm","fits",".","Journal","of","Applied","Statistics",",","41","(","6",")",",","1247-1259.","http",":","\/\/doi.org\/10.1080\/02664763.2013.864263","Parameters","--","--","--","--","--","partitions",":","int",",","list","(","default","=","2",")","Number","of","partitions","to","split","data","in",".","If","``","partitions=1","``",",",":","class",":","`","SubsetIndex","`","reduces","to","standard","K-Fold",".","folds",":","int","(","default","=","2",")","Number","of","splits","to","create","in","each","partition",".","``","folds","``","can","not","be","1","if","``","n_partition",">","1","``",".","Note","that","if","``","folds","=","1","``",",","both","the","train","and","test","set","will","index","the","full","data",".","X",":","array-like","of","shape","[","n_samples",",","]",",","optional","the","training","set","to","partition",".","The","training","label","array","is","also",",","accepted",",","as","only","the","first","dimension","is","used",".","If","``","X","``","is","not","passed","at","instantiation",",","the","``","fit","``","method","must","be","called","before","``","generate","``",",","or","``","X","``","must","be","passed","as","an","argument","of","``","generate","``",".","raise_on_exception",":","bool","(","default","=","True",")","whether","to","warn","on","suspicious","slices","or","raise","an","error",".","Examples","--","--","--","--",">",">",">","import","numpy","as","np",">",">",">","from","mlens.index","import","SubsetIndex",">",">",">","X","=","np.arange","(","10",")",">",">",">","idx","=","SubsetIndex","(","3",",","X=X",")",">",">",">",">",">",">","print","(","'Expected","partitions","of","X",":","'",")",">",">",">","print","(","'","J","=","1",":","{","!","r","}","'.format","(","X","[","0:4","]",")",")",">",">",">","print","(","'","J","=","2",":","{","!","r","}","'.format","(","X","[","4:7","]",")",")",">",">",">","print","(","'","J","=","3",":","{","!","r","}","'.format","(","X","[","7:10","]",")",")",">",">",">","print","(","'SubsetIndexer","partitions",":","'",")",">",">",">","for","i",",","part","in","enumerate","(","idx.partition","(","as_array=True",")",")",":","...","print","(","'","J","=","{","}",":","{","!","r","}","'.format","(","i","+","1",",","part",")",")",">",">",">","print","(","'SubsetIndexer","folds","on","partitions",":","'",")",">",">",">","for","i",",","(","tri",",","tei",")","in","enumerate","(","idx.generate","(",")",")",":","...","fold","=","i","%","2","+","1","...","part","=","i","\/\/","2","+","1","...","train","=","np.hstack","(","[","np.arange","(","t0",",","t1",")","for","t0",",","t1","in","tri","]",")","...","test","=","np.hstack","(","[","np.arange","(","t0",",","t1",")","for","t0",",","t1","in","tei","]",")",">",">",">","print","(","``","J","=","%","i","|","f","=","%","i","|","``","...","``","train",":","%","15r","|","test",":","%","r","''","%","(","part",",","fold",",","train",",","test",")",")","Expected","partitions","of","X",":","J","=","1",":","array","(","[","0",",","1",",","2",",","3","]",")","J","=","2",":","array","(","[","4",",","5",",","6","]",")","J","=","3",":","array","(","[","7",",","8",",","9","]",")","SubsetIndexer","partitions",":","J","=","1",":","array","(","[","0",",","1",",","2",",","3","]",")","J","=","2",":","array","(","[","4",",","5",",","6","]",")","J","=","3",":","array","(","[","7",",","8",",","9","]",")","SubsetIndexer","folds","on","partitions",":","J","=","1","|","f","=","1","|","train",":","array","(","[","2",",","3","]",")","|","test",":","array","(","[","0",",","1",",","4",",","5",",","7",",","8","]",")","J","=","1","|","f","=","2","|","train",":","array","(","[","0",",","1","]",")","|","test",":","array","(","[","2",",","3",",","6",",","9","]",")","J","=","2","|","f","=","1","|","train",":","array","(","[","6","]",")","|","test",":","array","(","[","0",",","1",",","4",",","5",",","7",",","8","]",")","J","=","2","|","f","=","2","|","train",":","array","(","[","4",",","5","]",")","|","test",":","array","(","[","2",",","3",",","6",",","9","]",")","J","=","3","|","f","=","1","|","train",":","array","(","[","9","]",")","|","test",":","array","(","[","0",",","1",",","4",",","5",",","7",",","8","]",")","J","=","3","|","f","=","2","|","train",":","array","(","[","7",",","8","]",")","|","test",":","array","(","[","2",",","3",",","6",",","9","]",")"]
docstr: Build global test folds for each split of every partition.
  
  This method runs through each partition and fold to register all the
  test set indices across partitions. For each test fold ``i``, the test
  set indices are thus the union of fold ``i`` indices across all ``J``
  partitions.
docstr_tok: ["Build","global","test","folds","for","each","split","of","every","partition",".","This","method","runs","through","each","partition","and","fold","to","register","all","the","test","set","indices","across","partitions",".","For","each","test","fold","``","i","``",",","the","test","set","indices","are","thus","the","union","of","fold","``","i","``","indices","across","all","``","J","``","partitions","."]
code: def _build_test_sets(self):
      """Build global test folds for each split of every partition.
  
      This method runs through each partition and fold to register all the
      test set indices across partitions. For each test fold ``i``, the test
      set indices are thus the union of fold ``i`` indices across all ``J``
      partitions.
      """
      partitions = self.partitions
      n_samples = self.n_samples
      folds = self.folds
  
      # --- Create global test set folds ---
      # In each partition, the test set spans all partitions
      # Hence we must run through all partitions once first to
      # register
      # the global test set fold for each split of the folds
  
      # Partition sizes
      p_len = partition(n_samples, partitions)
  
      # Since the splitting is sequential and deterministic,
      # we build a
      # list of global test set indices. Hence, for any split, the
      # test index will be a tuple of indexes of test folds from each
      # partition. By concatenating these slices, the full test fold
      # is constructed.
      tei = [[] for _ in range(folds)]
      p_last = 0
      for p_size in p_len:
          p_start, p_stop = p_last, p_last + p_size
  
          t_len = partition(p_stop - p_start, folds)
  
          # Append the partition's test fold indices to the
          # global directory for that fold number.
          t_last = p_start
          for i, t_size in enumerate(t_len):
              t_start, t_stop = t_last, t_last + t_size
  
              tei[i] += [(t_start, t_stop)]
              t_last += t_size
  
          p_last += p_size
  
      return tei
  
file_path: mlens/mlens/index/subsemble.py
imports: from __future__ import division
  import numpy
  from ._checks import check_subsample_index
  from .base import BaseIndex, partition, make_tuple, prune_train
label_desc: Use image node_id 4 for calling the SubsetIndex obj's underlying member method code with example usage: obj._build_test_sets() and returns: tei