class_docstr: The ArrayIterator class iterates over minibatches of data that
  have been preloaded into memory in the form of numpy arrays. This may be used when
  the entire dataset (e.g. CIFAR-10 or MNIST) is small enough to fit in memory. For example::
  
      X = np.random.rand(10000, 3072)
      y = np.random.randint(0, 10, 10000)
      train = ArrayIterator(X=X, y=y, nclass=10, lshape=(3, 32, 32))
  
  The above will create the ArrayIterator object. This object implements python's __iter__
  method, and returns one minibatch of data, formatted as tuple of (input, label), with
  each iteration. The minibatch size is controlled by the generated backend.
  
  X should be an ndarray of shape (# example, # features). For images, the features should be
  formatted in (channel, height, width) order. The `lshape` keyword indicates the local shape of
  the images in (channel, height, width) format.
  
  For classification tasks, the labels `y` should be integers from 0 to K-1, where K is the total
  number of classes. When `y` is not provided, the input features themselves will be returned
  as the target values (e.g. autoencoder).
  
  In regression tasks, where `y` is not a categorical label, set `make_onehot` to `False`.
  For example::
  
      X = np.random.rand(1000, 1)
      y = 2*X + 1
      train = ArrayIterator(X=X, y=y, make_onehot=False)
  
  For more information, see the Loading data section of the documentation.
docstr: During initialization, the input data will be converted to backend tensor objects
  (e.g. CPUTensor or GPUTensor). If the backend uses the GPU, the data is copied over to the
  device.
  
  Args:
      X (ndarray, shape: [# examples, feature size]): Input features of the
          dataset.
      y (ndarray, shape:[# examples, 1 or feature size], optional): Labels corresponding to
          the input features. If absent, the input features themselves will be returned as
          target values (e.g. autoencoder)
      nclass (int, optional): The number of classes in labels. Not necessary if
          labels are not provided or where the labels are non-categorical.
      lshape (tuple, optional): Local shape for the input features
          (e.g. # channels, height, width)
      make_onehot (bool, optional): True if y is a categorical label that has to be converted
          to a one hot representation.
docstr_tok: ["During","initialization",",","the","input","data","will","be","converted","to","backend","tensor","objects","(","e.g",".","CPUTensor","or","GPUTensor",")",".","If","the","backend","uses","the","GPU",",","the","data","is","copied","over","to","the","device",".","Args",":","X","(","ndarray",",","shape",":","[","#","examples",",","feature","size","]",")",":","Input","features","of","the","dataset",".","y","(","ndarray",",","shape",":","[","#","examples",",","1","or","feature","size","]",",","optional",")",":","Labels","corresponding","to","the","input","features",".","If","absent",",","the","input","features","themselves","will","be","returned","as","target","values","(","e.g",".","autoencoder",")","nclass","(","int",",","optional",")",":","The","number","of","classes","in","labels",".","Not","necessary","if","labels","are","not","provided","or","where","the","labels","are","non-categorical",".","lshape","(","tuple",",","optional",")",":","Local","shape","for","the","input","features","(","e.g",".","#","channels",",","height",",","width",")","make_onehot","(","bool",",","optional",")",":","True","if","y","is","a","categorical","label","that","has","to","be","converted","to","a","one","hot","representation","."]
code: def __init__(
      self,
      X,
      y=None,
      nclass=None,
      lshape=None,
      make_onehot=True,
      name=None,
  ):
      """
      During initialization, the input data will be converted to backend tensor objects
      (e.g. CPUTensor or GPUTensor). If the backend uses the GPU, the data is copied over to the
      device.
  
      Args:
          X (ndarray, shape: [# examples, feature size]): Input features of the
              dataset.
          y (ndarray, shape:[# examples, 1 or feature size], optional): Labels corresponding to
              the input features. If absent, the input features themselves will be returned as
              target values (e.g. autoencoder)
          nclass (int, optional): The number of classes in labels. Not necessary if
              labels are not provided or where the labels are non-categorical.
          lshape (tuple, optional): Local shape for the input features
              (e.g. # channels, height, width)
          make_onehot (bool, optional): True if y is a categorical label that has to be converted
              to a one hot representation.
  
      """
      # Treat singletons like list so that iteration follows same syntax
      super(ArrayIterator, self).__init__(name=name)
      X = X if isinstance(X, list) else [X]
      self.ndata = len(X[0])
      assert self.ndata >= self.be.bsz
      self.start = 0
      self.nclass = nclass
      self.ybuf = None
  
      if make_onehot and nclass is None and y is not None:
          raise AttributeError(
              "Must provide number of classes when creating onehot labels"
          )
  
      # if labels provided, they must have same # examples as the features
      if y is not None:
          assert all(
              [y.shape[0] == x.shape[0] for x in X]
          ), "Input features and labels must have equal number of examples."
  
          # for classifiction, the labels must be from 0 .. K-1, where K=nclass
          if make_onehot:
              assert (
                  y.max() <= nclass - 1 and y.min() >= 0
              ), "Labels must range from 0 to {} (nclass-1).".format(
                  nclass - 1
              )
  
              assert (
                  np.floor(y) == y
              ).all(), "Labels must only contain integers."
  
      # if local shape is provided, then the product of lshape should match the
      # number of features
      if lshape is not None:
          assert all(
              [x.shape[1] == np.prod(lshape) for x in X]
          ), "product of lshape {} does not match input feature size".format(
              lshape
          )
  
      # store shape of the input data
      self.shape = [x.shape[1] if lshape is None else lshape for x in X]
      if len(self.shape) == 1:
          self.shape = self.shape[0]
          self.lshape = lshape
  
      # Helpers to make dataset, minibatch, unpacking function for transpose and onehot
      def transpose_gen(z):
          return (
              self.be.array(z),
              self.be.iobuf(z.shape[1]),
              lambda _in, _out: self.be.copy_transpose(_in, _out),
          )
  
      def onehot_gen(z):
          return (
              self.be.array(z.reshape((-1, 1)), dtype=np.int32),
              self.be.iobuf(nclass),
              lambda _in, _out: self.be.onehot(_in, axis=0, out=_out),
          )
  
      self.Xdev, self.Xbuf, self.unpack_func = list(
          zip(*[transpose_gen(x) for x in X])
      )
  
      # Shallow copies for appending, iterating
      self.dbuf, self.hbuf = list(self.Xdev), list(self.Xbuf)
      self.unpack_func = list(self.unpack_func)
  
      if y is not None:
          self.ydev, self.ybuf, yfunc = (
              onehot_gen(y) if make_onehot else transpose_gen(y)
          )
          self.dbuf.append(self.ydev)
          self.hbuf.append(self.ybuf)
          self.unpack_func.append(yfunc)
  
file_path: neon/neon/data/dataiterator.py
imports: from builtins import zip
  import numpy
  from neon import NervanaObject
label_desc: Use image node_id 1 to create a new ArrayIterator object from inherited base classes: NervanaDataIterator with example: obj = ArrayIterator(X, y, nclass, lshape, make_onehot, name)