title: How to get mini-batches in pytorch in a clean and efficient way?
body: <p>I was trying to do a simple thing which was train a linear model with Stochastic Gradient Descent (SGD) using torch:</p>
  
  <pre><code>import numpy as np
  
  import torch
  from torch.autograd import Variable
  
  import pdb
  
  def get_batch2(X,Y,M,dtype):
      X,Y = X.data.numpy(), Y.data.numpy()
      N = len(Y)
      valid_indices = np.array( range(N) )
      batch_indices = np.random.choice(valid_indices,size=M,replace=False)
      batch_xs = torch.FloatTensor(X[batch_indices,:]).type(dtype)
      batch_ys = torch.FloatTensor(Y[batch_indices]).type(dtype)
      return Variable(batch_xs, requires_grad=False), Variable(batch_ys, requires_grad=False)
  
  def poly_kernel_matrix( x,D ):
      N = len(x)
      Kern = np.zeros( (N,D+1) )
      for n in range(N):
          for d in range(D+1):
              Kern[n,d] = x[n]**d;
      return Kern
  
  ## data params
  N=5 # data set size
  Degree=4 # number dimensions/features
  D_sgd = Degree+1
  ##
  x_true = np.linspace(0,1,N) # the real data points
  y = np.sin(2*np.pi*x_true)
  y.shape = (N,1)
  ## TORCH
  dtype = torch.FloatTensor
  # dtype = torch.cuda.FloatTensor # Uncomment this to run on GPU
  X_mdl = poly_kernel_matrix( x_true,Degree )
  X_mdl = Variable(torch.FloatTensor(X_mdl).type(dtype), requires_grad=False)
  y = Variable(torch.FloatTensor(y).type(dtype), requires_grad=False)
  ## SGD mdl
  w_init = torch.zeros(D_sgd,1).type(dtype)
  W = Variable(w_init, requires_grad=True)
  M = 5 # mini-batch size
  eta = 0.1 # step size
  for i in range(500):
      batch_xs, batch_ys = get_batch2(X_mdl,y,M,dtype)
      # Forward pass: compute predicted y using operations on Variables
      y_pred = batch_xs.mm(W)
      # Compute and print loss using operations on Variables. Now loss is a Variable of shape (1,) and loss.data is a Tensor of shape (1,); loss.data[0] is a scalar value holding the loss.
      loss = (1/N)*(y_pred - batch_ys).pow(2).sum()
      # Use autograd to compute the backward pass. Now w will have gradients
      loss.backward()
      # Update weights using gradient descent; w1.data are Tensors,
      # w.grad are Variables and w.grad.data are Tensors.
      W.data -= eta * W.grad.data
      # Manually zero the gradients after updating weights
      W.grad.data.zero_()
  
  #
  c_sgd = W.data.numpy()
  X_mdl = X_mdl.data.numpy()
  y = y.data.numpy()
  #
  Xc_pinv = np.dot(X_mdl,c_sgd)
  print('J(c_sgd) = ', (1/N)*(np.linalg.norm(y-Xc_pinv)**2) )
  print('loss = ',loss.data[0])
  </code></pre>
  
  <p>the code runs fine and all though my <code>get_batch2</code> method seems really dum/naive, its probably because I am new to pytorch but I have not found a good place where they discuss how to retrieve data batches. I went through their tutorials (<a href="http://pytorch.org/tutorials/beginner/pytorch_with_examples.html" rel="noreferrer">http://pytorch.org/tutorials/beginner/pytorch_with_examples.html</a>) and through the data set (<a href="http://pytorch.org/tutorials/beginner/data_loading_tutorial.html" rel="noreferrer">http://pytorch.org/tutorials/beginner/data_loading_tutorial.html</a>) with no luck. The tutorials all seem to assume that one already has the batch and batch-size at the beginning and then proceeds to train with that data without changing it (specifically look at <a href="http://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-variables-and-autograd" rel="noreferrer">http://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-variables-and-autograd</a>).</p>
  
  <p>So my question is do I really need to turn my data back into numpy so that I can fetch some random sample of it and then turn it back to pytorch with Variable to be able to train in memory? Is there no way to get mini-batches with torch?</p>
  
  <p>I looked at a few functions torch provides but with no luck:</p>
  
  <pre><code>#pdb.set_trace()
  #valid_indices = torch.arange(0,N).numpy()
  #valid_indices = np.array( range(N) )
  #batch_indices = np.random.choice(valid_indices,size=M,replace=False)
  #indices = torch.LongTensor(batch_indices)
  #batch_xs, batch_ys = torch.index_select(X_mdl, 0, indices), torch.index_select(y, 0, indices)
  #batch_xs,batch_ys = torch.index_select(X_mdl, 0, indices), torch.index_select(y, 0, indices)
  </code></pre>
  
  <p>even though the code I provided works fine I am worried that its not an efficient implementation AND that if I were to use GPUs that there would be a considerable further slow down (because my guess it putting things in memory and then fetching them back to put them GPU like that is silly).</p>
  
  <hr>
  
  <p>I implemented a new one based on the answer that suggested to use <code>torch.index_select()</code>:</p>
  
  <pre><code>def get_batch2(X,Y,M):
      '''
      get batch for pytorch model
      '''
      # TODO fix and make it nicer, there is pytorch forum question
      #X,Y = X.data.numpy(), Y.data.numpy()
      X,Y = X, Y
      N = X.size()[0]
      batch_indices = torch.LongTensor( np.random.randint(0,N+1,size=M) )
      pdb.set_trace()
      batch_xs = torch.index_select(X,0,batch_indices)
      batch_ys = torch.index_select(Y,0,batch_indices)
      return Variable(batch_xs, requires_grad=False), Variable(batch_ys, requires_grad=False)
  </code></pre>
  
  <p>however, this seems to have issues because it does not work if <code>X,Y</code> are NOT variables...which is really odd. I added this to the pytorch forum: <a href="https://discuss.pytorch.org/t/how-to-get-mini-batches-in-pytorch-in-a-clean-and-efficient-way/10322" rel="noreferrer">https://discuss.pytorch.org/t/how-to-get-mini-batches-in-pytorch-in-a-clean-and-efficient-way/10322</a></p>
  
  <p>Right now what I am struggling with is making this work for gpu. My most current version:</p>
  
  <pre><code>def get_batch2(X,Y,M,dtype):
      '''
      get batch for pytorch model
      '''
      # TODO fix and make it nicer, there is pytorch forum question
      #X,Y = X.data.numpy(), Y.data.numpy()
      X,Y = X, Y
      N = X.size()[0]
      if dtype ==  torch.cuda.FloatTensor:
          batch_indices = torch.cuda.LongTensor( np.random.randint(0,N,size=M) )# without replacement
      else:
          batch_indices = torch.LongTensor( np.random.randint(0,N,size=M) ).type(dtype)  # without replacement
      pdb.set_trace()
      batch_xs = torch.index_select(X,0,batch_indices)
      batch_ys = torch.index_select(Y,0,batch_indices)
      return Variable(batch_xs, requires_grad=False), Variable(batch_ys, requires_grad=False)
  </code></pre>
  
  <p>the error:</p>
  
  <pre><code>RuntimeError: tried to construct a tensor from a int sequence, but found an item of type numpy.int64 at index (0)
  </code></pre>
  
  <p>I don't get it, do I really have to do:</p>
  
  <pre><code>ints = [ random.randint(0,N) for i i range(M)]
  </code></pre>
  
  <p>to get the integers?</p>
  
  <p>It would also be ideal if the data could be a variable. It seems that it <code>torch.index_select</code> does not work for <code>Variable</code> type data.</p>
  
  <p>this list of integers thing still doesn't work:</p>
  
  <pre><code>TypeError: torch.addmm received an invalid combination of arguments - got (int, torch.cuda.FloatTensor, int, torch.cuda.FloatTensor, torch.FloatTensor, out=torch.cuda.FloatTensor), but expected one of:
   * (torch.cuda.FloatTensor source, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
   * (torch.cuda.FloatTensor source, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
   * (float beta, torch.cuda.FloatTensor source, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
   * (torch.cuda.FloatTensor source, float alpha, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
   * (float beta, torch.cuda.FloatTensor source, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
   * (torch.cuda.FloatTensor source, float alpha, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
   * (float beta, torch.cuda.FloatTensor source, float alpha, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
        didn't match because some of the arguments have invalid types: (int, torch.cuda.FloatTensor, int, torch.cuda.FloatTensor, torch.FloatTensor, out=torch.cuda.FloatTensor)
   * (float beta, torch.cuda.FloatTensor source, float alpha, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
        didn't match because some of the arguments have invalid types: (int, torch.cuda.FloatTensor, int, torch.cuda.FloatTensor, torch.FloatTensor, out=torch.cuda.FloatTensor)
  </code></pre>