How to use pytorch's cat function for K-Fold Validation (i.e. concatenate a list of pytorch chunks together)
I'm working on splitting up a data set for k-fold cross validation but having trouble with concatenating a list of tensors using Pytorch's stack/cat functions.
First, I split the training and test set into chunks using the .chunk method as follows
  x_train_folds = torch.chunk(x_train, num_folds)
  y_train_folds = torch.chunk(y_train, num_folds)

Where x_train is a tensor of torch.Size([5000, 3, 32, 32]) and y_train is a tensor of torch.Size([5000])
x_train_folds and y_train_folds are now tuples of num_folds tensors
Then, I need to setup a series of nested loops to iterate through the different values for K, and the various folds, while always excluding one fold from the training set to be used at test/validation time:
  for k in k_choices:
    k_to_accuracies[k] = [] # create empty space to append for a given k-value
    for fold in range(num_folds):
      # create training sets by excluding the current loop index fold and using that as the test set
      x_train_cross_val = torch.cat((x_train_folds[:fold], x_train_folds[fold+1:]), 0)
      y_train_cross_val = torch.cat((y_train_folds[:fold], y_train_folds[fold+1:]), 0)
      classifier = KnnClassifier(x_train_cross_val, y_train_cross_val)
      k_to_accuracies[k].append(classifier.check_accuracy(x_train_folds[fold], y_train_folds[fold], k=k))

As you can see, I'm always skipping one fold from the original training set to be used for validation. This is standard K-fold cross validation.
Unfortunately, I am getting the following error which I cannot seem to figure out:
TypeError: expected Tensor as element 0 in argument 0, but got tuple
As you can see in the API listing, .cat seems to need a tuple of tensors which is what I have.
https://pytorch.org/docs/stable/torch.html#torch.cat
Does anyone have any suggestions?
Much appreciated
-Drew

try:
x_train_cross_val = torch.cat((*x_train_folds[:fold], *x_train_folds[fold+1:]), 0)
y_train_cross_val = torch.cat((*y_train_folds[:fold], *y_train_folds[fold+1:]), 0)


torch.cat receives a tuple whose elements are torch.Tensor type. However, the elements in your tuple x_train_folds[:fold] are still tuple. So, you need to remove the tuple 'decorator' of your tensors.
