title: PLS in R: Predicting new observations returns Fitted values instead
body: <p>In the past few days I have developed multiple PLS models in R for spectral data (wavebands as explanatory variables) and various vegetation parameters (as individual response variables). In total, the dataset comprises of 56. The first 28 (training set) have been used for model calibration, now all I want to do is to predict the response values for the remaining 28 observations in the tesset. For some reason, however, R keeps on the returning the fitted values of the calibration set for a given number of components rather than predictions for the independent test set. Here is what the model looks like in short.</p>
  
  <pre><code># first simulate some data
  set.seed(123)
  bands=101
  data &lt;- data.frame(matrix(runif(56*bands),ncol=bands))
  colnames(data) &lt;- paste0(1:bands)
  data$height &lt;- rpois(56,10)
  data$fbm &lt;- rpois(56,10)
  data$nitrogen &lt;- rpois(56,10)
  data$carbon &lt;- rpois(56,10)
  data$chl &lt;- rpois(56,10)
  data$ID &lt;- 1:56
  
  data &lt;- as.data.frame(data)
  caldata &lt;- data[1:28,] # define model training set
  valdata &lt;- data[29:56,] # define model testing set
  
  # define explanatory variables (x)
  spectra &lt;- caldata[,1:101]
  
  # build PLS model using training data only
  library(pls)
  refl.pls &lt;- plsr(height ~ spectra, data = caldata, ncomp = 10, validation = 
  "LOO", jackknife = TRUE)
  </code></pre>
  
  <p>It was then identified that a model comprising of 3 components yielded the best performance without over-fitting. Hence, the following command was used to predict the values of the 28 observations in the testing set using the above calibrated PLS model with 3 components:</p>
  
  <pre><code>predict(refl.pls, ncomp = 3, newdata = valdata)
  </code></pre>
  
  <p>Sensible as the output may seem, I soon discovered that all this piece of code generates are the fitted values of the PLS model for the calibration/training data, rather than predictions. I discovered this because the below code, in which newdata = is omitted, yields identical results.</p>
  
  <pre><code>predict(refl.pls, ncomp = 3)
  </code></pre>
  
  <p>Surely something must be going wrong, although I cannot seem to find out what specifically is. Is there someone out there who can, and is willing to help me move in the right direction?</p>