title: Measuring VAR accuracy using accuracy() from forecast
body: <p>I'm trying to learn a vector autoregressive model using the <code>vars</code> package in R. This package doesn't have any way to measure the accuracy of the returned model. </p>
  
  <p>Specifically, I want to use MASE as defined in the <code>accuracy</code> function from the <code>forecast</code> package in R to compare forecasting with VAR with forecasting using Arima models on each component time series (I'm using 4 possibly correlated time series). <code>accuracy</code> doesn't recognize the <code>varest</code> object returned by <code>vars</code>. How can I get the MASE for each forecasted component? I want to calculate both in-sample and out-of-sample accuracy</p>
  
  <p>Code example:</p>
  
  <pre><code>library(vars)
  library(forecast)
  data(Canada)
  v&lt;- VAR(window(Canada, end=c(1998,4)), p=2)
  accuracy(v$varresult[[1]])
  </code></pre>
  
  <p>The argument of <code>accuracy</code> is an lm object and returns the training accuracy for series 1 as:</p>
  
  <pre><code>                       ME      RMSE       MAE           MPE      MAPE       MASE
  Training set 1.536303e-15 0.3346096 0.2653946 -1.288309e-05 0.0281736 0.03914555
  </code></pre>
  
  <p>I want to get the out-of-sample test accuracy using something like (not exactly this, since the forecast period needs to be specified):</p>
  
  <pre><code> accuracy(v$varresult[[1]], window(Canada[,1], start=c(1999,1)))
  </code></pre>
  
  <p>but this is not supported for lm objects and returns the error</p>
  
  <pre><code> Error in testaccuracy(f, x, test) : Unknown list structure
  </code></pre>
  
  <p>And if I use the values directly as follows, I don't get the MASE, which needs information about the training set. This is also prone to off-by-one errors because values are used and not <code>ts</code> objects, for which <code>accuracy</code> will match the stored times directly:</p>
  
  <pre><code> p&lt;-predict(v, n.ahead=8)
   accuracy(p$fcst[[1]][,"fcst"],window(Canada[,1], start=c(1999,1)))
  
               ME      RMSE       MAE         MPE       MAPE      ACF1 Theil's U
  Test set -0.1058358 0.8585455 0.7385238 -0.01114099 0.07694492 0.5655117  1.359761
  </code></pre>
  
  <p>Ideally, I should like to forecast it as:</p>
  
  <pre><code>fr&lt;-forecast(v$varresult[[1]], h=8)
  </code></pre>
  
  <p>but this can't work because it needs the other series for the prediction, and gives:</p>
  
  <pre><code>Error in eval(expr, envir, enclos) : object 'datamat' not found
  </code></pre>
  
  <p>I <em>could</em> try copying the functionality of <code>forecast.Arima</code> etc and try writing a <code>forecast.varresult</code> package, but is there a simpler way out?</p>