title: mlr3: using benchmark() with tuned models (i.e. AutoTuner objects)
body: <p>I would like to compare the performance of several machine learning algorithms (e.g., decisions trees from rpart, xgb, ...) including their hyperparameter tuning using mlr3. In other words, I would like to compare already tuned instances of different algorithms rather than the algorithms with their default hyperparameter values.</p>
  <p>mlr3 provides AutoTuner-Objects to carry out nested resampling and hyperparameter tuning. There is also a benchmark() function to conduct comparisons of several learners. The benchmark() function in turn uses benchmark_grid() to set up the benchmarking. According to <a href="https://rdrr.io/github/mlr-org/mlr3tuning/man/AutoTuner.html" rel="nofollow noreferrer">this manual</a> one can pass &quot;an AutoTuner object to mlr3::resample() or mlr3::benchmark()&quot;. I do not understand how I can pass an AutoTuner object to benchmark_grid().</p>
  <p>The following code (benchmarking a tuned decision tree with the default version; based on the code in <a href="https://mlr3book.mlr-org.com/" rel="nofollow noreferrer">this book</a>) is not working. It returns an error message: &quot;Error: Element with key 'rpart_tuned' not found in DictionaryLearner!&quot;</p>
  <pre><code>library(&quot;mlr3verse&quot;)
  
  ### Benchmarking including hyperparameter tuning
  
  # nested resampling:
  # - inner sampling: 5-fold CV
  # - outer sampling: manually defined hold-out sample
  
  # defining AutoTuner for the inner resampling
  learner = lrn(&quot;classif.rpart&quot;)
  resampling = rsmp(&quot;cv&quot;, folds = 5)
  # resampling = rsmp(&quot;holdout&quot;)
  measure = msr(&quot;classif.acc&quot;)
  search_space = ps(maxdepth = p_int(lower = 1, upper = 10))
  terminator = trm(&quot;none&quot;)
  tuner = tnr(&quot;grid_search&quot;, resolution = 5)
  
  rpart_tuned = AutoTuner$new(learner, resampling, measure, terminator, tuner, search_space)
  
  ## Outer re-sampling
  # hold-out sample with pre-defined partitioning into train and test set
  outer_resampling = rsmp(&quot;custom&quot;)
  train_sets = list(1:120)
  test_sets = list(121:150)
  outer_resampling$instantiate(task, train_sets, test_sets)
  
  ## Defining benchmark design
  
  design = benchmark_grid(
    tasks = tsks(c(&quot;iris&quot;)),
    learners = lrns(c(&quot;rpart_tuned&quot;, &quot;classif.rpart&quot;),
                    predict_type = &quot;prob&quot;, predict_sets = c(&quot;train&quot;, &quot;test&quot;)),
    resamplings = outer_resampling
  )
  </code></pre>