title: Should I tune a learner with the whole dataset?
body: <p>Recently I was learning about using mlr3 package, and I got some problems:</p>
  <p>(1) I read <a href="https://mlr-org.com/docs/cv-vs-predict/" rel="nofollow noreferrer">The &quot;Cross-Validation - Train/Predict&quot; misunderstanding</a>. This sounds to me that: I should focus on the average unbiased estimate performance of the model as the result of nested sampling. To me, tuning is adding CV in the training set (i.e., find the best hyperparameters and the feature importance rank), just like the figure below.
  <a href="https://i.stack.imgur.com/JH04a.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/JH04a.png" alt="A Data Analysis Plan" /></a>
  However, many examples in the mlr3gallery (e.g., <a href="https://mlr3gallery.mlr-org.com/posts/2021-03-09-practical-tuning-series-tune-a-support-vector-machine/" rel="nofollow noreferrer">tuning a SVM</a>; <a href="https://mlr3gallery.mlr-org.com/posts/2021-03-11-practical-tuning-series-build-an-automated-machine-learning-system/" rel="nofollow noreferrer">tuning a graph</a>) seem to tune their learners with the whole dateset, which confused me. Is it appropriate to use all the data for tuning learner?</p>
  <p>(2) I wonder if there is any way to combine <em>Feature Selection</em> and <em>Model Tuning</em> together. I find the function in mlr <a href="https://mlr.mlr-org.com/articles/tutorial/nested_resampling.html#example-3-one-task-two-learners-feature-filtering-with-tuning-1" rel="nofollow noreferrer">feature-filtering-with-tuning</a>, but not in mlr3. I guess a pipeline graph might be help, but I could not find a tutorial.</p>
  <p>Thanks a lot! Any helps would be grateful!</p>